Introduction to drcarlate

This introduction has three purposes:

  • Describes the functions and structures included with drcarlate.

  • Show how to use the functions in drcarlate for numerical simulation to get results similar to those in Jiang et al.(2022).

  • Provides an example of research using drcarlate.

Section 1

There are six families of functions in drcarlate:

  • Data generation: these functions provide a way to generate random data according to the three DGPs in Jiang et al.(2022).

  • Statistics calculation: these functions are used to calculate the statistics and the LATE used in Jiang et al.(2022).

  • Estimation strategy: these functions provide a method for estimating treatment effects according to the three estimation strategies including L, NL and R presented in Jiang et al.(2022).

  • Output function: this function integrates the functions of the above three types of functions and provides a means of output results.

  • JLTZ function: this function calls all of the above functions and helps the user reproduce the results of the data simulation section of Jiang et al.(2022).

  • ATE functions: these functions are based on some of the above functions, focusing on calculating average treatment effect(ATE) under the full compliance condition, one characteristic of these functions is that they begin with the ATE prefix.

Data generation

This section contains two functions, FuncDGP and CovAdptRnd:

  • FuncDGP allows users to generate the corresponding data using the three algorithms declared in the data generation process in Jiang et al.(2022).

  • CovadptRnd provides four CAR schemes proposed in Jiang et al.(2022).

  • Combined with the above two functions, users can generate 3 x 4 = 12 sets of data.

# Parameter dgpflag declares three ways to generate random data.
# FuncDGP will generate random data according to the method specified in (i), (ii) or (iii) when dgpflag = 1, 2 or 3 respectively.

# Parameter rndflag declares four ways to randomly assign treatment effects.
# rndflag = 1 - SRS; rndflag = 2 - WEI; rndflag = 3 - BCD; rndflag = 4 - SBR
# Note that CovadpRnd is built into FuncDGP, so it is not necessary to use CovadpRnd alone in the actual operation of generating random data

# Let's take dgpflag=1 and rndflag=2 for example
random_dgp <- FuncDGP(dgptype = 1, rndflag = 2, n = 100, g = 4, pi = c(0.5, 0.5, 0.5, 0.5))
# We can see that the return value of FuncDGP is a list of nine matrices, We can easily extract what we need from it
Y <- random_dgp$Y
X <- random_dgp$X
S <- random_dgp$S
A <- random_dgp$A
Y1 <- random_dgp$Y1
Y0 <- random_dgp$Y0
D1 <- random_dgp$D1
D0 <- random_dgp$D0
D <- random_dgp$D

Calculated statistics

This section contains four functions including TrueValue, pihat, tau and stanE.

  • pihat, tau and stanE computes estimated treatment assignment probabilities, LATE and standard deviation respectively, see Jiang et al.(2022) for more details.
# compute estimated LATE
tauhat <- tau(muY1 = Y1, muY0 = Y0, muD1 = D1, muD0 = D0, A = A, S = S, Y = Y, D = D)

#compute estimated treatment assignment probabilities
pihat(A = A, S = S)
#>             [,1]
#>   [1,] 0.4883721
#>   [2,] 0.5500000
#>   [3,] 0.5500000
#>   [4,] 0.4883721
#>   [5,] 0.4883721
#>   [6,] 0.5500000
#>   [7,] 0.4883721
#>   [8,] 0.4883721
#>   [9,] 0.4883721
#>  [10,] 0.5500000
#>  [11,] 0.5200000
#>  [12,] 0.4883721
#>  [13,] 0.4883721
#>  [14,] 0.5833333
#>  [15,] 0.5200000
#>  [16,] 0.4883721
#>  [17,] 0.4883721
#>  [18,] 0.5833333
#>  [19,] 0.4883721
#>  [20,] 0.4883721
#>  [21,] 0.5833333
#>  [22,] 0.4883721
#>  [23,] 0.4883721
#>  [24,] 0.5200000
#>  [25,] 0.5200000
#>  [26,] 0.5500000
#>  [27,] 0.4883721
#>  [28,] 0.5200000
#>  [29,] 0.5200000
#>  [30,] 0.5500000
#>  [31,] 0.5500000
#>  [32,] 0.5833333
#>  [33,] 0.5833333
#>  [34,] 0.4883721
#>  [35,] 0.5200000
#>  [36,] 0.5500000
#>  [37,] 0.5500000
#>  [38,] 0.5833333
#>  [39,] 0.5200000
#>  [40,] 0.4883721
#>  [41,] 0.4883721
#>  [42,] 0.4883721
#>  [43,] 0.5200000
#>  [44,] 0.5500000
#>  [45,] 0.5500000
#>  [46,] 0.4883721
#>  [47,] 0.5200000
#>  [48,] 0.4883721
#>  [49,] 0.4883721
#>  [50,] 0.4883721
#>  [51,] 0.5200000
#>  [52,] 0.5200000
#>  [53,] 0.5200000
#>  [54,] 0.5833333
#>  [55,] 0.5200000
#>  [56,] 0.5200000
#>  [57,] 0.4883721
#>  [58,] 0.4883721
#>  [59,] 0.4883721
#>  [60,] 0.4883721
#>  [61,] 0.5500000
#>  [62,] 0.5200000
#>  [63,] 0.4883721
#>  [64,] 0.5833333
#>  [65,] 0.4883721
#>  [66,] 0.5200000
#>  [67,] 0.5500000
#>  [68,] 0.5500000
#>  [69,] 0.4883721
#>  [70,] 0.4883721
#>  [71,] 0.5200000
#>  [72,] 0.5200000
#>  [73,] 0.5833333
#>  [74,] 0.5200000
#>  [75,] 0.5200000
#>  [76,] 0.5500000
#>  [77,] 0.5833333
#>  [78,] 0.5500000
#>  [79,] 0.4883721
#>  [80,] 0.5500000
#>  [81,] 0.4883721
#>  [82,] 0.4883721
#>  [83,] 0.5500000
#>  [84,] 0.4883721
#>  [85,] 0.5500000
#>  [86,] 0.5200000
#>  [87,] 0.4883721
#>  [88,] 0.4883721
#>  [89,] 0.4883721
#>  [90,] 0.4883721
#>  [91,] 0.4883721
#>  [92,] 0.4883721
#>  [93,] 0.4883721
#>  [94,] 0.5200000
#>  [95,] 0.5500000
#>  [96,] 0.5833333
#>  [97,] 0.5833333
#>  [98,] 0.5200000
#>  [99,] 0.4883721
#> [100,] 0.5200000

# compute estimated standard deviation
stanE(muY1 = Y1, muY0 = Y0, muD1 = D1, muD0 = D0, A = A, S = S, Y = Y, D = D, tauhat = tauhat)
#> [1] 5.307009
  • TrueValue differs a little from the above three functions，it calculates the LATE of all four kinds of CAR schemes (rndflag = 1, 2, 3 and 4) under the specified DGP (dgpflag = 1, 2, or 3). So the user who runs Truevalue is supposed to get four values at once.
# let's take dgpflag = 1 for example.
true_value <- TrueValue(dgptype = 1, vIdx = 1:4, n = 100, g = 4, pi = c(0.5, 0.5, 0.5, 0.5))
true_tau <- true_value$tau
# SRS - WEI - BCD - SBR
true_tau
#> [1] 0.9218209 0.9211110 0.9208200 0.9244314

Estimation strategy

This section contains three functions including LogisticReg, feasiblePostLassoMatTool and LinearLogit.

  • LinearLogit provides users with three regression estimation combination strategies: L (modelflag = 1), NP (modelflag = 2), and R (modelflag = 3), see Jiang et al.(2022) for more details.

  • feasiblePostLassoMatTool is the computing engine for function LinearLogit when modelflag = 3.

  • LogisticReg provides a simple way to use logistic CDF.

# remember that we set dgpflag = 1 and rndflag = 2 before
LinearLogit(Y = Y, D = D, A = A, X = X, S = S, s = 4, modelflag = 1, iridge = 0.001)
#> $theta_0s
#>            [,1]
#> [1,] -0.3107327
#> [2,]  1.3760984
#> 
#> $theta_1s
#>            [,1]
#> [1,] -0.1730392
#> [2,]  1.4584989
#> 
#> $beta_0s
#>            [,1]
#> [1,] 0.10144099
#> [2,] 0.03317667
#> 
#> $beta_1s
#>            [,1]
#> [1,] -0.1406257
#> [2,]  0.3252247

Output function

  • Output is the most important integration function, which has the functions of generating random data, estimating LATE by regression analysis based on different estimation strategies: (1) NA (2) LP (3) LG (4) F (5) NP (6) R (when dgp = 3) (7) TSLS (8) R (when dgp = 1 or 2) and summarizing analysis results. This function is key to generating the simulation results in Jiang et al.(2022). See the paper for more details about all the estimation strategies.
# set random seed
set.seed(1)
# get true tau
true_tau <- TrueValue(dgptype = 1, vIdx = 1:4, n = 1000, g = 4, pi = c(0.5, 0.5, 0.5, 0.5))
#> currently at 1th sample for simulating true value
#> currently at 2th sample for simulating true value
#> currently at 3th sample for simulating true value
#> currently at 4th sample for simulating true value
#> currently at 5th sample for simulating true value
#> currently at 6th sample for simulating true value
#> currently at 7th sample for simulating true value
#> currently at 8th sample for simulating true value
#> currently at 9th sample for simulating true value
#> currently at 10th sample for simulating true value
#> currently at 11th sample for simulating true value
#> currently at 12th sample for simulating true value
#> currently at 13th sample for simulating true value
#> currently at 14th sample for simulating true value
#> currently at 15th sample for simulating true value
#> currently at 16th sample for simulating true value
#> currently at 17th sample for simulating true value
#> currently at 18th sample for simulating true value
#> currently at 19th sample for simulating true value
#> currently at 20th sample for simulating true value
#> currently at 21th sample for simulating true value
#> currently at 22th sample for simulating true value
#> currently at 23th sample for simulating true value
#> currently at 24th sample for simulating true value
#> currently at 25th sample for simulating true value
#> currently at 26th sample for simulating true value
#> currently at 27th sample for simulating true value
#> currently at 28th sample for simulating true value
#> currently at 29th sample for simulating true value
#> currently at 30th sample for simulating true value
#> currently at 31th sample for simulating true value
#> currently at 32th sample for simulating true value
#> currently at 33th sample for simulating true value
#> currently at 34th sample for simulating true value
#> currently at 35th sample for simulating true value
#> currently at 36th sample for simulating true value
#> currently at 37th sample for simulating true value
#> currently at 38th sample for simulating true value
#> currently at 39th sample for simulating true value
#> currently at 40th sample for simulating true value
#> currently at 41th sample for simulating true value
#> currently at 42th sample for simulating true value
#> currently at 43th sample for simulating true value
#> currently at 44th sample for simulating true value
#> currently at 45th sample for simulating true value
#> currently at 46th sample for simulating true value
#> currently at 47th sample for simulating true value
#> currently at 48th sample for simulating true value
#> currently at 49th sample for simulating true value
#> currently at 50th sample for simulating true value
#> currently at 51th sample for simulating true value
#> currently at 52th sample for simulating true value
#> currently at 53th sample for simulating true value
#> currently at 54th sample for simulating true value
#> currently at 55th sample for simulating true value
#> currently at 56th sample for simulating true value
#> currently at 57th sample for simulating true value
#> currently at 58th sample for simulating true value
#> currently at 59th sample for simulating true value
#> currently at 60th sample for simulating true value
#> currently at 61th sample for simulating true value
#> currently at 62th sample for simulating true value
#> currently at 63th sample for simulating true value
#> currently at 64th sample for simulating true value
#> currently at 65th sample for simulating true value
#> currently at 66th sample for simulating true value
#> currently at 67th sample for simulating true value
#> currently at 68th sample for simulating true value
#> currently at 69th sample for simulating true value
#> currently at 70th sample for simulating true value
#> currently at 71th sample for simulating true value
#> currently at 72th sample for simulating true value
#> currently at 73th sample for simulating true value
#> currently at 74th sample for simulating true value
#> currently at 75th sample for simulating true value
#> currently at 76th sample for simulating true value
#> currently at 77th sample for simulating true value
#> currently at 78th sample for simulating true value
#> currently at 79th sample for simulating true value
#> currently at 80th sample for simulating true value
#> currently at 81th sample for simulating true value
#> currently at 82th sample for simulating true value
#> currently at 83th sample for simulating true value
#> currently at 84th sample for simulating true value
#> currently at 85th sample for simulating true value
#> currently at 86th sample for simulating true value
#> currently at 87th sample for simulating true value
#> currently at 88th sample for simulating true value
#> currently at 89th sample for simulating true value
#> currently at 90th sample for simulating true value
#> currently at 91th sample for simulating true value
#> currently at 92th sample for simulating true value
#> currently at 93th sample for simulating true value
#> currently at 94th sample for simulating true value
#> currently at 95th sample for simulating true value
#> currently at 96th sample for simulating true value
#> currently at 97th sample for simulating true value
#> currently at 98th sample for simulating true value
#> currently at 99th sample for simulating true value
#> currently at 100th sample for simulating true value
#> currently at 101th sample for simulating true value
#> currently at 102th sample for simulating true value
#> currently at 103th sample for simulating true value
#> currently at 104th sample for simulating true value
#> currently at 105th sample for simulating true value
#> currently at 106th sample for simulating true value
#> currently at 107th sample for simulating true value
#> currently at 108th sample for simulating true value
#> currently at 109th sample for simulating true value
#> currently at 110th sample for simulating true value
#> currently at 111th sample for simulating true value
#> currently at 112th sample for simulating true value
#> currently at 113th sample for simulating true value
#> currently at 114th sample for simulating true value
#> currently at 115th sample for simulating true value
#> currently at 116th sample for simulating true value
#> currently at 117th sample for simulating true value
#> currently at 118th sample for simulating true value
#> currently at 119th sample for simulating true value
#> currently at 120th sample for simulating true value
#> currently at 121th sample for simulating true value
#> currently at 122th sample for simulating true value
#> currently at 123th sample for simulating true value
#> currently at 124th sample for simulating true value
#> currently at 125th sample for simulating true value
#> currently at 126th sample for simulating true value
#> currently at 127th sample for simulating true value
#> currently at 128th sample for simulating true value
#> currently at 129th sample for simulating true value
#> currently at 130th sample for simulating true value
#> currently at 131th sample for simulating true value
#> currently at 132th sample for simulating true value
#> currently at 133th sample for simulating true value
#> currently at 134th sample for simulating true value
#> currently at 135th sample for simulating true value
#> currently at 136th sample for simulating true value
#> currently at 137th sample for simulating true value
#> currently at 138th sample for simulating true value
#> currently at 139th sample for simulating true value
#> currently at 140th sample for simulating true value
#> currently at 141th sample for simulating true value
#> currently at 142th sample for simulating true value
#> currently at 143th sample for simulating true value
#> currently at 144th sample for simulating true value
#> currently at 145th sample for simulating true value
#> currently at 146th sample for simulating true value
#> currently at 147th sample for simulating true value
#> currently at 148th sample for simulating true value
#> currently at 149th sample for simulating true value
#> currently at 150th sample for simulating true value
#> currently at 151th sample for simulating true value
#> currently at 152th sample for simulating true value
#> currently at 153th sample for simulating true value
#> currently at 154th sample for simulating true value
#> currently at 155th sample for simulating true value
#> currently at 156th sample for simulating true value
#> currently at 157th sample for simulating true value
#> currently at 158th sample for simulating true value
#> currently at 159th sample for simulating true value
#> currently at 160th sample for simulating true value
#> currently at 161th sample for simulating true value
#> currently at 162th sample for simulating true value
#> currently at 163th sample for simulating true value
#> currently at 164th sample for simulating true value
#> currently at 165th sample for simulating true value
#> currently at 166th sample for simulating true value
#> currently at 167th sample for simulating true value
#> currently at 168th sample for simulating true value
#> currently at 169th sample for simulating true value
#> currently at 170th sample for simulating true value
#> currently at 171th sample for simulating true value
#> currently at 172th sample for simulating true value
#> currently at 173th sample for simulating true value
#> currently at 174th sample for simulating true value
#> currently at 175th sample for simulating true value
#> currently at 176th sample for simulating true value
#> currently at 177th sample for simulating true value
#> currently at 178th sample for simulating true value
#> currently at 179th sample for simulating true value
#> currently at 180th sample for simulating true value
#> currently at 181th sample for simulating true value
#> currently at 182th sample for simulating true value
#> currently at 183th sample for simulating true value
#> currently at 184th sample for simulating true value
#> currently at 185th sample for simulating true value
#> currently at 186th sample for simulating true value
#> currently at 187th sample for simulating true value
#> currently at 188th sample for simulating true value
#> currently at 189th sample for simulating true value
#> currently at 190th sample for simulating true value
#> currently at 191th sample for simulating true value
#> currently at 192th sample for simulating true value
#> currently at 193th sample for simulating true value
#> currently at 194th sample for simulating true value
#> currently at 195th sample for simulating true value
#> currently at 196th sample for simulating true value
#> currently at 197th sample for simulating true value
#> currently at 198th sample for simulating true value
#> currently at 199th sample for simulating true value
#> currently at 200th sample for simulating true value
#> currently at 201th sample for simulating true value
#> currently at 202th sample for simulating true value
#> currently at 203th sample for simulating true value
#> currently at 204th sample for simulating true value
#> currently at 205th sample for simulating true value
#> currently at 206th sample for simulating true value
#> currently at 207th sample for simulating true value
#> currently at 208th sample for simulating true value
#> currently at 209th sample for simulating true value
#> currently at 210th sample for simulating true value
#> currently at 211th sample for simulating true value
#> currently at 212th sample for simulating true value
#> currently at 213th sample for simulating true value
#> currently at 214th sample for simulating true value
#> currently at 215th sample for simulating true value
#> currently at 216th sample for simulating true value
#> currently at 217th sample for simulating true value
#> currently at 218th sample for simulating true value
#> currently at 219th sample for simulating true value
#> currently at 220th sample for simulating true value
#> currently at 221th sample for simulating true value
#> currently at 222th sample for simulating true value
#> currently at 223th sample for simulating true value
#> currently at 224th sample for simulating true value
#> currently at 225th sample for simulating true value
#> currently at 226th sample for simulating true value
#> currently at 227th sample for simulating true value
#> currently at 228th sample for simulating true value
#> currently at 229th sample for simulating true value
#> currently at 230th sample for simulating true value
#> currently at 231th sample for simulating true value
#> currently at 232th sample for simulating true value
#> currently at 233th sample for simulating true value
#> currently at 234th sample for simulating true value
#> currently at 235th sample for simulating true value
#> currently at 236th sample for simulating true value
#> currently at 237th sample for simulating true value
#> currently at 238th sample for simulating true value
#> currently at 239th sample for simulating true value
#> currently at 240th sample for simulating true value
#> currently at 241th sample for simulating true value
#> currently at 242th sample for simulating true value
#> currently at 243th sample for simulating true value
#> currently at 244th sample for simulating true value
#> currently at 245th sample for simulating true value
#> currently at 246th sample for simulating true value
#> currently at 247th sample for simulating true value
#> currently at 248th sample for simulating true value
#> currently at 249th sample for simulating true value
#> currently at 250th sample for simulating true value
#> currently at 251th sample for simulating true value
#> currently at 252th sample for simulating true value
#> currently at 253th sample for simulating true value
#> currently at 254th sample for simulating true value
#> currently at 255th sample for simulating true value
#> currently at 256th sample for simulating true value
#> currently at 257th sample for simulating true value
#> currently at 258th sample for simulating true value
#> currently at 259th sample for simulating true value
#> currently at 260th sample for simulating true value
#> currently at 261th sample for simulating true value
#> currently at 262th sample for simulating true value
#> currently at 263th sample for simulating true value
#> currently at 264th sample for simulating true value
#> currently at 265th sample for simulating true value
#> currently at 266th sample for simulating true value
#> currently at 267th sample for simulating true value
#> currently at 268th sample for simulating true value
#> currently at 269th sample for simulating true value
#> currently at 270th sample for simulating true value
#> currently at 271th sample for simulating true value
#> currently at 272th sample for simulating true value
#> currently at 273th sample for simulating true value
#> currently at 274th sample for simulating true value
#> currently at 275th sample for simulating true value
#> currently at 276th sample for simulating true value
#> currently at 277th sample for simulating true value
#> currently at 278th sample for simulating true value
#> currently at 279th sample for simulating true value
#> currently at 280th sample for simulating true value
#> currently at 281th sample for simulating true value
#> currently at 282th sample for simulating true value
#> currently at 283th sample for simulating true value
#> currently at 284th sample for simulating true value
#> currently at 285th sample for simulating true value
#> currently at 286th sample for simulating true value
#> currently at 287th sample for simulating true value
#> currently at 288th sample for simulating true value
#> currently at 289th sample for simulating true value
#> currently at 290th sample for simulating true value
#> currently at 291th sample for simulating true value
#> currently at 292th sample for simulating true value
#> currently at 293th sample for simulating true value
#> currently at 294th sample for simulating true value
#> currently at 295th sample for simulating true value
#> currently at 296th sample for simulating true value
#> currently at 297th sample for simulating true value
#> currently at 298th sample for simulating true value
#> currently at 299th sample for simulating true value
#> currently at 300th sample for simulating true value
#> currently at 301th sample for simulating true value
#> currently at 302th sample for simulating true value
#> currently at 303th sample for simulating true value
#> currently at 304th sample for simulating true value
#> currently at 305th sample for simulating true value
#> currently at 306th sample for simulating true value
#> currently at 307th sample for simulating true value
#> currently at 308th sample for simulating true value
#> currently at 309th sample for simulating true value
#> currently at 310th sample for simulating true value
#> currently at 311th sample for simulating true value
#> currently at 312th sample for simulating true value
#> currently at 313th sample for simulating true value
#> currently at 314th sample for simulating true value
#> currently at 315th sample for simulating true value
#> currently at 316th sample for simulating true value
#> currently at 317th sample for simulating true value
#> currently at 318th sample for simulating true value
#> currently at 319th sample for simulating true value
#> currently at 320th sample for simulating true value
#> currently at 321th sample for simulating true value
#> currently at 322th sample for simulating true value
#> currently at 323th sample for simulating true value
#> currently at 324th sample for simulating true value
#> currently at 325th sample for simulating true value
#> currently at 326th sample for simulating true value
#> currently at 327th sample for simulating true value
#> currently at 328th sample for simulating true value
#> currently at 329th sample for simulating true value
#> currently at 330th sample for simulating true value
#> currently at 331th sample for simulating true value
#> currently at 332th sample for simulating true value
#> currently at 333th sample for simulating true value
#> currently at 334th sample for simulating true value
#> currently at 335th sample for simulating true value
#> currently at 336th sample for simulating true value
#> currently at 337th sample for simulating true value
#> currently at 338th sample for simulating true value
#> currently at 339th sample for simulating true value
#> currently at 340th sample for simulating true value
#> currently at 341th sample for simulating true value
#> currently at 342th sample for simulating true value
#> currently at 343th sample for simulating true value
#> currently at 344th sample for simulating true value
#> currently at 345th sample for simulating true value
#> currently at 346th sample for simulating true value
#> currently at 347th sample for simulating true value
#> currently at 348th sample for simulating true value
#> currently at 349th sample for simulating true value
#> currently at 350th sample for simulating true value
#> currently at 351th sample for simulating true value
#> currently at 352th sample for simulating true value
#> currently at 353th sample for simulating true value
#> currently at 354th sample for simulating true value
#> currently at 355th sample for simulating true value
#> currently at 356th sample for simulating true value
#> currently at 357th sample for simulating true value
#> currently at 358th sample for simulating true value
#> currently at 359th sample for simulating true value
#> currently at 360th sample for simulating true value
#> currently at 361th sample for simulating true value
#> currently at 362th sample for simulating true value
#> currently at 363th sample for simulating true value
#> currently at 364th sample for simulating true value
#> currently at 365th sample for simulating true value
#> currently at 366th sample for simulating true value
#> currently at 367th sample for simulating true value
#> currently at 368th sample for simulating true value
#> currently at 369th sample for simulating true value
#> currently at 370th sample for simulating true value
#> currently at 371th sample for simulating true value
#> currently at 372th sample for simulating true value
#> currently at 373th sample for simulating true value
#> currently at 374th sample for simulating true value
#> currently at 375th sample for simulating true value
#> currently at 376th sample for simulating true value
#> currently at 377th sample for simulating true value
#> currently at 378th sample for simulating true value
#> currently at 379th sample for simulating true value
#> currently at 380th sample for simulating true value
#> currently at 381th sample for simulating true value
#> currently at 382th sample for simulating true value
#> currently at 383th sample for simulating true value
#> currently at 384th sample for simulating true value
#> currently at 385th sample for simulating true value
#> currently at 386th sample for simulating true value
#> currently at 387th sample for simulating true value
#> currently at 388th sample for simulating true value
#> currently at 389th sample for simulating true value
#> currently at 390th sample for simulating true value
#> currently at 391th sample for simulating true value
#> currently at 392th sample for simulating true value
#> currently at 393th sample for simulating true value
#> currently at 394th sample for simulating true value
#> currently at 395th sample for simulating true value
#> currently at 396th sample for simulating true value
#> currently at 397th sample for simulating true value
#> currently at 398th sample for simulating true value
#> currently at 399th sample for simulating true value
#> currently at 400th sample for simulating true value
#> currently at 401th sample for simulating true value
#> currently at 402th sample for simulating true value
#> currently at 403th sample for simulating true value
#> currently at 404th sample for simulating true value
#> currently at 405th sample for simulating true value
#> currently at 406th sample for simulating true value
#> currently at 407th sample for simulating true value
#> currently at 408th sample for simulating true value
#> currently at 409th sample for simulating true value
#> currently at 410th sample for simulating true value
#> currently at 411th sample for simulating true value
#> currently at 412th sample for simulating true value
#> currently at 413th sample for simulating true value
#> currently at 414th sample for simulating true value
#> currently at 415th sample for simulating true value
#> currently at 416th sample for simulating true value
#> currently at 417th sample for simulating true value
#> currently at 418th sample for simulating true value
#> currently at 419th sample for simulating true value
#> currently at 420th sample for simulating true value
#> currently at 421th sample for simulating true value
#> currently at 422th sample for simulating true value
#> currently at 423th sample for simulating true value
#> currently at 424th sample for simulating true value
#> currently at 425th sample for simulating true value
#> currently at 426th sample for simulating true value
#> currently at 427th sample for simulating true value
#> currently at 428th sample for simulating true value
#> currently at 429th sample for simulating true value
#> currently at 430th sample for simulating true value
#> currently at 431th sample for simulating true value
#> currently at 432th sample for simulating true value
#> currently at 433th sample for simulating true value
#> currently at 434th sample for simulating true value
#> currently at 435th sample for simulating true value
#> currently at 436th sample for simulating true value
#> currently at 437th sample for simulating true value
#> currently at 438th sample for simulating true value
#> currently at 439th sample for simulating true value
#> currently at 440th sample for simulating true value
#> currently at 441th sample for simulating true value
#> currently at 442th sample for simulating true value
#> currently at 443th sample for simulating true value
#> currently at 444th sample for simulating true value
#> currently at 445th sample for simulating true value
#> currently at 446th sample for simulating true value
#> currently at 447th sample for simulating true value
#> currently at 448th sample for simulating true value
#> currently at 449th sample for simulating true value
#> currently at 450th sample for simulating true value
#> currently at 451th sample for simulating true value
#> currently at 452th sample for simulating true value
#> currently at 453th sample for simulating true value
#> currently at 454th sample for simulating true value
#> currently at 455th sample for simulating true value
#> currently at 456th sample for simulating true value
#> currently at 457th sample for simulating true value
#> currently at 458th sample for simulating true value
#> currently at 459th sample for simulating true value
#> currently at 460th sample for simulating true value
#> currently at 461th sample for simulating true value
#> currently at 462th sample for simulating true value
#> currently at 463th sample for simulating true value
#> currently at 464th sample for simulating true value
#> currently at 465th sample for simulating true value
#> currently at 466th sample for simulating true value
#> currently at 467th sample for simulating true value
#> currently at 468th sample for simulating true value
#> currently at 469th sample for simulating true value
#> currently at 470th sample for simulating true value
#> currently at 471th sample for simulating true value
#> currently at 472th sample for simulating true value
#> currently at 473th sample for simulating true value
#> currently at 474th sample for simulating true value
#> currently at 475th sample for simulating true value
#> currently at 476th sample for simulating true value
#> currently at 477th sample for simulating true value
#> currently at 478th sample for simulating true value
#> currently at 479th sample for simulating true value
#> currently at 480th sample for simulating true value
#> currently at 481th sample for simulating true value
#> currently at 482th sample for simulating true value
#> currently at 483th sample for simulating true value
#> currently at 484th sample for simulating true value
#> currently at 485th sample for simulating true value
#> currently at 486th sample for simulating true value
#> currently at 487th sample for simulating true value
#> currently at 488th sample for simulating true value
#> currently at 489th sample for simulating true value
#> currently at 490th sample for simulating true value
#> currently at 491th sample for simulating true value
#> currently at 492th sample for simulating true value
#> currently at 493th sample for simulating true value
#> currently at 494th sample for simulating true value
#> currently at 495th sample for simulating true value
#> currently at 496th sample for simulating true value
#> currently at 497th sample for simulating true value
#> currently at 498th sample for simulating true value
#> currently at 499th sample for simulating true value
#> currently at 500th sample for simulating true value
#> currently at 501th sample for simulating true value
#> currently at 502th sample for simulating true value
#> currently at 503th sample for simulating true value
#> currently at 504th sample for simulating true value
#> currently at 505th sample for simulating true value
#> currently at 506th sample for simulating true value
#> currently at 507th sample for simulating true value
#> currently at 508th sample for simulating true value
#> currently at 509th sample for simulating true value
#> currently at 510th sample for simulating true value
#> currently at 511th sample for simulating true value
#> currently at 512th sample for simulating true value
#> currently at 513th sample for simulating true value
#> currently at 514th sample for simulating true value
#> currently at 515th sample for simulating true value
#> currently at 516th sample for simulating true value
#> currently at 517th sample for simulating true value
#> currently at 518th sample for simulating true value
#> currently at 519th sample for simulating true value
#> currently at 520th sample for simulating true value
#> currently at 521th sample for simulating true value
#> currently at 522th sample for simulating true value
#> currently at 523th sample for simulating true value
#> currently at 524th sample for simulating true value
#> currently at 525th sample for simulating true value
#> currently at 526th sample for simulating true value
#> currently at 527th sample for simulating true value
#> currently at 528th sample for simulating true value
#> currently at 529th sample for simulating true value
#> currently at 530th sample for simulating true value
#> currently at 531th sample for simulating true value
#> currently at 532th sample for simulating true value
#> currently at 533th sample for simulating true value
#> currently at 534th sample for simulating true value
#> currently at 535th sample for simulating true value
#> currently at 536th sample for simulating true value
#> currently at 537th sample for simulating true value
#> currently at 538th sample for simulating true value
#> currently at 539th sample for simulating true value
#> currently at 540th sample for simulating true value
#> currently at 541th sample for simulating true value
#> currently at 542th sample for simulating true value
#> currently at 543th sample for simulating true value
#> currently at 544th sample for simulating true value
#> currently at 545th sample for simulating true value
#> currently at 546th sample for simulating true value
#> currently at 547th sample for simulating true value
#> currently at 548th sample for simulating true value
#> currently at 549th sample for simulating true value
#> currently at 550th sample for simulating true value
#> currently at 551th sample for simulating true value
#> currently at 552th sample for simulating true value
#> currently at 553th sample for simulating true value
#> currently at 554th sample for simulating true value
#> currently at 555th sample for simulating true value
#> currently at 556th sample for simulating true value
#> currently at 557th sample for simulating true value
#> currently at 558th sample for simulating true value
#> currently at 559th sample for simulating true value
#> currently at 560th sample for simulating true value
#> currently at 561th sample for simulating true value
#> currently at 562th sample for simulating true value
#> currently at 563th sample for simulating true value
#> currently at 564th sample for simulating true value
#> currently at 565th sample for simulating true value
#> currently at 566th sample for simulating true value
#> currently at 567th sample for simulating true value
#> currently at 568th sample for simulating true value
#> currently at 569th sample for simulating true value
#> currently at 570th sample for simulating true value
#> currently at 571th sample for simulating true value
#> currently at 572th sample for simulating true value
#> currently at 573th sample for simulating true value
#> currently at 574th sample for simulating true value
#> currently at 575th sample for simulating true value
#> currently at 576th sample for simulating true value
#> currently at 577th sample for simulating true value
#> currently at 578th sample for simulating true value
#> currently at 579th sample for simulating true value
#> currently at 580th sample for simulating true value
#> currently at 581th sample for simulating true value
#> currently at 582th sample for simulating true value
#> currently at 583th sample for simulating true value
#> currently at 584th sample for simulating true value
#> currently at 585th sample for simulating true value
#> currently at 586th sample for simulating true value
#> currently at 587th sample for simulating true value
#> currently at 588th sample for simulating true value
#> currently at 589th sample for simulating true value
#> currently at 590th sample for simulating true value
#> currently at 591th sample for simulating true value
#> currently at 592th sample for simulating true value
#> currently at 593th sample for simulating true value
#> currently at 594th sample for simulating true value
#> currently at 595th sample for simulating true value
#> currently at 596th sample for simulating true value
#> currently at 597th sample for simulating true value
#> currently at 598th sample for simulating true value
#> currently at 599th sample for simulating true value
#> currently at 600th sample for simulating true value
#> currently at 601th sample for simulating true value
#> currently at 602th sample for simulating true value
#> currently at 603th sample for simulating true value
#> currently at 604th sample for simulating true value
#> currently at 605th sample for simulating true value
#> currently at 606th sample for simulating true value
#> currently at 607th sample for simulating true value
#> currently at 608th sample for simulating true value
#> currently at 609th sample for simulating true value
#> currently at 610th sample for simulating true value
#> currently at 611th sample for simulating true value
#> currently at 612th sample for simulating true value
#> currently at 613th sample for simulating true value
#> currently at 614th sample for simulating true value
#> currently at 615th sample for simulating true value
#> currently at 616th sample for simulating true value
#> currently at 617th sample for simulating true value
#> currently at 618th sample for simulating true value
#> currently at 619th sample for simulating true value
#> currently at 620th sample for simulating true value
#> currently at 621th sample for simulating true value
#> currently at 622th sample for simulating true value
#> currently at 623th sample for simulating true value
#> currently at 624th sample for simulating true value
#> currently at 625th sample for simulating true value
#> currently at 626th sample for simulating true value
#> currently at 627th sample for simulating true value
#> currently at 628th sample for simulating true value
#> currently at 629th sample for simulating true value
#> currently at 630th sample for simulating true value
#> currently at 631th sample for simulating true value
#> currently at 632th sample for simulating true value
#> currently at 633th sample for simulating true value
#> currently at 634th sample for simulating true value
#> currently at 635th sample for simulating true value
#> currently at 636th sample for simulating true value
#> currently at 637th sample for simulating true value
#> currently at 638th sample for simulating true value
#> currently at 639th sample for simulating true value
#> currently at 640th sample for simulating true value
#> currently at 641th sample for simulating true value
#> currently at 642th sample for simulating true value
#> currently at 643th sample for simulating true value
#> currently at 644th sample for simulating true value
#> currently at 645th sample for simulating true value
#> currently at 646th sample for simulating true value
#> currently at 647th sample for simulating true value
#> currently at 648th sample for simulating true value
#> currently at 649th sample for simulating true value
#> currently at 650th sample for simulating true value
#> currently at 651th sample for simulating true value
#> currently at 652th sample for simulating true value
#> currently at 653th sample for simulating true value
#> currently at 654th sample for simulating true value
#> currently at 655th sample for simulating true value
#> currently at 656th sample for simulating true value
#> currently at 657th sample for simulating true value
#> currently at 658th sample for simulating true value
#> currently at 659th sample for simulating true value
#> currently at 660th sample for simulating true value
#> currently at 661th sample for simulating true value
#> currently at 662th sample for simulating true value
#> currently at 663th sample for simulating true value
#> currently at 664th sample for simulating true value
#> currently at 665th sample for simulating true value
#> currently at 666th sample for simulating true value
#> currently at 667th sample for simulating true value
#> currently at 668th sample for simulating true value
#> currently at 669th sample for simulating true value
#> currently at 670th sample for simulating true value
#> currently at 671th sample for simulating true value
#> currently at 672th sample for simulating true value
#> currently at 673th sample for simulating true value
#> currently at 674th sample for simulating true value
#> currently at 675th sample for simulating true value
#> currently at 676th sample for simulating true value
#> currently at 677th sample for simulating true value
#> currently at 678th sample for simulating true value
#> currently at 679th sample for simulating true value
#> currently at 680th sample for simulating true value
#> currently at 681th sample for simulating true value
#> currently at 682th sample for simulating true value
#> currently at 683th sample for simulating true value
#> currently at 684th sample for simulating true value
#> currently at 685th sample for simulating true value
#> currently at 686th sample for simulating true value
#> currently at 687th sample for simulating true value
#> currently at 688th sample for simulating true value
#> currently at 689th sample for simulating true value
#> currently at 690th sample for simulating true value
#> currently at 691th sample for simulating true value
#> currently at 692th sample for simulating true value
#> currently at 693th sample for simulating true value
#> currently at 694th sample for simulating true value
#> currently at 695th sample for simulating true value
#> currently at 696th sample for simulating true value
#> currently at 697th sample for simulating true value
#> currently at 698th sample for simulating true value
#> currently at 699th sample for simulating true value
#> currently at 700th sample for simulating true value
#> currently at 701th sample for simulating true value
#> currently at 702th sample for simulating true value
#> currently at 703th sample for simulating true value
#> currently at 704th sample for simulating true value
#> currently at 705th sample for simulating true value
#> currently at 706th sample for simulating true value
#> currently at 707th sample for simulating true value
#> currently at 708th sample for simulating true value
#> currently at 709th sample for simulating true value
#> currently at 710th sample for simulating true value
#> currently at 711th sample for simulating true value
#> currently at 712th sample for simulating true value
#> currently at 713th sample for simulating true value
#> currently at 714th sample for simulating true value
#> currently at 715th sample for simulating true value
#> currently at 716th sample for simulating true value
#> currently at 717th sample for simulating true value
#> currently at 718th sample for simulating true value
#> currently at 719th sample for simulating true value
#> currently at 720th sample for simulating true value
#> currently at 721th sample for simulating true value
#> currently at 722th sample for simulating true value
#> currently at 723th sample for simulating true value
#> currently at 724th sample for simulating true value
#> currently at 725th sample for simulating true value
#> currently at 726th sample for simulating true value
#> currently at 727th sample for simulating true value
#> currently at 728th sample for simulating true value
#> currently at 729th sample for simulating true value
#> currently at 730th sample for simulating true value
#> currently at 731th sample for simulating true value
#> currently at 732th sample for simulating true value
#> currently at 733th sample for simulating true value
#> currently at 734th sample for simulating true value
#> currently at 735th sample for simulating true value
#> currently at 736th sample for simulating true value
#> currently at 737th sample for simulating true value
#> currently at 738th sample for simulating true value
#> currently at 739th sample for simulating true value
#> currently at 740th sample for simulating true value
#> currently at 741th sample for simulating true value
#> currently at 742th sample for simulating true value
#> currently at 743th sample for simulating true value
#> currently at 744th sample for simulating true value
#> currently at 745th sample for simulating true value
#> currently at 746th sample for simulating true value
#> currently at 747th sample for simulating true value
#> currently at 748th sample for simulating true value
#> currently at 749th sample for simulating true value
#> currently at 750th sample for simulating true value
#> currently at 751th sample for simulating true value
#> currently at 752th sample for simulating true value
#> currently at 753th sample for simulating true value
#> currently at 754th sample for simulating true value
#> currently at 755th sample for simulating true value
#> currently at 756th sample for simulating true value
#> currently at 757th sample for simulating true value
#> currently at 758th sample for simulating true value
#> currently at 759th sample for simulating true value
#> currently at 760th sample for simulating true value
#> currently at 761th sample for simulating true value
#> currently at 762th sample for simulating true value
#> currently at 763th sample for simulating true value
#> currently at 764th sample for simulating true value
#> currently at 765th sample for simulating true value
#> currently at 766th sample for simulating true value
#> currently at 767th sample for simulating true value
#> currently at 768th sample for simulating true value
#> currently at 769th sample for simulating true value
#> currently at 770th sample for simulating true value
#> currently at 771th sample for simulating true value
#> currently at 772th sample for simulating true value
#> currently at 773th sample for simulating true value
#> currently at 774th sample for simulating true value
#> currently at 775th sample for simulating true value
#> currently at 776th sample for simulating true value
#> currently at 777th sample for simulating true value
#> currently at 778th sample for simulating true value
#> currently at 779th sample for simulating true value
#> currently at 780th sample for simulating true value
#> currently at 781th sample for simulating true value
#> currently at 782th sample for simulating true value
#> currently at 783th sample for simulating true value
#> currently at 784th sample for simulating true value
#> currently at 785th sample for simulating true value
#> currently at 786th sample for simulating true value
#> currently at 787th sample for simulating true value
#> currently at 788th sample for simulating true value
#> currently at 789th sample for simulating true value
#> currently at 790th sample for simulating true value
#> currently at 791th sample for simulating true value
#> currently at 792th sample for simulating true value
#> currently at 793th sample for simulating true value
#> currently at 794th sample for simulating true value
#> currently at 795th sample for simulating true value
#> currently at 796th sample for simulating true value
#> currently at 797th sample for simulating true value
#> currently at 798th sample for simulating true value
#> currently at 799th sample for simulating true value
#> currently at 800th sample for simulating true value
#> currently at 801th sample for simulating true value
#> currently at 802th sample for simulating true value
#> currently at 803th sample for simulating true value
#> currently at 804th sample for simulating true value
#> currently at 805th sample for simulating true value
#> currently at 806th sample for simulating true value
#> currently at 807th sample for simulating true value
#> currently at 808th sample for simulating true value
#> currently at 809th sample for simulating true value
#> currently at 810th sample for simulating true value
#> currently at 811th sample for simulating true value
#> currently at 812th sample for simulating true value
#> currently at 813th sample for simulating true value
#> currently at 814th sample for simulating true value
#> currently at 815th sample for simulating true value
#> currently at 816th sample for simulating true value
#> currently at 817th sample for simulating true value
#> currently at 818th sample for simulating true value
#> currently at 819th sample for simulating true value
#> currently at 820th sample for simulating true value
#> currently at 821th sample for simulating true value
#> currently at 822th sample for simulating true value
#> currently at 823th sample for simulating true value
#> currently at 824th sample for simulating true value
#> currently at 825th sample for simulating true value
#> currently at 826th sample for simulating true value
#> currently at 827th sample for simulating true value
#> currently at 828th sample for simulating true value
#> currently at 829th sample for simulating true value
#> currently at 830th sample for simulating true value
#> currently at 831th sample for simulating true value
#> currently at 832th sample for simulating true value
#> currently at 833th sample for simulating true value
#> currently at 834th sample for simulating true value
#> currently at 835th sample for simulating true value
#> currently at 836th sample for simulating true value
#> currently at 837th sample for simulating true value
#> currently at 838th sample for simulating true value
#> currently at 839th sample for simulating true value
#> currently at 840th sample for simulating true value
#> currently at 841th sample for simulating true value
#> currently at 842th sample for simulating true value
#> currently at 843th sample for simulating true value
#> currently at 844th sample for simulating true value
#> currently at 845th sample for simulating true value
#> currently at 846th sample for simulating true value
#> currently at 847th sample for simulating true value
#> currently at 848th sample for simulating true value
#> currently at 849th sample for simulating true value
#> currently at 850th sample for simulating true value
#> currently at 851th sample for simulating true value
#> currently at 852th sample for simulating true value
#> currently at 853th sample for simulating true value
#> currently at 854th sample for simulating true value
#> currently at 855th sample for simulating true value
#> currently at 856th sample for simulating true value
#> currently at 857th sample for simulating true value
#> currently at 858th sample for simulating true value
#> currently at 859th sample for simulating true value
#> currently at 860th sample for simulating true value
#> currently at 861th sample for simulating true value
#> currently at 862th sample for simulating true value
#> currently at 863th sample for simulating true value
#> currently at 864th sample for simulating true value
#> currently at 865th sample for simulating true value
#> currently at 866th sample for simulating true value
#> currently at 867th sample for simulating true value
#> currently at 868th sample for simulating true value
#> currently at 869th sample for simulating true value
#> currently at 870th sample for simulating true value
#> currently at 871th sample for simulating true value
#> currently at 872th sample for simulating true value
#> currently at 873th sample for simulating true value
#> currently at 874th sample for simulating true value
#> currently at 875th sample for simulating true value
#> currently at 876th sample for simulating true value
#> currently at 877th sample for simulating true value
#> currently at 878th sample for simulating true value
#> currently at 879th sample for simulating true value
#> currently at 880th sample for simulating true value
#> currently at 881th sample for simulating true value
#> currently at 882th sample for simulating true value
#> currently at 883th sample for simulating true value
#> currently at 884th sample for simulating true value
#> currently at 885th sample for simulating true value
#> currently at 886th sample for simulating true value
#> currently at 887th sample for simulating true value
#> currently at 888th sample for simulating true value
#> currently at 889th sample for simulating true value
#> currently at 890th sample for simulating true value
#> currently at 891th sample for simulating true value
#> currently at 892th sample for simulating true value
#> currently at 893th sample for simulating true value
#> currently at 894th sample for simulating true value
#> currently at 895th sample for simulating true value
#> currently at 896th sample for simulating true value
#> currently at 897th sample for simulating true value
#> currently at 898th sample for simulating true value
#> currently at 899th sample for simulating true value
#> currently at 900th sample for simulating true value
#> currently at 901th sample for simulating true value
#> currently at 902th sample for simulating true value
#> currently at 903th sample for simulating true value
#> currently at 904th sample for simulating true value
#> currently at 905th sample for simulating true value
#> currently at 906th sample for simulating true value
#> currently at 907th sample for simulating true value
#> currently at 908th sample for simulating true value
#> currently at 909th sample for simulating true value
#> currently at 910th sample for simulating true value
#> currently at 911th sample for simulating true value
#> currently at 912th sample for simulating true value
#> currently at 913th sample for simulating true value
#> currently at 914th sample for simulating true value
#> currently at 915th sample for simulating true value
#> currently at 916th sample for simulating true value
#> currently at 917th sample for simulating true value
#> currently at 918th sample for simulating true value
#> currently at 919th sample for simulating true value
#> currently at 920th sample for simulating true value
#> currently at 921th sample for simulating true value
#> currently at 922th sample for simulating true value
#> currently at 923th sample for simulating true value
#> currently at 924th sample for simulating true value
#> currently at 925th sample for simulating true value
#> currently at 926th sample for simulating true value
#> currently at 927th sample for simulating true value
#> currently at 928th sample for simulating true value
#> currently at 929th sample for simulating true value
#> currently at 930th sample for simulating true value
#> currently at 931th sample for simulating true value
#> currently at 932th sample for simulating true value
#> currently at 933th sample for simulating true value
#> currently at 934th sample for simulating true value
#> currently at 935th sample for simulating true value
#> currently at 936th sample for simulating true value
#> currently at 937th sample for simulating true value
#> currently at 938th sample for simulating true value
#> currently at 939th sample for simulating true value
#> currently at 940th sample for simulating true value
#> currently at 941th sample for simulating true value
#> currently at 942th sample for simulating true value
#> currently at 943th sample for simulating true value
#> currently at 944th sample for simulating true value
#> currently at 945th sample for simulating true value
#> currently at 946th sample for simulating true value
#> currently at 947th sample for simulating true value
#> currently at 948th sample for simulating true value
#> currently at 949th sample for simulating true value
#> currently at 950th sample for simulating true value
#> currently at 951th sample for simulating true value
#> currently at 952th sample for simulating true value
#> currently at 953th sample for simulating true value
#> currently at 954th sample for simulating true value
#> currently at 955th sample for simulating true value
#> currently at 956th sample for simulating true value
#> currently at 957th sample for simulating true value
#> currently at 958th sample for simulating true value
#> currently at 959th sample for simulating true value
#> currently at 960th sample for simulating true value
#> currently at 961th sample for simulating true value
#> currently at 962th sample for simulating true value
#> currently at 963th sample for simulating true value
#> currently at 964th sample for simulating true value
#> currently at 965th sample for simulating true value
#> currently at 966th sample for simulating true value
#> currently at 967th sample for simulating true value
#> currently at 968th sample for simulating true value
#> currently at 969th sample for simulating true value
#> currently at 970th sample for simulating true value
#> currently at 971th sample for simulating true value
#> currently at 972th sample for simulating true value
#> currently at 973th sample for simulating true value
#> currently at 974th sample for simulating true value
#> currently at 975th sample for simulating true value
#> currently at 976th sample for simulating true value
#> currently at 977th sample for simulating true value
#> currently at 978th sample for simulating true value
#> currently at 979th sample for simulating true value
#> currently at 980th sample for simulating true value
#> currently at 981th sample for simulating true value
#> currently at 982th sample for simulating true value
#> currently at 983th sample for simulating true value
#> currently at 984th sample for simulating true value
#> currently at 985th sample for simulating true value
#> currently at 986th sample for simulating true value
#> currently at 987th sample for simulating true value
#> currently at 988th sample for simulating true value
#> currently at 989th sample for simulating true value
#> currently at 990th sample for simulating true value
#> currently at 991th sample for simulating true value
#> currently at 992th sample for simulating true value
#> currently at 993th sample for simulating true value
#> currently at 994th sample for simulating true value
#> currently at 995th sample for simulating true value
#> currently at 996th sample for simulating true value
#> currently at 997th sample for simulating true value
#> currently at 998th sample for simulating true value
#> currently at 999th sample for simulating true value
#> currently at 1000th sample for simulating true value
# see the output: size, iPert = 0
Output(ii = 1, tau = tauhat[1], dgptype = 1, rndflag = 1, n = 1000, g = 4, 
       pi = c(0.5, 0.5, 0.5, 0.5),
       iPert = 0, iq = 0.05, iridge = 0.001)
#> Currently at 1 th sample!
#> $vtauhat
#>          [,1]     [,2]     [,3]     [,4]     [,5] [,6]      [,7]     [,8] [,9]
#> [1,] 1.236257 1.011659 1.014877 1.034401 1.070561  NaN 0.9797705 1.116384  NaN
#>      [,10] [,11] [,12]
#> [1,]   NaN   NaN   NaN
#> 
#> $vsighat
#>          [,1]     [,2]     [,3]     [,4]     [,5] [,6]     [,7]     [,8] [,9]
#> [1,] 14.71728 10.93291 10.97075 10.81983 9.691528  NaN 11.10584 9.792131  NaN
#>      [,10] [,11] [,12]
#> [1,]   NaN   NaN   NaN
#> 
#> $vstat
#>           [,1]      [,2]      [,3]      [,4]      [,5] [,6]      [,7]      [,8]
#> [1,] 0.9127023 0.5789947 0.5862718 0.6515118 0.8453483  NaN 0.4791787 0.9846466
#>      [,9] [,10] [,11] [,12]
#> [1,]  NaN   NaN   NaN   NaN
#> 
#> $vdeci
#>      [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10] [,11] [,12]
#> [1,]    0    0    0    0    0  NaN    0    0  NaN   NaN   NaN   NaN

# see the output: power, iPert = 1
Output(ii = 1, tau = tauhat[1], dgptype = 1, rndflag = 1, n = 1000, g = 4, 
       pi = c(0.5, 0.5, 0.5, 0.5),
       iPert = 1, iq = 0.05, iridge = 0.001)
#> Currently at 1 th sample!
#> $vtauhat
#>          [,1]     [,2]     [,3]     [,4]     [,5] [,6]     [,7]     [,8] [,9]
#> [1,] 1.848939 1.270918 1.271093 1.313234 1.036465  NaN 1.266284 1.060324  NaN
#>      [,10] [,11] [,12]
#> [1,]   NaN   NaN   NaN
#> 
#> $vsighat
#>          [,1]     [,2]     [,3]     [,4]     [,5] [,6]     [,7]     [,8] [,9]
#> [1,] 14.14772 10.91436 10.91949 10.57321 9.554213  NaN 10.48309 9.306626  NaN
#>      [,10] [,11] [,12]
#> [1,]   NaN   NaN   NaN
#> 
#> $vstat
#>            [,1]     [,2]    [,3]     [,4]     [,5] [,6]     [,7]    [,8] [,9]
#> [1,] 0.08371788 1.566213 1.56497 1.490185 2.565178  NaN 1.644623 2.55235  NaN
#>      [,10] [,11] [,12]
#> [1,]   NaN   NaN   NaN
#> 
#> $vdeci
#>      [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10] [,11] [,12]
#> [1,]    0    0    0    0    1  NaN    0    1  NaN   NaN   NaN   NaN

JLTZ function

  • JLTZ provides users with a convenient way to reproduce the simulation results in Jiang et al.(2022), The user only needs to set parameters according to the corresponding DGP, sample size and Monte Carlo simulation times in the paper, and the results consistent with Jiang et al.(2022) can be obtained under the condition that the random number seed is set as 1. It should be noted that because the results in the original text are implemented in MATLAB, the results obtained by using JLTZ are not numerically the same as those in the paper, but they are very close.
# For example, if we wanted to get the results shown in Panel A in Table 1, we could run the following two functions separately.

# First of all, There are four strata data (g = 4), the probability of data being treated at each strata is equal to 0.5 (pi = c(0.5, 0.5, 0.5, 0.5)), the random data generation process follows the data generation process 1 (DGP = 1), the total sample size is 200 (n = 200), run 10,000 Monte Carlo simulations (iMonte = 10000), the confidence level for hypothesis testing is 5%, and finally，get the size of the Monte Carlo simulation (iPert = 0).
# Time Consumeing. This command will take about 1.4 hours to execute, so do not run it unless necessary.

# JLTZ(iMonte = 10000, dgptype = 1, n = 200, g = 4, pi = c(0.5, 0.5, 0.5, 0.5), iPert = 0, iq = 0.05, iridge = 0.001)

# Second of all, There are four strata data (g = 4), the probability of data being treated at each strata is equal to 0.5 (pi = c(0.5, 0.5, 0.5, 0.5)), the random data generation process follows the digital generation process 1 (DGP = 1), the total sample size was 200 (n = 200), run 10,000 Monte Carlo simulations (iMonte = 10000), the confidence level for hypothesis testing is 5%, and finally，get the power of the Monte Carlo simulation (iPert = 1).
# Time Consumeing. This command will take about 1.4 hours to execute, so do not run it unless necessary.

# JLTZ(iMonte = 10000, dgptype = 1, n = 200, g = 4, pi = c(0.5, 0.5, 0.5, 0.5), iPert = 1, iq = 0.05, iridge = 0.001)

ATE functions

  • ATEDGP is a version of FuncDGP under full compliance condition.
  • ATETrueValue is a version of TrueValue under full compliance condition.
  • ATEOutput is a version of Output under full compliance condition.
  • ATEJLTZ is a version of JLTZ under full compliance condition.
  • The arguments and results of the ATE- functions and the non-ATE functions are basically identical. It should be noted that the results of ATEOutput is a list containing four 1x4 vectors rather than a list containing four 1x12 vectors, which is the result of Output. For details, please refer to the help documentation for related functions.

Section 2

In this section we will show the user how to reproduce the results of the numerical simulation section of Jiang et al.(2022) by using the JLTZ function. Since it may take a long time to get the results of all three kinds of GDPS in Jiang et al. (2022), we will focus on only one case as an example.

Let’s consider a case like this:

  • First, the total sample was set at 200, which means size = 200.

  • And second, the number of Monte Carlo simulations was set at 10,000, which means iMonte = 10000.

  • Third, calculate the size of the hypothesis test instead of the power and consider a five percent confidence interval in hypothesis testing,, which means iPert = 0 and iq = 0.05.

  • Fourth, the data generation process follows DGP-1 in Jiang et al.(2022), which means dgptype = 1.

  • Fifth, the samples are divided into four strata, and the probability of the samples being processed in each strata is 0.5, which means g = 4 and pi = c(0.5, 0.5, 0.5, 0.5).

  • Sixth, the penalization parameter in ridge regression is 0.001, which means iridge = 0.001.

# With the above Settings in mind, we get the following JLTZ function.

# JLTZ(iMonte = 10000, dgptype = 1, n = 200, g = 4, pi = c(0.5,0.5,0.5,0.5), iPert = 0,iq = 0.05, iridge = 0.001)

# Since it takes about 1.4 hours to run this function, we'll give you the results directly that users can check them out themselves.

#        vProb_d1   vProb_d2   vProb_d3   vProb_d4
# [1,] 0.03139898 0.03548546 0.03071509 0.03139898
# [2,] 0.04356403 0.04189256 0.03951368 0.04356403
# [3,] 0.04307085 0.04238541 0.03919373 0.04307085
# [4,] 0.05622226 0.04632824 0.05327148 0.05622226
# [5,] 0.10833470 0.08854937 0.09486482 0.10833470
# [6,]        NaN        NaN        NaN        NaN
# [7,] 0.03435805 0.03334976 0.03455447 0.03435805
# [8,] 0.04586553 0.05076392 0.05087186 0.04586553

Section 3

In this section we further show an example of using the drcarlate package to analyze real data. For detailed information on this case, see Section 7 and Table 5 of Jiang et al.(2022).


# Set up ------------------------------------------------------------------
library(drcarlate)
library(pracma)

# Load data ---------------------------------------------------------------
# data_for_final.csv add covariates b_total_income log_b_total_income b_exp_total_30days
# edu_index b_asset_index b_health_index to data_for_tab4.csv

data_table <- drcarlate::data_table

data1 <- as.matrix(data_table)

colnames(data1) <- NULL

# create S for wave 1: number of stratnum, total 41 stratnum
n <- size(data1,1)
S <- zeros(n,1)
for (i in 1:41) {
  S[data1[, size(data1,2)-i+1] == 1] <- 41 - i + 1
}

data1 <- cbind(data1, S)

# create Y, X, S, D: number of outcomes considered is 9:
# save the results for NA, TSLS, L, NL, F
vtauhat <- NaN * ones(8,5)
vsighat <- NaN * ones(8,5)
n_vec <- NaN * ones(8,1)

iridge = 0.01 #tuning parameter for in ridge regression

for (i in 1:8) {
  if (i <= 4) {
  # data_used: 1st col- Y, 2nd col- A, 3rd col- D, 4~(end-1)th col- X,
  # last col is strata number
    data_used <- cbind(data1[, 4+(i-1)*3], data1[, 2:3], data1[, 5:6],
                       data1[, size(data1,2)-1-41], data1[, size(data1,2)])
  } else {
    data_used <- cbind(data1[, 4+(i-1)*3], data1[,2:3], data1[, (4+(i-1)*3+1):(4+(i-1)*3+2)],
                       data1[, size(data1,2)-1-41], data1[, size(data1,2)])
  }

  # delete the outcome variables with NaN
  data_used <- data_used[!is.nan(data_used[,1]), ]

  # check whether a strata has obs less than 10
  for (s in 1:41) {
    if (length(data_used[data_used[, size(data_used,2)] == s,1]) < 10) {
      # delete strata has obs less than 10
     data_used[data_used[, size(data_used,2)] ==s, ] <- matrix()
     data_used <- data_used[!is.na(data_used[,1]),]
    }
  }

  # update n
  n <- size(data_used,1)
  n_vec[i] <- n

  # find the unique value for stratnum
  stratnum <-  sort(base::unique(data_used[, size(data_used,2)]))

  # create A
  A <-  matrix(data_used[, 2])

  # create D
  D <- matrix(data_used[, 3])

  # create Y
  Y <- matrix(data_used[, 1])

  # create S
  S <- matrix(data_used[, size(data_used,2)])

  # create X
  # use baseline total income
  X <- data_used[,4:6]

  # make a index
  print(stringr::str_c("Now i equals to ", i, " !"))

  #%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
  #  %% No adjustment (z) NA
  #%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
  muY0_z <- zeros(n,1)
  muY1_z <- zeros(n,1)
  muD0_z <- zeros(n,1)
  muD1_z <- zeros(n,1)

  vtauhat[i,1] <- tau(muY1 = muY1_z, muY0 = muY0_z, muD1 = muD1_z, muD0 = muD0_z,
                      A = A, S = S, Y = Y, D = D, stratnum = stratnum)
  vsighat[i,1] <- stanE(muY1 = muY1_z, muY0 = muY0_z, muD1 = muD1_z, muD0 = muD0_z,
                        A = A, S = S, Y = Y, D = D, tauhat = vtauhat[i,1], stratnum = stratnum)/sqrt(n)

  #%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
  #  %% TSLS
  #%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
  mS_iv <- zeros(n, length(stratnum))

  for (j in 1:length(stratnum)) {
    s <- stratnum[j]
    mS_iv[S==s, j] <- 1
  }

  mX_iv <- cbind(D, X, mS_iv)
  mZ_iv <- cbind(A, X, mS_iv)
  K <- size(mX_iv,2)

  vPara_iv <- inv(t(mZ_iv) %*% mX_iv) %*% t(mZ_iv) %*% Y
  mE_iv2 <- diag(((Y - mX_iv %*% vPara_iv)^2)[,])
  m0_iv <- inv(t(mZ_iv) %*% mX_iv/n) %*% (t(mZ_iv) %*% mE_iv2 %*% mZ_iv/n) %*% inv(t(mX_iv) %*% mZ_iv/n)
  vtauhat[i,2] <- vPara_iv[1]
  vsighat[i,2] <- sqrt(m0_iv[1,1])/sqrt(n)

  #%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
  #  %% Linear+linear model (L)
  #%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
  muY0_a <- NaN*ones(n,1)
  muY1_a <- NaN*ones(n,1)
  muD0_a <- NaN*ones(n,1)
  muD1_a <- NaN*ones(n,1)
  for (j in 1:length(stratnum)) {
    s = stratnum[j]

    result <- LinearLogit(Y = Y, D = D, A = A, X = X, S = S, s = s, modelflag = 1, iridge = iridge)

    theta_0s_a <- result[["theta_0s"]]
    theta_1s_a <- result[["theta_1s"]]
    beta_0s_a <- result[["beta_0s"]]
    beta_1s_a <- result[["beta_1s"]]

    muY0_a[S==s] <- X[S==s,] %*% theta_0s_a
    muY1_a[S==s] <- X[S==s,] %*% theta_1s_a
    muD0_a[S==s] <- X[S==s,] %*% beta_0s_a
    muD1_a[S==s] <- X[S==s,] %*% beta_1s_a
  }

  vtauhat[i,3] <- tau(muY1 = muY1_a, muY0 = muY0_a, muD1 = muD1_a, muD0 = muD0_a,
                      A = A, S = S, Y = Y, D = D, stratnum = stratnum)
  vsighat[i,3] <- stanE(muY1 = muY1_a, muY0 = muY0_a, muD1 = muD1_a, muD0 = muD0_a,
                        A = A, S = S, Y = Y, D = D, tauhat = vtauhat[i,3], stratnum = stratnum)/sqrt(n)
  #%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
  #  %% Linear+logistic model (NL)
  #%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
  muY0_b <- NaN*ones(n,1)
  muY1_b <- NaN*ones(n,1)
  muD0_b <- NaN*ones(n,1)
  muD1_b <- NaN*ones(n,1)

  for (j in 1:length(stratnum)) {
    s = stratnum[j]

    result <- LinearLogit(Y = Y, D = D, A = A, X = X, S = S, s = s, modelflag = 2, iridge = iridge)

    theta_0s_b <- result[["theta_0s"]]
    theta_1s_b <- result[["theta_1s"]]
    beta_0s_b <- result[["beta_0s"]]
    beta_1s_b <- result[["beta_1s"]]

    muY0_b[S==s] <- X[S==s,] %*% matrix(theta_0s_b[2:length(theta_0s_b)]) + theta_0s_b[1]
    muY1_b[S==s] <- X[S==s,] %*% matrix(theta_1s_b[2:length(theta_0s_b)]) + theta_1s_b[1]
    muD0_b[S==s] <- LogisticReg(x = (X[S==s,] %*% matrix(beta_0s_b[-1]) + beta_0s_b[1]))
    muD1_b[S==s] <- LogisticReg(x = (X[S==s,] %*% matrix(beta_1s_b[-1]) + beta_1s_b[1]))
  }

  vtauhat[i,4] <- tau(muY1 = muY1_b, muY0 = muY0_b, muD1 = muD1_b, muD0 = muD0_b,
                      A = A, S = S, Y = Y, D = D, stratnum = stratnum)
  vsighat[i,4] <- stanE(muY1 = muY1_b, muY0 = muY0_b, muD1 = muD1_b, muD0 = muD0_b,
                        A = A, S = S, Y = Y, D = D,tauhat = vtauhat[i,4], stratnum = stratnum)/sqrt(n)

  #%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
  #  %% Further Efficiency Improvement
  #%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
  X_c <- cbind(X, muD1_b, muD0_b)
  muY0_c <- NaN*ones(n,1)
  muY1_c <- NaN*ones(n,1)
  muD0_c <- NaN*ones(n,1)
  muD1_c <- NaN*ones(n,1)

  for (j in 1:length(stratnum)) {
    s = stratnum[j]

    result <- LinearLogit(Y = Y, D = D, A = A, X = X_c, S = S, s = s, modelflag = 1, iridge = iridge)

    theta_0s_c <- result[["theta_0s"]]
    theta_1s_c <- result[["theta_1s"]]
    beta_0s_c <- result[["beta_0s"]]
    beta_1s_c <- result[["beta_1s"]]

    muY0_c[S==s] <- X_c[S==s,] %*% theta_0s_c
    muY1_c[S==s] <- X_c[S==s,] %*% theta_1s_c
    muD0_c[S==s] <- X_c[S==s,] %*% beta_0s_c
    muD1_c[S==s] <- X_c[S==s,] %*%beta_1s_c
  }

  vtauhat[i,5] <- tau(muY1 = muY1_c, muY0 = muY0_c, muD1 = muD1_c, muD0 = muD0_c,
                      A = A, S = S, Y = Y, D = D, stratnum = stratnum)
  vsighat[i,5] <- stanE(muY1 = muY1_c, muY0 = muY0_c, muD1 = muD1_c, muD0 = muD0_c,
                        A = A, S = S, Y = Y, D = D, tauhat = vtauhat[i,5], stratnum = stratnum)/sqrt(n)
  }
#> [1] "Now i equals to 1 !"
#> Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
#> Warning: glm.fit: algorithm did not converge
#> Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
#> Warning: glm.fit: algorithm did not converge
#> Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
#> [1] "Now i equals to 2 !"
#> Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
#> Warning: glm.fit: algorithm did not converge
#> Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
#> Warning: glm.fit: algorithm did not converge
#> Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
#> [1] "Now i equals to 3 !"
#> Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
#> Warning: glm.fit: algorithm did not converge
#> Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
#> Warning: glm.fit: algorithm did not converge
#> Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
#> [1] "Now i equals to 4 !"
#> Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
#> Warning: glm.fit: algorithm did not converge
#> Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
#> Warning: glm.fit: algorithm did not converge
#> Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
#> [1] "Now i equals to 5 !"
#> Warning: glm.fit: algorithm did not converge
#> Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
#> [1] "Now i equals to 6 !"
#> Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
#> Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
#> Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
#> Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
#> Warning: glm.fit: algorithm did not converge
#> Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
#> Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
#> [1] "Now i equals to 7 !"
#> Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
#> Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
#> Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
#> Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
#> Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
#> Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
#> [1] "Now i equals to 8 !"
#> Warning: glm.fit: algorithm did not converge
#> Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred

# show the LATE estimates and standard errors in Table 5 of Jiang et al.(2022). 
# LATE Estimates
vtauhat
#>             [,1]        [,2]        [,3]        [,4]        [,5]
#> [1,]   2.7866925   7.1528747   7.1694104   7.2542016   8.4805342
#> [2,]  20.5577389  21.1544145  22.1604884  22.3940102  22.7366686
#> [3,]  -0.2076838  -0.1735085  -0.2912842  -0.2947078  -0.3099807
#> [4,]  20.3993940  21.0965957  21.9244498  22.1835123  22.3253690
#> [5,] -10.8261902  -7.4558010  -9.0040607  -8.9405514  -8.6914634
#> [6,]  -1.9332078  -2.3327993  -1.2417154  -1.2579026  -1.5505684
#> [7,]  -3.6211274  -3.3462369  -1.4278123  -1.5513955  -1.4265200
#> [8,] -17.6431819 -14.3166215 -15.6652067 -15.8118165 -15.5119289

# Standard Errors
vsighat
#>           [,1]      [,2]      [,3]      [,4]      [,5]
#> [1,] 7.2902879 6.3684935 6.1971428 6.2712205 6.1394941
#> [2,] 3.0667814 3.0151296 2.9646743 3.0037782 2.9512801
#> [3,] 0.2233596 0.2241912 0.2119157 0.2144341 0.2094491
#> [4,] 3.0890583 3.0344251 2.9792857 3.0221352 2.9639732
#> [5,] 5.0034112 4.4034599 4.4007221 4.3728318 4.4028981
#> [6,] 1.9711334 1.8580997 1.7936724 1.8155761 1.8076464
#> [7,] 2.0402640 1.9989033 1.8654540 2.0344097 1.8689752
#> [8,] 6.1995757 5.3509212 5.1849623 5.2346367 5.1130911

Reference

Jiang L, Linton O B, Tang H, Zhang Y. Improving estimation efficiency via regression-adjustment in covariate-adaptive randomizations with imperfect compliance [J]. 2022.