Trial simulation with binary toxicity measure

Overview

For illustrative purposes, we start by showing how to simulate a single trial, followed by simulations for repeated trials (100 sims). See the README file and the full publication for greater overview of the design framework.

The examples we will be working with are the following: suppose we have a study with 5 dose levels to test, each of which has an associated “true” toxicity level and “true” mean efficacy (used for data generation). Additionally, we specify an acceptable and unacceptable rate of dose-limiting toxicities (DLTs). In stage 1, equal cohort sizes are assigned to each dose level (default is 3). In stage 2, we also assign equal cohort sizes to only acceptable doses (default changes to 1). We employ a continuous efficacy outcome and a binary toxicity endpoint. The design assumes a monotonically increasing dose-toxicity relationship (i.e, as the dose increases, so does toxicity), but no monotonicity is imposed for the dose-efficacy relationship (accommodates increasing, flat, plateau trends).

In scenario 1, we will work with a linear increasing dose-efficacy relationship; in scenario 2, we show simulations for a quadratic dose-efficacy relationship.

Scenario 1: Monotone dose-efficacy

library(iAdapt)

# Number of pre-specified dose levels
dose <- 5 

# Vector of true toxicities associated with each dose
dose.tox <- c(0.05, 0.10, 0.20, 0.35, 0.45)       

# Acceptable (p_yes) and unacceptable (p_no) DLT rates used for establishing safety
p_no <- 0.40                                     
p_yes <- 0.15    

# Likelihood-ratio (LR) threshold
K <- 2                                          

# Cohort size used in stage 1
coh.size <- 3 

# Vector of true mean efficacies per dose (here mean T-cell persistence per dose (%))
m <- c(5, 15, 40, 65, 80)   # MUST BE THE SAME LENGTH AS dose.tox                  

# Efficacy (equal) variance per dose
v <- rep(0.01, 5) 

# Total sample size (stages 1&2)                            
N <- 25                                        

# Stopping rule: if dose 1 is the only safe dose, allocate up to 9 pts before ending the trial to collect more information
stop.rule <- 9

Note that, as we’ve specified them, we have monotone increasing dose-toxicity and dose-efficacy curves, as shown below. If the unacceptable dose-limiting toxicity (DLT) rate is 0.40, (horizontal dotted line, figure a), then our target/optimal dose is dose 4 (green point) - the safe dose with the highest efficacy.

## null device 
##           1

One Trial Simulation

Stage 1: Establish the safety profile for all initial doses.

Stage 1 establishes the safety profiles of the predefined doses.

Function to generate and tabulate toxicities per dose level:

set.seed(3)
tox.profile(dose = dose, dose.tox = dose.tox, p1 = p_no, p2 = p_yes, K = K, coh.size = coh.size)
##      [,1] [,2] [,3] [,4]
## [1,]    1    0    1 2.84
## [2,]    2    1    2 0.75
## [3,]    3    0    3 2.84
## [4,]    4    1    4 0.75
## [5,]    5    2    5 0.20

The first column indicates the cohort number and third columns gives the dose assignment for all specified doses (in this case, 5). The second column gives the number of DLTs observed at that dose. The fourth column gives the likelihood ratio calculated from the observed DLTs. A dose is considered acceptably safe if the LR > 1/K and unacceptably safe if LR <= 1/K.

Now let’s see which doses the design selects as being acceptably safe using K=2.

Function to select only the acceptable toxic doses:

set.seed(3)
safe.dose(dose = dose, dose.tox = dose.tox, p1 = p_no, p2 = p_yes, K = K, coh.size = coh.size) 
## $alloc.safe
##      [,1] [,2]
## [1,]    1    0
## [2,]    2    1
## [3,]    3    0
## [4,]    4    1
## 
## $alloc.total
##  [1] 1 1 1 2 2 2 3 3 3 4 4 4 5 5 5
## 
## $n1
## [1] 15

We can see that the design selects only doses 1 through 4; $alloc.safe gives the dose assignment (first column) and number of DLTs (second column). $alloc.total gives the dose assignment for all enrolled patients ($n1 gives the total sample size used in stage 1, in this case 15 subjects), where we see that 3 patients were assigned to each dose as specified by coh.size.

Stage 1 is mainly used to establish safety, but efficacy outcomes are also collected for each dose.

Function to generate efficacy outcomes (here T-cell percent persistence) for each dose:

set.seed(3)
eff.stg1(dose = dose, dose.tox = dose.tox, p1 = p_no, p2 = p_yes, K = K, coh.size = coh.size, m, v, nbb = 100)
## $Y.safe
##  [1]  1  3  5  7  3  3 54 30 34 71 76 78
## 
## $d.safe
##  [1] 1 1 1 2 2 2 3 3 3 4 4 4
## 
## $tox.safe
## [1] 0 1 0 1
## 
## $n1
## [1] 15
## 
## $Y.alloc
##  [1]  1  3  5  7  3  3 54 30 34 71 76 78 88 80 85
## 
## $d.alloc
##  [1] 1 1 1 2 2 2 3 3 3 4 4 4 5 5 5

$Y.safe and $d.safe give the efficacy values and dose allocations for all subjects enrolled at acceptably safe doses; $tox.safe gives the number of DLTs for each dose level; $Y.alloc and $d.alloc gives the efficacy values and dose allocations for all subjects enrolled in stage 1 (safe and unsafe doses). Notice that the $Y.safe and $d.safe are subsets of $Y.alloc and $d.alloc.

Stage 2: Adaptive randomization based on efficacy outcomes.

If 2 or more doses are considered acceptable after stage 1, the remaining patients are randomized to these open doses until the total sample size N is reached. If only dose 1 is acceptable after stage 1, allocate up to 9 patients (stop.rule = 9). Toxicity is still being monitored (in the ‘background’) throughout stage 2, so acceptable doses (declared in stage 1) can still be discarded based on observed DLTs. The discarded dose and all levels above it cannot be revisited.

Function to fit a linear regression for the continuous efficacy outcomes, compute the randomization probabilities per dose and allocate the next subject to an acceptable safe dose that has the highest randomization probability:

set.seed(3)
rand.stg2(dose, dose.tox, p_no, p_yes, K, coh.size, m, v, N, stop.rule = stop.rule, cohort = 1, samedose = TRUE, nbb = 100) 
## $Y.final
##  [1]  1  3  5  7  3  3 54 30 34 71 76 78 88 80 85 49 28  2 57 11 11  0  6 33  0
## 
## $d.final
##  [1] 1 1 1 2 2 2 3 3 3 4 4 4 5 5 5 4 3 1 3 1 2 1 2 2 1
## 
## $n1
## [1] 15

Notice that after stage 1 (dose 5, efficacy value 85), patients were randomized only to dose 4 or lower (acceptably safe). The complete vectors of dose allocations and efficacy outcomes for the entire trial (N=25) can be used to compute the operating characteristics of the design in repeated simulations.

100 Trials Simulations

To simulate this trial 100 times, we can run the following:

sims = 1e2

set.seed(3)
simulations = sim.trials(numsims = sims, dose, dose.tox, p1 = p_no, p2 = p_yes, K, coh.size, m, v, N, stop.rule = stop.rule, cohort = 1, samedose = TRUE, nbb = 100)

$safe.d indicates whether each dose (column) was declared safe in stage 1 (1 = yes, 0 = no) for each trial (row).

head(simulations$safe.d)
##      [,1] [,2] [,3] [,4] [,5]
## [1,]    1    1    1    1    0
## [2,]    1    1    1    1    1
## [3,]    1    1    1    0    0
## [4,]    1    1    1    0    0
## [5,]    1    1    1    0    0
## [6,]    1    1    1    0    0
head(simulations$sim.Y)
##      [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10] [,11] [,12] [,13] [,14]
## [1,]    1    3    5    7    3    3   54   30   34    71    76    78    88    80
## [2,]    3    0    3   27   11   21   25   46   38    63    63    61    92    93
## [3,]    6    0    0   53    2    1   41   44   51    68    68    66    85    79
## [4,]   22    5    0    4   14    2   47   39   47    58    75    70    13    43
## [5,]   20    0    2   17   14   30   59   42   55    58    56    79    27     7
## [6,]    5    1    3   13   22    3   37   33   64    61    75    81    47    31
##      [,15] [,16] [,17] [,18] [,19] [,20] [,21] [,22] [,23] [,24] [,25]
## [1,]    85    49    28     2    57    11    11     0     6    33     0
## [2,]    59    68    53    43     6    10    13    19    21    21    21
## [3,]    75    42     1    11     4     0     6    17     3     0     9
## [4,]    38    36    25    37    51    44    35    57     0    26    41
## [5,]    47    26    27    11     0    36    11    33    42    13    48
## [6,]    38    59    21    31    55    28    42    20     9    41    26
head(simulations$sim.d)
##      [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10] [,11] [,12] [,13] [,14]
## [1,]    1    1    1    2    2    2    3    3    3     4     4     4     5     5
## [2,]    1    1    1    2    2    2    3    3    3     4     4     4     5     5
## [3,]    1    1    1    2    2    2    3    3    3     4     4     4     5     5
## [4,]    1    1    1    2    2    2    3    3    3     4     4     4     2     3
## [5,]    1    1    1    2    2    2    3    3    3     4     4     4     3     2
## [6,]    1    1    1    2    2    2    3    3    3     4     4     4     3     3
##      [,15] [,16] [,17] [,18] [,19] [,20] [,21] [,22] [,23] [,24] [,25]
## [1,]     5     4     3     1     3     1     2     1     2     2     1
## [2,]     5     5     3     3     2     2     1     2     2     2     2
## [3,]     5     3     2     2     2     2     2     2     2     1     2
## [4,]     3     3     3     3     3     3     3     3     1     3     3
## [5,]     3     3     3     2     1     3     2     2     3     2     3
## [6,]     3     3     3     3     3     2     3     2     2     3     3

$sim.Y gives the observed outcomes, where each column corresponds to 1 patient (maximum of N columns), and each row is a simulated trial. Correspondingly, $sim.d gives the dose allocation for each patient (column) in each trial (row).

To see the proportion of times we’ve designated each dose as safe in stage 1, we can simply take the column totals of the $safe.d matrix:

colSums(simulations$safe.d) / sims
## [1] 1.00 1.00 0.91 0.65 0.30

Simulation results can be summarized. For each dose level, the inter-quartile range (25th percentile, median, 75th percentile) for the percent of subjects treated and observed efficacy are given in tables.

sim.summary(simulations)
## $pct.treated
##      [,1] [,2] [,3] [,4] [,5]
## [1,]    1 12.0   16 16.0   17
## [2,]    2 16.0   20 28.4   25
## [3,]    3 19.4   24 36.0   28
## [4,]    4 12.0   16 28.0   19
## [5,]    5  0.0   12 16.0   11
## 
## $efficacy
##      [,1]   [,2] [,3]   [,4]  [,5]
## [1,]    1  1.000  2.0  3.500  2.52
## [2,]    2 10.000 14.0 17.000 13.60
## [3,]    3 37.000 40.0 44.000 40.49
## [4,]    4 60.625 64.0 69.000 64.62
## [5,]    5 78.375 81.5 86.125 81.66

Scenario 2: Non-monotone dose-efficacy

Here, all parameters are the same as in scenario 1, except for the true mean efficacies. Instead, they take on a parabolic nature.

# Vector of true mean efficacies per dose (here mean percent persistence per dose)
m <- c(15, 35, 80, 60, 40)   # MUST BE THE SAME LENGTH AS dose.tox

Now we have a non-monotone dose-efficacy curves, as shown below. If we have an acceptable dose-limiting toxicity (DLT) rate of 0.40, (horizontal dotted line, figure a), then our target/optimal dose would be dose 4 as we did in the previous scenario (red point). However, note that according to the true efficacies of each dose, the lesser dose number 3 is actually best (green point). Assuming a monotone relationship, we would be incorrectly pushing forward a more toxic and less effective dose level.

## null device 
##           1

One Trial Simulation

Stage 1: Establish the safety profile for all initial doses

Stage 1 establishes the safety profiles of the predefined doses. All output can be interpreted in the same way as in scenario 1. However, note that in this case we identify dose level 3 as the last acceptably safe dose.

Function to generate and tabulate toxicities per dose level.

set.seed(1)
tox.profile(dose = dose, dose.tox = dose.tox, p1 = p_no, p2 = p_yes, K = K, coh.size = coh.size)
##      [,1] [,2] [,3] [,4]
## [1,]    1    0    1 2.84
## [2,]    2    0    2 2.84
## [3,]    3    1    3 0.75
## [4,]    4    2    4 0.20

Dose level 4 had a LR=0.20 < 1/2; it was considered unsafe, so allocation stopped at this level (no dose 5 was assigned).

Function to select only the acceptable safe doses

set.seed(1)
safe.dose(dose = dose, dose.tox = dose.tox, p1 = p_no, p2 = p_yes, K = K, coh.size = coh.size) 
## $alloc.safe
##      [,1] [,2]
## [1,]    1    0
## [2,]    2    0
## [3,]    3    1
## 
## $alloc.total
##  [1] 1 1 1 2 2 2 3 3 3 4 4 4
## 
## $n1
## [1] 12

Function to generate efficacy outcomes (here percent persistence) for each dose

set.seed(1)
eff.stg1(dose = dose, dose.tox = dose.tox, p1 = p_no, p2 = p_yes, K = K, coh.size = coh.size, m, v, nbb = 100)
## $Y.safe
## [1] 11 15  8 30 21 16 84 76 79
## 
## $d.safe
## [1] 1 1 1 2 2 2 3 3 3
## 
## $tox.safe
## [1] 0 0 1
## 
## $n1
## [1] 12
## 
## $Y.alloc
##  [1] 11 15  8 30 21 16 84 76 79 57 47 79
## 
## $d.alloc
##  [1] 1 1 1 2 2 2 3 3 3 4 4 4

Stage 2: Adaptive randomization based on efficacy outcomes.

If 2 or more doses are considered acceptable after stage 1, the remaining patients are randomized to these open doses until the total sample size N is reached. If only dose 1 is acceptable after stage 1, allocate up to 9 patients (stop.rule=9)

Function to fit a linear regression for the continuous efficacy outcomes, compute the randomization probabilities per dose and allocate the next subject to an acceptable safe dose that has the highest randomization probability.

set.seed(1)
rand.stg2(dose, dose.tox, p_no, p_yes, K, coh.size, m, v, N, stop.rule = stop.rule, cohort = 1, samedose = TRUE, nbb = 100) 
## $Y.final
##  [1] 11 15  8 30 21 16 84 76 79 57 47 79 85 84 62 83 75 72 69 80 90 84 88 74 34
## 
## $d.final
##  [1] 1 1 1 2 2 2 3 3 3 4 4 4 3 3 3 3 3 3 3 3 3 3 3 3 2
## 
## $n1
## [1] 12

Notice that after stage 1 (dose 4, efficacy value 79), patients were randomized to only to dose 3 orlower (acceptably safe). The complete vectors of dose allocations and efficacy outcomes for the entire trial (N=25) can be used to compute the operating characteristics of the design in repeated simulations. The complete vectors of dose allocations and efficacy outcomes can be used to compute the operating characteristics of the design in repeated simulations.

To simulate this trial 100 times, we can run the following:

sims = 1e2

set.seed(1)
simulations = sim.trials(numsims = sims, dose, dose.tox, p1 = p_no, p2 = p_yes, K, coh.size, m, v, N, stop.rule = stop.rule, cohort = 1, samedose = TRUE, nbb = 100)

$safe.d indicates whether each dose (column) was declared safe in stage 1 (1 = yes, 0 = no) for each trial (row).

head(simulations$safe.d)
##      [,1] [,2] [,3] [,4] [,5]
## [1,]    1    1    1    0    0
## [2,]    1    1    1    0    0
## [3,]    1    1    1    1    1
## [4,]    1    1    1    1    0
## [5,]    1    1    1    0    0
## [6,]    1    1    1    1    0
head(simulations$sim.Y)
##      [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10] [,11] [,12] [,13] [,14]
## [1,]   11   15    8   30   21   16   84   76   79    57    47    79    85    84
## [2,]   17   10   14   32   32   31   82   65   82    63    46    39    69    81
## [3,]   11    1    4   28   15   28   91   83   75    49    52    63    42    40
## [4,]   18    9   10   18   18   30   88   79   86    51    53    37    27    39
## [5,]   12    2    7   40   36   28   84   77   90    42    65    71    34    25
## [6,]    2   28   10   18   46   22   95   80   80    73    42    72    34    30
##      [,15] [,16] [,17] [,18] [,19] [,20] [,21] [,22] [,23] [,24] [,25]
## [1,]    62    83    75    72    69    80    90    84    88    74    34
## [2,]     8    88    33    85    84     3    82    82    74    92    75
## [3,]    45     5    36    59    86    89    48    13    51    64    71
## [4,]    37    55    87    13    84    17    13    89    89     6    28
## [5,]    54    32    56    39    29    32    42     5    33    16    38
## [6,]    49    62    30    98    61    55    85    48    51    70    11
head(simulations$sim.d)
##      [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10] [,11] [,12] [,13] [,14]
## [1,]    1    1    1    2    2    2    3    3    3     4     4     4     3     3
## [2,]    1    1    1    2    2    2    3    3    3     4     4     4     3     3
## [3,]    1    1    1    2    2    2    3    3    3     4     4     4     5     5
## [4,]    1    1    1    2    2    2    3    3    3     4     4     4     5     5
## [5,]    1    1    1    2    2    2    3    3    3     4     4     4     2     2
## [6,]    1    1    1    2    2    2    3    3    3     4     4     4     5     5
##      [,15] [,16] [,17] [,18] [,19] [,20] [,21] [,22] [,23] [,24] [,25]
## [1,]     3     3     3     3     3     3     3     3     3     3     2
## [2,]     1     3     2     3     3     1     3     3     3     3     3
## [3,]     5     1     5     4     3     3     2     1     5     2     3
## [4,]     5     4     3     2     3     1     1     3     3     1     2
## [5,]     3     2     2     2     2     2     2     1     2     1     2
## [6,]     5     4     1     3     4     2     3     1     2     3     1

$sim.Y gives the observed outcomes, where each column corresponds to 1 patient (maximum of N columns), and each row is a simulated trial. Correspondingly, $sim.d gives the dose allocation for each patient (column) in each trial (row).

To see the rate at which we’ve designated each dose as safe in stage 1, we can simply take the column totals of the $safe.d matrix:

colSums(simulations$safe.d) / sims
## [1] 1.00 0.99 0.91 0.56 0.19

Simulation results are also summarized below.

sim.summary(simulations)
## $pct.treated
##      [,1] [,2] [,3] [,4] [,5]
## [1,]    1   12   16   24   20
## [2,]    2   16   22   28   25
## [3,]    3   20   28   37   30
## [4,]    4   12   16   20   16
## [5,]    5    0   12   12    9
## 
## $efficacy
##      [,1]  [,2]  [,3]   [,4]  [,5]
## [1,]    1  9.50 12.25 16.500 13.07
## [2,]    2 30.00 33.50 37.625 33.36
## [3,]    3 78.00 80.75 83.500 80.86
## [4,]    4 57.00 61.00 64.875 60.83
## [5,]    5 35.25 41.00 44.500 39.67