Customising “clinically relevant change”

The user can provide their own function to customise the computation of clinically relevant thresholds, using the delta_fun argument of MSprog(). The user-defined function must take the baseline value as argument, and return the corresponding threshold. This applies to two scenarios.

1. The outcome of interest is among the ones listed above, but we want to define the thresholds differently. For example, we want to change the minimum \(\delta\) for SDMT to, say, “either 3 points or 10% of the reference value”. In this case, we would define:

   my_sdmt_delta <- function(x) {min(c(x/10, 3))}

   Let’s compute the minimum clinically relevant change in SDMT score starting from a baseline of 50 using our custom function, and compare it with the default function:

   print(my_sdmt_delta(50)) # my delta
   #> [1] 3
   print(compute_delta(50, outcome="sdmt")) # default delta
   #> [1] 4

   We can then compute:

   output <- MSprog(...
                   outcome="sdmt", delta_fun=my_sdmt_delta,
                    ...)

   Specifying outcome="sdmt", the outcomes values will be checked internally to verify that they are in the correct range for SDMT.

2. The outcome of interest is not among the ones listed above. In this case, we set outcome="custom", and delta_fun as our desired custom-defined function. For custom outcomes, the direction of worsening must be specified as well (i.e., whether increasing or decreasing values of the outcome are interpreted as a worsening). This can be done by setting the worsening argument to either "increase" or "decrease". Note that the provided worsening argument is only used by MSprog() when outcome is set to "custom". Otherwise, worsening is automatically set to "increase" if outcome is set to "edss", "nhpt", "t25fw", and to "decrease" if outcome is set to "sdmt".

Additional options

• For roving baseline schemes, the re-baseline procedure can be made finer through the sub_threshold_rebl argument in MSprog(): setting sub_threshold_rebl="change", for instance, rebaseline can be triggered by any confirmed change, even if below the clinically meaningful threshold. In this case, the sub-threshold improvement events are used to move the baseline, but not considered in event count.

• When a roving baseline is used, the reference is moved after a previously detected event. The new reference may be set at the date of the previous event (proceed_from="event"), or at the first available confirmation visit for such event (proceed_from="firstconf", default).

• If the baseline visit occurs in the vicinity of a clinical relapse, the reference value may be overestimated. The relapse_to_bl argument allows to specify the minimum accepted distance (in days) of the baseline visit from the onset of a previous relapse. For instance, if baseline="fixed" and relapse_to_bl=30, the baseline visit is chosen as the first available visit satisfying the requirement of “no relapses within the preceding 30 days”. If two values are provided, they are interpreted as intervals before and after the event – e.g., relapse_to_bl=c(30, 2) implements the constraint of “no relapses within the preceding 30 days or within the following 2 days”. If relapse end dates are available (may be provided as an additional column in the relapse file whose name is specified by argument renddate_col in MSprog()), the first value of relapse_to_bl is overwritten by the relapse duration, unless it was set to 0 (in which case it stays 0 and the baseline is not moved based on the influence of previous relapses). If the baseline argument is set to a roving scheme, all the above applies to every baseline change as well.

• On top of the chosen baseline scheme, post-relapse re-baseline [9] can be applied by setting relapse_rebl=TRUE in MSprog(). If this is enabled, the onset of each relapse prompts a re-baseline to the next available visit following the relapse and out of its influence (as per relapse_to_bl). As an additional constraint, the updated baseline can be forced to have a score greater than or equal to the previous baseline by setting bl_geq=TRUE in MSprog().

• Due to fluctuations in the data, the new reference value following a re-baseline may fall at a local minimum or maximum of the disability trajectory. These may be excluded for consideration as baseline visits using the skip_local_extrema argument in MSprog().

As already mentioned above, extra caution should be used when applying any re-baseline scheme to randomised data, as moving the reference value based on post-randomisation events can introduce bias (especially if the occurrence of these events is influenced by the treatment). For clinical trial data, general recommendation is to use a fixed baseline (at the time of randomisation).

Multiple events

The event argument allows to specify which events to detect. By default, it is set to "firstCDW" (only detect the first CDW event). It can be set to "multiple" to sequentially detect all CDW or CDI events.

For example, extracting multiple EDSS events for subject 4 from toydata_visits with a fixed baseline would result in the following.

print(toydata_visits[toydata_visits$id==4, c("date", "EDSS")]) # EDSS visits
#> # A tibble: 7 × 2
#>   date        EDSS
#>   <date>     <dbl>
#> 1 2021-09-18   4.5
#> 2 2021-12-04   3.5
#> 3 2022-03-12   3.5
#> 4 2022-07-19   5  
#> 5 2022-10-05   5  
#> 6 2023-01-16   5.5
#> 7 2023-04-27   5

output <- MSprog(data=toydata_visits, 
                 subj_col="id", value_col="EDSS", date_col="date", outcome="edss", 
                 subjects=4,
                 relapse=toydata_relapses, 
                 event="multiple", baseline="fixed",  # <---
                 verbose=0)
# print(output$results, row.names=FALSE) # results
DT::datatable(output$results, rownames=F,
              options = list(dom="t", scrollX=T, scrollY=F, paging = FALSE)
              )

With these settings, the EDSS improvement at visit 2 (EDSS=3.5, confirmed at visit 3) does not trigger a re-baseline. The algorithm keeps searching for events from after the confirmation visit, evaluating the changes relative to the original baseline (EDSS=4.5). The subsequent EDSS worsening is therefore not detected. On the other hand, adopting a roving baseline scheme we get:

output <- MSprog(data=toydata_visits, 
                 subj_col="id", value_col="EDSS", date_col="date", outcome="edss", 
                 subjects=4,
                 relapse=toydata_relapses, 
                 event="multiple", baseline="roving",  # <---
                 verbose=0)
# print(output$results, row.names=FALSE) # results
DT::datatable(output$results, rownames=F,
              options = list(dom="t", scrollX=T, scrollY=F, paging = FALSE)
              )

The baseline has now been moved to the confirmation visit of the first CDI event (visit 3, EDSS=3.5). With respect to this baseline, a subsequent CDW event is detected at visit 4 (EDSS=5), confirmed at visit 5.

Other valid values for the event argument are:

• "first": only the very first confirmed event – CDI or CDW
• "firstCDI": first CDI event
• "firstPIRA": first PIRA event
• "firstRAW": first RAW event.

Note. if, for example, "event=firstPIRA", all non-PIRA events preceding the first PIRA are actually detected – although not reported. This allows to combine this scenario with a roving baseline scheme, where the baseline is moved after every confirmed non-PIRA event. For instance, if a RAW event occurs first, the subsequent worsening would be counted from the re-baselined outcome value after the RAW.

Additional options

• If conf_days is specified as a sequence of values (e.g., conf_days=c(7*12,7*24)), events are retained if confirmed by at least one visit falling within any of the specified periods (here, 12 or 24 weeks with their relative tolerance interval)⁵. Confirmation dates/values at all specified values will be included in the results table (if include_dates=T / include_values=T in MSprog()).

• If the confirmation visit occurs in the vicinity of a clinical relapse, it is typically considered invalid. The relapse_to_conf argument allows to specify the minimum accepted distance (in days) of a confirmation visit from the onset of a previous relapse. For example, relapse_to_conf=30 implements the constraint “no relapses within the preceding 30 days” for a visit to be used as confirmation. If two values are provided, they are interpreted as intervals before and after the event – e.g., relapse_to_conf=c(30, 2) implements the constraint “no relapses within the preceding 30 days or within the following 2 days”. If relapse end dates are available (may be provided as an additional column in the relapse file whose name is specified by argument renddate_col in MSprog()), the first value of relapse_to_conf is overwritten by the relapse duration, unless it was set to 0 (in which case it stays 0 and confirmation visits are not discarded based on the influence of previous relapses).

• By default, a disability worsening occurring at the last available assessment is ignored by MSprog(). This behaviour can be changed using the impute_last_visit argument. If set to TRUE, any disability worsening occurring at the last visit is retained even though unconfirmed. If set to a numeric value N, unconfirmed worsening events are included only if occurring within N days of follow up (e.g., in case of early discontinuation).

• In scenarios (e.g., a clinical trial) where disability course is compared between two cohorts with different relapse rates, there may be an imbalance in visit frequency between the two groups, due to unscheduled visits after relapses. This implies a higher probability of detecting outcome changes in the group with more assessments. Such issue is sometimes addressed by only using scheduled visits as confirmation visits [11]. It is possible to implement this with MSprog() by including an additional column in the longitudinal data, specifying which visits can (TRUE) or cannot (FALSE) be used as confirmation visits. The name of such column must be specified using the validconf_col argument.

  

  Figure 2. Example of data with unscheduled visits.

  In the example depicted in Figure 2, unscheduled visits can be excluded from consideration as confirmation visits by setting:

  output <- MSprog(...
                   validconf_col="scheduled",
                   ...)

Sustained CDW or CDI

In addition to event confirmation, some studies require events to be sustained for a specified period of time. The require_sust_days argument allows to specify, if desired, the length of such period. For example, if require_sust_days=7*48, an event is only retained if the change in the outcome measure from the current baseline is confirmed at all visits falling within the following 48 weeks. If the event is sustained for the entire follow-up, it is retained even if the follow-up period ends <48 weeks after the event. Setting require_sust_days=Inf, events are retained only when sustained for the entire follow-up duration.

The require_sust_days argument may be of use in the following scenarios.

• Suppose the maximum follow-up in the population is 96 weeks excluding the baseline visit. If one wants to detect CDWs sustained over the whole follow-up, coding it as “worsening confirmed over 96 weeks or more” (conf_days=7*96, conf_tol_days=c(0, Inf)) would discard CDWs of patients whose follow-up is shorter than 96 weeks. In a scenario where patients have different follow-up lengths, the require_sust_days argument offers a more reliable way of implementing this, e.g., as “worsening confirmed over 12 weeks or more and sustained for the whole follow-up period” (conf_days=7*12, conf_tol_days=c(0, Inf), require_sust_days=Inf).

• In observational data with coarsely- and unevenly-spaced visits, the require_sust_days argument may be used to detect sustained CDW (e.g., require_sust_days=7*96) while still requiring the presence of a confirmation visit within a specified interval from the event (e.g., conf_days=7*12, conf_tol_days=c(0,7*12)).

• Sustained disability worsening can be required to exclude transient accumulation of disability (e.g., as a consequence of relapses) followed by complete recovery.

Relapse-associated worsening (RAW)

RAW events are typically defined as CDW events occurring within a specified interval from the onset of a relapse. The length (in days) of such interval can be specified using the relapse_assoc argument in MSprog(). Common values are 30 or 90 days. Additionally, one may also provide a maximum distance from relapses whose onset is after the event. The logic is that the reported onset date of a relapse may be slightly delayed. The examples below illustrate the usage of the relapse_assoc argument:

• relapse_assoc=90 (equivalent to relapse_assoc=c(90, 0)): RAW = “a CDW event occurring within 90 days after the onset of a relapse”

• relapse_assoc=c(90, 7): RAW = “a CDW event occurring within 90 days after or within 7 days before the onset of a relapse”.

Progression independent of relapse activity (PIRA)

In the literature, PIRA is generally defined as CDW occurring in the absence of relapses within predefined intervals. In most cases (see, e.g., [8,9,12]), relapse-free intervals are anchored to (possibly a subset of):

• a visit preceding the event (e.g., baseline)
• the initial worsening
• an eligible confirmation visit.

The relapse_indep argument in MSprog() allows to specify custom relapse-free intervals. The auxiliary function relapse_indep_from_bounds() organises the given interval bounds into a named list to be given to MSprog() through the relapse_indep argument.

output <- MSprog(...
                 relapse_indep=relapse_indep_from_bounds(p0, p1, e0, e1, c0, c1),
                 ...)

where: p0 and p1 specify the interval around a visit preceding the event (can be the current reference, the last visit before event onset, or the last visit before event onset with a clinically meaningful score difference from it [^pira]); e0 and e1 specify the interval around the event; c0 and c1 specify the interval around the confirmation visit; see Figure 1. If both ends of an interval are 0 (e.g., if both p0=0 and p1=0), the checkpoint is ignored. To merge two intervals together, set both the right end of the first interval and the left end of the second interval to NULL (e.g., “between baseline and event onset”: p1=NULL and e0=NULL).

Figure 1. Relapse-free intervals characterising PIRA, as defined by arguments p0, p1, e0, e1, c0, c1.

For example, in [8], the authors recommend an absence of relapses during the 90 days before and 30 days after the event, and during the 90 days before and 30 days after confirmation for a CDW event to be considered as PIRA. This translates into: p0<-0, p1<-0, e0<-90, e1<-30, c0<-90, c1<-30. When a high specificity is desired, they recommend an absence of relapses in the whole period between reference and confirmation, which is implemented by setting: p0<-0, p1<-NULL, e0<-NULL, e1<-NULL, c0<-NULL, c1<-0.

For more details and examples on PIRA computation, please refer to the dedicated package vignette Assessing progression independent of relapse activity (PIRA) in MS.

Example

The following code detects multiple events with the RAW_PIRA flag enabled.

output <- MSprog(data=toydata_visits,
                 subj_col="id", value_col="EDSS", date_col="date", outcome="edss", 
                 event="multiple", RAW_PIRA=TRUE, baseline="roving", 
                 relapse=toydata_relapses) 
#> 
#> ---
#> Outcome: edss
#> Confirmation over: 84 days (-7 days, +730.5 days)
#> Baseline: roving
#> Baseline skipped if: <30 days from last relapse
#> Event skipped if: -
#> Confirmation visit skipped if: <30 days from last relapse
#> Events detected: multiple
#> 
#> *Please use `print(output)` to display full info on event detection criteria*
#> 
#> ---
#> Total subjects: 7
#> ---
#> Subjects with CDW: 5 (PIRA: 5; RAW: 1)
#> Subjects with CDI: 2
#> ---
#> CDW events: 6 (PIRA: 5; RAW: 1)
#> CDI events: 2

The output$event_count data frame (and the summary printed out by the function) now includes RAW and PIRA counts:

# print(output$event_count)
DT::datatable(output$event_count, 
              options = list(dom="t", scrollX=T, scrollY=F, paging = FALSE)
              )

In the output$results data frame, each CDW event is now further characterised by a CDW_type column:

# print(output$results, row.names=FALSE)
DT::datatable(output$results, rownames=F,
              options = list(dom="t", scrollX=T, scrollY=F, paging = FALSE)
              )

What to include in results

The results attribute provides extended info on each event for all subjects. The following arguments can be used to control the information included.

• include_dates: if TRUE, the results will include the dates of: event onset; baseline; last visit before event onset at clinically meaningful score difference from it; first confirmation visit.
• include_value: if TRUE, the results will include the outcome value at: event onset; baseline; last visit before event onset at clinically meaningful score difference from it; first confirmation visit.
• include_stable: if TRUE, subjects with no confirmed events are included in the results, with time2event = total follow up.

For example:

output <- MSprog(data=toydata_visits, 
                      subj_col="id", value_col="EDSS", date_col="date", 
                      outcome="edss", relapse=toydata_relapses, verbose=0,
                      include_dates=T, include_value=T, include_stable=F) # <---- !

# print(output$results, row.names=FALSE)
DT::datatable(output$results, rownames=F,
              options = list(dom="t", scrollX=T, scrollY=F, paging = FALSE)
              )

Printing progress info

The verbose argument controls the amount of info printed out by the MSprog() function. When setting verbose=0, the function prints no info. When setting verbose=1 (default), the function only prints out concise info. When setting verbose=2, the function prints out an extended log of the ongoing computations. See the example below.

output <- MSprog(data=toydata_visits, 
                 subj_col="id", value_col="EDSS", date_col="date", outcome="edss",
                 event="multiple", baseline="roving", 
                 relapse=toydata_relapses, verbose=2)
#> 
#> Subject #1: 8 visits, 0 relapses
#> Baseline at visit no.1
#> Searching for events from visit no.2 on
#> Found EDSS change at visit no.5 (2022-07-12); potential confirmation visits available: no.6, 7, 8
#> Found EDSS-CDW (visit no.5, 2022-07-12) confirmed at 84 days, sustained up to visit no.8 (2023-03-11)
#> Baseline at visit no.6
#> Searching for events from visit no.7 on
#> No edss change in any subsequent visit: end process
#> Event sequence: CDW
#> 
#> Subject #2: 10 visits, 2 relapses
#> Baseline at visit no.1
#> Searching for events from visit no.2 on
#> Found EDSS change at visit no.4 (2021-06-12); potential confirmation visits available: no.5, 6, 7, 8, 9, 10
#> Found EDSS-CDW (visit no.4, 2021-06-12) confirmed at 84 days, sustained up to visit no.5 (2021-09-04)
#> Baseline at visit no.5
#> Searching for events from visit no.6 on
#> Found EDSS change at visit no.8 (2022-05-19); potential confirmation visits available: no.9, 10
#> Found EDSS-CDW (visit no.8, 2022-05-19) confirmed at 84 days, sustained up to visit no.10 (2022-11-26)
#> Baseline at visit no.9
#> Searching for events from visit no.10 on
#> No edss change in any subsequent visit: end process
#> Event sequence: CDW, CDW
#> 
#> Subject #3: 7 visits, 1 relapse
#> Baseline at visit no.1
#> Searching for events from visit no.2 on
#> Found EDSS change at visit no.2 (2021-12-01); potential confirmation visits available: no.4, 5, 7
#> Not a confirmed change: proceed with search
#> Searching for events from visit no.3 on
#> No edss change in any subsequent visit: end process
#> Event sequence: -
#> 
#> Subject #4: 7 visits, 0 relapses
#> Baseline at visit no.1
#> Searching for events from visit no.2 on
#> Found EDSS change at visit no.2 (2021-12-04); potential confirmation visits available: no.3, 4, 5, 6, 7
#> Found EDSS-CDI (visit no.2, 2021-12-04) confirmed at 84 days, sustained up to visit no.3 (2022-03-12)
#> Baseline at visit no.3
#> Searching for events from visit no.4 on
#> Found EDSS change at visit no.4 (2022-07-19); potential confirmation visits available: no.5, 6, 7
#> Found EDSS-CDW (visit no.4, 2022-07-19) confirmed at 84 days, sustained up to visit no.7 (2023-04-27)
#> Baseline at visit no.5
#> Searching for events from visit no.6 on
#> No edss change in any subsequent visit: end process
#> Event sequence: CDI, CDW
#> 
#> Subject #5: 9 visits, 0 relapses
#> Baseline at visit no.1
#> Searching for events from visit no.2 on
#> Found EDSS change at visit no.3 (2021-11-01); potential confirmation visits available: no.4, 5, 6, 7, 8, 9
#> Found EDSS-CDW (visit no.3, 2021-11-01) confirmed at 84 days, sustained up to visit no.9 (2023-03-13)
#> Baseline at visit no.4
#> Searching for events from visit no.5 on
#> Found EDSS change at visit no.5 (2022-04-30); potential confirmation visits available: no.7, 8, 9
#> Not a confirmed change: proceed with search
#> Searching for events from visit no.6 on
#> Found EDSS change at visit no.8 (2022-12-15); potential confirmation visits available: no.9
#> Not a confirmed change: proceed with search
#> Searching for events from visit no.9 on
#> No edss change in any subsequent visit: end process
#> Event sequence: CDW
#> 
#> Subject #6: 7 visits, 1 relapse
#> Baseline at visit no.1
#> Searching for events from visit no.2 on
#> Found EDSS change at visit no.3 (2022-02-15); potential confirmation visits available: no.4, 5, 7
#> Found EDSS-CDI (visit no.3, 2022-02-15) confirmed at 84 days, sustained up to visit no.5 (2022-10-05)
#> Baseline at visit no.4
#> Searching for events from visit no.5 on
#> Found EDSS change at visit no.6 (2022-12-22); potential confirmation visits available: no.
#> Not a confirmed change: proceed with search
#> Searching for events from visit no.7 on
#> Found EDSS change at visit no.7 (2023-02-21); potential confirmation visits available: no.
#> Not a confirmed change: proceed with search
#> Searching for events from visit no.- on
#> No edss change in any subsequent visit: end process
#> Event sequence: CDI
#> 
#> Subject #7: 9 visits, 1 relapse
#> Baseline at visit no.1
#> Searching for events from visit no.2 on
#> Found EDSS change at visit no.3 (2021-09-11); potential confirmation visits available: no.4, 5, 6, 7, 8, 9
#> Not a confirmed change: proceed with search
#> Searching for events from visit no.4 on
#> Found EDSS change at visit no.5 (2022-03-17); potential confirmation visits available: no.6, 7, 8, 9
#> Found EDSS-CDW (visit no.5, 2022-03-17) confirmed at 84 days, sustained up to visit no.9 (2023-04-28)
#> Baseline at visit no.6
#> Searching for events from visit no.7 on
#> No edss change in any subsequent visit: end process
#> Event sequence: CDW
#> 
#> ---
#> Outcome: edss
#> Confirmation over: 84 days (-7 days, +730.5 days)
#> Baseline: roving
#> Baseline skipped if: <30 days from last relapse
#> Event skipped if: -
#> Confirmation visit skipped if: <30 days from last relapse
#> Events detected: multiple
#> 
#> *Please use `print(output)` to display full info on event detection criteria*
#> 
#> ---
#> Total subjects: 7
#> ---
#> Subjects with CDW: 5
#> Subjects with CDI: 2
#> ---
#> CDW events: 6
#> CDI events: 2

For further insight into the event detection process, all score changes from baseline that were not identified as valid events (e.g., not confirmed) are stored in output$unconfirmed. In this example, this looks like:

# print(output$unconfirmed, row.names=FALSE)
DT::datatable(output$unconfirmed, rownames=F,
              options = list(dom="t", scrollX=T, scrollY=F, paging = FALSE)
              )