Package 'CohortGenerator'

Description

Given a subset definition and cohort definition set, this function returns a modified cohortDefinitionSet That contains cohorts that's have parent's contained within the base cohortDefinitionSet

Also adds the columns subsetParent and isSubset that denote if the cohort is a subset and what the parent definition is.

Usage

addCohortSubsetDefinition(
  cohortDefinitionSet,
  cohortSubsetDefintion,
  targetCohortIds = NULL,
  overwriteExisting = FALSE
)

Arguments

Description

This function checks a data.frame to verify it holds the expected format for a cohortDefinitionSet's data types and can optionally fix data types that do not match the specification.

Usage

checkAndFixCohortDefinitionSetDataTypes(
  x,
  fixDataTypes = TRUE,
  emitWarning = FALSE
)

Arguments

Value

Returns a list() of the following form:

list( dataTypesMatch = TRUE/FALSE, x = data.frame() )

dataTypesMatch == TRUE when the supplied data.frame x matches the cohortDefinitionSet specification's data types.

If fixDataTypes == TRUE, x will hold the original data from x with the data types corrected. Otherwise x will hold the original value passed to this function.

Description

Set of subset definitions

Active bindings

Methods

Public methods

• CohortSubsetDefinition$new()

• CohortSubsetDefinition$toList()

• CohortSubsetDefinition$toJSON()

• CohortSubsetDefinition$addSubsetOperator()

• CohortSubsetDefinition$getSubsetQuery()

• CohortSubsetDefinition$getSubsetCohortName()

• CohortSubsetDefinition$setTargetOutputPairs()

• CohortSubsetDefinition$getJsonFileName()

• CohortSubsetDefinition$clone()

Method new()

Usage

CohortSubsetDefinition$new(definition = NULL)

Arguments

Method toList()

List representation of object to JSON

Usage

CohortSubsetDefinition$toList()

Method toJSON()

json serialized representation of object add Subset Operator

Usage

CohortSubsetDefinition$toJSON()

Method addSubsetOperator()

add subset to class - checks if equivalent id is present Will throw an error if a matching ID is found but reference object is different

Usage

CohortSubsetDefinition$addSubsetOperator(subsetOperator)

Arguments

Method getSubsetQuery()

Returns vector of join, logic, having statements returned by subset operations

Usage

CohortSubsetDefinition$getSubsetQuery(targetOutputPair)

Arguments

Method getSubsetCohortName()

Usage

CohortSubsetDefinition$getSubsetCohortName(
  cohortDefinitionSet,
  targetOutputPair
)

Arguments

Method setTargetOutputPairs()

Usage

CohortSubsetDefinition$setTargetOutputPairs(targetIds)

Arguments

Method getJsonFileName()

Usage

CohortSubsetDefinition$getJsonFileName(
  subsetJsonFolder = "inst/cohort_subset_definitions/"
)

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

CohortSubsetDefinition$clone(deep = FALSE)

Arguments

Description

A subset of type cohort - subset a population to only those contained within defined cohort to List

Super class

CohortGenerator::SubsetOperator -> CohortSubsetOperator

Active bindings

Methods

Public methods

• CohortSubsetOperator$toList()

• CohortSubsetOperator$getAutoGeneratedName()

• CohortSubsetOperator$clone()

Method toList()

List representation of object Get auto generated name

Usage

CohortSubsetOperator$toList()

Method getAutoGeneratedName()

name generated from subset operation properties

Usage

CohortSubsetOperator$getAutoGeneratedName()

Returns

character

Method clone()

The objects of this class are cloneable with this method.

Usage

CohortSubsetOperator$clone(deep = FALSE)

Arguments

Description

This is used as part of the incremental operations to hash a value to store in a record keeping file. This function leverages the md5 hash from the digest package

Usage

computeChecksum(val)

Arguments

Value

Returns a string containing the checksum

Description

A definition of subset functions to be applied to a set of cohorts

Usage

createCohortSubset(
  name = NULL,
  cohortIds,
  cohortCombinationOperator,
  negate,
  startWindow,
  endWindow
)

Arguments

Value

a CohortSubsetOperator instance

Description

Create subset definition from subset objects

Usage

createCohortSubsetDefinition(
  name,
  definitionId,
  subsetOperators,
  identifierExpression = NULL,
  operatorNameConcatString = "",
  subsetCohortNameTemplate = ""
)

Arguments

Description

This function creates an empty cohort table and empty tables for cohort statistics.

Usage

createCohortTables(
  connectionDetails = NULL,
  connection = NULL,
  cohortDatabaseSchema,
  cohortTableNames = getCohortTableNames(),
  incremental = FALSE
)

Arguments

Description

Create createDemographicSubset Subset

Usage

createDemographicSubset(
  name = NULL,
  ageMin = 0,
  ageMax = 99999,
  gender = NULL,
  race = NULL,
  ethnicity = NULL
)

Arguments

Description

This function creates an empty cohort set data.frame for use with generateCohortSet.

Usage

createEmptyCohortDefinitionSet(verbose = FALSE)

Arguments

Value

Invisibly returns an empty cohort set data.frame

Description

This function creates an empty cohort set data.frame for use with generateNegativeControlOutcomeCohorts.

Usage

createEmptyNegativeControlOutcomeCohortSet(verbose = FALSE)

Arguments

Value

Invisibly returns an empty negative control outcome cohort set data.frame

Description

Subset cohorts using specified limit criteria

Usage

createLimitSubset(
  name = NULL,
  priorTime = 0,
  followUpTime = 0,
  limitTo = "all",
  calendarStartDate = NULL,
  calendarEndDate = NULL
)

Arguments

Description

Create the results data model tables on a database server.

Usage

createResultsDataModel(
  connectionDetails = NULL,
  databaseSchema,
  tablePrefix = ""
)

Arguments

Details

Only PostgreSQL and SQLite servers are supported.

Description

A definition of subset functions to be applied to a set of cohorts

Usage

createSubsetCohortWindow(startDay, endDay, targetAnchor)

Arguments

Value

a SubsetCohortWindow instance

Description

Operators for subsetting a cohort by demographic criteria

Value

char vector Get auto generated name

Super class

CohortGenerator::SubsetOperator -> DemographicSubsetOperator

Active bindings

Methods

Public methods

• DemographicSubsetOperator$toList()

• DemographicSubsetOperator$mapGenderConceptsToNames()

• DemographicSubsetOperator$getAutoGeneratedName()

• DemographicSubsetOperator$toJSON()

• DemographicSubsetOperator$isEqualTo()

• DemographicSubsetOperator$getGender()

• DemographicSubsetOperator$getRace()

• DemographicSubsetOperator$getEthnicity()

• DemographicSubsetOperator$clone()

Method toList()

List representation of object Map gender concepts to names

Usage

DemographicSubsetOperator$toList()

Method mapGenderConceptsToNames()

Usage

DemographicSubsetOperator$mapGenderConceptsToNames(
  mapping = list(`8507` = "males", `8532` = "females", `0` = "unknown gender")
)

Arguments

Method getAutoGeneratedName()

name generated from subset operation properties

Usage

DemographicSubsetOperator$getAutoGeneratedName()

Returns

character

Method toJSON()

json serialized representation of object

Usage

DemographicSubsetOperator$toJSON()

Method isEqualTo()

Compare Subset to another

Usage

DemographicSubsetOperator$isEqualTo(criteria)

Arguments

Method getGender()

Gender getter - used when constructing SQL to default NULL to an empty string

Usage

DemographicSubsetOperator$getGender()

Method getRace()

Race getter - used when constructing SQL to default NULL to an empty string

Usage

DemographicSubsetOperator$getRace()

Method getEthnicity()

Ethnicity getter - used when constructing SQL to default NULL to an empty string

Usage

DemographicSubsetOperator$getEthnicity()

Method clone()

The objects of this class are cloneable with this method.

Usage

DemographicSubsetOperator$clone(deep = FALSE)

Arguments

Description

This function drops the cohort statistics tables.

Usage

dropCohortStatsTables(
  connectionDetails = NULL,
  connection = NULL,
  cohortDatabaseSchema,
  cohortTableNames = getCohortTableNames(),
  dropCohortTable = FALSE
)

Arguments

Description

This function retrieves the data from the cohort statistics tables and writes them to the inclusion statistics folder specified in the function call. NOTE: inclusion rule names are handled in one of two ways:

1. You can specify the cohortDefinitionSet parameter and the inclusion rule names will be extracted from the data.frame. 2. You can insert the inclusion rule names into the database using the insertInclusionRuleNames function of this package.

The first approach is preferred as to avoid the warning emitted.

Usage

exportCohortStatsTables(
  connectionDetails,
  connection = NULL,
  cohortDatabaseSchema,
  cohortTableNames = getCohortTableNames(),
  cohortStatisticsFolder,
  snakeCaseToCamelCase = TRUE,
  fileNamesInSnakeCase = FALSE,
  incremental = FALSE,
  databaseId = NULL,
  minCellCount = 5,
  cohortDefinitionSet = NULL,
  tablePrefix = ""
)

Arguments

Description

This function generates a set of cohorts in the cohort table.

Usage

generateCohortSet(
  connectionDetails = NULL,
  connection = NULL,
  cdmDatabaseSchema,
  tempEmulationSchema = getOption("sqlRenderTempEmulationSchema"),
  cohortDatabaseSchema = cdmDatabaseSchema,
  cohortTableNames = getCohortTableNames(),
  cohortDefinitionSet = NULL,
  stopOnError = TRUE,
  incremental = FALSE,
  incrementalFolder = NULL
)

Arguments

Value

A data.frame consisting of the following columns:

cohortId

The unique integer identifier of the cohort

cohortName

The cohort's name

generationStatus

The status of the generation task which may be one of the following:

COMPLETE

The generation completed successfully

FAILED

The generation failed (see logs for details)

SKIPPED

If using incremental == 'TRUE', this status indicates that the cohort's generation was skipped since it was previously completed.

startTime

The start time of the cohort generation. If the generationStatus == 'SKIPPED', the startTime will be NA.

endTime

The end time of the cohort generation. If the generationStatus == 'FAILED', the endTime will be the time of the failure. If the generationStatus == 'SKIPPED', endTime will be NA.

Description

This function generate a set of negative control outcome cohorts. For more information please see [Chapter 12 - Population Level Estimation](https://ohdsi.github.io/TheBookOfOhdsi/PopulationLevelEstimation.html) for more information how these cohorts are utilized in a study design.

Usage

generateNegativeControlOutcomeCohorts(
  connectionDetails = NULL,
  connection = NULL,
  cdmDatabaseSchema,
  tempEmulationSchema = getOption("sqlRenderTempEmulationSchema"),
  cohortDatabaseSchema = cdmDatabaseSchema,
  cohortTable = getCohortTableNames()$cohortTable,
  negativeControlOutcomeCohortSet,
  occurrenceType = "all",
  incremental = FALSE,
  incrementalFolder = NULL,
  detectOnDescendants = FALSE
)

Arguments

Value

Invisibly returns an empty negative control outcome cohort set data.frame

Description

Computes the subject and entry count per cohort. Note the cohortDefinitionSet parameter is optional - if you specify the cohortDefinitionSet, the cohort counts will be joined to the cohortDefinitionSet to include attributes like the cohortName.

Usage

getCohortCounts(
  connectionDetails = NULL,
  connection = NULL,
  cohortDatabaseSchema,
  cohortTable = "cohort",
  cohortIds = c(),
  cohortDefinitionSet = NULL,
  databaseId = NULL
)

Arguments

Value

A data frame with cohort counts

Description

This function supports the legacy way of retrieving a cohort definition set from the file system or in a package. This function supports the legacy way of storing a cohort definition set in a package with a CSV file, JSON files, and SQL files in the 'inst' folder.

Usage

getCohortDefinitionSet(
  settingsFileName = "Cohorts.csv",
  jsonFolder = "cohorts",
  sqlFolder = "sql/sql_server",
  cohortFileNameFormat = "%s",
  cohortFileNameValue = c("cohortId"),
  subsetJsonFolder = "inst/cohort_subset_definitions/",
  packageName = NULL,
  warnOnMissingJson = TRUE,
  verbose = FALSE
)

Arguments

Value

Returns a cohort set data.frame

Description

This function returns a data frame of the inclusion rules defined in a cohort definition set.

Usage

getCohortInclusionRules(cohortDefinitionSet)

Arguments

Description

This function returns a data frame of the data in the Cohort Inclusion Tables. Results are organized in to a list with 5 different data frames:

• cohortInclusionTable

• cohortInclusionResultTable

• cohortInclusionStatsTable

• cohortSummaryStatsTable

• cohortCensorStatsTable

These can be optionally specified with the outputTables. See exportCohortStatsTables function for saving data to csv.

Usage

getCohortStats(
  connectionDetails,
  connection = NULL,
  cohortDatabaseSchema,
  databaseId = NULL,
  snakeCaseToCamelCase = TRUE,
  outputTables = c("cohortInclusionTable", "cohortInclusionResultTable",
    "cohortInclusionStatsTable", "cohortInclusionStatsTable", "cohortSummaryStatsTable",
    "cohortCensorStatsTable"),
  cohortTableNames = getCohortTableNames()
)

Arguments

Description

This function creates a list of table names used by createCohortTables to specify the table names to create. Use this function to specify the names of the main cohort table and cohort statistics tables.

Usage

getCohortTableNames(
  cohortTable = "cohort",
  cohortSampleTable = cohortTable,
  cohortInclusionTable = paste0(cohortTable, "_inclusion"),
  cohortInclusionResultTable = paste0(cohortTable, "_inclusion_result"),
  cohortInclusionStatsTable = paste0(cohortTable, "_inclusion_stats"),
  cohortSummaryStatsTable = paste0(cohortTable, "_summary_stats"),
  cohortCensorStatsTable = paste0(cohortTable, "_censor_stats")
)

Arguments

Value

A list of the table names as specified in the parameters to this function.

Description

Returns ResultModelManager DataMigrationsManager instance.

Usage

getDataMigrator(connectionDetails, databaseSchema, tablePrefix = "")

Arguments

Value

Instance of ResultModelManager::DataMigrationManager that has interface for converting existing data models

Description

This function will attempt to check the recordKeepingFile to determine if a list of operations have completed by comparing the keys passed into the function with the checksum supplied

Usage

getRequiredTasks(..., checksum, recordKeepingFile)

Arguments

Value

Returns a list of outstanding tasks based on inspecting the full contents of the record keeping file

Description

Get specifications for CohortGenerator results data model

Usage

getResultsDataModelSpecifications()

Value

A tibble data frame object with specifications

Description

Get the subset definitions (if any) applied to a cohort definition set. Note that these subset definitions are a copy of those applied to the cohort set. Modifying these definitions will not modify the base cohort set. To apply a modification, reapply the subset definition to the cohort definition set data.frame with addCohortSubsetDefinition with 'overwriteExisting = TRUE'.

Usage

getSubsetDefinitions(cohortDefinitionSet)

Arguments

Value

list of cohort subset definitions or empty list

Description

This function will take a cohortDefinitionSet that inclusions the Circe JSON representation of each cohort, parse the InclusionRule property to obtain the inclusion rule name and sequence number and insert the values into the cohortInclusionTable. This function is only required when generating cohorts that include cohort statistics.

Usage

insertInclusionRuleNames(
  connectionDetails = NULL,
  connection = NULL,
  cohortDefinitionSet,
  cohortDatabaseSchema,
  cohortInclusionTable = getCohortTableNames()$cohortInclusionTable
)

Arguments

Value

A data frame containing the inclusion rules by cohort and sequence ID

Description

This function is used check if a string conforms to the lower camel case format.

Usage

isCamelCase(x)

Arguments

Value

TRUE if the string is in lower camel case

Description

This function checks a data.frame to verify it holds the expected format for a cohortDefinitionSet.

Usage

isCohortDefinitionSet(x)

Arguments

Value

Returns TRUE if the input is a cohortDefinitionSet or returns FALSE with warnings on any violations

Description

This function is used to check a data.frame to ensure all column names are in snake case format.

Usage

isFormattedForDatabaseUpload(x, warn = TRUE)

Arguments

Value

Returns TRUE if all columns are snake case format. If warn == TRUE, the function will emit a warning on the column names that are not in snake case format.

Description

This function is used check if a string conforms to the snake case format.

Usage

isSnakeCase(x)

Arguments

Value

TRUE if the string is in snake case

Description

This function will attempt to check the recordKeepingFile to determine if an individual operation has completed by comparing the keys passed into the function with the checksum supplied

Usage

isTaskRequired(..., checksum, recordKeepingFile, verbose = TRUE)

Arguments

Value

Returns TRUE if the operation has completed according to the contents of the record keeping file.

Description

operator to apply limiting subset operations (e.g. washout periods, calendar ranges or earliest entries)

Get auto generated name

Super class

CohortGenerator::SubsetOperator -> LimitSubsetOperator

Active bindings

Methods

Public methods

• LimitSubsetOperator$getAutoGeneratedName()

• LimitSubsetOperator$toList()

• LimitSubsetOperator$clone()

Method getAutoGeneratedName()

name generated from subset operation properties

Usage

LimitSubsetOperator$getAutoGeneratedName()

Returns

character To List

Method toList()

List representation of object

Usage

LimitSubsetOperator$toList()

Method clone()

The objects of this class are cloneable with this method.

Usage

LimitSubsetOperator$clone(deep = FALSE)

Arguments

Description

Migrate data from current state to next state

It is strongly advised that you have a backup of all data (either sqlite files, a backup database (in the case you are using a postgres backend) or have kept the csv/zip files from your data generation.

Usage

migrateDataModel(connectionDetails, databaseSchema, tablePrefix = "")

Arguments

Description

This function is used to centralize the function for reading .csv files across the HADES ecosystem. This function will automatically convert from snake_case in the file to camelCase in the data.frame returned as is the standard described in: https://ohdsi.github.io/Hades/codeStyle.html#Interfacing_between_R_and_SQL

Usage

readCsv(file, warnOnCaseMismatch = TRUE, colTypes = readr::cols())

Arguments

Value

A tibble with the .csv contents

Description

This function will record a task as completed in the recordKeepingFile

Usage

recordTasksDone(..., checksum, recordKeepingFile, incremental = TRUE)

Arguments

Description

Run a cohort generation and export results

Usage

runCohortGeneration(
  connectionDetails,
  cdmDatabaseSchema,
  tempEmulationSchema = getOption("sqlRenderTempEmulationSchema"),
  cohortDatabaseSchema = cdmDatabaseSchema,
  cohortTableNames = getCohortTableNames(),
  cohortDefinitionSet = NULL,
  negativeControlOutcomeCohortSet = NULL,
  occurrenceType = "all",
  detectOnDescendants = FALSE,
  stopOnError = TRUE,
  outputFolder,
  databaseId = 1,
  minCellCount = 5,
  incremental = FALSE,
  incrementalFolder = NULL
)

Arguments

Details

Run a cohort generation for a set of cohorts and negative control outcomes. This function will also export the results of the run to the 'outputFolder'.

Description

Create 1 or more sample of size n of a cohort definition set

Subsetted cohorts can be sampled, as with any other subset form. However, subsetting a sampled cohort is not recommended and not currently supported at this time. In the case where n > cohort count the entire cohort is copied unmodified

As different databases have different forms of randomness, the random selection is computed in R, based on the count for each cohort. This is, therefore, db platform independent

Note, this function assumes cohorts have already been generated.

Lifecycle Note: This functionality is considered experimental and not intended for use inside analytic packages

Usage

sampleCohortDefinitionSet(
  cohortDefinitionSet,
  cohortIds = cohortDefinitionSet$cohortId,
  connectionDetails = NULL,
  connection = NULL,
  tempEmulationSchema = getOption("sqlRenderTempEmulationSchema"),
  cohortDatabaseSchema,
  outputDatabaseSchema = cohortDatabaseSchema,
  cohortTableNames = getCohortTableNames(),
  n = NULL,
  sampleFraction = NULL,
  seed = 64374,
  seedArgs = NULL,
  identifierExpression = "cohortId * 1000 + seed",
  incremental = FALSE,
  incrementalFolder = NULL
)

Arguments

Value

sampledCohortDefinitionSet - a data.frame like object that contains the resulting identifiers and modified names of cohorts

Description

This function saves a cohortDefinitionSet to the file system and provides options for specifying where to write the individual elements: the settings file will contain the cohort information as a CSV specified by the settingsFileName, the cohort JSON is written to the jsonFolder and the SQL is written to the sqlFolder. We also provide a way to specify the json/sql file name format using the cohortFileNameFormat and cohortFileNameValue parameters.

Usage

saveCohortDefinitionSet(
  cohortDefinitionSet,
  settingsFileName = "inst/Cohorts.csv",
  jsonFolder = "inst/cohorts",
  sqlFolder = "inst/sql/sql_server",
  cohortFileNameFormat = "%s",
  cohortFileNameValue = c("cohortId"),
  subsetJsonFolder = "inst/cohort_subset_definitions/",
  verbose = FALSE
)

Arguments

Description

This is generally used as part of saveCohortDefinitionSet

Usage

saveCohortSubsetDefinition(
  subsetDefinition,
  subsetJsonFolder = "inst/cohort_subset_definitions/"
)

Arguments

Description

When running in incremental mode, we may need to update results in a CSV file. This function will replace the data in fileName based on the key parameters

Usage

saveIncremental(data, fileName, ...)

Arguments

Description

Representation of a time window to use when subsetting a target cohort with a subset cohort

Active bindings

Methods

Public methods

• SubsetCohortWindow$toList()

• SubsetCohortWindow$toJSON()

• SubsetCohortWindow$isEqualTo()

• SubsetCohortWindow$clone()

Method toList()

List representation of object To JSON

Usage

SubsetCohortWindow$toList()

Method toJSON()

json serialized representation of object Is Equal to

Usage

SubsetCohortWindow$toJSON()

Method isEqualTo()

Compare SubsetCohortWindow to another

Usage

SubsetCohortWindow$isEqualTo(criteria)

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

SubsetCohortWindow$clone(deep = FALSE)

Arguments

Description

Abstract Base Class for subsets. Subsets should inherit from this and implement their own requirements.

Active bindings

Methods

Public methods

• SubsetOperator$new()

• SubsetOperator$classname()

• SubsetOperator$getAutoGeneratedName()

• SubsetOperator$getQueryBuilder()

• SubsetOperator$publicFields()

• SubsetOperator$isEqualTo()

• SubsetOperator$toList()

• SubsetOperator$toJSON()

• SubsetOperator$clone()

Method new()

Usage

SubsetOperator$new(definition = NULL)

Arguments

Returns

instance of object Class Name

Method classname()

Class name of object Get auto generated name

Usage

SubsetOperator$classname()

Method getAutoGeneratedName()

Not intended to be used - should be implemented in subclasses Return query builder instance

Usage

SubsetOperator$getAutoGeneratedName()

Method getQueryBuilder()

Return query builder instance Public Fields

Usage

SubsetOperator$getQueryBuilder(id)

Arguments

Method publicFields()

Publicly settable fields of object Is Equal to

Usage

SubsetOperator$publicFields()

Method isEqualTo()

Compare Subsets - are they identical or not? Checks all fields and settings

Usage

SubsetOperator$isEqualTo(subsetOperatorB)

Arguments

Method toList()

convert to List representation To Json

Usage

SubsetOperator$toList()

Method toJSON()

convert to json serialized representation

Usage

SubsetOperator$toJSON()

Returns

list representation of object as json character

Method clone()

The objects of this class are cloneable with this method.

Usage

SubsetOperator$clone(deep = FALSE)

Arguments

See Also

CohortSubsetOperator

DemographicSubsetOperator

LimitSubsetOperator

Description

Requires the results data model tables have been created using the createResultsDataModel function.

Usage

uploadResults(
  connectionDetails,
  schema,
  resultsFolder,
  forceOverWriteOfSpecifications = FALSE,
  purgeSiteDataBeforeUploading = TRUE,
  tablePrefix = "",
  ...
)

Arguments

Description

This function is used to centralize the function for writing .csv files across the HADES ecosystem. This function will automatically convert from camelCase in the data.frame to snake_case column names in the resulting .csv file as is the standard described in: https://ohdsi.github.io/Hades/codeStyle.html#Interfacing_between_R_and_SQL

This function may also raise warnings if the data is stored in a format that will not work with the HADES standard for uploading to a results database. Specifically file names should be in snake_case format, all column headings are in snake_case format and where possible the file name should not be plural. See isFormattedForDatabaseUpload for a helper function to check a data.frame for rules on the column names

Usage

writeCsv(
  x,
  file,
  append = FALSE,
  warnOnCaseMismatch = TRUE,
  warnOnFileNameCaseMismatch = TRUE,
  warnOnUploadRuleViolations = TRUE
)

Arguments

Value

Returns the input x invisibly.