DataRobot now includes the ability to make time series projects via the API.
Time series projects, like OTV projects, use datetime partitioning, and all the workflow changes that apply to other datetime partitioned projects also apply to them. Unlike other projects, time series projects produce different types of models which forecast multiple future predictions instead of an individual prediction for each row.
DataRobot uses a general time series framework to configure how time series features are created and what future values the models will output. This framework consists of a Forecast Point (defining a time a prediction is being made), a Feature Derivation Window (a rolling window used to create features), and a Forecast Window (a rolling window of future values to predict). These components are described in more detail below.
Time series projects will automatically transform the dataset provided in order to apply this framework. During the transformation, DataRobot uses the Feature Derivation Window to derive time series features (such as lags and rolling statistics), and uses the Forecast Window to provide examples of forecasting different distances in the future (such as time shifts). After project creation, a new dataset and a new feature list are generated and used to train the models. This process is reapplied automatically at prediction time as well in order to generate future predictions based on the original data features.
The timeUnit and timeStep used to define
the Feature Derivation and Forecast Windows are taken from the datetime
partition column, and can be retrieved for a given column in the input
data by using GetFeatureInfo.
To set up a time series project, use the new time series specific
parameters found in
CreateDatetimePartitionSpecification:
TreatAsExponential.DifferenicngMethod."ROW", will define the window with a number of rows.When using datasets to a time series project, the dataset might look
something like the following, if time is the datetime
partition column, target is the target column, and
temp is an input feature. If the dataset was uploaded with
a forecast point of “2017-01-08” and during partitioning the feature
derivation window start and end were set to -5 and -3 and the forecast
window start and end were set to 1 and 3, then rows 1 through 3 are
historical data, row 6 is the forecast point, and rows 7 though 9 are
forecast rows that will have predictions when predictions are
computed.
| row | time | target | temp |
|---|---|---|---|
| 1 | 2017-01-03 | 16443 | 72 |
| 2 | 2017-01-04 | 3013 | 72 |
| 3 | 2017-01-05 | 1643 | 68 |
| 4 | 2017-01-06 | NA | NA |
| 5 | 2017-01-07 | NA | NA |
| 6 | 2017-01-08 | NA | NA |
| 7 | 2017-01-09 | NA | NA |
| 8 | 2017-01-10 | NA | NA |
| 9 | 2017-01-11 | NA | NA |
On the other hand, if the project instead used holiday
as a known in advance input feature, the uploaded dataset might look
like the following:
| row | time | target | holiday |
|---|---|---|---|
| 1 | 2017-01-03 | 16443 | TRUE |
| 2 | 2017-01-04 | 3013 | FALSE |
| 3 | 2017-01-05 | 1643 | FALSE |
| 4 | 2017-01-06 | NA | FALSE |
| 5 | 2017-01-07 | NA | FALSE |
| 6 | 2017-01-08 | NA | FALSE |
| 7 | 2017-01-09 | NA | TRUE |
| 8 | 2017-01-10 | NA | FALSE |
| 9 | 2017-01-11 | NA | FALSE |
Here’s a simple example of using this data in a time series project:
The Feature Derivation window represents the rolling window that is
used to derive time series features and lags, relative to the Forecast
Point. It is defined in terms of
featureDerivationWindowStart and
featureDerivationWindowEnd which are integer values
representing datetime offsets in terms of the timeUnit
(e.g. hours or days).
The Feature Derivation Window start and end must be less than or
equal to zero, indicating they are positioned before the forecast point.
Additionally, the window must be specified as an integer multiple of the
timeStep which defines the expected difference in time
units between rows in the data.
Enough rows of historical data must be provided to cover the span of
the effective Feature Derivation Window (which may be longer than the
project’s Feature Derivation Window depending on the differencing
settings chosen). The effective Feature Derivation Window of any model
can be checked via the
effectiveFeatureDerivationWindowStart and
effectiveFeatureDerivationWindowEnd attributes of a
datetime model. See GetDatetimeModel.
The window is closed, meaning the edges are considered to be inside the window.
This information is added to your
CreateDatetimePartitionSpecification call like so:
The Forecast Window represents the rolling window of future values to
predict, relative to the Forecast Point. It is defined in terms of the
forecastWindowStart and forecastWindowEnd,
which are positive integer values indicating datetime offsets in terms
of the timeUnit (e.g. hours or days).
The Forecast Window start and end must be positive integers,
indicating they are positioned after the forecast point. Additionally,
the window must be specified as an integer multiple of the
timeStep which defines the expected difference in time
units between rows in the data.
The window is closed, meaning the edges are considered to be inside the window.
This information is added to your
CreateDatetimePartitionSpecification call like so:
In time series projects, a new set of modeling features is created after setting the partitioning options. If a featurelist is specified with the partitioning options, it will be used to select which features should be used to derived modeling features; if a featurelist is not specified, the default featurelist will be used.
These features are automatically derived from those in the project’s
dataset and are the features used for modeling - note that
ListFeaturelists and ListModelingFeaturelists
will return different data in time series projects. Modeling
featurelists are the ones that can be used for modeling and will be
accepted by the backend, while regular featurelists will continue to
exist but cannot be used. Modeling features are only accessible once the
target and partitioning options have been set. In projects that don’t
use time series modeling, once the target has been set, both modeling
and regular features and featurelists will behave the same.
Prediction datasets are uploaded as normal predictions. However, when
uploading a prediction dataset, a new parameter
forecastPoint can be specified. The forecast point of a
prediction dataset identifies the point in time relative which
predictions should be generated, and if one is not specified when
uploading a dataset, the server will choose the most recent possible
forecast point. The forecast window specified when setting the
partitioning options for the project determines how far into the future
from the forecast point predictions should be calculated.
For example:
When setting up a time series project, input features could be identified as known in advance features. These features are not used to generate lags, and are expected to be known for the rows in the forecast window at predict time (e.g. “how much money will have been spent on marketing”, “is this a holiday”).
To start a time series project, use
CreateDatetimePartitionSpecification and specify the
feaureSettings. (Note that this is for illustrative
purposes only - this project will not actually build because the 10 data
points are smaller than the 100 datapoint minimum required.)
partition <- CreateDatetimePartitionSpecification(datetimePartitionColumn = "timestamp",
useTimeSeries = TRUE,
featureSettings = list("featureName" = "holiday",
"knownInAdvance" = TRUE))
project <- StartProject(data,
projectName = "test-TimeSeries",
target = "target",
partition = partition,
metric = "RMSE")If you have another known in advance feature (e.g.,
weekend), just add it to feature settings as a list of
lists:
Some parts of time series require specifying durations. When date ranges are specified with a start and an end date, the end date is exclusive, so only dates earlier than the end date are included, but the start date is inclusive, so dates equal to or later than the start date are included. If the start and end date are the same, then no dates are included in the range.
Durations are specified using a subset of ISO8601. Durations will be of the form PnYnMnDTnHnMnS where each “n” may be replaced with an integer value. Within the duration string,
Example Durations:
The API also supports multiseries, or data with
multiple time series delineated by multiseries ID columns. To create
this, create a project, then create a datetime partition specification
that specifies the datetimePartitionColumn (the column with
your date in it) and the multiseriesIdColumns (a list of
columns specifying the ids that delineate the multiseries).
data <- read.csv(system.file("extdata", "multiseries.csv", package = "datarobot"))
partition <- CreateDatetimePartitionSpecification(datetimePartitionColumn = "timestamp",
useTimeSeries = TRUE,
multiseriesIdColumns = "series_id")
project <- StartProject(data,
projectName = "test-TimeSeries",
target = "target",
partition = partition,
metric = "RMSE",
mode = AutopilotMode$Manual,
targetType = "Regression")For each model, prediction intervals estimate the range of values DataRobot expects actual values of the target to fall within. They are similar to a confidence interval of a prediction, but are based on the residual errors measured during the backtesting for the selected model.
Note that because calculation depends on the backtesting values, prediction intervals are not available for predictions on models that have not had all backtests completed. Additionally, prediction intervals are not available when the number of points per forecast distance is less than 10, due to insufficient data.
In a prediction request, users can specify a prediction intervals
size, which specifies the desired probability of actual values falling
within the interval range. Larger values are less precise, but more
conservative. For example, specifying a size of 80 will result in a
lower bound of 10% and an upper bound of 90%. More generally, for a
specific predictionIntervalsSize, the upper and lower
bounds will be calculated as follows:
predictionIntervalsSize / 2)predictionIntervalsSize / 2)To view prediction intervals data for a prediction, the prediction
needs to have been created using Predict and specifying
includePredictionIntervals = TRUE. The size for the
prediction interval can be specified with the
predictionIntervalsSize parameter for the same function,
and will default to 80 if left unspecified. Specifying these fields will
result in prediction interval bounds being included in the retrieved
prediction data for that request. See Predict for more
details.
DataRobot does a lot of good work to automatically derive features
that may be useful (e.g., lags). You can always see these features
clearly by calling GetTimeSeriesFeatureDerivationLog.
However, from time to time, it may be useful to disable DataRobot’s
automatic feature engineering for a particular feature (e.g., so you can
derive lags yourself manually). To do this, we can use the
featureSettings to turn off derived features for a
particular base feature: