Timeseries Forecasting

Forecasting sales data with Arize

This example runs through how to setup a time series forecasting model in the Arize platform. In the timeseries forecasting use cases a model is run on specific run-date predicting data for a forecasting date in the future. This example will be predicting supply of a product from today, for each day, 10 days in the future.

This setup can be modified for predicting:

• End of quarter sales run every day until end of the quarter

• Predicting a single day in the future 100 days out - run every day

• Predicting sales of a product every day looking out over the next 30 days

Example of Timeseries Forecasting

The example above shows a supply model that makes 10 predictions for each day looking forward from the model run-date. The lag captures how many days ahead of the model run-date the specific prediction is for, where lag 0 is the actual day of the run.

Common Observability Data for Timeseries Model

The common data that is tracked for timeseries models includes:

• Forecast Date: The date and time of the predicted event or observation.

• Run Date: The date on which your model ran and generated a prediction.

• Lag: The number of days between the run date and the forecast date.

For example, if you run a model on Monday to predict the temperature on Friday, the run date would be Monday's date, the forecast date would be a timestamp for a time on Friday and the lag would be four days.

Setting up TimeSeries Data

The above picture shows how the data is mapped into the Arize platform. The timestamp is the forecast date, the run-date & lag is sent in as a tag and the actual prediction is sent in as the prediction label.

Common Performance Metrics

The common metrics for timeseries forecasting are MAPE, MAE, RMSE, MSE, R-Squared, and Mean Error, with filter options for run-date and lag.

In the Arize Dashboard shown below, it's clear that we see an over prediction event first, then an under prediction event later. We can clearly see the magnitude of these errors based on our custom configuration.

Timeseries MAPE

The above MAE shows predictions vs actuals for various forecast dates. In many scenarios teams want filtered by Lag < 10 looking at MAE for predictions only 10 days out.

Click here for all valid model types and metric combinations.

Schema for forecasting models

Timeseries forecasting models are characterized by three fields:

• forecast timestamp: the date and time of the predicted event or observation and is passed into the timestamp field. (data type: integer unix timestamp in seconds)

• run_date: the date on which the model was run and the prediction was generated and is optionally passed in as a tag. (recommended data type: str or integer unix timestamp)

• lag: the number of days between the forecast timestamp and run date and is optionally passed in as a tag (recommended data type: int)

You will likely need to extend your model's delayed actuals join window. Reach out to [e-mail] for help with this.

For example, if you run a model on Monday to predict the temperature on Friday, the run date would be Monday's date, the forecast timestamp would be a timestamp for a time on Friday and the lag would be four days.

Timeseries Code Example

Python Batch

Google Colaboratory

Example Row

state	pos_approved	run_date	lag	predicted_thermal	reported_thermal	forecast_ts
ca	true	'12_16_2022'	3	74.5	80.2	1671572541

# feature & tag columns can be optionally defined with typing:
feature_columns = TypedColumns(
    inferred=["name"],
    to_int=["pos_approved"],
    to_str=["state"],
)

schema = Schema(
    prediction_id_column_name="prediction_id",
    feature_column_names=feature_column_names,
    timestamp_column_name="forecast_ts",
    prediction_label_column_name="predicted_thermal",
    actual_label_column_name="reported_thermal",
    feature_column_names=feature_columns,
    tag_column_names=[
        "run_date",
        "lag",
    ],
)

response = arize_client.log(
    dataframe=df,
    schema=schema,
    model_id="time-series-batch-ingestion-tutorial",
    model_version="1.0.0",
    model_type=ModelTypes.REGRESSION,
    metrics_validation=[Metrics.REGRESSION],
    environment=Environments.PRODUCTION
    )

Python Single Record

Google Colaboratory

# features & tags can be optionally defined with typing
features = {
    'score': TypedValue(value='88', type=ArizeTypes.FLOAT),
    'state': 'ca',
    'pos_approved': TypedValue(value=False, type=ArizeTypes.INT)
}  

tags = {
    'run_date': '12_16_2022',
    'lag': 3
}
    
response = arize_client.log(
    model_id='sample-model-1'
    model_version='1.0.0',
    environment=Environments.PRODUCTION,
    model_type=ModelTypes.REGRESSION,
    prediction_id='1',
    prediction_timestamp= 1671572541,
    features=features,
    prediction_label=74.5,
    actual_label=80.2,
    tags=tags
)

Data Connector

Download an example Parquet file: Open parquet reader here.

https://storage.cloud.google.com/arize-assets/documentation-sample-data/data-ingestion/time-series-assets/time_series_data.parquetstorage.cloud.google.com

Learn how to upload files via various Data Connectors:

Google Cloud Storage (GCS)AWS S3Azure Blob StorageGoogle BigQuery

Other Examples

The Colab example below shows how a timeseries model is setup in the Arize platform.

Google Colaboratory

This Colab example shows how to configure a timeseries with multiple quantile forecasts in Arize, and how to configure Pinball Loss as a Custom Metric.

Google Colaboratory