Arize AI
Search…
Data Ingestion
Integrations
User Guides
On-Premise Deployment
2. Set a Model Baseline
A baseline is the reference data or benchmark used to compare model performance against for monitoring purposes. Baselines can be training data, validation data, prior time periods of production data,
Overview
The Arize platform automatically detects drift, data quality issues, or anomalous performance degradations with highly configurable monitors based on both common KPIs and custom metrics. However, in order to be able to compare shifts, perform analysis, and root cause performance degradation, teams need to set a baseline, either from training, validation, or prior time periods in production.
Choosing a Baseline
Choosing a baseline depends on your team's specific use case. See below for some considerations that may go into choosing your baseline.
- If you are not expecting changes between training and production: use training dataset as your baseline
- If you are expecting large changes between training and production (e.g. upsampling fraud in training, imbalanced):
- 1.use a specific/fixed time range from your production data set, such as the initial model launch period where you're actively monitoring its performance
- 2.use a validation dataset, this allows for monitoring to start from day 0
- If you are monitoring for highly fluctuating changes on a regular time interval (e.g. click-through rate): set a rolling window on your production dataset (e.g. every week, every two weeks)
Setting up a Baseline
In order to get metrics such as prediction and feature drift, we first need to set up a baseline to compare our model to. You can set up a baseline by clicking "Set up Model" within the blue bar at the top, going to the Datasets tab and clicking "Configure Baseline", or by going to the Config tab.
Production Baselines
Select parts of your production data using fixed or moving time ranges.
Pre-production Baselines
Select a dataset upload as a reference point.
​
​
Last modified 1mo ago
Copy link