Arize AI
Search…
4. Model Environments
Environments are the different prediction streams for a model.

Overview

Environments are the different prediction streams for a model.
  • Training Environment: Data used to build the model. The response of the model to the training data.
  • Validation Environment: Data used to test the model. The response of the model to various validation datasets.
    • We support multiple batches of validation data. This means there can be batch1, batch2, etc.
  • Production Environment: Deployed model. The stream of production inferences
Ex: Model A has 3 versions. v1 includes training, validation, and production datasets. v1 validation dataset includes Batch A, B, & C.

Code Example

Each model environment is defined in the code when the data is sent into the platform:
1
response = arize.log(
2
model_id='sample-model-1',
3
model_version='v1',
4
model_type=ModelTypes.NUMERIC,
5
environment=Environments.PRODUCTION,
6
...
7
prediction_value=100,
8
actual_value=90
9
)
Copied!

Examples of Model Environments across Arize

Baseline
Drift
Performance
Business Impact
Dashboards
In order to be able to compare shifts, perform analysis, and root cause performance degradation, teams need to set a baseline, either from training, validation, or prior time periods in production. When setting the model baseline, users can choose between pre-production or production datasets. For more information on setting a baseline, visit here.
Setting up a Baseline from Production or Pre-Production Environments
These baselines can then be used in monitors.
Using Baselines on Drift Monitors
Similar to drift monitors, the model's Drift tab shows a distribution between the current (production) distribution, and that of the baseline selected (production, validation, or training).
Distribution comparison between production and baseline in Model Drift Tab
In the model's Performance tab, you can choose to view performance for a specific model environment, or even compare different model environments.
Model environment filter in Model Performance Tab
The Business Impact tab supports environments. You can view the business impact of your model's predictions at different thresholds and compare across environments.
Model environment comparison in Model Business Impact Tab
Arize dashboards support model performance and data comparisons across different environments.
Example Use Case: Compare the distribution of my feature "fico_range" in production vs validation test set. The example below shows the distribution of "fico_range" from the validation test set which was testing against an extreme condition (orange graph) versus production (blue graph).
Validation versus Production Plot Lines in Dashboard Widget
Questions? Email us at [email protected] or Slack us in the #arize-support channel