Arize AI
Search…
11. Actuals
Actuals can take the form of scores, labels, and values depending on model type

Overview

Score Categorical Models

  • Actual Score: The ground truth score
  • Actual Label: The ground truth label
  • Actual Numeric Sequence: The ground truth sequence of actual relevance scores (only applicable for ranking models)
Numeric/Regression Models
  • Actual Value: The numeric value that the actual is for regression models
Actuals (outputs of a model) can take the form of scores, labels, and values depending on model type

Code Example

1
Code Example
2
# Declare the schema of the dataframe you're sending (feature columns, predictions, timestamp, actuals)
3
schema = Schema(
4
prediction_id_column_name="prediction_id",
5
...
6
actual_label_column_name="actual_label",
7
actual_score_column_name="actual_score",
8
actual_numeric_sequence_column_name="actual_relevance_scores",
9
tag_column_names=tags,
10
)
11
12
# Log the dataframe with the schema mapping
13
response = arize_client.log(
14
model_id="sample-model-1",
15
model_version= "v1",
16
model_type=ModelTypes.SCORE_CATEGORICAL,
17
environment=Environments.PRODUCTION,
18
dataframe=test_dataframe,
19
schema=schema,
20
)
Copied!

FAQ

  1. 1.
    What if I don't get back actuals for a while?
Arize supports logging actuals together with the prediction as well as sending in actuals delayed. In the case of sending delayed actuals, we will match them to their corresponding prediction using the prediction_id filed that both share and must match. This join operation takes place once per day.
Arize looks back 14 days to match an actual to its corresponding prediction.
For examples, check out our Examples for Delayed Actuals Tutorial.
Questions? Email us at [email protected] or Slack us in the #arize-support channel
Copy link