Tabular Skills
Get Model Insights
Suggested Prompt: "Give me an insight"
Use When: General inquiries about model behavior
Description: Provides a high-level analysis of a model's performance metrics, including trends over time, prediction volumes, and prediction drift. Ideal for general inquiries but less suited for detailed debugging or specific issues.
Check for Missing Data
Suggested Prompt: "What dimensions have high percent empty?"
Use When: Identifying potential data quality issues that could impact model inputs
Description: Analyzes input data to report on the percentage of missing data in features and tags, highlighting any sudden spikes or changes.
Detect Data Drift
Suggested Prompt: "Analyze my inputs for drift"
Use When: You want to understand which dimensions are drifting in your model, especially when performance metrics are unavailable
Description: Helps pinpoint sudden input quality issues by examining both features and tags for signs of data drift, comparing current distributions to a baseline to detect significant shifts.
Cohort Performance Analysis (get_performance_slices_over_time)
Suggested Prompt: "What are the worst performing slices?"
Use When: Tracking performance in specific data segments
Description: Analyzes model performance across different cohorts or slices of data to identify poorly performing segments. This function provides insights into model behavior over a specific period, but it is not designed for tracking performance trends over time.
Assess Feature Data Quality
Suggested Prompt: "What features have data quality issues?"
Use When: You want to understand data quality at a high level
Description: Assists machine learning engineers in debugging issues by conducting a detailed analysis of dataset metrics, focusing on dimensions related to the user's investigation. Identifies critical changes, such as drift or cardinality variations, and provides actionable suggestions to further investigate and resolve identified issues.
Evaluate Distribution Shifts
Suggested Prompt: "Analyze distribution shift for "
Use When: You want to understand a given dimension's distribution
Description: Analyzes a given dimension's distribution to understand which slices have had significant shifts in their percentage of the distribution.
Review Cardinality Trends
Suggested Prompt: "Analyze changes in feature cardinality"
Use When: You want to analyze changes in the cardinality of your features
Description: Analyzes changes in the cardinality of features and tags over time, alerting to any unusual variations that might indicate data quality problems. Particularly valuable when direct performance metrics from the model are not available.
Last updated