Tabular Skills

Get Model Insights

• Suggested Prompt: "Give me an insight"

• Use When: General inquiries about model behavior

• Description: Provides a high-level analysis of a model's performance metrics, including trends over time, prediction volumes, and prediction drift. Ideal for general inquiries but less suited for detailed debugging or specific issues.

Check for Missing Data

• Suggested Prompt: "What dimensions have high percent empty?"

• Use When: Identifying potential data quality issues that could impact model inputs

• Description: Analyzes input data to report on the percentage of missing data in features and tags, highlighting any sudden spikes or changes.

Detect Data Drift

• Suggested Prompt: "Analyze my inputs for drift"

• Use When: You want to understand which dimensions are drifting in your model, especially when performance metrics are unavailable

• Description: Helps pinpoint sudden input quality issues by examining both features and tags for signs of data drift, comparing current distributions to a baseline to detect significant shifts.

Cohort Performance Analysis (get_performance_slices_over_time) - NEEDS TO BE FIXED

• Suggested Prompt: "What are the worst performing slices?"

• Use When: Tracking performance in specific data segments

• Description: Analyzes model performance across different cohorts or slices of data to identify poorly performing segments. This function provides insights into model behavior over a specific period, but it is not designed for tracking performance trends over time.

Assess Feature Data Quality

• Suggested Prompt: "What features have data quality issues?"

• Use When: You want to understand data quality at a high level

• Description: Assists machine learning engineers in debugging issues by conducting a detailed analysis of dataset metrics, focusing on dimensions related to the user's investigation. Identifies critical changes, such as drift or cardinality variations, and provides actionable suggestions to further investigate and resolve identified issues.

Evaluate Distribution Shifts

• Suggested Prompt: "Analyze distribution shift for "

• Use When: You want to understand a given dimension's distribution

• Description: Analyzes a given dimension's distribution to understand which slices have had significant shifts in their percentage of the distribution.

Review Cardinality Trends

• Suggested Prompt: "Analyze changes in feature cardinality"

• Use When: You want to analyze changes in the cardinality of your features

• Description: Analyzes changes in the cardinality of features and tags over time, alerting to any unusual variations that might indicate data quality problems. Particularly valuable when direct performance metrics from the model are not available.