Generative Skills

AI Search

Available via the filter bar

AI Search consists of four powerful skills to help you curate your data:

1. Column Search

   • Description: Refined semantic search within a single column based on user criteria.

   • Suggested Prompt: “Find me confused inputs”

   • Use When: Copilot will automatically select this skill when you need to search within a specific column.

2. Table Search

   • Description: Search across multiple or all columns within a table to find patterns and outliers.

   • Suggested Prompt: “Find inputs that reference pricing that are hallucinated”

   • Use When: Copilot will automatically select this skill when you want to search across mutliple columns or entire table.

3. Text to Filter

   • Description: Generates query filters from natural language commands to refine your data search. Use keyword Filter to trigger this skill.

   • Suggested Prompt: "Filter by input contains SDK"

   • Use When: You want copilot to construct a query filter for you.

4. LLM Analysis Lite

   • Description: Lightweight analysis skill to find patterns and provide filter suggestions.

   • Suggested Prompt: "What are the top 5 types of questions asked?"

   • Use When: You need a lightweight analysis of your data.

Create Custom Evaluations

Available via the Task Creation Modal (Mini Chat) or in the main chat

• Suggested Prompt: "Write a custom evaluation that meets the following criteria:<criteria>"

• Use When: You need a tailored eval for your application.

• Description: Writes a tailored eval for your application. Specify your goal or let Copilot analyze your data and make suggestions.

Diagnose RAG Issues

Debug retrieval directly from trace details or in the main chat

• Suggested Prompt: "Debug retrieval step of RAG app for <trace_id>"

• Use When: You need to refine your retrieval process.

• Description: Evaluate and refine your retrieval process. Analyzes responses to ensure relevance and accuracy, and provides improvement strategies.

Optimize Prompts

Available directly in the prompt playground and via the main chat

• Suggested Prompt: "Optimize my prompt to <goal>"

• Use When: You want to fine-tune your prompts.

• Description: Optimize your prompt to address specific issues or enhance overall response quality.

Summarize Evaluation Metrics

• Suggested Prompt: "Summarize my <eval_name> eval"

• Use When: You want to understand the performance of your LLM application.

• Description: Assesses and summarizes your evaluation metrics, offering targeted suggestions for enhancing your LLM application.