Ask or search…

Prompt Engineering

Iterate on low performance prompt templates in Arize
Prompt analysis is an important component in troubleshooting your LLM's performance. Often, LLM performance can be improved simply by comparing different prompt templates, or iterating on the one you have.
Follow these steps to find prompt templates that need to be improved, and how to iterate and test out new prompt templates.

Identify Responses with Poor Feedback / Evaluation Scores

Select the evaluation metric you want to measure on Performance Tracing.
On the Table view, make sure the User Feedback and prompt_template_name columns are selected in the Primary Columns selector. Then sort the User Feedback column from lowest score to highest.

Identify the Template Associated with Poor Responses

Click on a row you want to investigate further, to open up the side panel.
The side panel will show you information such as the prompt template used, and any other relevant variables such as the prompt instruction.
Prediction Details
Click on either "Edit Template" or the "Prompt Playground" tab to iterate and compare responses.
Navigate to Prompt Playground

Iterate and Compare Responses Across Prompt Templates

Choose to edit the prompt variables, LLM parameters, or the prompt template itself. Then re-run the prompt to easily compare responses for analysis prior to implementing a new or revised prompt template.
Edit Prompt Template, Params, or Variables
How do I set this up?
Check out our Integrations to see how to gain access to LLM's in Arize.