Prompt Hub

Overview

The Prompt Hub is a centralized repository for managing, iterating, and deploying prompt templates within the Arize platform. It serves as a collaborative workspace for users to refine and store templates for various use cases, including production applications and experimentation.

Key features of the Prompt Hub include:

• Template Management: Users can save templates directly from the Prompt Playground along with associated LLM parameters, function definitions, and metadata required to reproduce specific LLM calls.

• Version Control: Every saved template supports versioning, enabling users to track updates, experiment with variations, and revert to previous versions if needed.

• Collaboration and Reusability: Saved templates can be shared across teams, facilitating collaboration and consistency in production workflows. Templates can also be reloaded into the Prompt Playground or accessed via APIs for seamless integration into codebases and online tasks.

• Evaluation and Optimization: By saving outputs as experiments, users can compare templates, compute evaluation metrics, and analyze performance both quantitatively and qualitatively.

The Prompt Hub provides a comprehensive solution for managing prompt workflows, ensuring traceability, reusability, and integration into production pipelines.

Save New Prompt to Hub

Any playground template can also be saved to the Prompt Hub, making it especially valuable for production use cases and collaboration. The Prompt Hub stores the template along with LLM parameters, function definitions, and other metadata necessary to reproduce the LLM call.

User provides a unique Prompt Name, description, version description, and tags, which will be saved as metadata associated with the prompt in the Prompt Hub.

The Prompt Hub listing page displays all prompts saved in the space. Users can search for prompts by name using the search bar and sort them by criteria such as 'Last Updated' or 'Name' for easy navigation.

By clicking on a specific prompt, the user can view its metadata, version history, and the associated prompt template and LLM parameters for the selected version.

Load Prompt in Hub to Playground

A saved prompt can be loaded into the Prompt Playground for additional iteration and version updates. This can be done directly from the prompt version view, the Prompt Hub listing page, or within the Prompt Playground itself, allowing for seamless editing and refinement.

User selects 'Edit in Prompt Playground' to load Version 2 of the RAG Prompt w/out Hallucinations into the Prompt Playground.

User selects the play button to load the latest version of the prompt into the playground.

User selects 'Load Prompt' to load a prompt name and version into the playground.

Edit Prompt and Save Version Update

Once the prompt is loaded into the Prompt Playground, the user can freely make changes and test the updated template on a dataset, following the same workflow described earlier. In the example below, the system template is modified to instruct the LLM to provide concise, one-sentence responses. Instead of creating a new prompt, the user updates the version of the existing prompt directly, preserving the connection to the original template from the Prompt Hub.

Users have the option to save their changes as a new prompt or update the existing prompt by creating a new version, ensuring flexibility in managing and refining templates.

Users provide a description of their changes, which is saved as metadata attached to the new version of the prompt for improved traceability and context.