Guardrails
Last updated
Last updated
Copyright © 2023 Arize AI, Inc
Guardrails for Large Language Models (LLMs) are essential for ensuring real-time safety, context management, compliance, and user experience. While offline evaluation is a great tool for in-depth analysis of patterns of failure modes, Guardrails provide immediate correction of inappropriate content. Guardrails can be applied to either user input messages (e.g. jailbreak attempts) or LLM output messages (e.g. answer relevance). If a message in a LLM chat fails a Guard, then the Guard will take a corrective action, either providing a default response to the user or prompting the LLM to generate a new response.
We offer three types of Guards:
Dataset Embeddings Guard: Provided few shot examples of "bad" messages, Guard against similar inputs based on the cosine distance between embeddings.
General LLM Guard: Provide a prompt for an LLM Evaluator to classify the input as "pass" or "fail".
Few Shot LLM Guard: Provided few shot examples, an LLM Evaluator will classify the input as "pass" or "fail".
Users have the option to instantiate the Guards with off-the-shelf prompts / datasets from Arize, or customize the Guard with their own prompts / datasets.
Arize offers an off-the-shelf ArizeDatasetEmbeddings Guard. Given any dataset of "bad" examples, this Guard will protect against similar messages in the LLM chat.
This Guard works in the following way:
The Guard computes embeddings for chunks associated with a set of few shot examples of "bad" user prompts or LLM messages (we recommend using 10 different prompts to balance performance and latency).
When the the Guard is applied to a user or LLM message, the Guard computes the embedding for the input message and checks if any of the few shot "train" chunks in the dataset are close to the message in embedded space.
If the cosine distance between the input message and any of the chunks is within the user-specified threshold (default setting is 0.2), then the Guard intercepts the LLM call.
Refer to the repo and Python notebook tutorial for details, which can be loaded into Colab.
By default, the ArizeDatasetEmbeddings Guard will use few shot examples from a public dataset of jailbreak prompts. We benchmarked the performance of our model on this dataset and recorded the following results:
True Positives: 86.43% of 656 jailbreak prompts failed the DatasetEmbeddings guard.
False Negatives: 13.57% of 656 jailbreak prompts passed the DatasetEmbeddings guard.
False Positives: 13.95% of 2000 regular prompts failed the DatasetEmbeddings guard.
True Negatives: 86.05% of 2000 regular prompts passed the DatasetEmbeddings guard.
1.41 median latency for end-to-end LLM call on GPT-3.5.
Note that the "regular prompts" in the dataset consist of role play prompts that are designed to resemble jailbreak prompts.
By comparison, the arxiv paper associated with the dataset explains that jailbreak prompts have a 68.5% attack success rate (ASR) on the GPT-4 model.
Refer to validator/benchmark_guard_on_dataset.py for implementation details.
(Coming soon) This is similar to the ArizeDatasetEmbeddings Guard, but instead of chunking and embedding the dataset to compute the cosine distance to input messages, we use the dataset as few shot examples for an LLM prompt. Provided the dataset, the LLM Guard uses the prompt to evaluate whether an incoming message is similar to the dataset.
(Coming soon) All off-the-shelf LLM evaluators from Arize will be offered as Guards. Users can also instantiate the Guard with a custom prompt.
We recommend two different types of corrective actions when input does not pass the Guard, which can be passed into the Guard upon instantiation:
default response: Instantiate the Guard with on_fail="fix" if you want the Guard to use a user-defined hard-coded default LLM response.
LLM reask: Instantiate the Guard with on_fail="reask" to re-prompt the LLM when the Guard fails. Note that this can introduce additional latency in your application.
Additional details in a tutorial (coming soon).
Refer to the Python notebook tutorial for an example on how to integrate OTEL tracing with your Guard.
In addition to real-time intervention, Arize offers tracing and visualization tools to investigate chats where the Guard was triggered.
Below we see the following information in the Arize UI for a Jailbreak attempt flagged by the DatasetEmbeddings Guard:
Each LLM call and guard step that took place under the hood.
The error message from the Guard when it flagged the Jailbreak attempt.
The validator_result: "fail"
The validator_on_fail: "exception"
The cosine_distance: 0.15, which is the cosine distance of the closest embedded prompt chunk in the set of few shot examples of jailbreak prompts.
The text corresponding to the most_similar_chunk.
The text corresponding to the input_message.
Users have the option to connect their guards to Arize Production Monitoring. In the example below, we see a user create a monitor that sends an alert every time the Guard fails. These alerts can be connected to slack, PagerDuty, email, etc.
For additional support getting started with Guards, please refer to the following resources:
Python notebook tutorial (ArizeDatasetEmbeddings Guard with OTEL tracing in Arize UI)
ArizeDatasetEmbeddings guard class
guard_fail to count the number of times that the Guard fails.
guard_fail metric.