MistralAI

Instrument LLM calls made using MistralAI's SDK

Google Colab

Follow our colab guide

tutorials/python/llm/tracing/mistral/mistral-tracing.ipynb at main · Arize-ai/tutorialsGitHub

See the source code

MistralAI is a leading provider for state-of-the-art LLMs. The MistralAI SDK can be instrumented using the openinference-instrumentation-mistralai package.

Python

In this example we will instrument a small program that uses the MistralAI chat completions API and observe the traces in Arize.

pip install openinference-instrumentation-mistralai mistralai arize-otel opentelemetry-sdk opentelemetry-exporter-grpc

Set the MISTRAL_API_KEY environment variable to authenticate calls made using the SDK.

export MISTRAL_API_KEY=[your_key_here]

In a python file, setup the MistralAIInstrumentor and configure the tracer to send traces to Arize.

# Import open-telemetry dependencies
from arize.otel import register

# Setup OTel via our convenience function
tracer_provider = register(
    space_id = "your-space-id", # in app space settings page
    api_key = "your-api-key", # in app space settings page
    project_name = "your-project-name", # name this to whatever you would like
)

# Import openinference instrumentor to map Mistral traces to a standard format
from openinference.instrumentation.mistralai import MistralAIInstrumentor

# Turn on the instrumentor
MistralAIInstrumentor().instrument(tracer_provider=tracer_provider)

To test, run the following code and observe your traces in Arize.

from mistralai.client import MistralClient
from mistralai.models.chat_completion import ChatMessage

client = MistralClient()
response = client.chat(
    model="mistral-large-latest",
    messages=[
        ChatMessage(
            content="Who won the World Cup in 2018?",
            role="user",
        )
    ],
)
print(response.choices[0].message.content)