Portkey

Portkey AI Gateway Tracing & Observability

Portkey is an AI Gateway and Control Panel that provides production-ready features for AI applications including observability, reliability, and cost management. Learn how to instrument the Portkey SDK using the openinference-instrumentation-portkey package for comprehensive LLM tracing and monitoring.

Quick Start: Portkey Python Integration

Installation & Setup

Install the required packages for Portkey AI Gateway tracing:

pip install openinference-instrumentation-portkey portkey-ai arize-otel

Instrumentation Setup

Configure the PortkeyInstrumentor and tracer to send traces to Arize for LLM observability:

# Import open-telemetry dependencies
from arize.otel import register

# Setup OTel via our convenience function
tracer_provider = register(
    space_id = "your-space-id", # in app space settings page
    api_key = "your-api-key", # in app space settings page
    project_name = "your-project-name", # name this to whatever you would like
)

# Import openinference instrumentor to map Portkey traces to a standard format
from openinference.instrumentation.portkey import PortkeyInstrumentor

# Turn on the instrumentor
PortkeyInstrumentor().instrument(tracer_provider=tracer_provider)

Example: Basic Portkey AI Gateway Usage

Test your Portkey integration with this example code and observe traces in Arize:

import os
from openai import OpenAI
from portkey_ai import PORTKEY_GATEWAY_URL, createHeaders

# Set up your API keys
os.environ["OPENAI_API_KEY"] = "your-openai-api-key"
os.environ["PORTKEY_API_KEY"] = "your-portkey-api-key"  # Optional for self-hosted

client = OpenAI(
    api_key=os.environ.get("OPENAI_API_KEY"),
    base_url=PORTKEY_GATEWAY_URL,
    default_headers=createHeaders(
        provider="openai",
        api_key=os.environ.get("PORTKEY_API_KEY")
    )
)

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "What is artificial intelligence?"}]
)

print(response.choices[0].message.content)

Start using your LLM application and monitor traces in Arize.

What is covered by the Instrumentation

Arize provides comprehensive observability for Portkey's AI Gateway capabilities, automatically tracing:

Multi-Provider LLM Management

• Multiple Provider Calls: Track requests across different LLM providers (OpenAI, Anthropic, Cohere) through Portkey's unified interface

• Provider Switching: Monitor seamless switching between AI providers

• Cost Optimization: Track usage and costs across different LLM providers

Reliability & Performance Monitoring

• Fallback and Retry Logic: Monitor automatic fallbacks and retry attempts when primary services fail

• Load Balancing: Observe how requests are distributed across multiple models or providers

• Latency Tracking: Monitor response times and performance metrics

Intelligent Caching & Optimization

• Semantic Caching: See cache hits and misses for semantic caching to optimize costs

• Request Deduplication: Track duplicate request handling

• Performance Optimization: Identify bottlenecks and optimization opportunities