Connect managed agents to Gateway

The setup has four parts: add your company's provider key, create a Cortex API key, point your agents at Gateway, and verify. It takes about 15 minutes. Your agents keep working as they do today.

Before you begin

Step 1. Add your company's provider key

Gateway calls the model provider with your company's key. Without it, requests through Gateway fail. Do this once for each provider your agents use.

1. Open Gateway → Providers.
2. Select Add provider, then choose the provider your agents use.
3. Confirm Endpoint URL matches the approved endpoint for that account.
4. Enter your company's key in Company key.
5. Confirm Provider enabled is on.
6. Select Save provider.

The refreshed provider list shows the provider as enabled with a Stored credential. For field details and the connection check, see Connect a provider.

Step 2. Create a Cortex API key

Each person gets their own key, so Cortex can show who made each request. Give an agent that runs on its own a separate key.

1. Open Gateway → API Keys.
2. Select Create API key.
3. Enter a name you will recognize, such as my-agent. The name is optional.
4. Select Create API key.
5. Select Copy key and store it in your password manager. Cortex shows the key once.
6. Confirm I've saved this key, then select Done.

The key starts with cct_ and appears in the key list. Treat it like a password.

Step 3. Point your agents at Gateway

Choose the option that matches how your agents reach their model provider today. Replace <your-api-key> with the key from step 2. Use an exact model ID from your organization's model catalog.

Option A. Agents call the provider directly

Set the Gateway URL and your key where the agent starts, in place of the provider's own URL and key. Use the variable names your agent or SDK reads. The Gateway URL depends on the API style your agent speaks.

The examples below use Anthropic and OpenAI-compatible APIs. They show the pattern. Other providers follow the same steps with their own variables.

Example: Anthropic API (Claude CLI)

export ANTHROPIC_BASE_URL=https://agent.chaoslabs.co/gateway
export ANTHROPIC_AUTH_TOKEN="<your-api-key>"
claude --model claude-opus-5-5 -p "Say hi"

Example: OpenAI-compatible API (OpenAI SDKs)

export OPENAI_BASE_URL=https://agent.chaoslabs.co/gateway/v1
export OPENAI_API_KEY="<your-api-key>"

Then send any request with your chosen model ID. The result is a normal provider response. For ready-made examples in cURL and the OpenAI and Anthropic SDKs, see Send your first request. On API Keys, Connect a client shows the setup for your client.

Option B. Agents go through LiteLLM

Keep LiteLLM in place and point it at Gateway instead of the provider. This was tested with LiteLLM 1.93.0. Add one entry for each model your agents use.

Each agent sends its own key (recommended). Each person's usage appears under their own name.

1. Point each model at Gateway and turn on header forwarding in your LiteLLM config.yaml. This example uses an Anthropic model:

model_list:
  - model_name: claude-opus-5-5
    litellm_params:
      model: anthropic/claude-opus-5-5
      api_base: https://agent.chaoslabs.co/gateway
      api_key: placeholder
 
general_settings:
  forward_client_headers_to_llm_api: true

For an OpenAI-compatible model, set model: openai/<model-id> and api_base: https://agent.chaoslabs.co/gateway/v1.

2. Restart the LiteLLM proxy so it loads the new config.
3. Start each agent with that person's key in the x-chaos-capture-token header. This example uses the Claude CLI. Other agents set the same header through their own settings:

export ANTHROPIC_BASE_URL=<your LiteLLM address>
export ANTHROPIC_CUSTOM_HEADERS="x-chaos-capture-token: <your-api-key>"
claude --model claude-opus-5-5 -p "Say hi"

One key for all agents. This is simpler, but all usage appears under that one key in Cortex. Put a single key in the LiteLLM config, restart the proxy, then start agents as you do today. This example uses an Anthropic model:

model_list:
  - model_name: claude-opus-5-5
    litellm_params:
      model: anthropic/claude-opus-5-5
      api_base: https://agent.chaoslabs.co/gateway
      api_key: placeholder
      extra_headers:
        Authorization: "Bearer <your-api-key>"

Step 4. Verify the connection

1. Run the test prompt from step 3, or ask your agent anything. A normal reply from the model means the request went through Gateway.
2. Open Gateway → API Keys. The Last used column for your key shows today's date.
3. Within a few minutes, the request appears in Gateway Overview with its cost and model. Only admins see Overview. Other users can check step 2.

Troubleshoot a failed request

Find the symptom, then work through the checks in order.

The request does not reach Gateway

Typical messages: "unsupported relay endpoint", a timeout, or no request in Last used.

• Check the base URL against the API style your agent speaks. In the examples, it ends in /gateway for the Anthropic API and /gateway/v1 for an OpenAI-compatible API. Do not add a path that is already there.
• In LiteLLM, confirm api_base is the Gateway URL, then restart the proxy so it loads the config.
• Confirm the agent starts with the variables from step 3 set.

Authentication fails

Typical messages: "an API key is required", "this relay requires an x-chaos-capture-token header", "the API key is not a valid Cortex API key", "the API key is not active".

• "An API key is required" or a missing header: Gateway did not receive your key. Send the key as an authorization token, not as an API key header. In LiteLLM, turn on forward_client_headers_to_llm_api and check the header from step 3.
• "Not a valid Cortex API key": the key was copied incompletely. Copy it again. It starts with cct_. If you lost it, create a new one.
• "Not active": the key was revoked. Create a new key in step 2.

The model is not found

Typical messages: "this model is not available through Gateway" or "no pricing row naming a routable provider".

• Check the model ID against your organization's model catalog. Use the exact ID.
• Confirm the provider is enabled with a stored key. See step 1.
• In LiteLLM, confirm the model and model_name entries match the model your agent requests.

Revoke a key

If a key leaks or you retire an agent, revoke its key.

1. Open Gateway → API Keys.
2. Find the key and select Revoke in its row.
3. Select Revoke key to confirm.

The key stops working within about a minute. You cannot undo a revoke, but you can create a new key.