Routing controls

Egress resolves an explicit model through the organization's pricing sheet, then forwards through an enabled provider. Auto Router can replace a model on qualified client paths using managed configuration. These are separate decisions.

Provider selection

The organization manages provider credentials and the model-to-provider mapping. A pricing row must name a provider that Egress can route, and that provider must have an enabled configuration. Public catalog membership alone does not establish gateway availability.

Control	Where it is applied
Model-to-provider mapping	The organization's pricing sheet supplies the provider for the requested model.
Provider credentials	Managed configuration supplies the enabled provider credential.
Native request shape	The caller uses a request format accepted by that provider; Egress does not translate every SDK format between providers.
Auto Router replacement	Managed tier settings and compatibility checks determine whether a qualified request can be rewritten.

Request-level provider allow lists, exclude lists, ordering, and capability filters are not implemented as Cortex gateway controls. A JSON routing object does not constrain provider selection and can be forwarded to the upstream provider as an unknown field. Do not use it to enforce a data-handling rule.

Example request

Use the client-native body with an explicit organization-enabled model:

{
  "model": "<organization-enabled-model>",
  "messages": [{ "role": "user", "content": "Draft the incident update." }]
}

Provider and model fallbacks

Egress does not accept an ordered provider or fallback-model list for automatic retries. Smart Router's fallback to the original request means a replacement was not applied; it is not a retry after a provider failure.

If your application needs retries, implement and bound them in the caller. Distinguish each provider fallback from a model fallback in your own execution record.

Application behavior	What to preserve
Provider fallback	The selected model, failed provider, failure class, and authorized alternate connection.
Model fallback	The original model, replacement model, reason, native request compatibility, and each attempt's usage.
Streaming failure	Whether output has already reached the client. Do not replay a stream after emitting partial output.

Retry only errors that your application and provider allow. Account for SDK retries when setting the total attempt budget. A failed or timed-out request can still incur provider charges.

Performance preferences and caching

Managed Auto Router maps complexity tiers to configured targets. It does not accept request-level quality, cost, latency, throughput, or prompt-caching objectives.

Use Models to compare base token prices, and your own workload measurements to choose targets. The public price catalog contains no quality scores or measured latency. Advanced routing shows a caller-owned selection recipe.

Prompt caching follows the native provider's supported request fields and behavior. A Smart Router decision cache stores routing decisions; it is separate from provider prompt caching and does not prove a billed cache hit.

Policy constraints

Enforce access and data-handling requirements with the organization's provider and security configuration. The managed allowed_models setting gates replacement targets, but a rejected replacement can preserve the original request. It does not prevent an otherwise admitted original model from being forwarded.

If an application imposes a stricter candidate set and no eligible route remains, stop before sending a request. Do not assume an unsupported request control will make the gateway fail closed.

Continue with routing

• Auto Router: configure tier targets and inspect the available decision evidence.
• Advanced routing: build application-owned request, selection, and deliberation workflows.
• Egress Gateway: connect the client and verify forwarding.