Guardrails at runtime

How the AI Gateway applies live guardrails to every request and answer, and what your application sees when one fires.

Every request your applications send through the AI Gateway, and every reply, is checked against live guardrails before it goes on. Which guardrails depends on whose key it is:

• A key attached to an AI system is checked against that AI system’s live version. If the AI system has no live version, its requests are not checked, and the organization’s guardrails do not stand in for it.
• An organization key and a person’s key are checked against the organization’s live version, if it has one. So is a request that carries the caller’s own provider key, unless the virtual key sent with it is attached to an AI system: then that AI system’s guardrails apply.

An AI system’s guardrails take effect from its next request after you apply them. The same goes for attaching a key to an AI system: the next request uses the AI system’s guardrails.

The order of checks

1. Before the model

   Guardrails that apply to What people send check every message in the request, whatever its role, your system prompt included, and the arguments of every tool call in it. A block stops the request: it never reaches the model provider. A mask replaces the sensitive text before it is sent. A request is checked once, even when Altrum then tries a fallback model.

2. After the model

   Guardrails that apply to What the model answers check every answer the model returns, including a refusal and the arguments of a tool call. A block withholds the answer; the model provider still charged for it, so its tokens and cost count toward budgets and show in the Request log. A mask replaces the sensitive text before your application receives it.

Long text is checked in parts. A request with more than 50 parts of text is refused with status 413:

This request has more text than can be checked against the guardrails (more than 50 parts). Send less text at once.

An answer with more text than that cannot be checked, so it is withheld.

A check that cannot be made stops the request

If Altrum cannot check a request, for any reason, the request is not sent to the model. If it cannot check an answer, the answer is withheld. A failed check never lets unchecked text through. A guardrail setup that Altrum cannot read is treated the same way: the request is refused, not let through.

A key whose owner has no live guardrails is not checked, and the request is recorded as unguarded.

Streaming

• When every guardrail on answers is set to Log only, the answer streams to your application as the model writes it. Altrum checks the whole answer once the stream ends and records what it found. The last event of the stream waits for that check.
• When any guardrail on answers can Block or Mask, Altrum holds the answer, checks it whole, and then streams it to your application. The first words arrive later, and no unchecked text ever reaches the caller. A stream that is cut off while it is held releases nothing.
• When an answer is blocked, the caller receives one error and no end marker.

A mask on an answer replaces the sensitive text wherever the answer repeats it, whichever API your application called.

Tool calls

Calls to MCP tools made through Altrum are checked with the same guardrails: the arguments before the tool runs, and the result before it returns to the model. Listing the tools a key may use is not checked. Tool arguments that a mask would have to change are blocked instead, because they cannot be masked safely. See MCP tools.

What Altrum records

For every request: whose guardrails applied and which version, whether the request and the answer were checked, and how the request ended. For every check that found something: which guardrail fired, in which direction, what it did (logged, blocked or masked) and the kind of thing it found, such as a data type, a topic name, or the word you configured. Never the text that matched. See Request logs.

To see how often your guardrails fire, where and on what, as a rate of the requests they checked, open Reporting, Guardrail violations. It also shows what guardrails set to Log only found, so you can decide whether to block. See Reporting.

What your application sees

A blocked request, a blocked answer and a check that could not be made each return an error in the format of the API your application called, with its code: in type and error.type for the OpenAI format, in error.type for the Anthropic format and in error.status for the Google format. It names the guardrails that fired and never your text. Altrum does not try fallback models after any of these. “This AI system” reads “this organization” when the organization’s guardrails applied.

Status	Code	Message
400	guardrail_blocked	A guardrail on this AI system stopped the request (<guardrails>). Remove or change the flagged content and send it again.
400	guardrail_blocked	A guardrail on this AI system would have to change structured tool arguments, which cannot be masked safely, so the request was stopped. Remove the sensitive data from the tool call and send it again.
422	response_blocked	A guardrail on this AI system stopped the reply (<guardrails>). Your request was fine: change it, or ask for something different.
422	response_blocked	A guardrail on this AI system would have to change structured tool arguments in the reply, which cannot be masked safely, so the reply was withheld.
503	guardrail_check_failed	Altrum could not check this request against the guardrails on this AI system, so it was not sent to the provider. Try again in a moment, and contact support if it keeps happening. (for an answer: Altrum could not check the reply ..., so it was withheld.)
413	payload_too_large	See above.

Handle them as you would a provider’s refusal. All the codes are in Errors.