Unthought.

Guardrails and failure behavior

Untrusted content treated as data rather than instruction, outputs validated before use, and nothing irreversible triggered without confirmation.

The interesting failures here are not wrong answers. They are the confidently wrong answer that nobody thought to check, the instruction hidden inside a document the system was asked to summarize, and the output that was acted on automatically because acting on it automatically was the whole point of building it.

Content the business did not author is treated as data and never as instruction. A document, a web page, an email, a support ticket or a filename may contain text addressed to the system, and the separation between instructions the business wrote and material the system is processing is enforced structurally instead of by asking the model politely to maintain it.

Output is validated before anything consumes it. A response is checked against its schema and against whatever the domain requires — a value inside a permitted range, a reference that resolves, a decision drawn from a closed set — and a response that fails is rejected rather than passed along in the hope that the next component is defensive.

Uncertainty has a specified behavior instead of being left to chance. What the system does when it has no good answer, and where that path leads, is designed alongside the successful path, because a system with no way to express doubt will express confidence by default.

Nothing irreversible happens without confirmation. Sending, refunding, deleting, publishing and committing are gated behind an explicit step, and the gate is a property of the surrounding code, not an instruction in a prompt, which is a request, not a constraint.

What goes in is governed as carefully as what comes out. Personal data, credentials and client-confidential material reaching a vendor is a disclosure regardless of how the response is handled, and the boundary is drawn before the feature is built rather than audited after somebody notices what the logs contain.

Rate and spend limits are set per user and in aggregate. A loop that feeds a model its own output can consume a month of budget in an afternoon without anything technically failing, and a limit is the only thing standing between an ordinary bug and an invoice nobody approved.

Inputs and outputs are logged with a deliberate retention period. An incident that cannot be reconstructed cannot be explained to whoever is asking, and the question always arrives after the window in which anyone was still storing the evidence.

Refusal is designed for the domain, not inherited from a vendor's defaults. What the system must decline — legal advice, medical interpretation, a commitment about price or eligibility — is enumerated as part of the specification, because the vendor's boundaries were drawn for everyone and this business is answerable for its own.

Exclusions

What this does not cover.

  • Guardrails implemented purely as prompt instructions, with no validation of what comes back.
  • Systems that treat retrieved or user-supplied content as instructions to follow.
  • Automatic execution of irreversible actions on the strength of model output alone.