> ## Documentation Index
> Fetch the complete documentation index at: https://anthale.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# LLM Output Validation Guide

> Validate LLM output before it reaches users, tools, or storage so generated content stays within your Anthale policy and application rules.

Use this guide when model output reaches a user, triggers a tool, generates code, or gets stored for later reuse. The failure mode is simple: the model produces something plausible, and the system trusts it too early.

## Assumptions

* You know what outputs the workflow is supposed to produce.
* Your application can reject, transform, or hold unsafe output.
* Anthale policies can run on the output path for the relevant workflow.

## Validate the output

<Steps>
  <Step title="Define the allowed output shape">
    Decide whether the output should be natural language, a structured object, code, a query, or a tool call. You cannot
    validate a shape you never defined.
  </Step>

  <Step title="Require structured data where the workflow allows it">
    Use schemas, enums, allowlisted fields, and deterministic parsing for any output that drives automation or writes to
    another system.
  </Step>

  <Step title="Run Anthale on the output path">
    Evaluate the response before it reaches a user or downstream system. This is where Anthale can return `detect`,
    `redact`, or `block` based on the policy for that boundary.
  </Step>

  <Step title="Use sanitized content after a redaction">
    If Anthale returns `redact`, keep only the sanitized content in the next step of the workflow. Do not send or store
    the original output.
  </Step>

  <Step title="Validate side-effecting payloads separately">
    Treat code, SQL, shell commands, and tool arguments as proposals until your application validates them against
    allowlists and business rules.
  </Step>

  <Step title="Test with adversarial and malformed cases">
    Check how the workflow behaves when the model produces hidden instructions, sensitive data, malicious links, or an
    invalid structured response.
  </Step>
</Steps>

<Tip>
  Output validation is not the same as prompt injection protection, but the two often meet at the same boundary.
  Injection changes what the model tries to do. Output validation decides whether the system will accept it.
</Tip>

## What this looks like in code

<CodeGroup>
  ```python title="Python" theme={null}
  import json
  from os import environ

  from anthale import Anthale

  client = Anthale(api_key=environ["ANTHALE_API_KEY"])
  OUTPUT_POLICY_ID = environ["ANTHALE_OUTPUT_POLICY_ID"]

  ALLOWED_ACTIONS = {"answer", "escalate"}

  def validate_support_response(model_output_json: str) -> dict:
  response = client.organizations.policies.enforce(
  policy_identifier=OUTPUT_POLICY_ID,
  direction="output",
  messages=[{"role": "assistant", "content": model_output_json}],
  metadata={"boundary": "final_output", "feature": "support-agent"},
  )
  if response.action == "block":
  raise RuntimeError("Anthale blocked the model output.")

      safe_output = response.redacted_messages[0]["content"] if response.action == "redact" else model_output_json
      payload = json.loads(safe_output)

      if payload["action"] not in ALLOWED_ACTIONS:
          raise RuntimeError("The model returned an unsupported action.")

      if payload["action"] == "escalate" and not payload["ticketId"].startswith("TICKET-"):
          raise RuntimeError("Escalations must reference a valid ticket ID.")

      return payload

  ```

  ```text title="TypeScript" theme={null}
  import Anthale from "anthale"

  const client = new Anthale({ apiKey: process.env.ANTHALE_API_KEY })
  const OUTPUT_POLICY_ID = process.env.ANTHALE_OUTPUT_POLICY_ID!

  const ALLOWED_ACTIONS = new Set(["answer", "escalate"])

  type SupportResponse = {
    action: string
    message: string
    ticketId?: string
  }

  export async function validateSupportResponse(modelOutputJson: string): Promise<SupportResponse> {
    const response = await client.organizations.policies.enforce(OUTPUT_POLICY_ID, {
      direction: "output",
      messages: [{ role: "assistant", content: modelOutputJson }],
      metadata: { boundary: "final_output", feature: "support-agent" },
    })

    if (response.action === "block") {
      throw new Error("Anthale blocked the model output.")
    }

    const safeOutput = response.action === "redact" ? response.redacted_messages[0].content : modelOutputJson
    const payload = JSON.parse(safeOutput) as SupportResponse

    if (!ALLOWED_ACTIONS.has(payload.action)) {
      throw new Error("The model returned an unsupported action.")
    }

    if (payload.action === "escalate" && !payload.ticketId?.startsWith("TICKET-")) {
      throw new Error("Escalations must reference a valid ticket ID.")
    }

    return payload
  }
  ```
</CodeGroup>

Anthale decides whether the output can continue, but your application still validates the payload shape and business
meaning before another system consumes it.

## Common failure patterns

* Raw model output is passed directly into a tool, database, or shell.
* Anthale returns `redact`, but the original output is still logged or forwarded elsewhere.
* Output checks exist for user-facing text but not for code, queries, or machine-readable payloads.
* Teams validate syntax but not business meaning, destination, or entitlement.

## Apply this in Anthale

* [Create a first policy](/docs/quickstart/first-policy) when you want to test output-path enforcement with a working Anthale policy.
* [See the enforcement API](/docs/api-reference) when you need the exact output-direction contract before wiring production traffic.
* [Request access](https://anthale.com/#request-access) when you want to review an output validation path with Anthale in place.

## Related pages

* [Actions and Evaluation Flow](/docs/learn/policies/actions-and-evaluation-flow)
* [Gate Tool Actions](/docs/learn/secure-ai-systems/gate-tool-actions)
* [Content Moderation](/docs/learn/guardrails/content-moderation)

## Next steps

Continue with [Secure Memory and State](/docs/learn/secure-ai-systems/secure-memory-and-state) or [Content Moderation](/docs/learn/guardrails/content-moderation).