Shipping an AI agent to production: the guardrails that matter
A demo agent takes an afternoon. A production agent needs evaluation, permissions, fallbacks and cost control. Here is the checklist we use.
By the Anirmo engineering team
Language models make it easy to build an impressive prototype. The distance between that prototype and something a business can rely on every day is mostly engineering discipline, not model choice.
1. Define what good looks like
Before tuning prompts, we collect real examples and write down the expected result for each. This evaluation set becomes the test suite for the agent. Every change to prompts, tools or models runs against it, so quality is measured instead of guessed.
2. Give the agent narrow permissions
An agent should only be able to do what its job requires. Read access where reading is enough, write access behind explicit tools with validation, and no direct access to anything that moves money or deletes data without a human approval step.
3. Design the handoff to a human
The best agents know when to stop. We define clear rules for escalation, such as low confidence, sensitive topics or angry customers, and pass the full context to a person so nobody has to start over.
4. Plan for failure
Models time out, APIs change and outputs occasionally break format. Production agents need:
- 01Timeouts and retries with backoff
- 02Output validation against a schema, with a safe fallback
- 03Logging of inputs, tool calls and outputs for debugging and audits
- 04A kill switch that routes everything to humans in one click
5. Watch the cost
Token costs scale with usage. We track cost per task from day one, cache what can be cached, and use smaller models for simple steps so the economics still work at ten times the volume.