PraxonEvery agent demo runs the same script. A request comes in, the model reasons through it, calls a tool, gets a clean 200 back, and lands the right answer in under ten seconds. It reads as impressive because the input was pre-selected to cooperate.
Point the same agent at production traffic and the failure modes show up immediately: an ambiguous request with no single correct interpretation, a tool call returning a 5xx, a downstream API rate-limiting mid-task, a user changing intent halfway through a multi-turn flow. None of that surfaces in a demo, because a demo is a curated happy path. All of it surfaces in the first week of real usage.
Past the toy-project stage, the model stops being the hard part of an agent build. Call it the easy 20%. The other 80% is infrastructure that never makes a launch post:
- Failure containment. When a tool call fails, does the agent retry with exponential backoff, silently fabricate a plausible-looking result, or surface the failure to a human? Most agent frameworks default to naive immediate retry or silent failure unless you explicitly wire in the third path.
- Idempotency. Webhooks fire more than once under at-least-once delivery guarantees. A retry can land after a timeout that actually succeeded upstream. Without an idempotency key on every side-effecting call, an agent sends the same email twice, double-books the slot, double-charges the invoice. This is the single most common production defect in agent systems, and it's structurally invisible in a demo, because demos don't retry.
- Escalation paths. A production agent needs a defined threshold for handing off to a human instead of guessing. A demo agent skips that, because "the agent paused to ask a clarifying question" doesn't make for a compelling clip.
- Observability. When something breaks three weeks post-launch, can you reconstruct what the agent saw, what it decided, and why, from structured logs and traces? If the answer is grepping through raw model completions, that's not observability. That's a haystack with no index.
None of this is a knock on model capability. Tool-use and reasoning performance have genuinely improved. But "the model can execute this task" and "the system around the model can run unattended in production" are separate claims with separate engineering requirements, and public discussion mostly stays on the first because it demos well.
The teams shipping agents that survive contact with production aren't running the cleverest prompt. They're the ones who instrumented the system, tested it against adversarial and malformed input on purpose, and built a defined answer for what happens when a dependency fails at 2am with nobody watching.
That instrumentation work doesn't show up in a demo. It's also the entire difference between a proof of concept and a system a business can depend on.