PraxonWhen we scope an AI automation for a client, the exciting part lasts about five minutes. "It parses the incoming message, classifies intent, and drafts a response": that layer is genuinely fun to build and demo. What actually determines whether the system is still running unattended six months later is everything downstream of that, and it's the part that requires deliberate discipline to prioritize, because none of it is interesting.
What's the defined behavior when the payload arrives in a format the parser doesn't handle? What happens when the CRM's API returns 503s for four minutes during business hours? What happens when the automation fires twice because the webhook provider retried after a slow response it interpreted as a failure? Who gets alerted when the pipeline silently stops processing, and on what timescale: a paging alert within minutes, or a customer complaint three weeks later?
None of that is a model-quality problem. It's the identical set of failure-mode concerns that has always separated production-grade software from a fragile prototype, and the introduction of an LLM into the pipeline hasn't changed the underlying engineering requirements. It's changed how easy it is to ship something that looks complete before any of those failure modes have been addressed.
That ease is the actual risk. Building a business automation a few years ago required enough upfront engineering effort that failure-mode analysis happened by necessity, because reaching a working prototype meant you'd already touched most of the system's surface area. Now a demo-quality pipeline ships in an afternoon, which is a genuine productivity gain, but it means "what's the failure behavior here" has to become a deliberate design step instead of something absorbed along the way.
The automations still running unattended a year after launch consistently share the same unglamorous engineering traits:
- Every external call has an explicitly chosen failure policy, not a library default: a bounded timeout, retry with exponential backoff, or a hard fail with an alert, decided deliberately per integration rather than inherited from whatever the SDK ships with.
- Every side-effecting action, sending an email, writing a record, booking a slot, is implemented against an idempotency key, so a duplicate trigger is provably safe to replay.
- Failures that exceed automatic recovery route to an actual notification a human sees, not a log line in a file nobody tails, and unrecoverable events land in a dead-letter queue instead of vanishing silently.
- Someone ran adversarial and malformed input through the system deliberately, before a live customer became the first person to do it by accident.
None of this argues against moving fast to a working prototype. That instinct is correct. It argues against conflating "the prototype completed a happy-path run" with "this is production-ready." The gap between those two states is almost entirely this unglamorous engineering, and for anything meant to run unattended, that engineering is the actual deliverable, not an add-on to it.