Automation Creates New Failure Modes
Automation removes certain human errors but introduces new types of failures. It changes the nature of problems rather than eliminating them.
Automation Creates New Failure Modes
Automation removes certain human errors. No typos from tired hands. No forgotten steps. No inconsistency from mood or context.
It introduces new types of failures.
We encountered:
- Retry loops: A task fails, retries, fails again, retries again. Without proper backoff and circuit breakers, the system can spiral.
- Duplicate tasks: The same work runs twice—or more—because of queue quirks or unclear idempotency.
- State drift: Agents and humans modify state in different orders. The system believes one thing; reality is another.
- Silent failures: A task "succeeds" but produces wrong output. No exception, no alert—just bad data downstream.
Automation changes the nature of problems rather than eliminating them. You trade one class of failure for another. The key is designing for the new class.