5 min read

The Trust Trap

The Trust Trap

In hospitals, monitoring equipment generates between 150 and 350 alarms per patient per day. Between 72% and 99% of those alarms are false. Clinicians know this. So they start ignoring them — not out of negligence, but out of mathematical necessity. You cannot meaningfully respond to 300 alerts a day when 290 are noise.

The FDA documented 566 deaths from ignored alarms in a three-year window. Not because the alarms failed. Because they worked too well. They fired for everything. The safety mechanism became the danger.

Software development just built the same trap.

The Numbers That Don't Add Up

The Sonar 2026 Developer Survey found that 96% of developers don't fully trust AI-generated code. That sounds healthy. Skepticism should be a safety net — a natural check on the system.

But 84% use AI tools anyway (Stack Overflow). And AI now accounts for 42% of all committed code. So 96% distrust doesn't mean careful review of the 42%. It means every piece of code in the codebase is suspect. There is no trusted baseline to compare against, no clean signal to separate from noise. Everything requires checking. And that volume is simply beyond what human review can absorb.

The result: only 48% of developers always verify AI code before committing it. The rest ship code they've already told you they don't trust.

This is alarm fatigue. The mechanism is identical.

Three Forces, One Trap

What makes this a trap — not just a bad habit — is that three forces lock it in place and each one feeds the others.

YOU CAN'T NOT USE IT 84% adoption, 29% trust 60% fear career consequences more code to review YOU CAN'T CHECK IT ALL 31.3% PRs merged unreviewed Review time up 441.5% unchecked code + overload YOU'RE LOSING THE ABILITY 39% more errors at 4+ tools 88% burnout, <40% comprehension more pressure to adopt AI THE TRAP

Force 1: You can't not use it

Sixty percent of employees fear AI will make colleagues question their competence if they don't use it. Ninety-five percent use AI tools weekly. The social cost of abstaining now exceeds the quality cost of participating. No one is being ordered to use AI. They're being measured in ways that make not using it untenable.

Force 2: You can't check it all

The Faros AI Engineering Report — 22,000 developers, 4,000 teams, two years of telemetry — shows what happens when adoption outpaces review capacity. Pull requests merged without any review: up 31.3%. Median time in review: up 441.5%. Incidents-to-PR ratio: up 242.7%. Code churn: up 861%.

AI-generated PRs wait 4.6 times longer for review. Reviewers can tell. They're overwhelmed. Forty-three percent of AI-generated code changes need debugging in production even after passing QA. The review backlog isn't a workflow problem. It's a throughput impossibility.

Force 3: You're losing the ability to check what you do review

The BCG "brain fry" study — 1,488 workers — found that using four or more AI tools produces a productivity decline. Fourteen percent more mental effort. Thirty-nine percent more major errors. Thirty-nine percent higher intent to quit.

UC Berkeley tracked 200 employees over eight months: AI users work more hours, not fewer. Task scope expands. Boundaries blur. Burnout offsets the gains. Eighty-eight percent of heavy AI users report increased burnout.

And then the deepest cut. O'Reilly's research on comprehension debt: developers who delegate to AI score below 40% on comprehension tests for their own codebases. An independent BCG study of 1,200 professionals found that workers detect fewer errors when reviewing AI output than when reviewing human output. The tool that's supposed to help you code is degrading your ability to evaluate code.

Why Distrust Doesn't Save You

Here's the structural insight, and why the alarm fatigue parallel is precise rather than metaphorical.

If 10% of developers distrusted AI code, verification would work. You'd spot-check the suspicious output, the way a clinician might investigate one unusual alarm per shift. The load is manageable. The skepticism is productive.

At 96%, the math inverts. Every piece of code is suspect. Forty-two percent of the codebase is AI-generated and climbing toward sixty-five percent by 2027. Verification isn't a targeted check anymore — it's the entire workload. And the entire workload can't fit inside an already-full workday, especially when 71% of developers already feel like middlemen in their own process.

So people stop checking. Not because they trust the code. Because the act of checking everything is physically impossible.

The safety mechanism doesn't fail because it's broken. It fails because it fires for everything. Distrust at 96% isn't a safety net. It's a weight.

In hospitals, they eventually learned this. Johns Hopkins turned off low-priority alarms and cut alarm volume by more than half. Boston Medical Center dropped from one million alarms per week to 400,000. The solution wasn't more vigilance. It was fewer alarms — accepting that universal alerting is worse than targeted silence.

Software hasn't learned this yet. The response to unreviewed AI code is more process: formal review requirements (93% of organizations have them, only 56% enforce them), new verification tools, additional checklists. More alarms. The same strategy that killed 566 patients.

The Trap Closes

In March 2026, Amazon suffered outages traced to AI-assisted code changes deployed without proper approval. Six hours of downtime. An estimated 6.3 million lost orders. They launched a 90-day code safety reset across 335 systems.

That's the cycle completing. Career pressure pushes adoption. Adoption floods the review pipeline. The pipeline can't absorb the volume. Unreviewed code ships. Production breaks. And the organizational response is more AI to fix the AI — which feeds more code into the system that already can't check what it has.

Every force in the trap reinforces the others. The developer who skips review today has a slightly worse codebase to navigate tomorrow, slightly less comprehension of what's running, slightly more pressure to use AI to manage the complexity that AI introduced. The trap doesn't spring. It tightens.

Ninety-six percent distrust. Eighty-four percent adoption. Thirty-one percent of pull requests merged without any review. These aren't three separate problems. They're one mechanism, and the mechanism is: universal skepticism, under sufficient load, produces the same outcome as universal trust.