The model arrived after lunch. A full architecture, capability map, and dependency diagram that once took three months. Clean. Consistent. Beautifully formatted. Nobody could find a single thing wrong with it.
That was the problem.
Wrong work used to look wrong. It was rough, late, and full of gaps that made reviewers lean in. Now wrong work arrives polished, and polish is easy to mistake for truth.
The Scarce Thing Is Changing
My friend Raj Ramesh recently made a sharp point about how AI is changing enterprise architecture (LINK): when AI makes EA artifacts cheap, the scarce thing becomes the understanding of the enterprise behind them.
I would take it one step further. What gets scarce is assurance.
Gartner predicts AI agents will help deliver 50% of enterprise architecture outcomes by 2028. If half of what we produce comes from a machine, the question stops being how fast we can produce it. It becomes who can vouch for it.
What Air Traffic Control Understands
Every pilot needs charts. Nobody would fly without them. But nobody confuses the chart with the airspace.
The sky stays safe because of what happens after the chart is drawn: verification of what is actually out there, strict separation rules, clear handoffs, and a named person accountable for every aircraft. The chart is where safety starts. It was never where safety lives.
That lesson applies to all of us who build models and diagrams, myself included. If the work ends at the diagram, we will have problems. AI just brought them closer.
What Years of Modeling Taught Me
I have spent most of my career in systems engineering and modeling, and one lesson keeps repeating. A model has never carried a program by itself. What carries a program is the confidence behind the model: someone checked it against reality, someone owns it, and someone can explain what it does not cover.
When that confidence is missing, the diagram becomes decoration, and the program gets surprised late. AI did not change that lesson. It removed the friction that used to expose the gap early.
Three Tests Every AI Deliverable Should Pass
Plausible. Does it look right? AI passes this test every time. Most organizations stop here.
Verified. Is it right, checked against the real system and not against another document? Systems engineering separates building it right from building the right thing. A review of generative AI in software architecture found rigorous testing of outputs was typically missing from the studies.
Accountable. Who answers for it? That means a named owner and a trail showing what AI drafted and what a human changed.
Plausible is the only test AI passes by default.
Picture how it plays out. An AI drafts a dependency map for a few dozen systems in an afternoon. It passes the first test easily: every box is labeled, every line is clean, and it speaks your team's vocabulary. Then someone checks it against the real environment and finds a connection that does not exist. Two systems share a similar name, and the model assumed they were related. The map was plausible. It was never verified. And when the program office asks who signed off, nobody can say, because that was never assigned.
That map was about to inform a funding decision.
Before You Sign Off
This is not a threat to architects. It is the reason architecture matters more.
In a 2026 SAP LeanIX survey of 228 enterprise architects, 63% said their practice should lead AI agent governance, yet only 34% said they are involved today. Gartner also expects over 40% of agentic AI projects to be canceled by the end of 2027 due to cost, unclear value, or inadequate risk controls.
Three habits close the gap.
Label what AI drafted. Every deliverable should say where AI helped and where a human took over. It takes one line and saves hours of debate later.
Verify what would hurt most. You cannot check everything. Pick the five claims that would do the most damage if wrong and test them against the real system, not another document.
Put a name on it. One accountable person per deliverable, recorded where the next reviewer will see it. If no name is there, no one has checked it.
For those of us serving government, this is the heart of the matter. What is the agency actually paying for? Increasingly, it is not the artifact. It is the assurance behind it.
The Question Behind the Question
The artifact was never the hard part. It was just the part we could see.
AI will soon hand every organization a thousand answers that look right. The only question is whether anyone can say which ones are.
Sources
- Gartner, Predict 2026: AI Agents Will Redefine How EA Works, November 2025.
- Gartner, Over 40% of Agentic AI Projects Will Be Canceled by End of 2027, June 2025.
- SAP LeanIX, Agentic AI Survey 2026.
- Generative AI for Software Architecture: Applications, Challenges, and Future Directions, Journal of Systems and Software, vol. 231.