Why your AI agents fail even when they follow instructions
You tuned the prompts. You tested the logic. The agent still outputs something wrong. The culprit probably isn’t your instructions. It’s the documents those instructions are drawing from.
That’s the core argument from MarTech writer Margaret Lee: AI agents follow instructions faithfully and still produce bad results when the underlying knowledge is vague, inconsistent, or incomplete. Better prompting doesn’t fix a broken source of truth.
The Three-Layer Framework
Lee describes a reliable source of truth as having three distinct layers:
- A knowledge base that defines the facts and rules your agent should apply.
- Examples that show the agent how to apply those rules in practice.
- A sandbox for testing the agent against real inputs before it goes live.
Each layer does different work. The knowledge base sets the boundaries. The examples translate abstract rules into concrete behavior. The sandbox is where you find out whether the first two actually worked.
What “Accurate Documentation” Actually Means Here
According to the framework, building a solid source of truth requires four things: accurate documentation, clear rules, examples of good output, and a mechanism to test whether agents apply the information correctly.
That last piece is the one most teams skip. Writing the knowledge base feels like the work. Testing it against real agent behavior is where you find out whether it holds up.
The Operator Takeaway
If you’re running any AI automation that touches customer-facing output, internal decisions, or anything that costs money when wrong, the documentation layer deserves as much attention as the agent architecture itself. Garbage in, garbage out applies to knowledge bases just as much as it applies to training data.
Start by asking: if a new hire read only the documents your agent reads, would they make the same decisions you want the agent to make? If the answer is no, the agent won’t either.
I’m founder of BizStack at Cagri Sarigoz LLC and a passionate advocate for entrepreneurs.
With over 14 years in tech, marketing, and AI, including my role as Head of SEO at CitizenShipper and co-founder of TaleBot at Intale AI, I’m dedicated to sharing genuine, useful product insights and tips.
At BizStack, I aim to cut through the digital noise to provide clear, actionable advice.
And more than all else, I’m a father to a (always) little girl and a husband.
Contact me at [email protected] for assistance.
