The Maker’s Bias: Why Your Democratized AI Strategy is Burning Tokens
Here’s a peculiar psychological shift I’ve noticed lately.
The way someone evaluates an AI agent changes entirely depending on one simple factor: whether they bought it, or whether they built it.
If you are a developer, you already know this phenomenon.
Developers have always loved building side projects. They will spend weekends writing custom scripts to automate some tiny personal workflow. The solution may not be efficient. It may not scale. It may not be useful to anyone else.
But they built it, so they love it.
Now that same energy is spreading across entire organizations.
With tools like Cursor, Claude, and plug-and-play MCP servers, the “developer side project” mindset is no longer limited to developers. Building an AI agent has become easy enough that non-technical employees are now spinning them up across departments to automate their daily tasks.
And in many cases, that is a good thing.
The problem is what happens when you zoom out.
Across the organization, leaders are still struggling to answer a basic question:
Is all this AI activity actually moving the productivity needle?
The “Unlimited Assistants” Problem
Where are all these tokens going?
Are they being used productively?
Because agents cost money. Letting every employee create unchecked, self-serve AI workflows is the equivalent of telling everyone in the company:
“Go ahead and hire a personal assistant. We will pay their salary. And we won’t measure what they actually do.”
In the pre-AI world, if an organization wanted to improve productivity, it usually created some kind of centralized change-management, automation, or process-standardization team.
These teams were often unpopular. They faced adoption friction, training challenges, and political pushback because the automation was being imposed on employees from the outside.
But because they were centralized, their ROI was at least evaluated with some distance. Someone had to justify the investment. Someone had to measure the impact.
Today, the dynamic has flipped.
Using AI is a status symbol. Building AI workflows is even better. There is no central team forcing adoption. People want to build.
Adoption is through the roof.
But that creates a new blind spot:
The Maker’s Bias
When someone builds an agent that works for them, they often think it is the greatest thing in the world.
And that makes sense. It feels empowering. It solves a real annoyance. It gives them control over their own workflow.
But it also means they will often over-index its impact.
They may not factor in the token burn. Or the API cost. Or the time spent tinkering with prompts. Or the maintenance cost when the workflow breaks. Or the fact that five other people in the company may have built slightly different versions of the same thing.
What feels like a massive productivity win at the individual level may still be deeply sub-optimal at the business level.
This is the core tension of AI democratization.
You want the creativity of bottom-up building.
But you cannot rely on bottom-up enthusiasm to measure business impact.
The Fix: Execute Bottom-Up, Measure Top-Down
If you are starting an AI democratization project in your organization, you should absolutely allow people to build their own agents.
You need that bottom-up execution to capture the long tail of productivity. The people closest to the work often know exactly where the friction is.
But you cannot measure success bottom-up.
You have to measure it top-down.
Before you give teams the keys to the kingdom, you need a clear North Star. With AI, everything can look like success to everybody. That means you need a rigid definition of what actual productivity means.
The impact cannot only be a second- or third-order effect.
“My team saves two hours a week drafting emails” sounds great. But it is a trap if those two hours do not translate into something measurable.
Did output increase?
Did cost go down?
Did quality improve?
Did cycle time reduce?
Did revenue per employee improve?
Did margins expand?
A time saving only matters if the saved time is converted into more throughput, lower cost, better decisions, or better customer outcomes.
Otherwise, the organization may simply be subsidizing a very expensive hobby.
The real value of AI agents will come when they operate close to strategic decision boundaries: the places where speed, judgment, cost, and revenue actually meet.
But that is a topic for another post.
The takeaway is simple:
Before you democratize AI, define the KPI.
Let your teams build.
But make sure your very first agent is the one measuring the math.
Thanks for reading Bits of AI! Subscribe for free to receive new posts and support my work.