The 501st Agent Is Not the Problem

The hard part of enterprise AI is knowing what the first 500 are doing.

Making an agent is exciting, and oh so in vogue! You describe a job, connect a few tools, give it some knowledge, and watch it do something that would have taken a person twenty minutes. When I was designing Docusign’s Agent Studio, that creation experience mattered enormously. People needed to understand what they were building and feel enough control to trust it.

Then I started thinking about the person who comes back six months later and finds 500 agents in the account. Some are drafts. Some are published. Some ran this morning. Some have not run in weeks. A colleague edited one yesterday, but the production version may still be older. A customer says an outcome is wrong, although every technical step in the execution log is green.

What does that person do first?

The enterprise design problem is no longer only how to make an agent. It is how to understand, manage, and recover from a system of agents when no one can watch all of them at once.

What changed while I wasn’t looking?

At Arista Networks, I worked on network-management problems where the administrator’s attention was always scarce. A network can have thousands of devices, and the person responsible for it cannot stare at every chart all day. The interface needs to answer a more useful set of questions: Is anything down? Has traffic changed? Was there a security event? What happened while I was away? What deserves my attention now?

That is an observability problem, but it is also a product-design problem. Dumping every event into a chronological feed does not solve it. Neither does turning every event into a notification. If everything asks for attention, eventually nothing gets it. A good management surface has to compress a noisy system into a credible summary, then let someone move from the summary to the evidence behind it.

Agent platforms need the same discipline, with a harder definition of health. When an agent fails to call a tool, the failure is relatively legible. When it calls every tool successfully and reaches the wrong conclusion, the dashboard may still look cheerful. The system ran. That is not the same as saying the system worked.

Deloitte’s 2026 survey of 3,235 leaders involved in AI programs illustrates the scale of the management gap. Twenty-three percent said their organizations were already using agents at least moderately; 74% expected to reach that level by 2027. Only 21% reported a mature model for agent governance at the time of the survey. The expectation is a forecast by respondents, not an observed adoption rate, and governance maturity is self-reported. Still, the contrast explains why management cannot be designed as an afterthought. Source: Deloitte

Imagine an administrator investigating a bad result from Tuesday afternoon. Today that may mean finding the incident time, scanning a long activity stream, identifying which agents triggered, checking which agents changed, comparing production versions, and reconstructing the path manually. Each step is possible. The experience as a whole is still poor. The product should be able to say, in effect: These three agents ran in the affected workflow. One was published two hours earlier. Its tool permissions changed. Here is the version that produced the result, the steps it took, and the human corrections made afterward.

That is much more useful than a red badge with the number 47 in it.

Execution is not the same thing as success

Infrastructure has its own ambiguities, but at least a failed device or a latency spike has a reasonably stable technical definition. Agent outcomes are contextual. An agent can return a valid response, satisfy a schema, and complete its run while misunderstanding what the user wanted. It may choose the wrong document, summarize the wrong exception, or make a plausible recommendation that ignores one important constraint. Its logs can be immaculate while its work is not.

This is why I would separate technical success from outcome success in the product. Technical success asks whether the agent executed its steps. Outcome success asks whether it achieved the user’s intended result. The first can often be instrumented automatically. The second may require context that the platform does not yet have, plus a human willing to judge the result. Pretending those are the same metric makes the interface simpler and the system less trustworthy.

For now, I would rather give teams a useful evidence trail than an overconfident “agent health score.” Show trigger history, exact execution steps, the knowledge and tools available at the time, the published version, and any human override. Ask for correct/incorrect feedback where someone is already reviewing an outcome. Track those signals over time, by task and by version, so a team can see whether quality is drifting. If evaluation improves enough to compare user intent with actual result reliably, use it. Until then, make uncertainty visible instead of hiding it behind a green check.

Microsoft reports that the number of active agents in its Microsoft 365 ecosystem grew 15-fold year over year, and 18-fold among large enterprises. This is platform-specific telemetry, not a count of all enterprise agents. But it captures the direction of travel: a management experience that works for a few hand-built agents may become unusable quickly as the inventory grows. Source: Microsoft Work Trend Index 2026

The builder still matters

There is a tempting version of this argument in which the agent builder becomes the least important screen. I do not buy it. Making an agent with natural language may become almost effortless, but easy generation does not eliminate the need for direct manipulation.

If Photoshop could generate an entire image from a prompt, I would still sometimes want the lasso tool. I might need to color grade one corner, move one object, or tweak a detail because I can see what is wrong more precisely than I can describe it. I may also just have a preference. Spending another dozen prompts to negotiate with the machine is not always better than touching the work myself.

Agent builders have the same need for control, precision, learning, and taste. A person may want to inspect a tool connection, remove a knowledge source, reorder a workflow, or adjust a handoff without asking an assistant to reinterpret the whole system. The canvas helps them build a mental model. It gives them a place to curate the agent’s behavior and understand the consequences of a change.

The shift is not from creation to governance. Creation remains an essential part of the product, while management becomes equally consequential as the system grows. A polished builder that leaves a team unable to explain what went into production has only solved the beginning of the job.

Don’t make the agent a person

I understand why people reach for human metaphors. A friendly avatar makes an abstract system approachable. I like playful companions; a little Tamagotchi energy can be charming. But the metaphor starts doing damage when the interface implies a level of understanding or reliability the system does not have.

An agent is not a person. It can hallucinate, drift, and behave differently across similar inputs. “Copilot” is an appealing name, but it can also suggest a peer who will safely fly the plane while you sleep. An employee metaphor brings its own baggage: now the product seems to have assigned competence, intention, and perhaps accountability to software that infers rather than understands in the human sense. We should be careful about what trust the design asks people to extend.

Imagine applying for a role you care about and discovering halfway through an interview that the “person” asking about your experience is AI. The conversation might function technically. It might even ask sensible questions. But the company has also outsourced a relationship-building moment that many candidates would reasonably expect a human to own. That is a product decision, not just an efficiency decision.

Agents can be approachable and delightful without pretending to be sentient colleagues. The interface should make their scope, limits, and human owner clearer, not cover those things with a more convincing face.

Governance should live where the work happens

In cybersecurity products, I developed a preference for putting a summarized analytical layer close to the objects it describes. If someone is looking at a table of devices, policies, or alerts, they should not have to leave that context to understand its health. Give them the overview, then let them inspect the rows that explain it.

Agent platforms need both a central management view and local management inside the workflows where agents live. A central view can answer: Which agents changed recently? Which triggered today? Which are failing? Which have no clear owner? Which permissions or evaluations need review? The local view can answer a different question: What is this specific agent doing in this workflow, and is it helping?

That overlap is not waste. Not everyone navigates the whole platform. A workflow owner may never open the administrator’s control plane; an administrator may not know the workflow’s business context. The important information should be available at both scales, with enough consistency that the views do not contradict each other. Governance should not be a miserable annex people visit only after an incident.

Microsoft’s 2026 survey found that respondents it calls “Frontier Professionals” were more likely than other AI-using workers to say agent workflows, human handoffs, and quality standards were documented and repeatable: 26% versus 19% at team level, 29% versus 17% at function level, and 25% versus 14% at organization level. These are self-reports from different groups, not proof that documentation causes better outcomes. They do show that even among advanced users, repeatable practice is far from universal. Source: Microsoft Work Trend Index 2026

Atlassian’s recent survey of more than 1,100 engineers and engineering leaders points in a similar direction, though it concerns software delivery rather than enterprise agents generally: 94% of engineering leaders reported some organizational AI use, while 88% said they needed a governed engineering system of work and 19% said they had built one. The vendor framing and survey population matter, but the practical question is familiar: how does a team connect an agent’s work to the intent, standards, and evidence needed to review it? Source: Atlassian

Everyone owns the failure

My background is in architecture, where the idea of one person single-handedly causing a building failure would be an odd starting assumption. The architect, structural engineer, developer, owner, contractors, and inspectors each shape different parts of the outcome. That does not mean nobody is accountable. It means accountability has to match the structure of the work.

Software organizations often reach for a more convenient story after an incident: find the person whose name was nearest the failure. In cybersecurity that can mean the CISO becomes the visible fall person for a breach that involved product decisions, resourcing, operations, and organizational incentives. I do not want to reproduce that pattern with agents. The creator should be able to explain design intent, but a production failure may also implicate the people who set permissions, approved deployment, supplied knowledge, defined evaluation, or decided where human review was unnecessary.

The inverse question is useful too. When an agent helps a team do excellent work, who gets the credit? The builder? The workflow owner? The person who curated its knowledge? The team that noticed the result was wrong and corrected it? A model that recognizes only one person when something breaks, and nobody when it works, is not a serious accountability model.

The product should support shared accountability with an intelligible record of decisions and changes. It should help the organization learn without turning every incident into a hunt for a culprit.

The version that went to lunch

These questions were on my mind while we worked on Agent Studio. An agent might receive skills, tools, and knowledge sources. It is tempting to give it everything. More context sounds like more capability. But could too much knowledge make the agent less purposeful or less accurate? Could it make permissions harder to reason about? Even before measuring those effects, the design has to let someone understand what context an agent actually had when it acted.

Then there is the ordinary mess of collaboration. Someone publishes an agent. Another person starts editing it and goes to lunch halfway through. The draft is unfinished, but the production agent must keep running. Which version does a user see? Which version did yesterday’s run use? Can the editor compare changes, revert a bad publication, or pass the draft to someone else? If a colleague remixes the agent, whose intent does the new version represent?

Those sound like edge cases until the platform has dozens of editors and hundreds of agents. At that point they are Tuesday. Version history, draft-versus-published state, ownership, change attribution, and a clear production record are not merely governance checkboxes. They are what let someone make a change without wondering whether they are about to interrupt a live business process.

Guardrails also cannot be treated as something the industry will discover only in its “next generation.” Organizations already constrain agents. An agent in Docusign should not suddenly order hamburgers. The harder question is whether today’s boundaries are the right ones as people discover misuse, edge cases, unexpected behavior, and new ways to connect tools. Guardrails have to evolve through observation and review. The product should show both what an agent was allowed to do and what it actually did.

Maybe the control plane is the product

I still want a builder that feels good. I want people to make agents, test ideas, adjust the details, and learn by manipulating the system directly. The creation experience is where possibility becomes concrete.

But if a platform has 500 agents, the quality of the 501st creation flow may matter less than whether someone can understand the first 500. They need to know what changed while they were away, which version is running, what happened during a disputed result, and whether a technically successful execution actually helped a person do the job. They need enough signal to act without spending the whole day reading notifications or reconstructing an incident from scraps.

That is the next product-design challenge I see in enterprise agents: not simply increasing their capability, but designing the management system that lets people inspect, trust, change, and recover from what those agents do at scale. The builder gives an organization new possibilities. The surrounding system determines whether it can live with them.