AI agents are becoming digital coworkers. So much so that Gartner predicts that by 2028, an average global Fortune 500 enterprise will have over 150,000 agents in use – up from less than 15 just last year. Even if that projection turns out to be way too optimistic and the actual numbers will land somewhere around 50%, it describes a workforce larger than most of the companies deploying it, assembled in only three years.
Most organizations are somewhere in the early part of this curve right now, and that’s exactly the moment to get governance right. It’s much easier to define the policy framework at the start than to retrofit it later on. Otherwise, organizations risk discovering gaps when scaling, during an audit, or worse, when one becomes a serious security incident.
This guide explores how organizations can deploy AI agents securely and enable innovation without losing control.
- What is an AI agent?
- How do AI agents differ from other non-human identities (NHIs)?
- What can go wrong when AI agents aren’t governed?
- Three AI agent governance questions most organizations haven’t answered.
- How to govern AI agents: an 8-step guide.
- How AI agent governance works in midPoint.
What is an AI agent?
An AI agent is a software identity powered by a large language model (LLM) that has been given access to credentials and business systems so it can complete tasks autonomously. Unlike a chatbot that simply answers questions, an AI agent can perform work on your behalf. It can call APIs, query databases, read and write files, invoke external services, and even coordinate with other agents to complete more complex workflows.
To do that, an agent authenticates with credentials such as OAuth tokens, service accounts, and workload identities, allowing it to operate as a trusted user inside your environment. Once an AI agent receives credentials and permissions, it becomes another identity that must be governed throughout its lifecycle.
How do AI agents differ from other non-human identities (NHIs)?
AI agents are a type of non-human identity but they behave very differently from the static machine identities that IAM teams have managed for years.
Traditional identity governance focuses on granting access and reviewing it periodically. AI agents require continuous oversight because what matters isn’t just who has access, it’s what they’re trying to do right now.
What can go wrong when AI agents aren’t governed?
Unlike what we see in sci-fi movies, most governance failures don’t begin with a malicious AI agent. What’s more likely is that they start with an ordinary business problem that slowly snowballs into an identity problem.
Sensitive data exposure
Unlike a human employee reading a customer record on a screen, an AI agent often sends portions of that information into a language model as part of its reasoning process. Governance therefore needs to extend beyond controlling access to databases. Organizations should also consider what information an AI agent can access and process in the first place, and whether personally identifiable information should be masked before it is sent to an LLM for summarization.
Privilege accumulation
As organizations discover new use cases, agents tend to receive additional permissions without much review. What began as read-only CRM access gradually expands into access to multiple systems, while no one periodically asks whether those permissions are still required. Unlike employees, whose access is often reviewed during certification campaigns, AI agents are frequently overlooked because they aren’t seen as traditional identities.
Lack of ownership
AI agents also have a habit of outliving the projects that created them because no one clearly owns them. A developer builds an agent during a pilot, changes teams a few months later, and the agent continues operating because the business depends on it. Over time, the risk of everyone forgetting about these agents compounds.
Shadow AI
Not every AI agent is deployed through an approved IT project. Often, employees are creating agents themselves to solve everyday business problems, often without realizing the security implications. Just as Shadow IT introduced unmanaged applications, Shadow AI introduces unmanaged AI identities. If security teams don’t know an agent exists, they can’t govern its access and monitor its activity.
It’s worth noting that the instinct to clamp down makes this worse. Gartner’s own guidance on agent sprawl is that blocking or restricting AI agents isn’t a long-term solution, because employees who can’t work in sanctioned tools will go around the controls and adopt shadow AI instead — which presents far greater risk than the thing you were trying to prevent.
Compliance: EU AI Act, GDPR, NIS2, DORA, and CRA
As AI agents become part of everyday business processes, governance also becomes a compliance issue. An agent with access to sensitive data and regulated systems is subject to many of the same expectations as human employees.
| Regulation | Primary focus | Relevance to AI agent governance |
|---|---|---|
| GDPR | Protection of personal data | Governs how personal data is processed, including by any model provider in the chain – applies the moment an agent touches customer or employee data, regardless of where the model itself runs. |
| EU AI Act | Risk-based regulation of AI systems | Categorizes AI by its potential danger to society. Simple tools like internal document summarizers face minimal restrictions, whereas agents that make or heavily influence decisions about people face strict legal obligations. This tiering dictates the exact level of human oversight and transparency required. |
| NIS2 | Cybersecurity risk management for critical entities | Requires access control, logging, and demonstrable accountability across essential and important entities, and makes senior management personally responsible for compliance – including for AI-driven operations. |
| DORA | Operational resilience for financial entities | Imposes comparable expectations to NIS2 on financial entities, with particular attention to third-party ICT risk – directly relevant wherever an agent or model runs on outsourced infrastructure. |
| CRA | Cybersecurity of products with digital elements | Introduces mandatory cybersecurity requirements for products with digital elements, including systems that incorporate AI components. Manufacturers must build in secure-by-design principles throughout the product lifecycle before it reaches the market. |
Three AI agent governance questions most organizations haven’t answered
The risks above are the visible ones, and most organizations will recognize at least one from their own environment. Underneath them sit three harder problems that don’t often come up in discussions and where the genuinely difficult work is. They’re worth settling before you start building.
Whose authority is the agent acting under?
When an agent does something on behalf of a user, does it use its own permissions or theirs?
An agent acting for a user should be limited to the intersection of what the agent may do and what that user may do. In practice this means token exchange and on-behalf-of flows rather than static service accounts. It’s the single most important technical decision in agent identity, and it’s much easier to design in than to retrofit.
What happens when an agent’s instructions come from an attacker?
Your agent has perfectly scoped permissions and a clean audit trail, however, it reads an email containing text crafted to look like an instruction and decides to follow it. This is prompt injection, and the reason it belongs in a governance conversation is what it does to your threat model.
Identity governance unfortunately can’t solve this problem, but it can limit the damage. If the agent’s permissions are scoped to the intersection of its role and the user it’s acting for, an injected instruction can only misuse what’s inside that narrow overlap. That is why enforcement has to exist at the moment of action, not only at the moment of provisioning.
What happens when AI agents call other agents?
Agent A invokes agent B, which calls agent C. Does authority remain the same at each handoff, or does access just keep piling up so the last agent in the chain ends up able to do more than anyone meant for it to? And if something goes wrong six months from now, could you actually trace back and figure out who allowed it and reconstruct the chain? Most organizations can’t answer this yet. If your architecture involves multi-agent orchestration, answer it before you scale.
How to govern AI agents: an 8-step guide
Fortunately, none of these questions require reinventing identity governance. They require applying familiar identity principles to a new type of identity, and the following framework is a practical place to start.
1. Find the agents you already have
Every step below assumes you know an agent exists, so start by confirming that. Survey the business, check what’s been connected to your SaaS platforms, look at OAuth grants and API key usage, and ask teams directly what they’ve built. Discovery is uncomfortable and almost always produces surprises. Do it first.
2. Give every agent its own identity
The most common mistake is running agents as extensions of an existing user or application. If you do that, it will be impossible to separate what the person did from what the agent did. Each agent gets its own identity, with a lifecycle that runs from creation through to decommissioning.
3. Authenticate and scope agents like agents, not like users
Avoid shared service accounts and long-lived API keys. Use short-lived credentials and modern standards: NIST identifies SPIFFE/SPIRE as a universal approach to authenticating software workloads, while OAuth 2.0 can provide delegated authorization when an agent needs to act on someone’s behalf. Whatever credentials remain should live in a managed secrets store with automatic rotation, never in a config file or prompt template. Then scope what’s granted to the task – instead of broad access, restrict the agent to exact actions, specific datasets, and precise APIs.
4. Govern the tools, not just the data
An agent’s real capability comes from the tools connected to it. Every integration and MCP server it can reach expands what it can do, and each one (hopefully) arrived through some approval process. Maintain an inventory of what each agent is connected to, who approved it, and what scopes it was granted. This is where most of the risk actually is.
5. Enforce guardrail and human checkpoints where decisions happen
Granting access is only half of the story, as runtime guardrails determine what happens once an agent reaches the data. For high-stakes decisions, keep a human in the loop. Standards like CIBA let the agent pause, request approval through a separate channel, and resume once granted, without anyone watching a screen.
6. Make someone accountable
If we’re treating AI agents as our little work helpers, then every agent needs a designated manager – be it a person or a team – who approves new permissions, responds to access reviews, answers audit questions, and decides when the agent retires. Ownership should be a required field at creation, and it should be reassigned when people move roles. Without it, everything else on this list eventually degrades.
7. Test before you trust
Run new agents in a sandbox with tightly scoped permissions and evaluate how they behave, ideally putting it up against deliberately hostile input. Grant production access when they’ve earned it, and treat every subsequent expansion as a decision requiring a rationale and an approval rather than a configuration change. This prevents permissions from silently piling up over time.
8. Review continuously, monitor at runtime, and know how to stop
Agents belong in the same certification campaigns as employees, with owners confirming entitlements are still justified. Log every significant action and the reasoning behind why it was permitted or denied. And know (before you need it) whether revoking an agent’s credentials kills active sessions immediately or just blocks new ones. When an agent is no longer needed, retire it properly: revoke credentials and tool access, save its logs, and document the shutdown.
How AI agent governance works in midPoint
Building a governance program for AI might sound daunting, but if you already use a mature identity governance platform, you likely have most of the foundations in place. You don’t necessarily need a separate, specialized tool to govern AI agents; platforms like midPoint can manage them much like any other non-human identity.
The Service object type exists precisely for this. Services represent non-person identities, and because they carry credentials in much the same way users do, they can represent machine agents and similar things that behave like humans. Services are also abstract roles, so they can grant privileges to other entities.
Archetypes provide the granular typing. An archetype describes what kind of thing an object is, (e.g. “Employee”, “Application”, “AI Agent”) and can be applied to Users, Roles, Orgs, and Services alike. Each archetype can be defined by policies, letting you treat AI agents differently from standard service accounts or human employees without building a parallel governance framework.
MidPoint roles can form hierarchies through inducement relations, and access rights can be assigned and unassigned automatically according to organizational structure. Roles assigned to an organizational unit can provide access to identities within that unit, while inducements can be used to propagate privileges through a hierarchy.
Approval workflows, access certification and audit are core platform capabilities available to any identity type. Certification campaigns can automatically revoke access if a reviewer denies it, and comprehensive audit logs make meeting regulatory requirements much easier to manage.
Our partner Inalogy has extended this approach to address the specific requirements of AI agent governance, and a detailed guide can be found in this article. They have extended midPoint with these three attributes:
- Business Roles define who the agent is
- Application Roles define what it may do, enforcing permissions on every single action rather than just at provisioning
- Skill Roles define what it knows. This is an agent-specific tier: a skill is a set of instructions, procedures and guardrails loaded into the agent’s context. Without the role, the agent never sees those instructions at all.
Inducement ties the three together as assigning a specific business role automatically triggers the corresponding application access and skill sets.
See how Midpoint brings human and non-human identities together under one identity governance layer.
The payoff of getting it right
AI agents are already inside most environments, and building a governance framework now, while the count is on the lower end, will not only be much easier than trying to play catch up later, it’s also the right thing to do.
It won’t stop innovation either – the Larridin State of Enterprise AI Q1 2026 report found that organizations with formalized AI risk and compliance policies are 2.2x more likely to demonstrate ROI than those without. The bottom line is, companies that get AI agent governance right will reap the benefits of deploying AI agents faster, safer and with more confidence.
Frequently Asked Questions
+Is AI agent governance legally required, or is it just best practice?
There isn’t one single AI-specific law that mandates AI agent governance. Regulations such as the EU AI Act, GDPR and NIS2 can already create obligations around AI, data protection, security, and accountability. Even where there isn’t a specific rule for AI agents, existing requirements may still apply. Good governance is therefore both a compliance consideration and a practical security best practice.
+If an AI agent causes a breach or makes a bad decision, who’s actually liable?
Responsibility sits with the organization that deployed the agent. Exactly who is liable depends on the circumstances and the laws that apply. That’s why it’s important to have comprehensive audit logs of each agent and clear ownership.
+Should IAM own AI agent governance, or does it belong to security or legal?
In practice it tends to split naturally across teams, with IAM handling the identity lifecycle, while security takes on security and monitoring and legal keeps track of interpreting regulations. AI governance is a joint effort throughout the organization and not something that one department can solve on its own.
About Evolveum:Evolveum is the organization behind midPoint, the leading open source IGA platform recognized as a complete IGA by both Gartner and KuppingerCole. MidPoint bridges the gap between IT and business, making it an ideal choice for organizations seeking digital transformation to enhance security and efficiency.