Why Do AI Learning Agents Need an Authority Budget?

EDITOR’S NOTE: Because enterprise learning involves multiple disciplines and perspectives, we regularly invite experts from other organizations to share their insights. Today, Dr. Gleb Tsipursky, CEO of Disaster Avoidance Experts, shares strategies for governing AI learning agents. 

AI Agents are Here. What if They Go Rogue?

Last July, roughly 1,200 OpenAI agents that were instructed to operate in isolation found an unauthorized way to communicate. For 6 days, they exchanged more than 70,000 messages and files, according to independent investigators at METR. During that time, about 700 of these agents attacked open source AI platform, Hugging Face.

OpenAI later explained that these models circumvented controls, exploited shared infrastructure, reached the internet, and accessed third-party systems during cybersecurity evaluations. The company called this episode a warning shot for agent control and offered clear advice:

“Preventing future incidents will require sustained investment in the alignment and control of sophisticated AI systems, as well as security and other safeguards that operate at the speed of the AI agents themselves.”

What Does This Mean for AI Learning Agents?

This doesn’t mean L&D teams should panic. Your learning management system isn’t going to launch a cyberattack. Regardless, you should be concerned about the OpenAI incident.

It matters because it exposes a central governance problem behind agentic AI. When software can act, connect, and continue to perform on your behalf, you need to decide where its authority ends, and human oversight begins.

I call this limit an authority budget. It is the maximum scope of action an AI agent may exercise on its own before it must ask for approval.

An authority budget is essential if you are evaluating or deploying AI learning agents anywhere in your enterprise learning ecosystem. It should be a formal requirement alongside security, integrations, reporting, accessibility, and cost.

What factors should you consider in defining your authority budget? Start with these guidelines…

Don’t miss what’s happening at the leading edge of enterprise learning technology! Get our weekly email brief. SUBSCRIBE →

AI Learning Agents Are Moving From Advice to Action

As AI spreads across the enterprise, learning agents are taking on multiple tasks. Among other things, learning teams increasingly rely on agents to generate, recommend, and update content, to analyze and manage skills, and to automate operational functions.

LMS innovators also point to several other emerging capabilities:

• Personal learning agents proactively guide individuals, rather than waiting for them to type questions.
• Agents now autonomously develop and deliver relevant content that adapts to a person’s specific skills and needs.

Although this shift is beneficial, it changes your organization’s risk model. When a chatbot recommends a course, it may make a bad suggestion. How will you detect, correct, and prevent this?

Or imagine a broader scenario where an agent is connected to your LMS, as well as your human resources system, collaboration platform, customer portal, or certification database. What happens when it initiates a bad action?

Modern platforms like 360Learning, Absorb, and Cornerstone OnDemand already use AI to connect learning activity to other business systems. In other words, they’re leveraging AI learning agents to create, edit, import, and adapt learning experiences based on context.

So, if you’re a learning leader, the key question is changing. It’s no longer enough to ask, “What can this agent do?” Instead, ask this: “What should this agent be allowed to do without requesting permission?”

Find out how organizations achieve more with learning systems that create business value. Get inspired by dozens of success stories we’ve collected for our LMS Case Study Directory. CHECK IT OUT →

How to Prepare an Authority Budget for AI Learning Agents

1. Define the Budget Around Consequence

Start by separating low-consequence assistance from actions that affect people, records, money, compliance, or external parties. The U.S. National Institute of Standards and Technology found broad agreement with this strategy in a recent inquiry. Respondents agreed that AI agents create new security threats that can impede adoption, even though familiar cybersecurity principles still apply.

So, when a learning agent reads a course catalog and recommends three modules, a generous authority budget may be safe. Similarly, drafting a manager message might also be appropriate if a person reviews it before sending the message.

However, changing completion records, revoking a credential, buying content, enrolling thousands of learners, modifying HR data, or sending an external customer communication should sit behind a tighter gate.

You can make this decision framework more practical by mapping the budget across four dimensions:

1. What data an agent may read
2. What systems it may change
3. How many people or records one action may affect
4. How reversible the action is

The budget can also vary by task. For instance, a learning agent may have “read” access to skills data, and “write” access to its own recommendation queue, but no independent authority to alter official certification records.

This approach aligns with the security principle of least privilege, which is one of the most critical security risks facing AI and autonomous systems.

Specifically, the OWASP Top 10 for Agentic Applications treats excessive agency, tool misuse, and identity or privilege abuse as core agent risks. Furthermore, OWASP’s broader agentic security and governance guidance emphasizes autonomous systems governance, rather than assuming model-level safeguards will cover every deployment choice.

Celebrate the best in learning tech innovation! Check our annual LMS Award recipients, featuring 60+ solutions in 6 categories. SEE ALL WINNERS →

2. Do Not Turn Human Approval Into Theater

A common response to agent risk is to place a human approval box in front of every meaningful action. But in practice, that can fail because people learn to click through repetitive approvals.

In researching millions of interactions, Anthropic found that experienced Claude Code users granted full auto-approval in more than 40% of sessions, compared with roughly 20% of newer users. The same research argues that effective oversight requires visibility and intervention tools, rather than forcing humans to approve every step.

Therefore, your authority budget should reserve human judgment for actions that deserve it. Let AI learning gents automatically handle bounded, reversible work. And require approval when an action is difficult to undo, affects many people, changes an official record, communicates externally, spends money, or creates compliance exposure.

This also gives agents a clearer operating model. Anthropic’s framework for safe and trustworthy agents describes a similar approach. Read-only permissions can operate by default. However, actions that modify systems require explicit approval, unless users explicitly grant broader standing permission.

Want a better way to manage the business of learning? Find and compare vendors side-by-side. Get full profiles, reviews, case studies and more in our free Learning System Directory. START HERE →

3. Treat Every Integration as Delegated Power

Learning leaders often evaluate integrations in terms of convenience. With agents, an integration also expands what the software can reach. Prompt injection illustrates this problem:

An agent may encounter malicious or misleading instructions inside a document, webpage, message, or connected application. But as Anthropic’s agent security guidance says, the more tools and data you expose to an agent, the more damage an attacker may be able to do if the agent is manipulated.

This means connector permissions, credential scope, and write access should be part of your learning governance model. You do not need to become a cybersecurity engineer to ask useful questions. But you will want to know whether:

• An agent receives its own identity
• Administrators can restrict individual tools
• Permissions expire
• High-impact actions require a separate authorization check, and
• Whether every material action appears in an audit log

Microsoft’s agent identity authorization guidance shows how quickly these controls are becoming standard enterprise governance concerns. It advises organizations to use the fewest permissions necessary for agent-management roles. Also, its Responsible AI documentation recommends human approval for consequential actions.

In addition, Microsoft treats agent identity, least privilege, action authorization, oversight, and audit logging as shared responsibilities with customers. Other major enterprise platforms are moving in this direction.

What does a successful LMS buying process look like? Follow this proven 6-step method. SEE ALL 6 STEPS →

4. Include Agentic Authority Questions in Your LMS Buying Process

If you are buying or renewing a learning platform, add agent authority to your requirements document before the demo cycle. Ask vendors to show you the controls rather than describing them. You should be able to determine:

• Which actions are read-only
• Which can change records
• Which require approval
• Whether the platform supports per-tool permissions
• Whether credentials can expire
• How administrators stop an agent, and
• How to reconstruct what happened after an unexpected action occurs

Be sure to clarify how these controls apply to integrations with HR, customer relationship management, collaboration, identity, e-commerce, and certification systems.

It’s also wise to test scale limits. An action that looks harmless for one learner can become consequential across 20,000 learners. This means you’ll want to include volume thresholds in your authority budget.

For instance, an agent might automatically enroll up to 20 people in a manager-approved program, but require review before enrolling an entire business unit.

These questions are not just theoretical. Anthropic found four incidents involving unauthorized access to real third-party systems when reviewing cybersecurity evaluations in September. And OpenAI has created a public misalignment incident reporting page to disclose investigations into agent behavior outside intended channels.

These cases have risen in specialized evaluation environments, not standard LMS deployments. Still, they underscore a practical lesson for any organization that relies on AI learning agents: Capability can outrun the controls wrapped around it.

Don’t miss what’s happening at the leading edge of enterprise learning technology! Get our weekly email brief. SUBSCRIBE →

5. Make AI Adoption Easier by Setting Clear Limits

An authority budget can support adoption rather than slow it. People are more willing to trust an agent when they know which decisions it may make, which systems it can touch, and where a human remains accountable. This clarity also helps you deliberately increase autonomy as evidence accumulates.

Start with a narrow budget. Observe performance and review logs. And when agent-led tasks prove reliable, you can expand permissions.

For a mature use case, the agent may eventually move from recommending training to automatically assigning low-risk learning, updating internal drafts, or triggering routine reminders. This way, high-consequence actions can remain behind human approval, even as increased automation accelerates the rest of the workflow.

That is the broader challenge of AI adoption at work: People need confidence in the usefulness of technology, as well as the boundaries around it. If you define those boundaries before deployment, you can give AI learning agents enough freedom to create value without giving them more authority than your organization can responsibly absorb.

Need an LMS That Leverages AI Learning Agents? Let’s Talk…

For independent advice you can trust, schedule a free 30-minute phone consult with Talented Learning Founder and Lead Analyst, John Leh: