The UX Dilemma: Over-Restriction Kills Your Agent’s Value
Real-World Implementation Pattern
Unified Containment Permissions
Building Containment That Scales
Every organization deploying AI agents risks the same nightmare scenario: what if my agent goes rogue? It’s not exactly paranoia as it’s already happened to some companies. When Replit’s AI agent deleted a prod database, the company determined it as a catastrophic failure. No malicious actors were involved, just a non-deterministic system with too much access and not enough guardrails.
Unfortunately, tightening down on security is not an easy task. There’s a paradox at play: giving an AI agent broad permissions is obviously risky, but locking it down so tightly that it needs approval for every action defeats the point of automation.
How much agency should an AI agent actually have? And how should permissions be used to achieve that ideal allowance?
By implementing AI containment strategies, organizations could allow agents to operate autonomously and sharply mitigate the damage of a failure. To make this possible, centralized authorization must enforce a designated policy.
There is a lot of nuance in how much authorization an AI agent should have. For example, an insecure AI agent that runs wild through your systems, potentially leaking data, is a terrible user experience. An over-restricted agent that constantly requires human intervention loses much of its value proposition. Users adopt AI agents to save time and automate workflows, not to create new bottlenecks.
This friction compounds when multiple manual approvals are required for what should be an autonomous workflow. Imagine an agent designed to handle customer support requests that needs approval to (i) access the database, (ii) draft a response, (iii) update the customer’s state, and (iv) send the email to finish the interaction. At that point, it would be faster for a human to just handle the ticket directly. The promise of automation evaporates under the weight of excessive guardrails.
The question that haunts product teams is one that users will inevitably ask in overly restricted systems: “Why can’t this agent just do what I ask?” When an AI agent feels more like a burden than a benefit, adoption stalls. The challenge isn’t how do I implement security measures? It’s more how do I implement them in a way that preserves the agent’s ability to deliver intelligent automation?
Effective AI agent containment implements smart controls that allow agents to operate autonomously while staying within safe boundaries.
Permissioning can be context aware, moving beyond binary access controls to reflect the layered protocols of real-world operations.
Static approach: “Customer support agent has read access to customer records.”
Reality needed: “Agent can read customer records for customers they’re actively helping, in their assigned region, during business hours, excluding PII unless escalated.”
This is an example of relation-based access controls (ReBAC). Policies written in Polar allow constraints that incorporate relationships, temporal conditions, and request context directly into the authorization check. These constraints should be shaped around the specific circumstances of each agent.
Just-in-time (JIT) permissioning grants time-bounded, scope-bounded privileges at execution time. JIT grants are elevated only when a specific high-risk operation requires it
JIT offers a middle ground between constant human oversight and unrestricted autonomy. Rather than requiring approval for every action or granting blanket permissions, JIT permissioning allows agents to operate independently for routine tasks, escalating only high-risk operations for human review.
Remember, not all actions carry equal risk. An agent processing a routine customer inquiry shouldn’t need the same oversight as one attempting to delete production data or access financial records.
Structure your agent’s capabilities in tiers:
This approach preserves workflow velocity for most agent actions while creating targeted checkpoints for operations that require human judgment. It reduces the human involvement to the absolute minimum while keeping your agent as autonomous as possible.
Many attacks are made by finding a weakness and exploiting it at scale. However, the blast radius of these attacks can be drastically reduced by implementing usage-based restrictions. For example, a health care AI agent may have a use case to read one or a few patients’ data at a specific time, but if it attempts to scan the whole database, your system should alarm on that agent and require human intervention to make sure nothing malicious is happening.
While this does not prevent small attacks on your system, it drastically reduces their impact, making unintentional leaks a lot faster to clean up.
To the same tune as the last one, security systems should alarm on behavioral anomalies that certain agents exhibit. Example anomalies to alarm on could be:
These anomalies can be evaluated pre-execution or in-flight to halt or escalate risky behavior. Setting up these alarms creates a safety net that allows agents to operate with greater autonomy while maintaining organizational security, catching genuine threats without overwhelming teams with false positives.
A typical authorization sequence for a payment-processing agent looks like:
Agent initiates action: “Transfer $5,000 to vendor account.”
Authorization flow:
This layered approach means that multiple independent checks must pass before the agent can execute sensitive operations, creating defense in depth while maintaining speed for legitimate workflows. When all checks pass, the agent completes the transfer autonomously. If any check fails, the system either denies the action or escalates to a human for review.
It can be very difficult to implement these checks as permissioning becomes more complex and multiple systems need these permissions. Without an authorization layer (e.g. Oso), permissions code can get drawn out into hundreds of lines of spaghetti code that pulls data from different sources (e.g. application database, authorization database) and runtime features (e.g. IP address, MAC address, geography, time). Consequently, humans are often over-provisioned. With agents, the risk multiplies. Agents do hundreds of actions in a minute; being over-provisioned could be catastrophic.
While agents differ from humans in velocity of work, their authorization logic doesn’t need to be fundamentally different. But it does need to be tighter. You should be able to define permissions once and enforce them everywhere. Otherwise, you might face cascading failures:
Inconsistency: User has permission in the app but not in the RAG system → agent can’t retrieve context needed to answer questions
Gaps: The agent authorized to read data through API but not authorized to access same data in vector database → half-working features
Redundancy: Same permission logic implemented 5 different ways across 5 systems → 5 places to make errors
Drift: Permissions updated in application, but not in agent monitoring system → security policy violations go undetected
The solution is treating authorization as a cross-cutting concern with central coordination. Define your permission model once, then enforce it consistently across every system your agents interact with.
The aforementioned strategies above all require consistent enforcement across every system your agents touch.
Oso enables you to define your security logic once and enforce it everywhere. Rather than rebuilding permission checks separately in your application, RAG system, vector database, and monitoring tools, you create a single source of truth. Update a policy once, and that change propagates across all systems automatically. With Oso, you can define fine-grained policies using Polar, monitor agent behavior in real time, and maintain full audit trails—all from a unified platform.
Containment depends on consistent, context-aware authorization enforced at every boundary where an agent can take action. Smart authorization isn’t about blocking your agents from being useful; it’s about giving them the freedom to operate within safe, well-defined boundaries.
If your team wants to streamline the containment and security process, consider booking a demo with Oso to see how to build permissioning that scales.
How is securing AI agents different from securing human access?
While the authorization logic can be similar, agents operate at machine speed without breaks, can be manipulated through prompt injection, and lack human judgment about context. A human with excessive permissions might access 50 records inappropriately if operating manually; an agent could exfiltrate 50,000 in seconds. The scale and speed of potential damage are orders of magnitude higher.
Won’t adding all these security layers slow down my agent and ruin the user experience?
Not if implemented correctly. The goal is to automate security checks at the same speed your agent operates. Low-risk actions (and some medium-risk actions) will still happen instantly without human intervention. Only sensitive or suspicious operations require approval, preserving velocity for 90%+ of agent actions while protecting against catastrophic mistakes.
Can’t I just use good prompts to control what my agent does?
Unfortunately, an AI agent can’t distinguish between context and data, leaving it vulnerable to attacks even when set up correctly. Prompts are by no means security controls; they should guide agent behavior, but authorization policies must enforce hard boundaries that can’t be talked around, no matter how clever the prompt.