When AI can retrieve data, invoke tools, and take action, least privilege becomes part of AI architecture

A financial-services organization deploys an AI assistant to answer customer questions. Engineering connects it to internal documents through retrieval-augmented generation, customer records in a CRM, a ticketing system, an email tool, and a workflow engine.

Each connection makes the assistant more useful.

Each connection also gives it authority.

The document repository determines what information the agent may retrieve. The CRM determines which customer records it may access. The email integration determines whom it may contact. The workflow systems determine which actions it may initiate.

Organizations tend to describe these as AI capabilities. Security should recognize what they also are: permissions.

Once an AI system can retrieve protected information, invoke tools, call APIs, modify records, or initiate transactions, its guardrails cannot consist only of instructions about how the model should behave.

Instructions influence what a model should do.

Access controls constrain what it can do.

That distinction matters. AI security cannot depend on the model making the right decision every time. The damage a bad decision can cause depends on the authority available to act on it.

The Security Model Changes When AI Can Act

Consider three increasingly consequential AI deployments.

A generative system produces a response. Its immediate consequence is what it tells someone.

A retrieval-enabled system can obtain information not contained in the user’s prompt. Its consequences now include what information it can reach and where that information can surface.

An agentic system can select and invoke tools that affect external systems. Its consequences can extend from information into action.

A hallucinating chatbot can mislead someone. An overprivileged agent could disclose protected data, modify records, send communications, execute code, invoke APIs, or trigger financial workflows.

The sophistication of the failure is only part of the risk. What matters just as much is what the system allows the agent to do after the failure occurs.

This makes authority architecture part of AI security: least privilege, scoped permissions, temporary access, approval thresholds, revocation, auditability. Agentic AI creates a machine actor whose reasoning can determine when and how granted capabilities are exercised. The principles for governing privileged actors still apply.

What Slack AI Taught Us About Retrieval and Trust

In August 2024, security researchers at PromptArmor disclosed an indirect prompt-injection vulnerability affecting Slack AI.

The attack exploited an important characteristic of retrieval-enabled AI: information retrieved to answer a question becomes part of the material the model processes.

Researchers demonstrated that malicious instructions placed in Slack content could be retrieved alongside legitimate information. Slack AI could then follow those instructions while processing information available to the victim, including content from private channels.

PromptArmor demonstrated an exfiltration technique in which Slack AI could generate a malicious link containing sensitive information from a private channel; completing the exfiltration required the victim to click that link. Slack subsequently changed the system to address the reported attack.

The broader lesson matters more than the individual exploit.

Information can be authorized for retrieval without being trustworthy as an instruction source.

Those are different security properties.

A document can simultaneously be authorized for retrieval, relevant to the query, and hostile to the AI system processing it. Security therefore has two questions to answer:

May the agent retrieve this information?

What authority should anything contained in that information have over the agent?

The second question becomes more important when retrieval is connected to tools. A malicious document influencing a summarization system can compromise an answer. The same document influencing an agent with powerful APIs can potentially influence actions outside the model.

Prompt-injection defenses matter. So does constraining what a successful injection can accomplish.

EchoLeak Shows Why User Access Should Not Automatically Become Agent Access

In 2025, Aim Security disclosed EchoLeak, CVE-2025-32711, a critical vulnerability affecting Microsoft 365 Copilot. The vulnerability received a CVSS score of 9.3. Microsoft addressed it before public disclosure, and researchers reported no evidence of exploitation in the wild.

EchoLeak demonstrated a zero-click attack chain in which malicious instructions embedded in an email could influence Copilot when that content was later retrieved. The researchers showed how the chain could cause sensitive Microsoft 365 information available to Copilot to be exfiltrated without additional victim interaction.

The details matter. The architectural lesson is larger.

An AI assistant becomes valuable because it can operate across information the user is permitted to access. That same breadth determines what becomes reachable if the AI system is manipulated.

This exposes an important distinction: user permission should not automatically equal agent permission.

Employees accumulate permissions as their roles require different resources over time. Their total entitlement may represent everything they could legitimately need across months or years. An AI agent usually performs a much narrower task.

If an agent can operate across everything the user can access, its authority reflects the user’s maximum entitlement, not the minimum the current task requires. That can defeat the purpose of least privilege at the task level.

The better question is not only “Can this user access the information?” but “Does this agent need this information for this task?”

Purpose should shape how much authority the agent receives.

The Chevrolet Chatbot Shows the Value of Separating Speech From Authority

A much less serious incident provides an unusually clean illustration of the same principle.

In December 2023, Chris Bakke manipulated a ChatGPT-powered chatbot on the Chevrolet of Watsonville dealership website. He instructed the chatbot to agree with whatever the customer said and to append language claiming the response constituted a binding offer. He then offered one dollar for a 2024 Chevrolet Tahoe.

The chatbot agreed.

The exchange went viral. No vehicle changed hands for one dollar.

That difference is the useful part of the story.

The chatbot’s behavior had been successfully manipulated, but it did not appear to possess the transactional authority required to turn its words into an executed vehicle sale.

It could say yes.

It could not complete the transaction.

Now change one architectural fact: the same manipulated chatbot also has permission to alter pricing, create contracts, reserve inventory, invoke payment APIs, and finalize transactions.

The prompt does not change.

The authority does.

Model compromise determines behavior. Granted authority determines consequence.

Preventing manipulation matters. But architecture should also ensure that manipulation does not automatically provide the authority to create material harm.

The Agent Authority Stack

Saying “the agent has access to Salesforce” or “the agent can use email” describes a connection. It does not describe an authority architecture. A security review should examine six dimensions.

Data authority. What can it retrieve? Document repositories, customer records, email, source code, financial data, and nonpublic personal information all carry different risk. Access should reflect the task, not the full extent of what the requesting user can reach.

Tool authority. What systems can it use? Search, email, ticketing, CRM, browser automation, code execution, payment services, and administrative systems carry very different consequences.

Action authority. What can it do through them? Reading a customer record, changing it, and deleting it involve the same application. They are three different permissions.

Scope authority. What can those actions affect? Permission to update one customer’s case is different from permission to update the entire database. Write access to a development environment is different from write access to production.

Autonomy authority. What can it do without human approval? Meaningful distinctions exist between recommending an action, drafting it, requesting approval, executing a reversible action, and executing an irreversible one.

Temporal authority. How long does it retain the permission? Standing access persists indefinitely. Temporary access exists only for a session, transaction, or task.

Together, these dimensions define the agent’s authority envelope: what it may retrieve, which capabilities it may invoke, what those capabilities may affect, how independently it may act, and how long that authority persists. The goal is not the smallest possible envelope. It is an envelope no larger than the task requires.

Behavioral Guardrails and Authority Guardrails Solve Different Problems

Many AI controls try to influence model behavior: system instructions, content filters, prompt-injection defenses, red-team exercises. All are useful. None should be confused with access control.

A behavioral guardrail says: do not modify production without approval. An authority control makes production write access unavailable until approval occurs.

A behavioral guardrail says: do not disclose customer information unnecessarily. An authority control prevents the agent from retrieving it outside the authorized task.

A behavioral guardrail says: do not send an unauthorized message. An authority control permits drafting but blocks transmission until approval exists.

A behavioral guardrail says: do not execute transactions above a threshold. An authority control causes the downstream API to reject them regardless of what the model requests.

The distinction is fundamental.

Behavioral controls reduce the probability of a bad decision.

Authority controls limit what a bad decision can accomplish.

Mature AI architecture needs both.

Security already applies this logic elsewhere. Organizations do not give every employee administrative access and rely on policy to prevent misuse. Privileged access is constrained because people make mistakes, credentials get compromised, and controls fail. Adding AI does not change that reasoning. It extends it.

Design for Guardrail Failure

One of the strongest tests of an AI system ignores how trustworthy the model appears. Assume the model is compromised. Assume the prompt-injection defense failed. Assume the content filter missed the attack. Assume the model follows the attacker’s instructions perfectly.

What can it actually do?

That answer describes the system’s blast radius.

All three examples reveal the same principle from different directions. Slack AI showed how untrusted retrieved content could influence model behavior. EchoLeak showed how malicious instructions interacting with an AI system’s access to enterprise information can create a path to sensitive data. The Chevrolet chatbot showed the other side: its conversational behavior was successfully manipulated while limited authority contained the consequence.

Do not make the model your final security boundary.

If a compromised agent can retrieve an entire customer database when it needs one record, its data authority is too broad. If it can modify production when its task requires only reading, its action authority is too broad. If it can execute a consequential transaction when its job requires only recommending one, its autonomy authority is too broad.

The question is not whether we trust the model. It is what remains protected when that trust is misplaced.

Governance Has to Reach the Permission Layer

Agentic AI exposes a control-boundary problem. A single agent can retrieve documents, query customer records, invoke APIs, draft communications, and initiate workflows across systems historically governed by separate functions: application security, identity and access management, data governance, privacy, compliance, and engineering.

AI governance cannot stop at policy.

“Do not expose customer information” is a policy. Restricting which records the agent can retrieve is a control.

“Require human review for consequential decisions” is a policy. Blocking the API call until approval exists is a control.

“Limit autonomous financial activity” is a policy. A transaction ceiling enforced outside the model is a control.

Governance becomes materially stronger when requirements survive translation into technical enforcement: scoped credentials, temporary permissions, approval gates, transaction limits, revocable access, and logging sufficient to reconstruct who requested an action, which agent performed it, which tool it invoked, and what the action changed.

Questions Every AI Security Review Should Ask

Before an agentic system ships, security should be able to answer:

• What information can the agent retrieve, and does that scope reflect the task or the user’s maximum entitlement?
• Which retrieved sources can contain untrusted instructions?
• Which tools can the agent invoke, and which operations does each tool expose?
• Are read and write permissions separated?
• Are credentials standing or temporary?
• Does the agent need those permissions for the current task?
• Which actions require human approval, and which are irreversible?
• Can the agent invoke another agent, and what authority does that agent hold?
• Can retrieved content influence tool execution?
• Can sensitive information reach external models or systems?
• Can access be revoked immediately?
• Can every consequential action be traced to the requesting user, agent, and tool?

And most importantly:

• If every behavioral guardrail failed at once, what could this agent actually accomplish?

That question should have an acceptable answer before the system ships.

Give the Agent Keys, Not the Keyring

The goal of agentic AI security is not a machine we can trust with unlimited authority. It is a system that remains safe when trust in the machine fails.

For decades, security programs have operated on a simple assumption: legitimate actors should not receive more authority than they need merely because we expect them to behave correctly. We constrain administrators, separate duties, scope service accounts, expire elevated privileges, require additional approval for consequential actions, log privileged activity, and revoke access when its justification disappears. Agentic AI does not create an exception to those principles. It creates another actor to which they must apply.

That changes how we should think about AI guardrails. A system prompt can tell an agent which doors it should open, but an access control determines which keys it actually holds. The objective is not to build an AI agent that can be trusted with every key and then instruct it to choose wisely. Give it only the keys required for the task, only when required, and take them back when the task is finished. When the agent makes the wrong decision, the doors it never needed should remain locked.

About the Author

Kimly Hong is a Principal Cybersecurity and GRC Consultant with more than ten years of experience building enterprise security programs across regulated financial services, hospitality, and technology environments. Her work spans governance, risk, and compliance program design, third-party risk management, access governance, and incident response readiness. She has built these programs from the ground up across complex, multi-region environments and currently consults across financial services, SaaS, and retail organizations. Connect on LinkedIn to continue the conversation.