The first enterprise AI tools wrote text: a summary, a draft email, an answer to a question. A person decided whether anything came of that text. An AI agent goes further. It opens tickets, reads files, sends email, and writes to databases, all as part of one task, in several steps, following its own plan.
That moves the security question too. With an agent, you have to think through which actions it can carry out, with whose rights, and who will notice if it does something nobody asked it to do. This article works through these questions from the design side.
How an agent differs from a chat interface
An agent is given a goal, for example “review the open tickets and close the duplicates.” From that, the language model plans steps. At each step it picks one of the tools available to it, calls it, reads the result, and uses that to choose the next step. A tool can be an API call, a file operation, a search, or a code execution environment.
Two things change as a result. First, the model's output becomes a command. If the model is wrong or misreads the task, the error hits a system right away, and there is no draft in between for anyone to read. Second, everything the agent reads while it works ends up in the model's input: a web page, an incoming email, an uploaded document, a tool's response. The language model does not reliably separate data from instructions in this content, so an instruction hidden in a piece of text can turn into a tool call. This is indirect prompt injection. That is why everything the agent reads, tool responses included, has to be treated as untrusted external input.
How much damage such an error or hijacking causes depends on how much the agent can reach. The 2025 edition of the OWASP Top 10 for LLM Applications covers this as an item separate from prompt injection, under the name Excessive Agency: the agent gets more functions, more permissions, or more autonomy than its task justifies. The article Security risks of enterprise AI adoption gives the wider picture. Here we cover the controls specific to agents.
Least privilege for agents
The principle of least privilege works the same way as for a person or a service: only as much access as the task needs. For an agent, it is worth thinking through at three levels.
Which tools it gets
The agent should see only the tools that belong to its task. A general-purpose tool, such as arbitrary shell commands or HTTP requests to any URL, can be used for almost anything, so a narrow task calls for a purpose-built function instead: “close ticket by ID,” “search documents in the project folder.”
What it can do with a tool
Actions have to be narrowed within a tool as well. Summarizing email takes read access to one mailbox, with no send or delete. The scope should be fixed in the tool's code or in the permission behind it: which project, which folder, which tables. A prohibition written into the system prompt (“never delete anything”) only influences the model's behavior, and a clever input can override it. This rule applies to every control in this article: the limit is enforced by the tool layer or the permission system, independently of the model.
For how long and how much
Access should be short-lived, using a token valid for one run or a few minutes. Set an upper limit on the number of steps and tool calls, and on runtime too. Otherwise a loop that keeps calling itself, or a manipulated task, can keep working unsupervised for a long time.
| Task | Sufficient | Not needed |
|---|---|---|
| Summarizing incoming email | Read access to one mailbox | Sending, deleting, other mailboxes |
| Answering from internal documents | Search in the repositories the person asking can also see | Read access to every share, write access |
| Sorting tickets | Labeling and status changes in one project | Deleting tickets, project settings |
| Building a report from a database | Read-only access to the required tables | Write access, schema changes, the whole database |
Who the agent runs as: identity and credentials
An agent can access systems in two ways: with its own identity or on behalf of the user. The two carry different risks, and the choice has to be made at the start of the design.
Its own service account
The agent works with a service account that does not depend on who gave it the task. For background processes this is the natural choice, for example an overnight data cleanup job. Trouble starts when an agent that serves several users reads data through such an account. If the account can see every document, a user can get, through their questions, at content they could not open with their own rights. The agent then bypasses the existing permission system.
On behalf of the user
The other route is delegated access: the agent works with the rights of the user who started the request, or a narrowed subset of them. OAuth 2.0 delegation patterns exist for this, among them the IETF's RFC 8693 token exchange standard, which lets a service exchange the user's token for one with a narrower scope, issued for the system being called. The agent then sees and does at most what the user can. If the target system logs both the token's subject and the actor working on the subject's behalf, the log also shows at whose request the action took place.
Where the credentials live
A password, API key, or token should never go into the prompt, and the model should not see it at all. Whatever the model can see, an injection attack can make it print. Secrets belong in a secrets manager, and the tool layer attaches them to the call after the model has decided what it wants to call. Give every agent its own identity instead of a shared account, and when one agent hands a subtask to another, the second one should not automatically inherit all of the caller's rights. The identity needs an owner who reviews its rights regularly and can revoke them immediately when needed.
Human approval for irreversible actions
Narrow rights reduce the possible damage, but an agent still has to write somewhere to do its job. Where an action cannot be undone, or has an effect outside the organization, a person has to sign off on it. It helps to group actions by risk:
- Reading and searching: can run without approval if the scope is narrow.
- Reversible writes, such as saving a draft or adding a label: usually fine as well, with logging.
- Actions with an external effect or that cannot be reversed: sending email to an external address, deleting data, starting a payment or an order, changing permissions, changing the configuration of a production system, deploying code to production. These need approval.
The approver has to see the actual action. If they only get the agent's own summary (“I'll send the reply to the client”), they are approving what the model claims. The approval screen should show the real parameters: the recipient, the full text of the email, the list of records to be deleted, the configuration before and after the change.
The approval is also requested by the tool layer, independently of what the model decides. If every small step asks for confirmation, though, approving turns into routine clicking, and the request that really is dangerous gets lost among the many harmless ones. So approval is needed only where the damage cannot be repaired. If no answer arrives within the set time, the action should not go ahead.
Vetting tools and integrations
Every tool is a piece of code that the model calls, and its description goes into the model's input. It follows that the description itself influences the model: a malicious or carelessly written tool description can slip instructions to the model, or lead it to misuse another tool.
Tools are often provided by separate servers over open protocols, the Model Context Protocol (MCP) being one example. This makes integration easier, and it also creates a supply chain: the code of a third-party tool server runs with the organization's data and credentials. A component like this has to be handled the same way as any other external software.
- It comes from a known source and runs at a pinned version, and someone reviews each update before it goes to production.
- It is clear which operations it offers and what permissions it requests in the system behind it.
- The tool validates input on its own side: schema, allowed values, rejection of identifiers outside its scope. It does not count on the model sending the right parameters.
- Code runs in a sandbox, outbound network traffic goes only to allowed addresses, and only the working directory is visible in the file system.
Before adding a new tool, it is worth asking one question: in the worst case, what damage could this tool do with its existing rights if a malicious document is steering the agent?
Logging: every step should be traceable
An agent can make dozens of decisions during a single task, and afterwards you need to know why each of them happened. For that, every element of a run should go into the log under the same correlation ID:
- the request: who started it, when, and with what task;
- the identity: which account it ran with or on whose behalf, and with what token scope;
- every tool call with its parameters and result, failed and rejected calls included;
- approvals: who approved, when, and exactly what they saw at the time;
- external sources: which document, web page, or email the agent read, because that is often where the trace of an injection turns up.
The log itself holds sensitive data, since the questions and the content read in are all in it. Access to it has to be restricted, and secrets and personal data masked. The agent must not be able to change or delete the log, so it is worth copying it to separate storage or a SIEM. Retention and the choice of sources are covered in detail in the article Logging: what to collect for an incident.
The log is also the basis for alerting. Things worth watching for: an unusually high number of tool calls, out-of-scope requests and denied permissions, a sudden rise in calls to a rarely used tool, and a run that reaches the configured step limit.
Related service: Secure AI solution development →