Retrieved text should not acquire authority.

What is prompt injection, and where is the boundary?#

Prompt injection occurs when text without legitimate authority is interpreted as an instruction that changes an AI-enabled workflow's behavior. OWASP distinguishes direct injection through submitted input from indirect injection through material the system later reads, such as documents or tool results. The important boundary is not simply external versus internal storage. A customer-created ticket in an internal database can still contain instructions from someone who has no authority over the assistant's tools or access.

Map the actors before selecting a guardrail. Identify the user who requested the task, the people who authored each retrieved item, and the service identities that tools use. Then describe a forbidden consequence: an unauthorized record read, an unapproved message, or a write outside the intended workspace. A suspicious sentence and a completed unauthorized action are different observations. The assessment should establish which boundary is enforced, using your own authorized environment and synthetic information, without experimenting against another organization's assistant.

A model proposal is not an authorization decision#

Permission belongs to the application and the resource owner, not to the generated explanation. OWASP's authorization guidance calls for permission checks on every request. Apply that principle to tool invocations: validate the authenticated principal, target resource, tenant, action and relevant conditions when the operation executes. A model can nominate arguments, but it should not be able to turn an arbitrary document identifier into access merely by describing the request as helpful or urgent.

Design separate capabilities for separate consequences. A retrieval tool that reads permitted support records is narrower than a general file browser. Preparing a response draft is narrower than transmitting it. In a proposed approval flow, bind approval to the exact destination, content or change, environment and version being reviewed. If those values change, the prior approval should no longer silently authorize execution. Review is useful only when the reviewer can inspect the proposed consequence and the downstream system still enforces the reviewer's authority.

Use architecture to constrain authority#

The CaMeL research paper explores separating trusted control flow from untrusted data and checking security policies around tool calls. Its value here is architectural: do not rely exclusively on a model to remember which text may grant authority. A research evaluation of a particular implementation does not demonstrate that a different deployed agent is immune to prompt injection, nor does the architectural idea remove the need to review policy definitions and tool adapters.

For an ordinary application, start with boundaries you can explain and test. Keep credentials out of model-readable context. Retrieve only records the user may access. Limit network destinations and expose task-specific operations instead of a general command tool. Treat tool results as potentially untrusted too: an approved retrieval operation can return a customer-authored instruction. Check outputs before they reach an interpreter, browser or downstream service. These are proposed design choices to reduce consequence, not a claim that one wrapper or prompt template solves every injection path.

Compare controls by the boundary they actually enforce#

Several controls can be useful while doing different jobs. A clear system instruction describes intended behavior. A classifier may flag suspicious text. A constrained tool adapter determines whether a particular operation is permitted. Confusing these roles creates false confidence: a low classifier score does not confer permission, and an allowed operation does not prove the content is truthful. Use the following comparison when defining responsibility between the model, the application and a reviewer.

Prefer a small set of observable boundaries over a collection of features with no defined failure state. For each chosen control, document what it receives, what it can deny, who can change its configuration, and what happens when it is unavailable. Consider both misuse and legitimate tasks: a defense that blocks every document may prevent unauthorized actions, but it has not produced a useful document assistant. Record the operational tradeoff instead of hiding it behind an aggregate score.

Fictional worked example: a support-summary assistant#

Consider a fictional internal product called Ledger Support. An operator asks its assistant to summarize test case North-17 and prepare an internal note. The assistant can retrieve records for the operator's assigned test tenant and create drafts in that tenant. It cannot send external messages. The development team uses two invented tenants and disposable records; no production customer material or live credentials enter the evaluation. These assumptions define the authorized laboratory exercise rather than describing a client incident.

A North-17 attachment includes a request to obtain another tenant's case and add its details to an outgoing message. The attachment is customer-authored content, not an instruction from the operator. The application records its provenance and sends the model only the permitted case data. If the model proposes the cross-tenant read anyway, the retrieval adapter checks the requested resource against the operator's tenant and denies it. If it proposes transmission, no send capability exists in this workflow. The evidence is the denied request and unchanged dummy state, not a claim that the model never considered the request.

Fictional worked example: prove the permitted task still works#

The team next runs an ordinary attachment with no misleading instructions. Ledger Support must summarize North-17 accurately and save a draft in the correct tenant. It then runs the misleading attachment and checks the same authorized result, denied cross-tenant access and absence of outgoing messages. An instrumented test destination records zero unauthorized sends; the draft store records only the intended North note. The test report includes model and tool-adapter versions, fixture versions, selected policy and relevant event identifiers so another engineer can reproduce the exact scenario.

A second laboratory scenario changes the draft target after a reviewer has approved it. The executor rejects the stale proposal version and requests fresh review rather than accepting a broadly worded permission to continue. A third scenario makes retrieval unavailable: the assistant states that the summary lacks required evidence and does not fabricate it. These fictional tests cover distinct boundaries. Passing them supports those configured cases, not universal immunity across every document format, tool or model release. A new send capability or broader retrieval scope requires additional evaluation.

Test actions, state changes and permitted task completion#

Prompt-injection evaluation should inspect the execution trace as well as the final response. Define an intended task, forbidden effect and authoritative observation for each scenario. Capture tool requests, permission decisions, selected destinations and changes to disposable state. A refusal displayed after an unauthorized tool call is not successful prevention. Conversely, awkward wording without any authority crossing may be a product-quality problem rather than a demonstrated data-exposure finding.

Build a small regression set around your actual inputs: a support attachment, a retrieved document, a tool response and an ordinary user correction. Vary authorship and access, not just suspicious wording. Include a denied tool request, an expired approval, an unavailable policy service and a legitimate task that needs the allowed capability. Keep test information synthetic and assessments explicitly authorized. Document which observables you could not inspect, since an uninstrumented outbound integration cannot be declared unaffected merely because the assistant's visible answer looks normal.

• Identify the author and authority of every instruction and content source.
• State the forbidden consequence and where its occurrence would be observed.
• Validate user, tenant, resource, action and destination outside model reasoning.
• Test denied requests and inspect dummy state after the apparent refusal.
• Check ordinary task completion so controls do not merely disable the product.
• Version fixtures, policies, models and adapters for repeatable review.

Limits and tradeoffs of constrained AI workflows#

Narrow authority can mean more product design work. A task-specific tool may require several explicit operations where a general file manager would be easier to demonstrate. Access-filtered retrieval may omit useful context unless record ownership is modeled well. Human review can add delay, and reviewers may misunderstand a proposal if evidence is incomplete. Those costs should be assessed against the consequence of a mistaken operation, not against a desire to describe the assistant as fully autonomous.

A constrained workflow can still produce incorrect summaries, misleading citations or private information in an allowed location. Authorization controls address who may perform an action; they do not settle factual accuracy or every confidentiality question. Review the intended output audience, retention, rendering and external integrations separately. Avoid claims that all injection can be detected by a phrase list or that one successful evaluation guarantees future model behavior. Security conclusions should name the tested policy, capability and scenario, with uncertainty retained.

Next steps: map one workflow before broadening the agent#

Choose one existing assistant task and draw its content-to-action path. Mark user instructions, retrieved items, model output, tool requests and the point where an operation changes state. Write an action contract for that final point: authorized principal, resource, destination, allowed change, approval requirement and verification result. If a team cannot explain the contract, adding a classifier is unlikely to resolve the underlying uncertainty.

Implement the smallest useful capability set and evaluate it with ordinary and misleading synthetic inputs. Record a bounded finding for each failure: which input reached which authority boundary and what effect occurred. Then decide whether to remove a capability, strengthen a downstream check or improve the review experience. Repeat this review when tools, access scopes or workflow objectives change. The durable objective is an understandable authority model, not a marketing promise that the language model has become impossible to manipulate.