Prompt injection is a hypothetical system-design threat here, but the mechanism is real: untrusted content attempts to influence model behavior as if it were an authorized instruction. The problem becomes more serious when an assistant can retrieve private data, call tools, browse the network, or perform side effects.
How would you defend an AI assistant against prompt injection?
Why the common answer is incomplete
Most candidates answer: “Add a system prompt telling the model to ignore malicious instructions.”
That instruction is useful context, but it is not a security boundary. Models process instructions and data in the same representation and can be manipulated by content inside documents, web pages, emails, or tool output. No prompt can grant the model reliable knowledge of which downstream actions the authenticated user is allowed to perform.
Security must be enforced by the application around the model.
A production approach
1. Define trust boundaries and assets
List every input source: user text, retrieved documents, web content, tool output, memory, and developer instructions. Mark which sources are trusted, tenant-scoped, or fully untrusted. Identify assets such as private documents, credentials, external communication, financial actions, and administrative tools. Threat modeling should describe what an attacker could cause the system to read, reveal, or change.
2. Keep policy separate from retrieved data
Place application rules in controlled instructions and clearly delimit untrusted content as data. Do not concatenate retrieved documents into an instruction block. Preserve source identity and avoid passing irrelevant markup, scripts, or hidden text. This separation helps the model, but the application must still assume that retrieved content can influence generation.
3. Enforce authorization outside the model
Derive user and tenant identity from authenticated server context, never from model output or prompt text. Every retrieval query and tool call must apply authorization at the data or service boundary. The model can propose read_document(123), but the document service decides whether the current user may access it. A refusal sentence is not access control.
4. Give the agent least privilege
Expose narrow, task-specific tools rather than general network, shell, or database access. Separate read and write capabilities. Use allowlisted destinations and parameter schemas. Run untrusted code or document processing in isolated environments with network, filesystem, CPU, and time limits. Do not place long-lived secrets in model-visible context or tool responses.
5. Validate every proposed action
Treat tool calls as untrusted requests. Validate schema, business rules, authorization, rate limits, and destination before execution. Bind approvals to the exact action, parameters, and user. Require a fresh confirmation for high-impact writes or external messages. If the proposed action changes after approval, invalidate the approval rather than letting the model reinterpret consent.
6. Constrain information flow
Return only the minimum tool result needed for the next step. Redact credentials, internal headers, and unrelated records. Prevent the assistant from combining data across tenants or security domains. Apply output filtering for known secret formats and policy violations, while recognizing that filters complement rather than replace authorization. Sensitive operations may require a deterministic response template instead of free-form generation.
7. Detect suspicious instruction patterns
Classifiers, heuristics, and model-based checks can identify content that requests policy disclosure, credential exfiltration, hidden tool use, or instruction override. Use the signal to restrict tools, request confirmation, isolate content, or decline the task. Detection will have false positives and false negatives, so it should reduce privileges rather than serve as the sole gate.
8. Test end-to-end abuse paths
Build adversarial evaluations across direct user prompts, poisoned retrieved documents, malicious tool output, encoded instructions, multilingual attacks, and multi-turn persistence. Verify the actual effect: whether restricted data was returned or a prohibited tool ran. Re-run tests when prompts, models, tools, retrieval, or permissions change. Include benign controls so stronger defenses do not make the assistant unusable.
Engineering trade-offs
Strict tool allowlists reduce blast radius but limit open-ended usefulness. Human confirmation protects high-impact actions while adding friction and habituation risk. Content filtering can block legitimate security research. Sandboxing and per-tool services improve isolation but increase operational cost. Rich audit logs help incident response while creating sensitive data that needs retention and access controls.
Choose controls from the potential impact of each capability. A summarizer with no private data needs a different boundary than an assistant that can send email or change infrastructure.
Common mistakes
- Treating the system prompt as the primary security control.
- Allowing the model to choose the authenticated tenant or user identity.
- Giving a generic browser, shell, or SQL tool broad privileges.
- Passing complete tool responses when one field is sufficient.
- Approving a vague intent rather than the exact side effect.
- Testing whether the model says “no” instead of whether the forbidden action occurred.
- Logging secrets and poisoned content without access or retention controls.
Failure handling and observability
Log tool proposals, policy decisions, approvals, denials, tenant-safe resource identifiers, and action outcomes. Do not log credentials or full sensitive payloads. Track denied tool calls, invalid arguments, cross-tenant attempts, suspicious-content detections, confirmation rates, and post-approval action changes. Alert on unusual tool sequences, bulk reads, repeated authorization failures, and new outbound destinations.
When a control fails or uncertainty is high, reduce capability: answer from public information, disable writes, ask the user to confirm a precise action, or route to a human. Revoke affected tokens, preserve an incident trace, and review whether the boundary failed at retrieval, authorization, tool validation, or output handling. Do not retry a denied action through a differently worded model prompt.
Interview insight
The interviewer is not testing whether you know a magic prompt. They are testing whether you understand trust boundaries, least privilege, authorization, isolation, and auditability. A production design assumes the model can be influenced and ensures that influence cannot directly cross a security boundary.