Secure Implementation

Retrofitting security onto an agent after it is built is expensive, incomplete, and often architecturally impossible without rebuilding significant parts of the system. The patterns in this module are most effective when applied from the initial design. This is not theoretical: the incidents that have attracted public attention in 2024-2025 almost exclusively involved agents with insufficient input validation, overly broad permissions, or no human approval gates for consequential actions.

The UK Government's AI Cyber Security Code of Practice (2024) states this principle directly: security requirements should be addressed during the design phase of an AI system, not added retrospectively. The Code of Practice is the UK's voluntary framework for organisations developing or deploying AI in production, and it aligns closely with the NIST AI Risk Management Framework (AI RMF) 1.0 Manage function (specifically Manage 2.4: Risk treatments are applied, monitored, and adjusted).

This module is organised around the five controls that address the highest-priority risks from Module 16: input validation, output filtering, least privilege, human approval gates, sandboxing for code execution, and audit logging. A defence-in-depth (layered defence) approach requires all five to work together. No single control is sufficient on its own.

Input validation is the process of checking all inputs to an agent system before they are processed, to ensure they conform to expected formats and do not contain malicious content. For agents, this means validating both direct user inputs and content retrieved from external sources: emails, web pages, documents, and tool results.

User input validation should check for common injection patterns using regular expressions, enforce maximum message lengths to prevent context-window flooding attacks, and log any rejected inputs for later review. Pattern matching alone is insufficient: attackers continuously develop new phrasings that bypass known patterns. However, it is a valuable first layer that catches automated and unsophisticated attacks without incurring inference costs.

Retrieved content sanitisation is more important and more commonly neglected. Before any content retrieved from an external source enters the agent's context window, it should have HTML and XML tags stripped (which may contain hidden instructions), zero-width Unicode characters removed (a common technique for hiding injected text from human readers while keeping it visible to the model), and length truncated to a reasonable limit. Content that exceeds the limit should be truncated with a clear marker so the agent knows the document was not fully processed.

Agent outputs should be validated before they affect downstream systems. This is particularly important for outputs that will be used as parameters in tool calls, fed into databases, or rendered in user interfaces. OWASP Agentic AI Top 10 AA03 (Unsafe Agent Output) specifically addresses the risk of unvalidated agent outputs propagating into downstream systems.

Schema validation is the most effective output filtering technique. Rather than trying to detect malicious content in free text, require the agent to produce structured output (JSON conforming to a defined schema) and validate it against that schema before use. If the agent's output does not conform, reject it and request a retry or escalate to a human reviewer.

Tools like Pydantic (for Python) make schema validation straightforward: define the expected output structure, instantiate the model with the agent's output, and catch validation errors. Pydantic validates field types, value ranges, and custom constraints (such as requiring an email address to contain an @ symbol) in a single step.

Tool call allow-lists are a complementary control. Rather than trying to detect which tool calls are malicious, define an explicit allow-list of which tools the agent is permitted to call, and block everything not on the list. This is the OWASP AA02 (Excessive Agency) mitigation applied at the output layer: the agent cannot call a tool that is not on its permitted list, even if it has been successfully prompted to try.

The principle of least privilege states that every component should have the minimum permissions necessary to perform its function, and nothing more. For agents, this means giving each agent access only to the specific tools, data sources, and API credentials it needs for its defined task. An agent that summarises documents should not have access to an email sending tool. An agent that reads order status should not have payment processing credentials.

In practice, least privilege for agents covers three dimensions:

1. Tool scope: each agent instance should be initialised with only the tools required for its task. Do not create a single general-purpose agent with all available tools and trust the model to use them appropriately.
2. Credential scope: create separate API credentials for each agent role, with only the permissions that role requires. A customer support agent's database credentials should connect to a read-only replica, not the primary read-write database.
3. Data scope: if the agent reads documents, limit it to documents within the scope of the task. Do not give a summarisation agent access to the entire document store.

This approach directly implements OWASP Agentic AI AA02 (Excessive Agency) mitigation and aligns with NIST AI RMF Manage 2.4, which requires that AI risk treatments be applied and monitored, including limiting the capability of AI systems to the minimum required for their task.

For irreversible or high-impact actions, require explicit human confirmation before the agent executes the tool. This pattern is sometimes called human-in-the-loop (HITL) control, and it is the primary mitigation for OWASP AA04 (Overreliance on Agents).

The key design decision is which actions require approval. Classify every tool in the agent's tool set by risk level. Read-only operations on reversible data (searching a knowledge base, retrieving an order status) can typically run without approval. Write operations with reversible consequences (creating a support ticket, drafting an email for human review) may require a confirmation step. Irreversible or financially consequential operations (deleting records, sending email without review, processing payments, bulk communications) should always require explicit human approval before execution.

In production, the approval mechanism typically routes the pending action to a dashboard, a Slack channel, or an email, with a time-limited window for the approver to confirm or reject. If the window expires without a response, the action should be cancelled and the agent should notify the user that the action requires human review.

Human approval gates were absent in the 2023 LangChain email exfiltration scenario described at the start of this module. Had a human been required to approve thesend_email tool call before execution, the attack would have been stopped at that point, and the exfiltration would have been surfaced as an anomalous approval request.

If your agent can execute code (a common capability in data analysis and automation agents), that code must run in an isolated environment that cannot affect the host system or network. Sandboxing is the practice of running untrusted code inside a restricted execution environment that limits what the code can do.

The strongest approach is Docker containerisation. Each code execution request runs in a fresh container that has no network access, a read-only filesystem except for a temporary writable directory, and explicit CPU and memory limits. The container is destroyed after the code finishes executing. This means that even if an attacker successfully injects malicious code, the code cannot reach the network, cannot read files outside the temporary directory, and cannot persist anything to the host system.

Key parameters for a secure code execution container include: --network noneto prevent any network communication, --read-only to make the filesystem immutable, --memory 128m and --cpus 0.5 to prevent resource exhaustion attacks, and a subprocess timeout to kill executions that run longer than the permitted window. The container image should be a minimal base image with only the required runtime installed and no additional utilities that could be used to escalate privileges or probe the host environment.

Every agent action that affects the real world should be logged with sufficient detail to reconstruct what happened and why. OWASP Agentic AI AA10 (Inadequate Audit Logging) ranks this in the top ten because without thorough logs, incident response and forensic analysis after a security event are nearly impossible.

A complete audit log entry for an agent tool call should include: a timestamp in UTC (Coordinated Universal Time), the session identifier, the agent name and version, the tool called, a summary of the tool inputs (but not secrets or personal data in plaintext), the result status (success or error), and the identity of any human approver. This provides the information needed to answer: what did the agent do, when, on whose behalf, using what inputs, and who authorised it?

Audit logs must be tamper-evident. If an attacker can delete or modify log entries, they can cover the evidence of what the agent was manipulated into doing. In practice, tamper-evidence is achieved by writing logs to append-only storage (such as an immutable log service or a write-once-read-many storage bucket) and using cryptographic chaining (each log entry includes a hash of the previous entry) so that any deletion or modification breaks the chain.

Audit logs should not contain secrets, full personal data, or complete tool result payloads. Logging these creates a high-value target: a single log file would contain all the sensitive data the agent processed. Instead, log summaries and truncated representations, and ensure the log retention and access policy complies with applicable data protection law (such as the UK GDPR or EU GDPR).