06 Insecure Output Handling

Large Language Models do more than answer questions.

Modern AI applications use LLMs to generate:

• HTML
• SQL queries
• Shell commands
• Source code
• API parameters
• URLs
• JSON
• Infrastructure configurations
• Cloud commands
• Agent actions

This creates an important security boundary.

Consider:

User
 ↓
LLM
 ↓
Generated Output
 ↓
Application
 ↓
Enterprise System

The LLM output is now becoming input to another system.

If the application automatically trusts and executes that output, model mistakes or attacker manipulation may create real security impact.

This is the problem of Insecure Output Handling.

The fundamental principle is:

LLM output should be treated as untrusted input whenever another system consumes it.

Learning Objectives

By the end of this lesson, you should be able to:

• Explain Insecure Output Handling.

• Understand why LLM output is a trust boundary.

• Recognize dangerous output-to-execution patterns.

• Understand risks involving HTML, SQL and shell commands.

• Identify risks in AI-generated code.

• Understand agent tool-call risks.

• Recognize output handling risks in cloud automation.

• Apply structured output validation.

• Use allowlisting and policy enforcement.

• Apply sandboxing and least privilege.

• Understand when human approval should be required.

• Perform safe output-handling assessments.

• Document findings professionally.

What Is Insecure Output Handling?

Insecure Output Handling occurs when an application accepts LLM-generated output and passes it to another component without sufficient validation or security controls.

Conceptually:

LLM
 ↓
Generated Output
 ↓
Trusted Automatically
 ↓
Interpreter / Tool / Application
 ↓
Potential Security Impact

The downstream component may be:

• Web browser

• Database

• Operating system

• API

• Cloud platform

• CI/CD pipeline

• Infrastructure automation

• AI agent tool

The security problem is not necessarily that the model generated incorrect information.

The problem is:

Another system trusted that information as an instruction.

Output Becomes Input

This is one of the most important concepts in AI application security.

From the LLM perspective:

Generated Text

From the next component’s perspective:

Input

Therefore:

User Input
    ↓
   LLM
    ↓
LLM Output
    ↓
New Untrusted Input
    ↓
Downstream System

Every transition creates another trust boundary.

Why LLM Output Should Not Be Trusted

LLMs are probabilistic systems.

Their output may be influenced by:

User Input
+
System Prompt
+
Conversation History
+
RAG Content
+
External Websites
+
Documents
+
Tool Results

Some of these sources may be attacker-controlled.

Therefore:

Untrusted Input
      ↓
LLM
      ↓
Generated Output

does not magically become:

Trusted Output

The LLM is not a security sanitizer.

The Dangerous Assumption

A common architecture assumes:

LLM Generated It
      ↓
Therefore It Is Safe

A safer assumption is:

LLM Generated It
      ↓
Treat It as Untrusted
      ↓
Validate
      ↓
Authorize
      ↓
Execute

Why This Matters More With AI Agents

A chatbot may produce:

Text

An agent may produce:

Action

For example:

User
 ↓
AI Agent
 ↓
LLM
 ↓
Tool Call
 ↓
Cloud Platform

If the LLM output controls the tool call, insecure output handling may directly affect enterprise infrastructure.

Output Handling Attack Chain

Consider:

Attacker
   ↓
Manipulates AI Input
   ↓
LLM Generates Dangerous Output
   ↓
Application Trusts Output
   ↓
Downstream System Executes It
   ↓
Security Impact

This combines two security problems:

Input Manipulation
       +
Unsafe Output Consumption

Defense should exist at both boundaries.

Scenario 1 — Generated HTML

Suppose an AI application generates content that will be displayed on a website.

Architecture:

User
 ↓
LLM
 ↓
Generated HTML
 ↓
Browser

If the application renders model-generated content as executable HTML without appropriate controls, the browser may interpret more than plain text.

The safer pattern is:

LLM Output
    ↓
Sanitization / Encoding
    ↓
Safe Rendering

The application should decide what content is allowed.

HTML Output Security

Depending on the application, controls may include:

• Output encoding

• HTML sanitization

• Content Security Policy

• Restricted rendering

• Safe Markdown processing

The model should not determine which browser capabilities are safe.

Scenario 2 — Generated SQL

Consider an AI analytics assistant.

User
 ↓
Natural Language Question
 ↓
LLM
 ↓
SQL Query
 ↓
Database

Example business requirement:

Show sales totals for last month.

The model generates SQL.

If the application automatically executes arbitrary generated SQL:

LLM
 ↓
SQL
 ↓
Production Database

the risk may include unintended:

• Data access

• Data modification

• Expensive queries

• Destructive operations

Safer SQL Architecture

A stronger pattern is:

User
 ↓
LLM
 ↓
Structured Query Intent
 ↓
Validation
 ↓
Approved Query Builder
 ↓
Read-Only Database Role
 ↓
Database

Security controls may include:

• Read-only database identities

• Approved schemas

• Approved tables

• Query validation

• Query timeouts

• Result limits

• Row-level security

This reduces dependence on model behavior.

Least Privilege Matters

Suppose the AI only needs analytics access.

Give it:

SELECT

rather than:

SELECT
INSERT
UPDATE
DELETE
DROP

Even if the model generates an unsafe operation, the database authorization layer should reject it.

Scenario 3 — Shell Commands

AI assistants may generate operating-system commands.

Architecture:

User
 ↓
LLM
 ↓
Shell Command
 ↓
Operating System

This becomes extremely dangerous if the application automatically executes arbitrary generated commands.

The model output is effectively becoming:

Executable Instructions

Weak Shell Architecture

LLM
 ↓
Generated Command
 ↓
shell=True
 ↓
Operating System

There is little separation between model reasoning and system execution.

Safer Architecture

Prefer:

LLM
 ↓
Requested Operation
 ↓
Allowlisted Action
 ↓
Validated Parameters
 ↓
Restricted Execution Environment

For example, instead of allowing the model to generate arbitrary shell commands:

restart nginx

the application might expose:

restart_service(service_name)

and validate:

service_name ∈ Approved Services

This dramatically reduces capability.

Commands vs Actions

This distinction is important.

Avoid:

LLM
 ↓
Generate Arbitrary Command

Prefer:

LLM
 ↓
Choose Approved Action
 ↓
Provide Validated Parameters

The application defines capability.

The model chooses only among approved options.

Scenario 4 — AI-Generated Code

Developers increasingly use AI to generate:

• Python

• JavaScript

• Java

• Infrastructure code

• Scripts

• CI/CD configuration

Generated code may contain:

• Security weaknesses

• Incorrect validation

• Unsafe dependencies

• Hard-coded secrets

• Insecure configurations

• Logic errors

Therefore:

AI-generated code should go through the same security lifecycle as human-written code.

Secure AI Coding Workflow

Developer
   ↓
AI-Generated Code
   ↓
Human Review
   ↓
Static Analysis
   ↓
Dependency Scanning
   ↓
Secret Scanning
   ↓
Testing
   ↓
Code Review
   ↓
CI/CD

AI assistance should not bypass established software engineering controls.

Scenario 5 — Generated URLs

An AI application may generate URLs.

LLM
 ↓
Generated URL
 ↓
Application Fetches URL

If the application automatically retrieves arbitrary model-generated locations, additional risks may appear.

Controls may include:

• Allowed protocols

• Domain allowlists

• Destination validation

• Network restrictions

• Redirect validation

The model should not independently determine which network destinations are trusted.

Scenario 6 — Generated API Requests

Consider:

User
 ↓
AI Agent
 ↓
LLM
 ↓
API Request
 ↓
Enterprise Service

The LLM may generate:

Method
Endpoint
Parameters
Body

If all values are trusted automatically, the model effectively controls the API client.

Safer API Pattern

LLM
 ↓
Structured Tool Request
 ↓
Schema Validation
 ↓
Authorization
 ↓
Approved Endpoint
 ↓
API

For example:

Tool:
get_security_alert

Parameters:
alert_id

rather than allowing:

method
url
headers
body

to be freely generated.

Scenario 7 — AI Agent Tool Calls

Modern AI agents frequently use structured tools.

Example:

AI Security Agent
      ↓
LLM
      ↓
Tool Call:
get_alert(alert_id)

Structured tools are generally safer than arbitrary execution.

However, they still require validation.

Tool Parameters Are Untrusted

Suppose the model generates:

delete_user(
    username="..."
)

The application should not assume:

LLM selected delete_user
        ↓
Therefore deletion is authorized

Instead:

LLM Tool Request
      ↓
Parameter Validation
      ↓
User Authorization
      ↓
Policy Enforcement
      ↓
Approval if Required
      ↓
Execution

The LLM Should Propose, Not Authorize

This principle is extremely useful:

The LLM may propose an action. The application should decide whether that action is allowed.

Architecture:

LLM
 ↓
Proposed Action
 ↓
Security Controls
 ↓
Approved Action
 ↓
Tool

Scenario 8 — Cloud Automation

Consider an AI Cloud Security Assistant.

Engineer
   ↓
AI Assistant
   ↓
LLM
   ↓
Cloud Command
   ↓
AWS / Azure / Google Cloud

If generated commands are automatically executed using administrator privileges, a model mistake may affect production infrastructure.

Weak Cloud Architecture

LLM
 ↓
Cloud CLI Command
 ↓
Administrator Credentials
 ↓
Production

This creates a very large blast radius.

Stronger Cloud Architecture

LLM
 ↓
Recommended Change
 ↓
Policy Validation
 ↓
Human Approval
 ↓
CI/CD or IaC Workflow
 ↓
Restricted Deployment Role
 ↓
Production

The AI assists the engineer.

It does not bypass enterprise change control.

Scenario 9 — Infrastructure as Code

AI may generate:

• Terraform

• CloudFormation

• Kubernetes YAML

• Helm configuration

Generated infrastructure should not move directly into production.

A stronger workflow:

AI-Generated IaC
      ↓
Code Review
      ↓
Security Scanning
      ↓
Policy-as-Code
      ↓
Terraform Plan / Deployment Preview
      ↓
Approval
      ↓
Deployment

This integrates AI into existing DevSecOps controls.

Scenario 10 — Kubernetes

Consider:

AI Assistant
   ↓
Generated Kubernetes YAML
   ↓
kubectl apply

The generated workload might unintentionally request:

• Privileged containers

• Host networking

• Host filesystem mounts

• Excessive service account permissions

A stronger workflow:

Generated YAML
      ↓
Schema Validation
      ↓
Policy-as-Code
      ↓
Security Review
      ↓
Deployment

AI output should not bypass Kubernetes admission controls.

Structured Output

LLMs increasingly generate structured output.

Example:

{
  "action": "disable_user",
  "username": "test-user"
}

Structured output improves parsing.

But:

Valid JSON does not mean authorized action.

The application still needs to validate:

Schema
+
Values
+
Authorization
+
Business Policy

Schema Validation

Suppose a tool expects:

action:
    read_alert
    close_alert

The application should reject:

delete_database

even if the LLM generates it.

This is allowlisting.

Allowlisting

Allowlisting defines what the model is permitted to request.

Example:

Allowed Actions:

read_alert
get_asset
search_runbook
create_draft

Everything else:

DENY

This is much safer than trying to enumerate every dangerous action.

Validate Parameters

Even approved actions may contain dangerous parameters.

Example:

read_file(path)

If the model controls:

path

the application must ensure the path falls within the permitted scope.

Think:

Approved Tool
      +
Unvalidated Parameters
      =
Still Dangerous

Parameter Constraints

Controls may include:

• Type validation

• Length limits

• Character restrictions

• Allowed values

• Allowed resource IDs

• Path restrictions

• Domain restrictions

The exact controls depend on the tool.

Output Encoding

When model output enters another interpretation context, encode it appropriately.

Examples:

HTML Context
→ HTML Encoding

URL Context
→ URL Validation

Database
→ Parameterized Query

Shell
→ Avoid String Command Construction

Traditional secure coding practices remain highly relevant.

Avoid Interpreter Chaining

A dangerous architecture may look like:

User Input
 ↓
LLM
 ↓
Generated Python
 ↓
Python Interpreter
 ↓
Shell
 ↓
Operating System

Every interpreter increases potential complexity.

Prefer fewer interpretation layers.

Sandboxing

If generated code must execute, consider a restricted environment.

Generated Code
      ↓
Sandbox
      ↓
Limited CPU
Limited Memory
Limited Files
Limited Network
No Production Credentials

The sandbox reduces blast radius.

Sandbox Controls

Depending on the environment, controls may include:

• No privileged execution

• Filesystem isolation

• Network restrictions

• Resource limits

• Temporary storage

• No production secrets

• Execution timeout

Sandboxing is particularly important for code-execution AI systems.

Network Isolation

Generated output may attempt to interact with network services.

Consider:

AI-Generated Code
      ↓
Sandbox
      ↓
Internet?
Enterprise Network?
Cloud Metadata?

Network access should be explicitly designed rather than automatically allowed.

Credential Isolation

Do not place powerful credentials inside the execution environment unless required.

Bad:

AI Code Sandbox
      ↓
Cloud Administrator Credentials

Better:

AI Code Sandbox
      ↓
No Cloud Credentials

or:

Restricted Task-Specific Identity

Human-in-the-Loop

Some actions should require human approval.

Examples:

• Production deployments

• IAM modifications

• Resource deletion

• External communications

• Financial operations

• Security-control changes

Architecture:

LLM
 ↓
Proposed Action
 ↓
Human Review
 ↓
Approve?
 ├── No → Stop
 └── Yes
       ↓
     Execute

Approval Must Be Meaningful

Avoid presenting users with:

AI wants to execute something.
Approve?

Provide:

Action:
Disable account TEST-USER

Reason:
Repeated authentication failures

Impact:
User will lose access

Target:
TEST-USER

Approve?

The reviewer needs enough context to make an informed decision.

Defense in Depth

Secure output handling may include:

LLM Output
    ↓
Structured Schema
    ↓
Validation
    ↓
Allowlisting
    ↓
Authorization
    ↓
Policy Enforcement
    ↓
Human Approval
    ↓
Least-Privilege Tool
    ↓
Execution
    ↓
Monitoring

No single layer should carry the entire security responsibility.

Prompt Injection + Insecure Output Handling

These vulnerabilities can form an attack chain.

Attacker
   ↓
Prompt Injection
   ↓
LLM Behavior Manipulated
   ↓
Dangerous Output Generated
   ↓
Application Trusts Output
   ↓
Execution

Prompt Injection affects:

What the model generates

Insecure Output Handling affects:

What the application does with it

Both should be addressed.

Indirect Prompt Injection + Tool Execution

Consider:

Malicious Website
      ↓
AI Browser Agent
      ↓
Indirect Prompt Injection
      ↓
LLM Generates Tool Request
      ↓
Application Executes Request

A strong tool-authorization layer can break this attack chain.

LLM Tool Request
      ↓
Authorization
      ↓
DENIED

This demonstrates why output validation is critical even when input defenses exist.

Output Validation vs Prompt Engineering

Developers may attempt:

System Prompt:
"Never generate dangerous commands."

This may improve behavior.

But it should not replace:

Command Validation
+
Authorization
+
Least Privilege

Remember:

Prompt engineering guides output. Security controls govern what happens to output.

AI Security Engineer Assessment Methodology

When reviewing an AI application, map every place where LLM output is consumed.

Step 1 — Identify Output Destinations

Ask:

Where does model output go?

Possible destinations:

Browser
Database
Shell
API
Cloud Platform
Agent Tool
CI/CD
File System
Email

Step 2 — Identify Interpretation

For each destination ask:

Is output displayed?

Parsed?

Interpreted?

Executed?

Risk generally increases as you move from:

Display
   ↓
Parse
   ↓
Interpret
   ↓
Execute

Step 3 — Identify Validation

Determine whether output is:

Schema Validated

Sanitized

Encoded

Allowlisted

Policy Checked

Step 4 — Identify Privilege

Ask:

Which identity executes the action?

What permissions does it have?

This is critical.

Step 5 — Identify Autonomy

Determine whether execution is:

Automatic

Human Approved

Workflow Approved

More autonomy generally increases potential impact.

Step 6 — Define Expected Behavior

Example:

AI SOC Assistant

Allowed:
Read alerts
Search runbooks

Not Allowed:
Delete alerts
Disable logging
Modify IAM

This creates a testable security boundary.

Step 7 — Use Synthetic Actions

Do not test destructive behavior against production.

Use:

TEST-USER
TEST-RESOURCE
TEST-DATABASE

and restricted environments.

Step 8 — Test Output Manipulation

In an authorized lab, determine whether input manipulation can influence downstream output.

Focus on whether the application:

• Validates the output

• Enforces allowed actions

• Validates parameters

• Enforces authorization

Step 9 — Test Compensating Controls

Example:

LLM Generates Unauthorized Action
        ↓
Policy Engine
        ↓
DENY

This is a successful security control.

The model may behave unexpectedly while the application remains secure.

Step 10 — Assess Business Impact

Ask:

Could data be accessed?

Could data be modified?

Could code execute?

Could infrastructure change?

Could messages be sent?

Could security controls be disabled?

This determines severity.

Example — AI SOC Assistant

Consider:

SOC Analyst
   ↓
AI Assistant
   ↓
LLM
   ↓
Security Tool

Required capability:

Read Alert

Available tools:

read_alert
disable_user
delete_alert
modify_firewall

This violates capability minimization.

Better Design

SOC Assistant
      ↓
read_alert
search_runbook
create_incident_draft

Higher-risk operations should exist in separate controlled workflows.

Example — AI Cloud Assistant

Requirement:

Explain Cloud Misconfiguration

Weak implementation:

LLM
 ↓
Generate AWS CLI
 ↓
Automatically Execute
 ↓
Production

Better:

LLM
 ↓
Explain Finding
 ↓
Recommend Remediation
 ↓
Engineer Review
 ↓
IaC Change
 ↓
Security Validation
 ↓
Normal Deployment Pipeline

The AI supports existing enterprise processes rather than replacing security controls.

Example — AI Database Assistant

Requirement:

Business Analytics

Strong architecture:

User
 ↓
AI
 ↓
Approved Query Interface
 ↓
Read-Only Database
 ↓
Row-Level Security
 ↓
Limited Results

This is stronger than allowing arbitrary model-generated SQL.

Insecure Output Handling Finding Template

Finding:
LLM-Generated Tool Actions Are Executed Without Validation

Affected Component:
AI Operations Agent

Expected Behavior:
The agent should execute only approved read-only actions.

Observed Behavior:
Model-generated tool requests are passed directly
to the execution layer without independent policy validation.

Security Impact:
Manipulated or incorrect model output could cause
unauthorized operations against connected systems.

Root Cause:
LLM output is treated as trusted execution instructions.

Recommendation:
Implement structured tool schemas, action allowlisting,
parameter validation, deterministic authorization,
least-privilege identities and approval for high-risk actions.

Another Example Finding

Finding:
AI-Generated SQL Executes Using Excessive Database Permissions

Affected Component:
AI Analytics Assistant

Expected Behavior:
The assistant should perform read-only analytics queries.

Observed Behavior:
Generated SQL is executed directly using an identity
with data-modification privileges.

Potential Impact:
Incorrect or manipulated model output could modify
or delete production data.

Recommendation:
Use a read-only database identity, restrict accessible
schemas, validate generated queries, enforce query limits
and use approved query-building mechanisms.

Risk Assessment

Consider:

Output Control
      +
Interpreter
      +
Privilege
      +
Autonomy
      +
Business Impact

A useful conceptual model is:

Output Handling Risk
        ≈
Likelihood of Unsafe Output
        ×
Execution Capability
        ×
Privilege
        ×
Impact

Example Risk Comparison

Scenario A

LLM Output
 ↓
Displayed as Plain Text

Potential risk:

Relatively Limited

Scenario B

LLM Output
 ↓
Rendered as Active Web Content

Potential risk increases.

Scenario C

LLM Output
 ↓
Executed as SQL
 ↓
Production Database

Potential risk increases significantly.

Scenario D

LLM Output
 ↓
Cloud Administrator Tool
 ↓
Production

Potential impact may be very high.

The destination matters.

Output Handling Security Checklist

Output Inventory

• Model output destinations identified.

• Interpreters identified.

• Execution paths documented.

• Trust boundaries mapped.

Web Output

• Output safely encoded.

• Active content sanitized.

• Browser security controls applied.

Database

• Generated SQL restricted.

• Read-only identities used where possible.

• Database authorization enforced.

• Query limits applied.

Operating System

• Arbitrary shell execution avoided.

• Approved operations exposed instead.

• Parameters validated.

• Execution sandboxed where required.

APIs

• Endpoints allowlisted.

• Parameters validated.

• Authorization enforced independently.

• Sensitive operations controlled.

Agents

• Tool schemas defined.

• Tool access minimized.

• Tool parameters validated.

• User authorization enforced.

• High-risk actions require approval.

Code

• AI-generated code reviewed.

• Static analysis performed.

• Dependencies scanned.

• Secrets scanned.

• Tests performed.

Cloud

• AI does not bypass change management.

• Deployment identities follow least privilege.

• IaC validation exists.

• Production changes require appropriate controls.

Execution

• Sandboxing used where required.

• Network access restricted.

• Credentials minimized.

• Resource limits configured.

Monitoring

• Tool requests logged.

• Authorization decisions logged.

• Executed actions auditable.

• Security teams can investigate abnormal activity.

Common Beginner Mistakes

Mistake 1 — Trusting LLM Output

LLM output is untrusted whenever another system consumes it.

Mistake 2 — Fixing Only the Prompt

Instructions such as:

Never generate dangerous commands.

are not sufficient security controls.

Mistake 3 — Allowing Arbitrary Shell Execution

Expose restricted operations instead of a general-purpose shell.

Mistake 4 — Giving Agents Administrator Permissions

This dramatically increases blast radius.

Mistake 5 — Assuming Structured JSON Is Safe

Structured output still requires validation and authorization.

Mistake 6 — Validating the Tool but Not Parameters

An approved tool can still be abused with unsafe parameters.

Mistake 7 — Automatically Deploying AI-Generated Code

AI-generated code should follow normal engineering and security review.

Mistake 8 — Ignoring Downstream Authorization

The target system should still enforce permissions.

Mistake 9 — Giving Code Sandboxes Production Credentials

Sandbox isolation is weakened if powerful credentials remain available.

Mistake 10 — Treating Human Approval as a Checkbox

Reviewers need enough context to understand the proposed action.

AI Security Engineer Perspective

When assessing an AI application, do not stop at:

What does the model generate?

Continue the data flow:

Where does that output go?

Who interprets it?

Is it parsed?

Is it executed?

What identity executes it?

What permissions exist?

Is the action validated?

Is authorization checked?

Can a human stop it?

What is the blast radius?

This turns AI testing into real security engineering.

Interview Perspective

You may be asked:

What is Insecure Output Handling?

A strong answer is:

Insecure Output Handling occurs when an application trusts LLM-generated output and passes it to another component such as a browser, database, shell, API or agent tool without sufficient validation. Because model output may be incorrect or attacker-influenced, it should be treated as untrusted input at the next system boundary.

Another question may be:

How would you secure an AI agent that executes tools?

A strong answer is:

I would expose narrowly scoped structured tools rather than arbitrary execution, validate tool parameters, enforce user authorization independently from the model, apply least privilege to the tool identity, allowlist permitted actions and require approval for high-impact operations.

Another question may be:

Why isn’t a system prompt enough to prevent dangerous output?

A strong answer is:

A system prompt guides probabilistic model behavior but does not provide deterministic security enforcement. The downstream application must validate and authorize model-generated actions independently before execution.

Another question may be:

How would you safely use AI-generated SQL?

A strong answer is:

I would prefer a constrained query interface or validated query-building mechanism, execute through a read-only database identity, restrict accessible schemas and tables, apply row-level security and query limits, and never rely solely on the model to generate safe SQL.

Key Takeaways

LLM output should be treated as:

Untrusted Input

whenever it enters another system.

The dangerous pattern is:

LLM
 ↓
Generated Output
 ↓
Automatically Trusted
 ↓
Execution

A stronger architecture is:

LLM
 ↓
Structured Output
 ↓
Validation
 ↓
Allowlisting
 ↓
Authorization
 ↓
Policy Enforcement
 ↓
Approval Where Required
 ↓
Least-Privilege Execution

Insecure Output Handling becomes especially important when output controls:

HTML
SQL
Shell
Code
APIs
Cloud Infrastructure
AI Agent Tools

Remember:

The LLM may recommend or propose an action, but security controls must determine whether that action is allowed to happen.

What’s Next?

➡️ 07 — Excessive Agency and AI Agent Security

You now understand why model-generated output should never automatically become trusted execution.

The next step is to examine what happens when AI systems are intentionally given the ability to act.

Modern AI agents may:

Read Email
Search Documents
Query Databases
Call APIs
Create Tickets
Modify Cloud Resources
Execute Workflows

This introduces the problem of Excessive Agency.

In the next lesson, you will learn:

• What agency means in AI systems

• AI assistants vs AI agents

• Excessive functionality

• Excessive permissions

• Excessive autonomy

• Agent identity

• Tool design

• Tool allowlisting

• Least privilege

• Human-in-the-loop controls

• Agent blast radius

• Multi-agent risks

• Agent monitoring

• Emergency controls

• Safe agent security testing

You will move from:

Can LLM Output Trigger an Action?

to:

How Much Power Should
an AI Agent Be Given?

➡️ Next: 07 — Excessive Agency and AI Agent Security