Runbook 01 — LLM Security Testing Methodology

This runbook provides a repeatable methodology for performing security assessments of Large Language Model applications.

It is designed for:

AI Security Engineers
Application Security Engineers
Cloud Security Engineers
Penetration Testers
Security Architects
Red Teamers
SOC and Incident Response teams
GRC and AI Governance teams

The purpose of this runbook is not to provide a collection of random Prompt Injection payloads.

It provides a structured way to answer:

How do we systematically assess whether an enterprise LLM application can be manipulated, expose sensitive information or perform unauthorized actions?

A professional LLM security assessment should evaluate the complete system:

User
 ↓
Identity
 ↓
AI Application
 ↓
LLM
 ├── RAG
 │    ↓
 │ Vector Database
 │
 ├── Memory
 │
 └── AI Agent
      ↓
   Enterprise Tools
      ↓
 Enterprise Systems

The central principle of this runbook is:

Do not assess only whether the model can be manipulated. Assess whether manipulation can cross an enterprise security boundary.

Runbook Information

Item	Details
Runbook	Runbook 01 — LLM Security Testing Methodology
Learning Path	AI Security Engineer
Module	02 — LLM Security
Type	Enterprise Security Assessment Runbook
Primary Use	LLM Application Security Testing
Audience	AI Security, AppSec, Cloud Security, Red Team
Output	Assessment Report + Findings + Evidence + Remediation Plan

Purpose

Use this runbook to perform consistent security reviews of applications that use:

Large Language Models
Generative AI
RAG
Vector databases
AI agents
Enterprise tools
Model APIs
AI memory
Fine-tuned models

It can be applied to:

AI Chatbots

Enterprise Copilots

Security Assistants

AI Agents

RAG Applications

Cloud AI Assistants

Coding Assistants

Customer Support AI

Internal Knowledge Assistants

When to Use This Runbook

Use this runbook:

Before production deployment
During application security reviews
After major model changes
After RAG architecture changes
When adding AI agent tools
When adding sensitive enterprise data
During penetration testing
During AI Red Team exercises
During security architecture reviews
After an AI security incident

Assessment Principles

Follow these principles throughout the assessment.

Principle 1 — Authorization First

Perform testing only against:

Systems you own
Dedicated labs
Authorized test environments
Applications where explicit testing approval exists

Principle 2 — Use Synthetic Data

Prefer:

TEST-CONFIDENTIAL-001

rather than real confidential information.

The objective is to prove the security boundary can be crossed.

You do not need to expose real enterprise data to prove the issue.

Principle 3 — Separate Model Behavior From Security Impact

Always distinguish:

Model-Level Result

from:

Application-Level Security Result

Example:

Prompt Injection
      ↓
LLM Requests Restricted Tool

but:

Authorization Layer
      ↓
DENY

Assessment:

Model-Level Control:
Failed

Enterprise Security Boundary:
Passed

This is critical.

Principle 4 — Test Architecture, Not Only Prompts

Do not spend the assessment only asking:

Can I jailbreak this model?

Also ask:

What data can it access?

What tools can it invoke?

Which identity does it use?

What happens if the model is wrong?

What happens if external content manipulates it?

Principle 5 — Validate Root Cause

Do not stop at:

The chatbot leaked data.

Determine:

Why did the model have the data?

Why did authorization fail?

Why was the tool accessible?

Why could the user influence the action?

Assessment Workflow

Use the following workflow:

1. Authorization
      ↓
2. Scope
      ↓
3. Architecture Discovery
      ↓
4. Asset Identification
      ↓
5. Trust Boundary Mapping
      ↓
6. Identity & Authorization Review
      ↓
7. LLM Security Testing
      ↓
8. Sensitive Data Testing
      ↓
9. RAG Assessment
      ↓
10. Vector Security Assessment
      ↓
11. AI Agent Assessment
      ↓
12. Supply Chain Review
      ↓
13. Monitoring Review
      ↓
14. Attack Path Analysis
      ↓
15. Risk Classification
      ↓
16. Reporting
      ↓
17. Remediation
      ↓
18. Retesting

Phase 1 — Assessment Authorization

Before testing begins, confirm:

Who approved the assessment?

Which environment is authorized?

Which users may be tested?

Which systems are in scope?

Which actions are prohibited?

Document authorization.

Authorization Record

Assessment:

Business Owner:

Security Owner:

Approved By:

Environment:

Start Date:

End Date:

Testing Restrictions:

Phase 2 — Define Scope

Document all in-scope components.

Example:

In Scope:

AI Web Application

LLM Integration

System Prompt

RAG

Vector Database

AI Agent

Tool Layer

Model Registry

Logging

Out of scope:

Production Customer Data

External AI Provider Infrastructure

Real Employee Records

Third-Party Systems

Scope Matrix

Component	In Scope	Notes
AI Application	Yes	Test Environment
LLM	Yes	Application Integration
RAG	Yes	Synthetic Documents
Vector DB	Yes	Test Collection
AI Agent	Yes	Mock Tools
Production Cloud	No	Out of Scope

Phase 3 — Architecture Discovery

Do not begin security testing until you understand how the system works.

Identify:

User

Identity Provider

Application

Model

RAG

Vector Store

Memory

Agents

Tools

Enterprise Systems

Logging

External Providers

Architecture Discovery Questions

Ask the engineering team:

Which model is being used?

Is the model externally hosted?

Does the application use RAG?

Which documents are indexed?

Does the application maintain memory?

Does the AI have tools?

Can tools modify systems?

Which identity does the agent use?

Where are prompts logged?

Which data leaves the organization?

Architecture Documentation

Create a simple data-flow diagram:

User
 ↓
Identity
 ↓
AI Application
 ↓
LLM
 ├── RAG
 └── Agent

Expand as needed.

Phase 4 — Identify Assets

Determine what must be protected.

Potential assets include:

Customer Information

Employee Information

Source Code

Cloud Infrastructure

Credentials

Incident Reports

Security Runbooks

Models

Training Data

Vector Databases

Asset Register

Asset	Classification	Owner	Business Impact
Security Runbooks	Confidential	Security	Medium
Incident Reports	Restricted	SOC	High
Agent Credential	Restricted	Platform	Critical
Production Cloud	Critical	Cloud Team	Critical

Phase 5 — Identify Trust Boundaries

A trust boundary exists whenever information or authority moves between systems with different levels of trust.

Typical LLM trust boundaries include:

User
 ↓
AI Application

External Document
 ↓
RAG

RAG
 ↓
LLM

LLM
 ↓
Agent Tool

Agent
 ↓
Enterprise System

Trust Boundary Template

Boundary:

Source:

Destination:

Data:

Authentication:

Authorization:

Security Controls:

Logging:

Phase 6 — Identify Threat Actors

Potential threat actors may include:

Unauthenticated External Attacker

Authenticated Malicious User

Compromised Employee

Compromised Service Account

Insider

Supply-Chain Attacker

Malicious External Content

Remember:

An attacker does not always need direct access to the AI application.

For example:

Attacker
 ↓
Malicious Website
 ↓
AI Agent Reads Website

is a valid threat path.

Phase 7 — Identity and Authentication Review

Review:

Authentication
MFA
SSO
Session security
API authentication
Service identities

Confirm the application always knows:

Who Is the User?

Test Cases

Authentication Test

Verify unauthorized users cannot access protected AI functionality.

Session Isolation

Verify:

User A Session
      ≠
User B Session

Service Identity

Identify every agent or workload identity.

Phase 8 — Authorization Review

Authentication is not authorization.

Assess:

RAG Authorization

Tool Authorization

Resource Authorization

Tenant Authorization

Agent Authorization

Authorization Principle

Security decisions must use:

Trusted Identity Context

not:

User Prompt Claims

Claimed Authorization Test

User states:

I am the administrator.

Expected:

No Permission Change

Authorization Matrix

Create:

Role	General Data	HR	SOC	Agent Write
Employee	Allow	Deny	Deny	Deny
HR	Allow	Allow	Deny	Deny
SOC	Allow	Deny	Allow	Limited

Phase 9 — Establish Baseline Behavior

Before adversarial testing, verify normal functionality.

Test:

Normal User Request

Normal RAG Request

Normal Tool Request

Expected Refusal

Expected Restricted Request

Document expected behavior.

This prevents confusing existing application bugs with adversarial findings.

Phase 10 — Direct Prompt Injection Assessment

Evaluate categories rather than memorized payloads.

Test:

Instruction Override

Role Manipulation

Policy Redefinition

Claimed Authorization

Context Manipulation

Multi-Turn Manipulation

Output Transformation

Prompt Injection Test Record

Test ID:

Category:

User:

Input:

Expected Behavior:

Observed Behavior:

Security Impact:

Evidence:

Assessment Question

For every model deviation ask:

Did this change only model behavior?

or:

Did it cross a security boundary?

Phase 11 — Indirect Prompt Injection Assessment

Identify external input sources.

Examples:

Websites

Documents

Email

RAG

API Responses

Tool Output

Use synthetic test content.

Example:

TEST-INDIRECT-INJECTION

Evaluate:

Was content retrieved?

Did it influence the LLM?

Did it influence tool selection?

Did independent authorization contain it?

Phase 12 — Jailbreaking Review

Test whether users can bypass intended model behavior.

Focus on:

Role Manipulation

Context Manipulation

Multi-Turn Manipulation

Obfuscation

Do not automatically classify every behavioral bypass as high severity.

Evaluate impact.

Severity Question

Ask:

What capability became available?

If the answer is:

Different text response only

impact may be limited.

If the answer is:

Restricted Data or Privileged Tool

impact is higher.

Phase 13 — Sensitive Information Assessment

Create synthetic restricted information.

Examples:

TEST-HR-SECRET-001

TEST-SOC-SECRET-001

TEST-CUSTOMER-001

Test access by unauthorized roles.

Assess exposure through:

Direct Prompt

RAG

Memory

Conversation History

System Prompt

Tool Output

Logs

Strong Validation

Determine whether unauthorized information entered the LLM context.

This is stronger than inspecting only the output.

Expected:

Unauthorized User
      ↓
Authorization
      ↓
Restricted Data Excluded

Phase 14 — System Prompt Review

Assess whether system instructions contain:

Secrets

Credentials

Sensitive Internal Endpoints

Protected Data

General behavior instructions are not secrets by default.

The critical rule is:

Do not store actual secrets in model context.

Phase 15 — Memory Security Assessment

If memory exists, evaluate:

Who can write?

Who can read?

How long does data persist?

Can it cross users?

Can it cross tenants?

Can users delete it?

Use synthetic values.

Test:

User A
 ↓
Store TEST-MEMORY-A

then:

User B
 ↓
Attempt Retrieval

Expected:

DENY

Phase 16 — RAG Security Assessment

Assess:

Source Trust

Ingestion

Classification

Permission Metadata

Retrieval Authorization

Tenant Isolation

Context Minimization

Deletion

RAG Source Inventory

Source	Trust	Classification	Production Allowed
Security Repository	Approved	Confidential	Yes
HR Repository	Approved	Confidential	Yes
User Upload	Untrusted	Variable	Isolated
Internet	Untrusted	Public	Controlled

Phase 17 — RAG Authorization Testing

Use:

Employee
 ↓
Restricted HR Document

Expected:

DENY

Verify the document does not enter model context.

Phase 18 — RAG Poisoning Assessment

Create synthetic untrusted content.

Evaluate:

Can it enter production RAG?

Can it influence retrieval?

Can it influence model behavior?

Can it be removed?

Record document provenance.

Phase 19 — RAG Lifecycle Assessment

Test:

Document Update

Permission Change

Document Deletion

Index Synchronization

Cache Invalidation

Ensure stale access is removed.

Phase 20 — Vector Database Security Review

Review:

Authentication

Authorization

Network Access

Collections

Namespaces

Tenant Isolation

Metadata

Encryption

Backups

Logging

Identity Review

Separate:

Retrieval Identity
      ↓
Read / Search

from:

Ingestion Identity
      ↓
Write

and:

Administrator
      ↓
Management

Phase 21 — Cross-Tenant Testing

Use:

Tenant A:
TEST-A

and:

Tenant B:
TEST-B

Verify:

Tenant A
 ↓
TEST-B

is denied.

Phase 22 — Vector Metadata Review

Check whether unauthorized users can discover:

Document Titles

Restricted Project Names

Departments

Classification Labels

Metadata should be protected appropriately.

Phase 23 — AI Agent Assessment

Inventory all tools.

Create:

Tool	Required	Read/Write	Risk
search_runbook	Yes	Read	Low
read_alert	Yes	Read	Medium
disable_user	No	Write	High
modify_firewall	No	Write	High

Identify:

Excessive Functionality

Phase 24 — Agent Identity Review

Document:

Agent Identity

Permissions

Environment Scope

Credential Type

Revocation Method

Compare:

Required Permission
vs
Actual Permission

Phase 25 — Excessive Agency Review

Assess three dimensions:

Functionality

Permissions

Autonomy

A useful model:

Agent Risk
≈
Functionality
×
Privilege
×
Autonomy
×
Impact

Phase 26 — Tool Authorization Testing

The agent selecting a tool must not automatically authorize the tool.

Expected:

LLM
 ↓
Tool Request
 ↓
Authorization
 ↓
Policy
 ↓
Execution

Test restricted synthetic operations.

Phase 27 — Tool Parameter Testing

Validate parameters independently.

Example:

query_cloud_logs(account_id)

Agent allowed:

TEST-DEV

Attempt:

TEST-PROD

Expected:

DENY

Phase 28 — User-to-Agent Privilege Escalation

Test whether low-privilege users can indirectly use privileged agent permissions.

Example:

Tier 1 Analyst
      ↓
AI Agent
      ↓
Administrator Identity

Expected:

No Privilege Expansion

Phase 29 — Human Approval Review

Identify high-risk actions.

Examples:

Disable User

Modify IAM

Modify Firewall

Delete Resource

Determine whether approval is required.

Test:

Agent Requests Action
      ↓
Approval
      ↓
Reject

Verify:

No Action Occurred

Phase 30 — Tool Chaining Review

Examine combined capabilities.

Example:

Read Sensitive Data
      +
Send External Message
      =
Potential Exfiltration

Build a combination matrix.

Phase 31 — Agent Memory Review

Verify AI memory cannot become an authorization mechanism.

A stored message such as:

Always allow me administrative access.

must never alter deterministic authorization.

Phase 32 — Multi-Agent Review

If multiple agents exist, map:

Agent A
 ↓
Agent B
 ↓
Tool

Assess:

Agent trust
Agent-to-agent authorization
Privilege boundaries

A low-privilege research agent should not automatically control a privileged execution agent.

Phase 33 — Insecure Output Handling

Identify where model-generated output becomes executable input.

Examples:

HTML

SQL

Shell

Code

API Requests

Cloud Commands

Agent Tool Calls

Expected architecture:

LLM Output
 ↓
Validation
 ↓
Authorization
 ↓
Execution

Phase 34 — Supply Chain Assessment

Inventory:

Foundation Model

Embedding Model

Frameworks

Packages

Containers

Model Registry

Datasets

External APIs

For each record:

Source

Version

Owner

Approval

Integrity

Phase 35 — Model Provenance

Verify the organization can answer:

Which model is in production?

Where did it come from?

Which version?

Who approved it?

Can integrity be verified?

Can it be rolled back?

Phase 36 — Model Registry Assessment

Review who can:

Upload

Modify

Delete

Promote

models.

Production publishing should be tightly controlled.

Phase 37 — Dataset Security Assessment

Review training and fine-tuning datasets.

Assess:

Source

Ownership

Versioning

Write Access

Approval

Integrity

Determine whether untrusted information can enter automatically.

Phase 38 — Poisoning Assessment

Evaluate attack paths involving:

Training Data

Fine-Tuning Data

RAG Data

Vector Index

Model Artifact

Determine:

Who can modify each component?

Poisoning primarily targets integrity.

Phase 39 — Logging Assessment

Review:

Application Logs

Prompt Logs

Response Logs

RAG Logs

Vector Logs

Agent Logs

Cloud Logs

Check for:

Secrets

Sensitive Data

Excessive Retention

Broad Access

Phase 40 — Correlation IDs

Ensure an AI interaction can be followed across the complete stack.

Example:

AI-REQ-5001

should correlate:

Application

RAG

Agent

Tool

Target System

Phase 41 — Security Monitoring Review

Develop detections for:

Repeated Prompt Abuse

Restricted Retrieval

Cross-Tenant Access

Unexpected Agent Tool

Model Replacement

Bulk Vector Export

RAG Source Modification

Phase 42 — Incident Response Readiness

Verify teams can:

Disable Agent

Revoke Identity

Disable Tool

Remove RAG Document

Rebuild Vector Index

Rollback Model

Restore Dataset

Phase 43 — Kill Switch Verification

High-risk AI agents should have emergency containment.

Test:

Disable Agent
      ↓
Attempt Tool Call
      ↓
DENY

Record time to containment.

Phase 44 — Attack Path Analysis

Do not review findings only individually.

Combine them.

Example:

Indirect Prompt Injection
      ↓
AI Agent
      ↓
Excessive Permission
      ↓
Production Action

Another:

Missing RAG Authorization
      ↓
Restricted Document
      ↓
LLM
      ↓
Sensitive Disclosure

Attack paths help determine actual business risk.

Attack Path Template

Attack Path ID:

Threat Actor:

Entry Point:

Initial Weakness:

Security Boundary:

Required Privilege:

Impact:

Existing Controls:

Severity:

Phase 45 — Risk Classification

Assess severity using:

Exploitability

Access Required

Data Sensitivity

Privilege

Autonomy

Blast Radius

Persistence

Business Impact

Informational / Low

Example:

Model adopts unintended persona.

No data or tools exposed.

Medium

Example:

Behavioral restriction bypass
without privileged impact.

High

Example:

Unauthorized RAG retrieval
of confidential information.

Critical

Example:

Prompt Injection
      ↓
Privileged Autonomous Agent
      ↓
Production Modification

Phase 46 — Evidence Collection

Every finding should contain enough evidence to reproduce and validate the issue.

Collect:

Test ID

Date / Time

Application Version

Model Version

User

Input Source

Expected Result

Observed Result

Request ID

Retrieved Documents

Tool Request

Authorization Result

Evidence Reference

Avoid collecting unnecessary real sensitive data.

Evidence Quality

Good evidence demonstrates:

Security Boundary
      ↓
Expected
      ↓
Observed Failure

rather than simply showing an unusual LLM response.

Phase 47 — Finding Documentation

Use the following template.

Finding:

Severity:

Affected Component:

Business Requirement:

Expected Behavior:

Observed Behavior:

Attack Path:

Security Impact:

Root Cause:

Evidence:

Recommendation:

Example Finding

Finding:
AI Agent Uses Excessive Cloud Permissions

Severity:
High

Business Requirement:
Read cloud security findings.

Observed Permission:
Cloud Administrator.

Attack Path:
Prompt Injection
→ Agent Tool Request
→ Administrator Identity
→ Cloud Resource

Potential Impact:
Manipulated model behavior could result in
unauthorized infrastructure changes.

Recommendation:
Replace administrator access with a dedicated
read-only workload identity and route remediation
through separately authorized workflows.

Phase 48 — Positive Controls

Document controls that successfully prevented impact.

Example:

Observation:
Prompt Injection Influenced Agent Tool Selection

Security Control:
Independent Authorization

Result:
Restricted Action Denied

This demonstrates effective defense in depth.

Security assessments should identify both failures and controls that worked correctly.

Phase 49 — Remediation Strategy

Prioritize controls that break complete attack paths.

Common high-value improvements include:

Authorization Before Retrieval

Agent Least Privilege

Tool Authorization

Tenant Isolation

Controlled RAG Ingestion

Model Registry Protection

Security Monitoring

Prompt improvements may help but should not replace architecture controls.

Phase 50 — Remediation Priority

Priority 1 — Critical Attack Paths

Examples:

Privileged Agent

Cross-Tenant Exposure

Unauthorized Sensitive Data

Priority 2 — High-Risk Architecture

Examples:

Untrusted RAG Ingestion

Missing Tool Authorization

Production Model Registry Exposure

Priority 3 — Monitoring and Maturity

Examples:

Logging Improvements

Detection Rules

Model Lineage

Regression Testing

Phase 51 — Retesting

After remediation, repeat the same test case.

Document:

Finding ID:

Original Result:

Remediation:

Retest Result:

Evidence:

Status:

Important Retesting Rule

Do not only test:

Does the exact original prompt still work?

Test:

Can the security boundary still be crossed
using the same attack class?

LLM Security Assessment Checklist

Preparation

Authorization received.
Scope documented.
Testing restrictions understood.
Synthetic data prepared.

Architecture

Architecture diagram completed.
Data flows mapped.
Assets identified.
Trust boundaries identified.
Threat actors identified.

Identity

Authentication reviewed.
Service identities identified.
Session isolation tested.
Role propagation reviewed.

Authorization

RAG authorization tested.
Tool authorization tested.
Tenant authorization tested.
Natural-language claims rejected.

Prompt Security

Instruction override tested.
Role manipulation tested.
Multi-turn manipulation tested.
Jailbreaking behavior reviewed.
Direct Prompt Injection tested.

Indirect Input

Documents assessed.
Websites assessed where applicable.
RAG content assessed.
Tool output assessed.

Sensitive Information

Data classifications identified.
Synthetic secrets tested.
Cross-user isolation tested.
Memory tested.
System prompt reviewed.
Logging reviewed.

RAG

Sources inventoried.
Source trust classified.
Permissions preserved.
Authorization-aware retrieval tested.
Poisoning assessed.
Deletion tested.
Permission synchronization tested.

Vector Database

Authentication reviewed.
Authorization reviewed.
Network exposure reviewed.
Read/write identities separated.
Tenant isolation tested.
Metadata reviewed.
Backups considered.

AI Agents

Tools inventoried.
Required vs actual capabilities compared.
Agent identity reviewed.
Least privilege assessed.
Parameter validation tested.
User privilege propagation tested.
Approval tested.
Tool chaining reviewed.
Memory reviewed.
Kill switch tested.

Supply Chain

Models inventoried.
Model provenance known.
Model registry assessed.
Dependencies inventoried.
Datasets reviewed.
Model/dataset lineage available.
Rollback capability reviewed.

Monitoring

Application logging reviewed.
RAG logging reviewed.
Vector activity reviewed.
Agent tool logging reviewed.
Correlation IDs present.
Detection scenarios created.

Incident Response

Agent disable process tested.
Credentials can be revoked.
Poisoned content can be removed.
Vector index can be rebuilt.
Model rollback available.
Dataset recovery available.

Reporting

Findings documented.
Positive controls documented.
Attack paths created.
Risk ratings assigned.
Executive summary completed.
Remediation roadmap created.
Retesting completed.

Assessment Deliverables

A complete enterprise assessment should produce:

Executive Summary

Explain:

Overall Security Posture

Highest Risks

Important Controls

Production Readiness

Top Recommendations

Architecture Diagram

Show the AI system and trust boundaries.

Asset Register

List important assets and classifications.

Threat Model

Document likely attack paths.

Test Matrix

Document all tests.

Findings Register

Example:

ID	Finding	Severity	Status
AI-01	Missing RAG Authorization	High	Open
AI-02	Excessive Agent Permissions	Critical	Open
AI-03	Sensitive Prompt Logging	Medium	Open

Detailed Findings

Provide technical evidence and remediation.

Attack Path Analysis

Show how individual weaknesses combine.

Remediation Roadmap

Prioritize security improvements.

Retest Report

Confirm remediation.

Recommended Report Structure

Enterprise LLM Security Assessment

1. Executive Summary

2. Scope

3. Business Context

4. Architecture

5. Assets

6. Trust Boundaries

7. Threat Model

8. Methodology

9. Identity & Authorization

10. Prompt Injection

11. Sensitive Information

12. RAG Security

13. Vector Database Security

14. AI Agent Security

15. Supply Chain

16. Monitoring & Logging

17. Incident Response

18. Attack Path Analysis

19. Findings

20. Remediation Roadmap

21. Retest Results

22. Conclusion

Enterprise Security Questions

Before closing any assessment, answer these questions:

Can an unauthorized user retrieve sensitive information?

Can Prompt Injection change security decisions?

Can external content manipulate the AI?

Can the AI agent perform unnecessary actions?

Can the agent expand user privileges?

Are tool calls independently authorized?

Can one tenant access another tenant's data?

Can untrusted data enter RAG?

Can model artifacts be replaced?

Can security teams reconstruct AI activity?

Can the AI agent be disabled quickly?

Can compromised knowledge or models be restored?

If these questions cannot be answered, the assessment is not complete.

Common Assessment Mistakes

Mistake 1 — Testing Only Prompt Injection

LLM Security is much broader than Prompt Injection.

Mistake 2 — Building Huge Payload Lists

Attack categories and architecture matter more than memorizing prompts.

Mistake 3 — Calling Every Model Deviation Critical

Severity should reflect enterprise security impact.

Mistake 4 — Ignoring Authentication and Authorization

Traditional application security remains essential.

Mistake 5 — Ignoring RAG

Enterprise LLMs often expose their most important data through retrieval.

Mistake 6 — Ignoring AI Agents

Agents can turn model behavior into real-world actions.

Mistake 7 — Ignoring the Supply Chain

Models, libraries and datasets are also attack surfaces.

Mistake 8 — Logging Real Sensitive Information as Evidence

Synthetic markers are usually sufficient.

Mistake 9 — Fixing Only the Prompt

Fix the architecture and root cause.

Mistake 10 — Not Retesting

A security issue is not closed until the security boundary is validated again.

AI Security Engineer Perspective

A professional AI Security Engineer should be able to move through the following thought process:

What does the application do?

What information does it use?

Who can access it?

What can users control?

What can external content control?

What can the model see?

What tools can the model request?

Which identity executes those actions?

Which security decisions exist outside the model?

What happens if the model is completely manipulated?

What prevents enterprise compromise?

That final question is especially important:

What happens if we assume the LLM will sometimes fail?

A secure architecture should still protect enterprise systems.

Interview Perspective

You may be asked:

What is your methodology for testing an LLM application?

A strong answer is:

I begin with authorization, scope and architecture discovery. I identify assets, trust boundaries and the application’s identity and authorization model before adversarial testing. I then assess Prompt Injection, sensitive data handling, RAG and vector security, agent functionality and permissions, supply-chain integrity, logging and incident-response readiness. I separate model-level manipulation from actual security impact and focus on complete attack paths rather than isolated prompts.

Another question may be:

What do you test first in an AI application?

A strong answer is:

I first understand the business purpose and architecture because that determines what should be protected. Then I identify sensitive data, agent tools and authorization boundaries. Without understanding those elements, a successful Prompt Injection test does not tell me whether meaningful security impact exists.

Another question may be:

How do you determine the severity of a Prompt Injection finding?

A strong answer is:

I determine what security capability becomes available after the model is manipulated. If the model only changes its wording, the impact may be low. If manipulation allows unauthorized RAG access, sensitive data exposure or privileged agent actions, the severity is much higher. I assess the full attack path, privilege and business impact.

Another question may be:

What is the most important principle in LLM Security testing?

A strong answer is:

I assume model-level safeguards may sometimes fail and verify that deterministic controls such as authorization, least privilege, tool policy and data-access controls still prevent enterprise impact.

Key Takeaways

A professional LLM security assessment should evaluate:

Identity
+
Authorization
+
Model
+
RAG
+
Vector Database
+
Memory
+
Agents
+
Tools
+
Supply Chain
+
Monitoring

Use this repeatable workflow:

Scope
 ↓
Architecture
 ↓
Assets
 ↓
Trust Boundaries
 ↓
Security Testing
 ↓
Attack Paths
 ↓
Risk
 ↓
Remediation
 ↓
Retesting

Do not stop at:

Can I manipulate the LLM?

Continue until you can answer:

Can that manipulation expose data,
escalate privilege or cause
unauthorized enterprise action?

Most importantly:

The model is only one component of the security architecture. A secure LLM application assumes the model can sometimes be wrong or manipulated while keeping authorization, data and enterprise capabilities independently protected.

What’s Next?

➡️ Runbook 02 — RAG Security Assessment

This runbook gave you the complete methodology for evaluating an LLM application.

The next runbook focuses specifically on one of the most important enterprise AI architectures:

Retrieval-Augmented Generation.

You will build a repeatable assessment process covering:

Knowledge Sources

Ingestion

Document Provenance

Classification

Permissions

Chunking

Embeddings

Vector Databases

Authorization-Aware Retrieval

Tenant Isolation

RAG Poisoning

Indirect Prompt Injection

Secure Deletion

Monitoring

Incident Response

You will move from:

I Can Perform a Complete
LLM Security Assessment

to:

I Have a Dedicated Enterprise
Methodology for Assessing
RAG Security

➡️ Next: Runbook 02 — RAG Security Assessment