05 AI Threat Actors and Attack Surface Identification

Once you understand the architecture, assets, data flows and trust boundaries of an AI system, the next question is:

Who might attack the system, why would they attack it, and where could they interact with it?

This is where threat actors and attack surface identification become important.

Threat modeling is not only about listing technical weaknesses.

It is about understanding:

• Who may attempt to cause harm

• What they are trying to achieve

• What access they already have

• What capabilities they possess

• Which entry points they can reach

• Which assets they may target

• How one weakness could lead to another

An AI Security Engineer should therefore develop the ability to connect:

Threat Actor
     ↓
Motivation
     ↓
Capability
     ↓
Entry Point
     ↓
Attack Surface
     ↓
Trust Boundary
     ↓
Target Asset
     ↓
Business Impact

Learning Objectives

By the end of this lesson, you should be able to:

• Explain what a threat actor is.

• Distinguish threat actors from attack techniques.

• Identify common threat actors affecting AI systems.

• Understand external and insider AI threats.

• Identify compromised identities and workloads.

• Understand supply-chain adversaries.

• Recognize indirect attackers using malicious content.

• Identify LLM attack surfaces.

• Identify RAG attack surfaces.

• Identify AI agent and tool attack surfaces.

• Identify model, dataset and software supply-chain surfaces.

• Map threat actors to assets and trust boundaries.

• Identify AI system entry points.

• Build an AI Attack Surface Register.

• Prioritize high-risk attack surfaces.

What Is a Threat Actor?

A threat actor is a person, group, organization or system capable of causing harm to an asset.

Examples include:

• External attackers

• Malicious insiders

• Compromised employees

• Cybercriminal groups

• Supply-chain attackers

• Competitors

• Nation-state actors

• Hacktivists

• Automated malicious services

A threat actor describes who may cause the threat.

It does not describe the method.

For example:

Threat Actor:
External Attacker

Technique:
Prompt Manipulation

Target:
Enterprise AI Assistant

These are different parts of the threat scenario.

Threat Actor vs Threat vs Technique

This distinction is important.

Threat Actor

Who may cause harm?

Example:

Compromised employee account

Threat

What undesirable event could occur?

Example:

Unauthorized access to confidential RAG data

Technique

How could the attacker attempt it?

Example:

Manipulating prompts and retrieval requests

Impact

What happens if successful?

Example:

Confidential enterprise information is disclosed

Together:

Compromised Employee
        ↓
Prompt / Retrieval Manipulation
        ↓
Unauthorized RAG Access
        ↓
Confidential Data Disclosure

Why Threat Actors Matter

Different attackers have different:

• Access

• Knowledge

• Resources

• Motivation

• Time

• Capabilities

For example, an anonymous external attacker may only access a public chatbot.

An internal developer may have access to:

• Source code

• AI configuration

• RAG ingestion

• Cloud infrastructure

• Model repositories

The attack surface available to these actors is very different.

Threat Actor Profile

For each important threat actor, consider:

Who Are They?

What Do They Want?

What Access Do They Have?

What Knowledge Do They Have?

What Capabilities Do They Have?

Which Entry Points Can They Reach?

Which Assets Could They Target?

This creates a much stronger threat model than simply writing:

Hacker

External Attackers

External attackers have no legitimate internal access at the beginning of the attack.

They may target:

• Public AI applications

• Public APIs

• Login portals

• File upload features

• Internet-facing model endpoints

• Cloud infrastructure

Potential objectives may include:

• Data theft

• Credential theft

• Resource abuse

• Service disruption

• Unauthorized system access

• Manipulation of AI behavior

Example:

External Attacker
      ↓
Public AI Application
      ↓
Prompt / API Abuse
      ↓
Sensitive Information

Anonymous Users

Some AI systems intentionally allow anonymous interaction.

Examples:

• Public chatbots

• Public support assistants

• Marketing assistants

An anonymous user may have limited identity but potentially large interaction volume.

Security concerns include:

• Automated abuse

• Resource exhaustion

• Data extraction attempts

• Prompt manipulation

• Content generation abuse

Controls may include:

• Rate limiting

• Abuse monitoring

• Usage limits

• Output controls

• Network protections

Authenticated External Users

Customer-facing AI systems may require authentication.

Example:

Customer
   ↓
Authentication
   ↓
AI Assistant

Authentication reduces anonymous access but does not remove risk.

A legitimate customer may attempt to:

• Access another customer’s information

• Abuse API functionality

• Manipulate prompts

• Exploit authorization weaknesses

Malicious Insiders

A malicious insider already has legitimate access.

Examples:

• Employee

• Contractor

• Administrator

• Developer

• Support engineer

Possible motives include:

• Financial gain

• Revenge

• Intellectual property theft

• Data theft

• Sabotage

Insiders may know:

• Architecture

• Security controls

• Internal data locations

• Administrative processes

This knowledge can increase their effectiveness.

Compromised Employee Accounts

The employee may not be malicious.

Instead:

Attacker
   ↓
Phishing / Credential Theft
   ↓
Employee Identity
   ↓
AI Application

The AI system sees a legitimate identity.

This is why:

Authentication does not automatically mean the request is trustworthy.

Controls such as:

• MFA

• Behavioral monitoring

• Authorization

• Least privilege

• Session security

remain important.

Privileged Administrators

Administrators may have access to:

• AI configuration

• Cloud infrastructure

• Identity systems

• Model deployment

• Secrets

• Logging

A compromised administrative account can have significantly greater impact.

Example:

Privileged Identity
      ↓
AI Configuration
      ↓
Model / Agent Permissions
      ↓
Enterprise Impact

Privileged access should therefore receive strong monitoring and control.

Developers

Developers may access:

• Source code

• API credentials

• System prompts

• Model configuration

• CI/CD pipelines

• Development data

Developer environments can become significant attack surfaces.

Potential risks include:

• Credential leakage

• Malicious code insertion

• Dependency compromise

• Unauthorized configuration changes

AI and ML Engineers

AI engineers may have access to:

• Models

• Training datasets

• Fine-tuning datasets

• Prompt templates

• Model registries

• AI infrastructure

This access makes their identities high-value targets.

Threat models should therefore consider compromise of trusted technical roles.

Supply-Chain Adversaries

The attacker may never interact directly with your production AI system.

Instead, they may compromise something the system trusts.

Examples:

Software Package
Model
Container Image
Dataset
Plugin
External API

The attack path becomes:

Attacker
   ↓
Trusted Dependency
   ↓
Enterprise AI Environment

This is a supply-chain attack.

Model Supply-Chain Threat Actor

Imagine:

External Model Repository
        ↓
Downloaded Model
        ↓
Enterprise AI Platform

If the model source is compromised, an attacker may influence the enterprise system before deployment.

Questions include:

• Where did the model come from?

• Is provenance known?

• Is integrity validated?

• Was the model approved?

Software Dependency Adversary

AI applications depend heavily on software packages.

Example:

Public Package Repository
        ↓
Python Dependency
        ↓
AI Application

A compromised dependency could lead to:

• Code execution

• Credential theft

• Data theft

• Backdoors

AI security therefore includes traditional software supply-chain security.

Dataset Adversary

Training and RAG systems rely on data.

An attacker may target:

Dataset
   ↓
Model / RAG
   ↓
AI Behavior

Potential objectives include:

• Poisoning

• Bias manipulation

• Incorrect responses

• Backdoor behavior

Malicious Content Authors

This is especially important in AI security.

An attacker may not interact directly with the AI application.

Instead, they create content that the AI later processes.

Example:

Attacker
   ↓
Malicious Web Page
   ↓
AI Browsing Agent
   ↓
LLM Context

or:

Attacker
   ↓
Malicious Document
   ↓
RAG Ingestion
   ↓
LLM

This creates an indirect attack path.

Indirect Attackers

An indirect attacker influences data consumed by the AI.

Potential sources include:

• Websites

• Emails

• Documents

• Support tickets

• Knowledge bases

• Uploaded files

• Code repositories

The attacker may never send a prompt directly.

This is one of the major differences between traditional applications and AI systems.

Example — Malicious Email

Consider:

External Attacker
      ↓
Malicious Email
      ↓
Enterprise Mailbox
      ↓
AI Email Assistant
      ↓
AI Agent
      ↓
Enterprise Tool

The email becomes the attack vector.

The employee may not even recognize that AI-specific instructions are embedded in the message.

Compromised Workloads

Threat actors do not always control human identities.

An application or workload could be compromised.

Example:

Compromised Application
        ↓
Internal AI API
        ↓
AI Platform

Service-to-service authorization remains important.

Do not assume:

Internal service = trusted service

Third-Party Providers

Third-party compromise can affect:

• AI model services

• SaaS platforms

• APIs

• Data providers

• Plugins

• cloud platforms

Organizations should understand the dependencies that exist outside their direct control.

What Is an Attack Surface?

An attack surface is the collection of points where an attacker can interact with, influence or potentially compromise a system.

For example:

AI Application Attack Surface

User Interface
APIs
File Uploads
LLM
RAG
Vector Database
AI Agent
Enterprise Tools
Cloud Infrastructure
Software Supply Chain

The more capabilities and integrations the system has, the larger the potential attack surface.

Entry Point vs Attack Surface

An entry point is a specific location where interaction begins.

Example:

Login Page
API Endpoint
File Upload
Chat Input

The attack surface is broader.

Example:

Public AI Application
   ├── Authentication
   ├── Chat Interface
   ├── API
   ├── Upload
   ├── Model
   └── RAG

Build an Attack Surface Map

Start with the architecture.

Example:

                         Users
                           │
                           ▼
                    Web Application
                           │
                  ┌────────┼────────┐
                  │        │        │
                  ▼        ▼        ▼
                 API    Upload     Auth
                  │
                  ▼
              AI Backend
                  │
         ┌────────┼────────┐
         │        │        │
         ▼        ▼        ▼
        LLM      RAG     AI Agent
                  │        │
                  ▼        ▼
             Vector DB   Tools

Each component may contain attack surfaces.

User Interface Attack Surface

The application interface may expose:

• Chat input

• Search

• Forms

• Uploads

• Login

• Session management

Potential threats include:

• Prompt manipulation

• Authentication abuse

• Session abuse

• Malicious file uploads

• Automated abuse

API Attack Surface

Modern AI systems heavily depend on APIs.

Potential attack surfaces include:

• Public endpoints

• Internal endpoints

• Model APIs

• Tool APIs

• Administrative APIs

Questions include:

• Is authentication required?

• Is authorization enforced?

• Are rate limits configured?

• Are input parameters validated?

• Are administrative APIs exposed?

LLM Attack Surface

The model interaction itself creates attack surfaces.

Examples:

• User prompts

• Conversation context

• System instructions

• Model configuration

• Model output

• External context

Possible threats include:

• Prompt injection

• Information disclosure

• Output manipulation

• Resource abuse

Prompt Surface

Prompt-related input may come from:

User
Uploaded File
RAG
Website
Email
API
Tool Response

All of these may influence model behavior.

Therefore:

The chat box is not the only prompt attack surface.

System Prompt Surface

System instructions may influence:

• Behavior

• Tool usage

• Restrictions

• Workflow logic

Potential concerns include:

• Unauthorized modification

• Exposure

• Unsafe dependency on prompt controls

Security should not rely on secrecy of system prompts alone.

Conversation History Surface

Conversation history may become part of the model context.

Possible concerns include:

• Sensitive information retention

• Cross-user leakage

• Persistent malicious instructions

• Privacy exposure

File Upload Surface

Uploaded files may create:

• Prompt injection opportunities

• Malware risk

• Resource exhaustion

• Sensitive data exposure

Example:

User
 ↓
Uploaded Document
 ↓
Document Parser
 ↓
LLM Context

Each stage should be considered.

RAG Attack Surface

RAG introduces several distinct attack surfaces.

Source Documents
      ↓
Ingestion
      ↓
Embedding
      ↓
Vector Database
      ↓
Retrieval
      ↓
LLM

You should review each stage separately.

RAG Source Surface

Ask:

• Who can add content?

• Can external content enter?

• Can compromised users modify data?

• Is source provenance known?

RAG Ingestion Surface

Potential issues include:

• Unauthorized ingestion

• Malicious content

• Poisoning

• Parser vulnerabilities

• Incorrect classification

Vector Database Surface

Potential attack paths include:

• Unauthorized queries

• Unauthorized writes

• Exposed endpoints

• Weak service identity

• Metadata leakage

Retrieval Surface

Ask:

• Is user authorization enforced?

• Can retrieval cross departments?

• Can sensitive documents be returned?

• Can filters be bypassed?

AI Agent Attack Surface

AI agents significantly increase attack surface because they connect model behavior to real actions.

Example:

AI Agent
   │
   ├── Email
   ├── Database
   ├── Cloud API
   ├── Ticketing
   └── Source Control

Every tool creates another potential attack path.

Agent Tool Surface

For each tool ask:

What Can It Do?

What Identity Does It Use?

What Permissions Does It Have?

What Input Controls the Tool?

Can an Attacker Influence That Input?

Is Execution Logged?

Agent Permission Surface

The potential impact depends greatly on permissions.

Compare:

Agent A
  ↓
Read Public Documentation

with:

Agent B
  ↓
Production Administrator

The second architecture creates a much larger blast radius.

External Tool Surface

Agents may integrate with:

• SaaS applications

• Public APIs

• Plugins

• Browsers

• Search services

Every external service adds:

• Trust dependency

• Data flow

• Credential

• Potential compromise path

Email Attack Surface

AI-enabled email workflows deserve special attention.

External Sender
      ↓
Email Content
      ↓
AI Assistant
      ↓
Agent
      ↓
Enterprise Action

Potential risks include:

• Indirect prompt injection

• Malicious attachments

• Sensitive information exposure

• Unauthorized automated actions

Web Browsing Surface

Browsing agents process uncontrolled external content.

Example:

Internet
   ↓
Website
   ↓
AI Agent

Potential concerns include:

• Manipulated webpage content

• Malicious instructions

• Unsafe links

• Untrusted downloads

Model Attack Surface

If the organization hosts or manages models directly, additional surfaces may include:

• Model repository

• Model registry

• Model files

• Model APIs

• Model deployment pipeline

Potential threats include:

• Model replacement

• Model tampering

• Model theft

• Unauthorized deployment

Model Registry Surface

Consider:

Developer
   ↓
Model Registry
   ↓
Production Deployment

Questions include:

• Who can publish?

• Who can approve?

• Is versioning controlled?

• Is integrity checked?

• Is activity logged?

Training Pipeline Attack Surface

If training models:

Training Data
      ↓
Pipeline
      ↓
Training Environment
      ↓
Model

Potential surfaces include:

• Data ingestion

• Training infrastructure

• Configuration

• Credentials

• Model output

Fine-Tuning Attack Surface

Fine-tuning creates similar concerns:

• Malicious data

• Unauthorized modifications

• Sensitive training information

• Model integrity

Dataset Attack Surface

Data may enter through:

• Internal datasets

• External datasets

• Public sources

• Third parties

• User submissions

Questions include:

• Who controls the data?

• Is provenance known?

• Can it be altered?

• Is sensitive data included?

Software Supply-Chain Attack Surface

AI systems commonly depend on:

• Python packages

• JavaScript packages

• Containers

• Libraries

• ML frameworks

• Plugins

This creates a large traditional software attack surface.

Container Attack Surface

AI workloads may run inside containers.

Potential surfaces include:

• Container images

• Registries

• Runtime privileges

• Mounted secrets

• Network interfaces

Kubernetes Attack Surface

For Kubernetes-based AI workloads:

Internet
 ↓
Ingress
 ↓
Service
 ↓
AI Pod
 ↓
Model / Data

Potential surfaces include:

• Kubernetes API

• RBAC

• Service accounts

• Secrets

• Admission configuration

• Container images

• Network policies

Cloud Attack Surface

Cloud-hosted AI systems may expose:

• IAM

• Storage

• AI services

• APIs

• Networks

• Secrets

• Logs

• Databases

Cloud misconfiguration can create AI security impact without any weakness in the model itself.

Identity Attack Surface

Identity is one of the largest attack surfaces.

Potential targets include:

• Users

• Administrators

• Developers

• Workload identities

• Agent identities

Attack scenarios may involve:

• Credential theft

• Token theft

• Excessive permissions

• Privilege escalation

• Account takeover

Secret Attack Surface

AI environments often contain many credentials.

Examples:

LLM API Key
Cloud Credential
Database Password
Agent Token
OAuth Token

Questions include:

• Where are they stored?

• Who can access them?

• Are they logged?

• Are they hardcoded?

• Can they be rotated?

Logging and Monitoring Surface

Security infrastructure itself can become a target.

An attacker may attempt to:

• Disable logging

• Delete logs

• Avoid detection

• Manipulate telemetry

Therefore logging components should also appear in the threat model.

CI/CD Attack Surface

Deployment pipelines may control:

• Application code

• AI configuration

• Models

• Container images

• Production infrastructure

A compromised pipeline can affect the entire AI platform.

Administrative Interfaces

Administrative interfaces are usually high-value surfaces.

Examples:

• Model configuration console

• Agent configuration

• Cloud administration

• Vector database administration

• Prompt management platform

Restrict and monitor administrative access carefully.

Attack Surface Changes Over Time

The attack surface is not static.

Consider:

Version 1:
Chatbot

Then:

Version 2:
Chatbot + RAG

Then:

Version 3:
Chatbot + RAG + AI Agent

Each stage introduces new attack surfaces.

Threat models must therefore be updated when architecture changes.

Connect Threat Actors to Attack Surfaces

Do not list threat actors and attack surfaces separately.

Connect them.

Example:

Threat Actor	Reachable Surface	Potential Target
Anonymous user	Public chatbot	Model behavior
Customer	RAG retrieval	Customer data
Employee	Internal AI assistant	Internal documents
Developer	CI/CD	AI application
Supply-chain attacker	Dependency	Production workload
Malicious content author	RAG document	AI context
Compromised agent	Enterprise APIs	Production systems

This begins forming realistic threat scenarios.

Connect Attack Surfaces to Assets

Example:

Attack Surface:
RAG Retrieval

        ↓

Asset:
Confidential HR Documents

Now ask:

Who can reach the retrieval surface?

What controls protect it?

What happens if those controls fail?

Connect Attack Surfaces to Trust Boundaries

Example:

User
  ↓
RAG Query
  ↓
========================
Authorization Boundary
========================
  ↓
Vector Database

The attack surface is:

Retrieval interface

The trust boundary is:

User-authorized context to sensitive data

Together they help identify the threat.

Build Threat Actor Profiles

Create a file:

AI Threat Actor Profiles.md

Use:

# AI Threat Actor Profiles

## TA-01 — External Attacker

### Description

### Motivation

### Existing Access

### Technical Capability

### Likely Entry Points

### Target Assets

### Potential Impact

### Relevant Controls

Repeat for major actors.

Build an AI Attack Surface Register

Create:

AI Attack Surface Register.md

Use:

ID	Component	Attack Surface	Exposure	Threat Actors	Assets at Risk	Trust Boundary	Priority
AS-01	Web App	Chat Interface	Public/Internal	User	AI Application	TB-01	Medium
AS-02	RAG	Retrieval API	Internal	Employee	Enterprise Data	TB-03	High
AS-03	AI Agent	Tool Interface	Internal	User/Attacker	Production Systems	TB-04	Critical
AS-04	CI/CD	Deployment Pipeline	Internal	Developer/Supply Chain	Production AI	TB-05	Critical

Attack Surface Exposure Levels

You may classify surfaces as:

Public

Directly reachable from the internet.

Partner

Accessible by trusted external organizations.

Internal

Accessible within the enterprise environment.

Privileged

Restricted to administrative or service identities.

Indirect

Not directly reachable, but attacker-controlled content may influence it.

That last category is especially useful for AI systems.

Direct vs Indirect Attack Surfaces

Direct

The attacker interacts with the system.

Attacker
   ↓
AI Chat Interface

Indirect

The attacker manipulates content the system later consumes.

Attacker
   ↓
Document
   ↓
RAG
   ↓
LLM

AI threat models should explicitly identify both.

Prioritize Attack Surfaces

High-priority attack surfaces often combine:

High Exposure
      +
Sensitive Asset
      +
Weak Trust Boundary
      +
High Privilege

Examples:

• Public AI agent with production access

• RAG retrieval containing restricted data

• AI agent using administrative credentials

• Uncontrolled model deployment pipeline

Attack Surface Risk Questions

For each surface ask:

Who Can Reach It?

How Easily Can They Reach It?

What Input Can They Control?

What Identity Is Used?

What Privilege Exists Behind It?

What Sensitive Assets Are Reachable?

What Security Controls Exist?

What Happens If Those Controls Fail?

How Would We Detect Abuse?

Example — Enterprise Knowledge Assistant

Architecture:

Employee
   ↓
AI Assistant
   ↓
RAG
   ↓
Vector Database
   ↓
Enterprise Documents

Threat actors:

Malicious Employee
Compromised Employee Account
Malicious Document Contributor
Supply-Chain Attacker

Attack surfaces:

Chat Interface
Authentication
RAG Retrieval
Knowledge Ingestion
Vector Database
LLM API

Assets:

Enterprise Documents
Employee Identity
API Credentials
Vector Database
Logs

Now the threat model begins to connect.

Example — AI Cloud Agent

Consider:

Employee
   ↓
AI Agent
   ↓
Cloud API
   ↓
Production Infrastructure

Threat actors:

• Compromised employee

• Malicious insider

• External attacker controlling agent context

• Compromised service

Attack surfaces:

• User prompt

• Agent tool interface

• Agent identity

• Cloud API

• Agent configuration

Assets:

• Production cloud

• Cloud credentials

• Customer data

• IAM configuration

This architecture deserves higher scrutiny because AI behavior can directly affect production.

Example — AI Coding Assistant

Architecture:

Developer
   ↓
AI Coding Assistant
   ↓
Source Repository
   ↓
CI/CD
   ↓
Production

Attack surfaces include:

• Developer prompts

• Repository content

• AI suggestions

• Extensions

• Dependencies

• CI/CD

Target assets include:

• Source code

• Credentials

• Build pipeline

• Production application

Example — AI SOC Assistant

Architecture:

SOC Analyst
   ↓
AI Assistant
   ↓
SIEM
   ↓
Security Logs

Potential attack actors:

• Compromised analyst

• Malicious insider

• External attacker influencing log content

• Supply-chain attacker

Attack surfaces include:

• Analyst prompts

• Security telemetry

• SIEM API

• AI agent tools

Sensitive assets include:

• Incident evidence

• Customer information

• Detection rules

• Security architecture

Attack Surface and Blast Radius

Attack surface answers:

Where can the attacker enter?

Blast radius answers:

How much can the attacker affect after entry?

Example:

Chat Interface
     ↓
AI Agent
     ↓
Read-Only Knowledge

Blast radius may be limited.

Compare:

Chat Interface
     ↓
AI Agent
     ↓
Administrator API

The entry point is similar.

The blast radius is dramatically different.

Reduce Attack Surface

Security architecture should reduce unnecessary exposure.

Examples:

Instead of:

Public Administrative API

prefer:

Private Administrative API

Instead of:

Agent Access to All Tools

prefer:

Agent Access to Required Tools Only

Instead of:

All Enterprise Documents

prefer:

Authorized Data Segment Only

Reduce Blast Radius

Even when attack surface cannot be removed, reduce potential impact using:

• Least privilege

• Data segmentation

• Network segmentation

• Scoped tools

• Read-only access

• Transaction limits

• Human approval

• Rate limits

Monitor High-Risk Attack Surfaces

Attack surfaces should also map to telemetry.

Example:

Surface	Detection Opportunity
Authentication	Failed logins, impossible travel
Chat interface	Abuse patterns
RAG retrieval	Unusual document access
Agent tools	Unexpected tool invocation
API	High request volume
Model registry	Unauthorized model changes
CI/CD	Unapproved deployments

This connects threat modeling with security operations.

Real-World AI Security Engineer Workflow

When reviewing an AI system:

Understand Architecture
        ↓
Identify Actors
        ↓
Identify Threat Actors
        ↓
Identify Entry Points
        ↓
Map Attack Surfaces
        ↓
Identify Assets Behind Each Surface
        ↓
Identify Trust Boundaries
        ↓
Identify Privilege
        ↓
Prioritize Surfaces
        ↓
Create Threat Scenarios

The next lessons will build directly on this process.

Common Threat Actor Mistakes

Using “Hacker” for Everything

Different threat actors have different access and capabilities.

Ignoring Insiders

Internal access may significantly change attack paths.

Ignoring Compromised Accounts

A legitimate identity can still represent attacker activity.

Ignoring Malicious Content

AI systems can be attacked indirectly.

Ignoring Supply Chain

The attacker may compromise something your AI system already trusts.

Common Attack Surface Mistakes

Looking Only at Public Interfaces

Internal and indirect surfaces matter too.

Looking Only at the LLM

RAG, agents, IAM, cloud and CI/CD may create larger risks.

Ignoring Agent Permissions

The same attack surface can have very different impact depending on agent privilege.

Ignoring Development Systems

Compromised development pipelines can become production compromise.

Ignoring Logging Systems

Attackers may target visibility and evidence.

AI Threat Actor Mindset

Whenever you review an architecture ask:

Who Might Want to Abuse This?

What Do They Want?

What Access Do They Already Have?

What Can They Influence?

Where Can They Enter?

Which Trust Boundary Can They Cross?

Which Asset Can They Reach?

What Privilege Exists Behind That Surface?

What Is the Potential Blast Radius?

Knowledge Check

Question 1

What is a threat actor?

Question 2

What is the difference between a threat actor and a technique?

Question 3

Why should compromised employee accounts be considered separately from malicious employees?

Question 4

What is an indirect AI attacker?

Question 5

How can malicious documents become an AI attack vector?

Question 6

What is an attack surface?

Question 7

What is the difference between an entry point and an attack surface?

Question 8

Why does RAG expand the AI attack surface?

Question 9

Why do AI agents significantly increase potential attack surface?

Question 10

Why should CI/CD and model repositories appear in AI threat models?

Question 11

What is the difference between attack surface and blast radius?

Question 12

How can least privilege reduce blast radius?

If you can explain these concepts clearly, you are ready to begin systematically categorizing threats.

Key Takeaways

AI threat actor analysis helps answer:

Who Could Attack?
       +
Why Would They Attack?
       +
What Access Do They Have?

Attack surface analysis helps answer:

Where Can They Interact?
       +
What Can They Influence?
       +
What Assets Are Reachable?

Together:

Threat Actor
      +
Attack Surface
      +
Trust Boundary
      +
Target Asset
      =
Threat Scenario

AI systems require special attention because attackers may interact:

• Directly through prompts and APIs

• Indirectly through documents, email and web content

• Through compromised identities

• Through software and model supply chains

• Through AI agents and tools

• Through underlying cloud infrastructure

The most important principle is:

Do not ask only where an attacker can enter. Ask what authority and sensitive assets exist behind that entry point.

That determines the true security impact.

What’s Next?

➡️ 06 — Applying STRIDE to AI Systems

You now understand:

• Who may attack AI systems.

• What motivates different threat actors.

• Where attackers may directly or indirectly interact with AI.

• How LLM, RAG and agent attack surfaces differ.

• How supply chains expand the attack surface.

• How attack surfaces connect to assets and trust boundaries.

• Why privilege determines potential blast radius.

The next step is to systematically ask:

What kinds of security threats can occur at each component and trust boundary?

In the next lesson, you will apply the STRIDE threat modeling methodology to AI environments.

You will learn how to analyze AI systems for:

• Spoofing

• Tampering

• Repudiation

• Information Disclosure

• Denial of Service

• Elevation of Privilege

You will apply STRIDE to:

• LLM applications

• RAG systems

• Vector databases

• AI agents

• APIs

• Identities

• Data flows

• Cloud infrastructure

The goal is to move from:

“I know who can attack and where they can enter.”

to:

“I can systematically identify the categories of threats that may occur across every important part of the AI architecture.”

➡️ Next: 06 — Applying STRIDE to AI Systems