Lesson 09 — Incident Detection

Learning Objectives

By the end of this lesson, you will be able to:

Understand Kubernetes Incident Detection
Learn the stages of incident detection
Explore enterprise detection architecture
Understand Indicators of Compromise (IoCs)
Learn how SOC teams validate incidents
Build enterprise detection workflows
Apply incident detection best practices

Why This Matters

Cyber attacks rarely begin with a single obvious event.

Instead, attackers typically perform multiple actions over time.

For example:

Reconnaissance
Credential theft
Privilege escalation
Persistence
Lateral movement
Data access
Data exfiltration

Each step generates security events.

The objective of Incident Detection is to identify these activities as early as possible before significant damage occurs.

Early detection reduces:

Financial impact
Operational disruption
Recovery time
Regulatory penalties
Customer impact

What is Incident Detection?

Incident Detection is the process of identifying malicious, suspicious or unauthorized activity occurring within Kubernetes environments.

It involves:

Collecting telemetry
Correlating events
Applying detection rules
Validating alerts
Prioritizing incidents
Escalating to responders

Incident Detection is one of the primary responsibilities of a Security Operations Centre (SOC).

Incident Detection Lifecycle

Security Event

↓

Telemetry Collection

↓

Correlation

↓

Threat Detection

↓

Incident Validation

↓

Severity Assessment

↓

SOC Investigation

↓

Incident Response

Each stage improves confidence before security teams begin response activities.

Security Events vs Security Incidents

Many security events occur every day.

Not every event becomes an incident.

Security Event	Security Incident
Failed login	Credential attack
Pod restart	Malware execution
New Deployment	Unauthorized deployment
High CPU usage	Cryptocurrency mining
Secret access	Secret theft

Incident detection separates normal operational events from genuine security threats.

Sources of Incident Detection

Enterprise detection relies on multiple data sources.

Applications

↓

Audit Logs

↓

Runtime Detection

↓

Metrics

↓

Cloud Logs

↓

Threat Intelligence

↓

SIEM

↓

Incident Detection

Combining telemetry improves detection accuracy.

Indicators of Compromise (IoCs)

Indicators of Compromise are signs that a system may have been compromised.

Examples include:

Suspicious IP addresses
Malware hashes
Unexpected processes
Reverse shells
Interactive container sessions
Privileged container execution
Credential misuse
Unexpected outbound traffic
Unauthorized Secret access
Container escape attempts

IoCs help analysts quickly identify malicious activity.

Indicators of Attack (IoAs)

Unlike IoCs, Indicators of Attack focus on attacker behaviour.

Examples include:

Repeated authentication failures
Multiple RBAC changes
Namespace enumeration
Pod discovery
Privilege escalation attempts
Persistent API requests
Multiple failed kubectl exec attempts
Unusual Service Account usage

Behaviour-based detection is especially valuable against unknown threats.

MITRE ATT&CK Mapping

Enterprise SOC teams often map Kubernetes detections to the MITRE ATT&CK Framework.

Examples include:

Kubernetes Activity	MITRE ATT&CK Technique
Credential Theft	Credential Access
Privilege Escalation	Privilege Escalation
Container Escape	Defence Evasion
Secret Access	Credential Access
API Discovery	Discovery
Data Exfiltration	Exfiltration

This helps analysts understand attacker tactics and improve detection coverage.

Detection Methods

Security teams use multiple detection techniques.

Method	Purpose
Signature-based	Detect known threats
Behaviour-based	Detect suspicious activity
Rule-based	Identify predefined conditions
Threat Intelligence	Detect known malicious indicators
Anomaly Detection	Identify unusual behaviour
Machine Learning	Detect unknown attack patterns

Modern SOCs combine several methods for stronger detection.

Kubernetes Incident Detection Architecture

Amazon EKS

↓

Audit Logs

↓

Falco

↓

Prometheus

↓

CloudWatch

↓

CloudTrail

↓

GuardDuty

↓

Inspector

↓

Enterprise SIEM

↓

Incident Detection Engine

↓

Security Operations Centre (SOC)

Each component contributes valuable information during incident detection.

Alert Validation

Every alert should be validated before escalation.

SOC analysts ask:

Is this activity expected?
Is it a false positive?
Which workload is affected?
Which user performed the action?
Is production impacted?
Has this behaviour occurred before?

Validation prevents unnecessary investigations.

Incident Prioritisation

SOC teams prioritise incidents based on risk.

Severity	Example
Critical	Container escape, ransomware
High	Privilege escalation, Secret theft
Medium	Suspicious API activity
Low	Failed login attempts
Informational	Normal deployment activity

Prioritisation ensures that the most serious threats receive immediate attention.

Event Correlation

Multiple low-risk events may together indicate a serious attack.

Failed Login

+

Service Account Creation

+

RBAC Change

+

Interactive Shell

+

Outbound Connection

↓

Critical Incident

Correlation reduces false positives and increases detection confidence.

Threat Intelligence Integration

Threat intelligence improves detection by providing context.

Examples include:

Known malicious IP addresses
Malicious domains
File hashes
Exploit indicators
Emerging attack techniques
Adversary tactics

Threat intelligence enables faster identification of known threats.

Amazon EKS Incident Detection Workflow

Amazon EKS

↓

Runtime Activity

↓

Falco

↓

CloudWatch

↓

Security Hub

↓

Enterprise SIEM

↓

Detection Rules

↓

Incident Created

↓

SOC Investigation

This workflow provides automated detection and rapid escalation.

Enterprise Example

A multinational airline operates hundreds of Amazon EKS clusters supporting online booking services.

The following sequence occurs:

Kubernetes Audit Logs record multiple failed administrator logins.
Falco detects an interactive shell inside a container.
Prometheus reports an unusual increase in CPU utilisation.
GuardDuty identifies communication with a known malicious IP address.
CloudTrail records suspicious IAM role activity.

The SIEM correlates these events into a single Critical Incident.

The SOC validates the alert, confirms malicious activity and immediately initiates incident response.

The compromised Pod is isolated before attackers can access customer booking data.

Common Detection Challenges

Cloud Security Engineers frequently encounter:

Excessive false positives
Missing telemetry sources
Weak correlation rules
Alert fatigue
Delayed event ingestion
Incomplete threat intelligence
Poor prioritisation
Missing detection coverage
Insufficient log retention
Inconsistent monitoring

Continuous improvement is necessary to maintain effective detection capabilities.

Enterprise Monitoring

SOC teams continuously monitor:

Authentication failures
Privileged access
Runtime alerts
Secret access
RBAC modifications
Container escapes
API activity
Network anomalies
CloudTrail events
Threat intelligence matches

Continuous monitoring enables rapid incident detection.

Enterprise Implementation Strategy

Step 1

↓

Enable Audit Logging

↓

Step 2

↓

Deploy Runtime Detection

↓

Step 3

↓

Collect Metrics

↓

Step 4

↓

Enable AWS Security Services

↓

Step 5

↓

Forward Events to SIEM

↓

Step 6

↓

Create Detection Rules

↓

Step 7

↓

Validate Alerts

↓

Step 8

↓

Continuously Improve Detection

Detection capabilities should evolve alongside the threat landscape.

Enterprise Detection Architecture

Amazon EKS

↓

Applications

↓

Audit Logs

↓

Falco

↓

Prometheus

↓

CloudWatch

↓

CloudTrail

↓

AWS Security Hub

↓

Amazon GuardDuty

↓

Amazon Inspector

↓

Enterprise SIEM

↓

Threat Intelligence

↓

Security Operations Centre (SOC)

This layered architecture provides comprehensive detection across Kubernetes and AWS.

Best Practices

As a Kubernetes Security Engineer:

Collect telemetry from every production cluster.
Correlate events from multiple security tools.
Prioritise incidents based on business impact.
Reduce false positives through continuous rule tuning.
Use threat intelligence to enrich detections.
Monitor privileged operations closely.
Validate alerts before escalation.
Regularly test detection rules using security exercises.
Map detections to the MITRE ATT&CK Framework.
Continuously review and improve detection coverage.

Effective incident detection depends on visibility, context and continuous refinement.

Real-World Scenario

A global healthcare provider hosts electronic medical record systems on Amazon EKS.

Monitoring systems detect:

A new privileged ClusterRoleBinding
Multiple failed administrator logins
Falco alerts for shell execution
Access to Kubernetes Secrets
Outbound communication with a known malicious IP address

The SIEM correlates these events into a Critical Incident.

Threat intelligence confirms the IP address is associated with ransomware operators.

The SOC immediately:

Isolates the affected namespace
Revokes compromised credentials
Preserves forensic evidence
Initiates the incident response plan
Restores workloads from trusted images

Rapid incident detection prevents unauthorized access to sensitive patient records.

Key Takeaways

After completing this lesson, you should understand:

What Incident Detection is
Security events versus security incidents
Indicators of Compromise (IoCs) and Indicators of Attack (IoAs)
Detection methods and event correlation
Threat intelligence integration
Amazon EKS incident detection architecture
Enterprise incident prioritisation
Incident detection best practices

Incident Detection is the bridge between monitoring and incident response. By combining Kubernetes Audit Logs, Falco runtime alerts, Prometheus metrics, AWS security services and SIEM correlation, organizations can rapidly identify malicious activity, prioritise threats and protect critical workloads running on Amazon EKS.

Knowledge Check

Question 1

What is the primary purpose of Incident Detection?

A. Build Kubernetes clusters
B. Identify malicious or suspicious activity and determine whether it represents a security incident
C. Schedule Pods
D. Replace Kubernetes Audit Logs

Answer: B

Question 2

What is the difference between a security event and a security incident?

A. They are the same.
B. A security event is any observable activity, while a security incident is a validated event that poses a threat.
C. Incidents occur only in production.
D. Events only occur in AWS.

Answer: B

Question 3

Which of the following is an Indicator of Compromise (IoC)?

A. Suspicious outbound connection to a known malicious IP address
B. Creating a ConfigMap
C. Scaling a Deployment
D. Installing Helm

Answer: A

Question 4

Why is event correlation important in Incident Detection?

A. It reduces storage costs.
B. It combines related events to identify high-confidence security incidents.
C. It replaces runtime monitoring.
D. It automatically patches Kubernetes clusters.

Answer: B

Question 5

Which combination represents enterprise best practice?

A. Collect comprehensive telemetry, correlate events in a SIEM, enrich alerts with threat intelligence, validate incidents before escalation and continuously improve detection rules.
B. Investigate every failed login as a critical incident.
C. Disable runtime monitoring to reduce alerts.
D. Depend on a single log source for all detections.

Answer: A

What’s Next?

In the next lesson, you will learn about Enterprise Security Monitoring, where you’ll explore how large organizations design, operate and continuously improve Security Operations Centres (SOCs) for Kubernetes and Amazon EKS, including enterprise monitoring architecture, governance, detection engineering and operational best practices.

➡️ Next Lesson: Lesson 10 — Enterprise Security Monitoring