Runbook 02 — Enterprise Kubernetes Environment Assessment

Runbook Information

Item	Details
Runbook ID	K8S-RB-02
Category	Enterprise Security Assessment
Difficulty	Advanced
Estimated Time	2–4 Hours
Environment	Amazon EKS / Azure AKS / Google GKE / On-Prem Kubernetes
Platform	Enterprise Kubernetes
Primary Role	Kubernetes Security Engineer
Team	Cloud Security
Escalation	Cloud Security Architect / Platform Engineering

Scenario

CloudNova Technologies has recently completed the deployment of a new enterprise Kubernetes platform.

Before the environment is approved for production workloads, the Cloud Security Team must perform a comprehensive security and operational assessment to ensure that the cluster aligns with organisational standards, security policies, and industry best practices.

You have been assigned to conduct the assessment and provide recommendations before production approval.

Objectives

Validate the Kubernetes environment across the following domains:

Cluster Architecture
Identity & Access Management
Node Security
Workload Security
Networking
Storage
Secrets Management
Monitoring & Logging
Backup & Disaster Recovery
Governance & Compliance
Operational Readiness

Success Criteria

The assessment is successful when:

Security controls are implemented appropriately.
No Critical findings remain unresolved.
High-risk issues have mitigation plans.
Compliance requirements are satisfied.
Platform governance is documented.
Operational monitoring is functional.

Assessment Workflow

Cluster Overview
        │
        ▼
Platform Architecture
        │
        ▼
Identity & RBAC
        │
        ▼
Node Security
        │
        ▼
Workload Security
        │
        ▼
Networking
        │
        ▼
Storage
        │
        ▼
Monitoring
        │
        ▼
Compliance
        │
        ▼
Assessment Report

Assessment Phase 1 — Platform Overview

Collect general platform information.

kubectl cluster-info

kubectl version

kubectl get nodes -o wide

Document:

Cluster Name
Kubernetes Version
Cloud Provider
Number of Control Plane Nodes
Number of Worker Nodes
Kubernetes Distribution
Container Runtime

Assessment Phase 2 — Cluster Architecture

Review:

kubectl get nodes

kubectl describe nodes

Verify:

Multi-node deployment
Worker node separation
Control Plane isolation
Availability Zones
Private networking
Cluster scaling capability

Assessment Checklist

Control	Status
HA Architecture
Multi-AZ
Private Nodes
Separate Workloads
Cluster Scaling

Assessment Phase 3 — Namespace Governance

Review:

kubectl get namespaces --show-labels

Confirm:

Naming standards
Environment labels
Owner labels
Business Unit labels
Pod Security labels

Example:

production

development

testing

security

monitoring

Assessment Phase 4 — Identity & Access Management

Review Service Accounts.

kubectl get sa -A

Review ClusterRoles.

kubectl get clusterroles

Review RoleBindings.

kubectl get rolebindings -A

kubectl get clusterrolebindings

Review Current Permissions.

kubectl auth can-i --list

Validate:

Least Privilege
Dedicated Service Accounts
No unnecessary ClusterAdmin access
Separation of Duties

Assessment Phase 5 — Node Security

Inspect Worker Nodes.

kubectl describe node <NODE>

Review:

Runtime
Labels
Taints
Allocatable Resources
Node Conditions

Security Checks

Confirm:

No privileged SSH access
Managed patching
Node image management
Secure bootstrap
Runtime updates

Assessment Phase 6 — Workload Security

Review Deployments.

kubectl get deployments -A

Inspect one Deployment.

kubectl describe deployment <DEPLOYMENT>

Validate:

Security Context
Non-root execution
Resource Limits
Health Probes
Dedicated Service Accounts
Read-only Filesystem
Dropped Capabilities

Review Pods.

kubectl get pods -A

Look for:

Privileged Containers

Host Network

Host PID

Host IPC

HostPath

Root User

Assessment Phase 7 — Secrets Management

Review Secrets.

kubectl get secrets -A

Validate:

Secrets exist only where required.
No plaintext credentials.
Encryption enabled.
Secret rotation process documented.
External Secret Manager implemented.

Examples:

AWS Secrets Manager
Azure Key Vault
Google Secret Manager
HashiCorp Vault

Assessment Phase 8 — ConfigMaps

Review:

kubectl get configmaps -A

Confirm:

Configuration separated from code.
No passwords inside ConfigMaps.
Environment-specific configuration.

Assessment Phase 9 — Storage

Review Storage Classes.

kubectl get storageclass

Review PVs.

kubectl get pv

Review PVCs.

kubectl get pvc -A

Validate:

Encryption
Dynamic Provisioning
Backup
Recovery
Access Modes

Assessment Phase 10 — Network Security

Review Services.

kubectl get svc -A

Review Network Policies.

kubectl get networkpolicy -A

Confirm:

Default Deny
East-West segmentation
Internal Services
Public Exposure reviewed

Review Ingress.

kubectl get ingress -A

Validate:

TLS
WAF
Public endpoints
Approved Domains

Assessment Phase 11 — Monitoring & Logging

Review:

kubectl get pods -n kube-system

Confirm:

Metrics Server
CoreDNS
kube-proxy

Review enterprise tooling.

Examples:

Prometheus
Grafana
Loki
Fluent Bit
Fluentd
OpenTelemetry

Cloud-native examples:

Amazon CloudWatch
Azure Monitor
Google Cloud Operations

Assessment Phase 12 — Audit Logging

Confirm Kubernetes Audit Logging.

Review:

API Activity
Authentication
RBAC Changes
Secret Access
Administrative Actions

Cloud integrations:

AWS CloudTrail
Azure Activity Logs
Google Cloud Audit Logs

Assessment Phase 13 — Backup & Disaster Recovery

Confirm:

etcd Backup
Persistent Volume Backup
Cluster Recovery Procedure
Restore Testing
Cross-Region Backup

Cloud examples:

AWS Backup
Azure Backup
Google Backup

Assessment Phase 14 — Compliance

Review alignment with:

CIS Kubernetes Benchmark
CIS Cloud Benchmark
NIST CSF
ISO 27001
SOC 2
PCI DSS
HIPAA

Confirm:

Policies documented
Security controls implemented
Exceptions approved

Assessment Phase 15 — Operational Readiness

Confirm:

Monitoring
Alerting
Incident Response
Runbooks
Change Management
Capacity Planning
Platform Ownership

Enterprise Security Checklist

Domain	Status
Architecture
IAM
RBAC
Workloads
Storage
Networking
Secrets
Monitoring
Logging
Backup
Compliance
Operations

Risk Rating Matrix

Critical

Examples

Public API Server
Unencrypted Secrets
ClusterAdmin granted broadly
No backups
etcd exposed

Immediate remediation required.

High

Examples

Missing Network Policies
Containers running as root
Privileged Pods
Public LoadBalancers

Medium

Examples

Missing labels
Missing health probes
High resource utilisation
Old Kubernetes version

Low

Examples

Documentation gaps
Minor warnings
Unused ConfigMaps
Legacy ReplicaSets

Recommendations

Prioritise improvements in the following order:

Identity & Least Privilege
Secrets Protection
Network Segmentation
Runtime Security
Backup & Recovery
Monitoring & Alerting
Governance
Compliance Automation

Assessment Report Template

Environment Name:

Assessment Date:

Assessor:

Cloud Provider:

Kubernetes Version:

Overall Status:

Architecture:

Identity:

RBAC:

Networking:

Storage:

Secrets:

Monitoring:

Logging:

Compliance:

Operational Readiness:

Critical Findings:

High Findings:

Medium Findings:

Recommendations:

Production Approval:

Approved / Conditionally Approved / Rejected

Best Practices

Always ensure:

Private Worker Nodes
Least Privilege RBAC
Dedicated Service Accounts
Network Policies
Encrypted Storage
External Secrets Management
Continuous Monitoring
Backup Validation
Compliance Reviews
Security Assessments before production deployment

Troubleshooting Quick Reference

Issue	Investigation
Worker Node NotReady	`kubectl describe node`
CrashLoopBackOff	`kubectl logs`
Pending Pods	Scheduler & Events
Storage Pending	PVC
DNS Issues	CoreDNS
Network Issues	Services & NetworkPolicies
Secret Errors	Secret Mounts
Permission Errors	RBAC

Evidence Collection

Collect evidence for:

Cluster Information
Node Health
Namespaces
Service Accounts
RBAC
Workload Security
Storage
Network Policies
Ingress
Monitoring
Logging
Backup
Compliance Controls
Risk Assessment
Final Report

Runbook Summary

This runbook provides a structured methodology for performing a comprehensive enterprise Kubernetes environment assessment.

By following this procedure, Cloud Security Engineers can validate architecture, identity, networking, workload security, storage, monitoring, compliance, and operational readiness before approving a Kubernetes environment for production use. The outcome is a documented assessment with risk ratings, remediation recommendations, and a clear production readiness decision.