Lesson 05 — Grafana
Learning Objectives
Section titled “Learning Objectives”By the end of this lesson, you will be able to:
- Understand what Grafana is
- Learn how Grafana visualizes Kubernetes metrics
- Explore Grafana architecture
- Understand dashboards and panels
- Learn how Grafana integrates with Prometheus
- Explore Amazon Managed Grafana
- Apply enterprise visualization best practices
Why This Matters
Section titled “Why This Matters”Collecting metrics is only the first step.
Security engineers, DevOps teams and Site Reliability Engineers (SREs) need an easy way to understand what those metrics mean.
Questions such as:
- Is the Kubernetes cluster healthy?
- Which Pods are consuming the most CPU?
- Which Nodes are under memory pressure?
- Are API response times increasing?
- Are there security incidents occurring?
can be answered much faster using dashboards rather than raw metric data.
Grafana transforms raw metrics into meaningful visualizations.
What is Grafana?
Section titled “What is Grafana?”Grafana is an open-source visualization and observability platform maintained by Grafana Labs.
It allows organizations to:
- Visualize metrics
- Build dashboards
- Create alerts
- Correlate metrics and logs
- Monitor Kubernetes clusters
- Observe cloud infrastructure
- Investigate incidents
Grafana is commonly paired with Prometheus for Kubernetes monitoring.
Grafana Architecture
Section titled “Grafana Architecture”Applications
↓
Exporters
↓
Prometheus
↓
Grafana
↓
Dashboards
↓
Operations TeamGrafana reads data from Prometheus and presents it through interactive dashboards.
How Grafana Works
Section titled “How Grafana Works”Grafana itself does not collect metrics.
Instead, it connects to external data sources.
Metrics
↓
Prometheus
↓
Grafana
↓
DashboardsGrafana acts as the visualization layer of the observability stack.
Data Sources
Section titled “Data Sources”Grafana supports numerous enterprise data sources.
| Data Source | Purpose |
|---|---|
| Prometheus | Kubernetes metrics |
| Amazon CloudWatch | AWS monitoring |
| Amazon OpenSearch | Log analytics |
| Loki | Log aggregation |
| Elasticsearch | Log storage |
| InfluxDB | Time-series metrics |
| MySQL | Operational data |
| PostgreSQL | Business data |
| Azure Monitor | Azure monitoring |
Organizations often combine multiple data sources into a single dashboard.
Dashboards
Section titled “Dashboards”Dashboards provide a real-time view of system health.
A dashboard may contain:
- CPU usage
- Memory usage
- Pod status
- Network traffic
- API latency
- Error rates
- Node health
- Storage utilization
Dashboards help engineers quickly identify abnormal behaviour.
Panels
Section titled “Panels”A dashboard consists of multiple panels.
Dashboard
├── CPU Usage
├── Memory Usage
├── Node Health
├── Pod Restarts
├── API Latency
└── Network TrafficEach panel displays a specific metric or group of related metrics.
Common Visualizations
Section titled “Common Visualizations”Grafana supports many visualization types.
Examples include:
- Line charts
- Bar charts
- Pie charts
- Heat maps
- Tables
- Gauges
- Stat panels
- Time-series graphs
- Histograms
- World maps
Choosing the correct visualization improves operational visibility.
Kubernetes Dashboard Example
Section titled “Kubernetes Dashboard Example”A typical Kubernetes dashboard displays:
Cluster Overview
↓
Nodes
↓
Pods
↓
Deployments
↓
CPU Usage
↓
Memory Usage
↓
Network Traffic
↓
Storage
↓
AlertsEngineers can assess cluster health within seconds.
Prometheus Integration
Section titled “Prometheus Integration”Grafana integrates directly with Prometheus.
Applications
↓
Exporters
↓
Prometheus
↓
Grafana
↓
DashboardPromQL queries retrieve the metrics displayed in Grafana.
Amazon Managed Grafana
Section titled “Amazon Managed Grafana”Amazon Managed Grafana is AWS’s fully managed Grafana service.
Benefits include:
- Managed infrastructure
- AWS IAM integration
- High availability
- Automatic scaling
- Simplified upgrades
- Integration with AWS services
This reduces operational overhead while providing enterprise-grade dashboards.
Amazon EKS Monitoring Architecture
Section titled “Amazon EKS Monitoring Architecture”Amazon EKS
↓
Prometheus
↓
Amazon Managed Service for Prometheus
↓
Amazon Managed Grafana
↓
Security Team
↓
Operations TeamThis architecture provides centralized monitoring across Kubernetes environments.
Alerting
Section titled “Alerting”Grafana supports alerting based on dashboard metrics.
Example workflow:
CPU Usage
↓
Threshold Exceeded
↓
Grafana Alert
↓
Email
↓
Slack
↓
Microsoft Teams
↓
SOC TeamAlerting enables proactive response to operational and security issues.
Log Correlation
Section titled “Log Correlation”Grafana can correlate metrics with logs.
Metric Spike
↓
Open Related Logs
↓
Investigate Incident
↓
Root Cause AnalysisThis reduces investigation time during outages or security incidents.
Security Dashboards
Section titled “Security Dashboards”Cloud Security Engineers often build dashboards for:
- Failed authentication
- Pod restarts
- Privileged containers
- Runtime alerts
- Kubernetes Audit Logs
- Network traffic
- Node health
- Resource exhaustion
- Vulnerability findings
- Compliance status
Security dashboards provide continuous visibility into cluster security.
Enterprise Dashboard Categories
Section titled “Enterprise Dashboard Categories”Typical enterprise dashboards include:
| Dashboard | Purpose |
|---|---|
| Cluster Health | Infrastructure overview |
| Node Health | Worker node monitoring |
| Application Performance | Service health |
| Security Monitoring | Runtime threats |
| Capacity Planning | Resource utilization |
| Network Monitoring | Traffic analysis |
| Cost Monitoring | Resource optimisation |
| Compliance Dashboard | Security posture |
Different teams consume different dashboards based on their responsibilities.
Enterprise Example
Section titled “Enterprise Example”A multinational banking organization operates more than 700 Amazon EKS clusters across several AWS Regions.
Grafana dashboards display:
- Kubernetes cluster health
- API Server latency
- Pod restart counts
- Memory utilization
- CPU utilization
- Runtime security alerts
- Audit log statistics
- Prometheus alerts
One morning, the SRE team notices a sharp increase in Pod restart rates on the production payments cluster.
Grafana highlights the issue immediately.
Engineers correlate the restart metrics with application logs and identify a faulty application deployment.
The deployment is rolled back before customers experience service disruption.
Common Dashboard Mistakes
Section titled “Common Dashboard Mistakes”Cloud Security Engineers frequently encounter:
- Too many panels on one dashboard
- Missing alert thresholds
- Inconsistent dashboard layouts
- Duplicate dashboards
- No security dashboards
- Poor naming conventions
- Missing documentation
- Overly complex queries
- No role-based access
- Unused dashboards
Well-designed dashboards should be simple, actionable and easy to understand.
Enterprise Monitoring
Section titled “Enterprise Monitoring”Organizations typically create dashboards for:
- Cluster health
- Node performance
- Pod health
- Namespace activity
- Storage utilization
- Network traffic
- Runtime security
- Kubernetes Audit Logs
- Amazon CloudWatch metrics
- Amazon EKS Control Plane metrics
Dashboards should provide both operational and security visibility.
Enterprise Implementation Strategy
Section titled “Enterprise Implementation Strategy”A recommended rollout:
Step 1
↓
Deploy Grafana
↓
Step 2
↓
Connect Prometheus
↓
Step 3
↓
Configure Data Sources
↓
Step 4
↓
Import Kubernetes Dashboards
↓
Step 5
↓
Create Security Dashboards
↓
Step 6
↓
Configure Alert Rules
↓
Step 7
↓
Enable RBAC
↓
Step 8
↓
Continuously Improve DashboardsOrganizations should begin with standard dashboards before creating custom visualizations.
Enterprise Observability Architecture
Section titled “Enterprise Observability Architecture”Applications
↓
Prometheus
↓
Amazon Managed Service for Prometheus
↓
Amazon Managed Grafana
↓
Amazon CloudWatch
↓
Amazon OpenSearch
↓
AWS Security Hub
↓
Enterprise SIEM
↓
Security Operations Centre (SOC)Grafana becomes the central visualization platform for enterprise observability.
Best Practices
Section titled “Best Practices”As a Kubernetes Security Engineer:
- Build separate dashboards for operations and security.
- Keep dashboards focused and easy to interpret.
- Use consistent naming conventions.
- Configure meaningful alert thresholds.
- Restrict dashboard access using IAM and RBAC.
- Integrate Prometheus, CloudWatch and OpenSearch.
- Regularly review dashboard usefulness.
- Monitor cluster, application and security metrics together.
- Correlate metrics with logs during investigations.
- Use Amazon Managed Grafana for enterprise Amazon EKS deployments.
Effective dashboards allow engineers to detect problems before users notice them.
Real-World Scenario
Section titled “Real-World Scenario”A global healthcare provider manages patient applications on Amazon EKS.
Grafana dashboards show:
- Increasing API response times
- Elevated CPU usage
- Multiple Pod restarts
- Rising memory consumption
At the same time, runtime alerts from Falco indicate suspicious process execution within one application.
Security analysts correlate the runtime alerts with Prometheus metrics and application logs through Grafana.
The compromised workload is isolated before it affects patient services.
Grafana significantly reduces investigation time by presenting all relevant operational information in one place.
Key Takeaways
Section titled “Key Takeaways”After completing this lesson, you should understand:
- What Grafana is
- Grafana architecture
- Dashboards and panels
- Prometheus integration
- Amazon Managed Grafana
- Alerting capabilities
- Log and metric correlation
- Enterprise dashboard design
- Observability best practices
Grafana is the visualization layer of the Kubernetes observability stack. By transforming Prometheus metrics into meaningful dashboards and integrating with AWS monitoring services, Grafana enables operations and security teams to monitor cluster health, investigate incidents and maintain reliable Amazon EKS environments.
Knowledge Check
Section titled “Knowledge Check”Question 1
Section titled “Question 1”What is the primary purpose of Grafana?
- A. Collect metrics directly from Kubernetes
- B. Visualize metrics and build monitoring dashboards
- C. Replace Prometheus
- D. Store Kubernetes Audit Logs
Answer: B
Question 2
Section titled “Question 2”Which monitoring platform is most commonly used as a data source for Grafana in Kubernetes?
- A. Prometheus
- B. Docker Hub
- C. Helm
- D. CoreDNS
Answer: A
Question 3
Section titled “Question 3”What is the basic building block of a Grafana dashboard?
- A. Namespace
- B. Panel
- C. Deployment
- D. ReplicaSet
Answer: B
Question 4
Section titled “Question 4”Which AWS service provides a fully managed Grafana environment?
- A. Amazon CloudFront
- B. Amazon Managed Grafana
- C. Amazon Lightsail
- D. Amazon Route 53
Answer: B
Question 5
Section titled “Question 5”Which combination represents enterprise best practice?
- A. Build focused dashboards, integrate Prometheus and CloudWatch, configure alerting, secure dashboard access with IAM and RBAC, and correlate metrics with logs and runtime security events.
- B. Place every metric on a single dashboard without alerting.
- C. Allow unrestricted access to all dashboards.
- D. Create dashboards without reviewing them regularly.
Answer: A
What’s Next?
Section titled “What’s Next?”In the next lesson, you will learn about Runtime Detection, exploring how enterprise organizations detect suspicious behaviour inside running Kubernetes workloads by combining Falco, Kubernetes Audit Logs, Prometheus metrics, Grafana dashboards and cloud-native security services in Amazon EKS.
➡️ Next Lesson: Lesson 06 — Runtime Detection