Cloud CI/CD Pipeline Lab
A modern cloud deployment should not depend on an engineer manually building, testing, and deploying every change.
Welcome to Lab 24 of the CompTIA Cloud+ practical lab sequence.
In the previous labs, you progressed from:
Manual Cloud Operations βAutomation & Scripting βInfrastructure as CodeYou can now automate operational tasks and define cloud infrastructure through code.
The next challenge is delivering changes safely and consistently.
A traditional workflow might look like:
Developer βWrites Code βManually Builds βManually Tests βUploads Files βManually Deploys βProductionThis creates opportunities for:
Human Error
Inconsistent Builds
Skipped Testing
Configuration Differences
Deployment Failures
Slow ReleasesCI/CD introduces a controlled pipeline:
Code βVersion Control βBuild βTest βSecurity Validation βArtifact βDeployment βVerification βMonitoringIn this lab, you will design and build a practical cloud CI/CD workflow.
π― Mission Information
Section titled βπ― Mission Informationβ| Item | Details |
|---|---|
| Lab | 24 β Cloud CI/CD Pipeline Lab |
| Difficulty | Intermediate |
| Estimated Time | 150β210 Minutes |
| Certification Alignment | CompTIA Cloud+ |
| Primary Focus | CI/CD and Cloud Deployment Automation |
| Previous Lab | 23 β Infrastructure as Code (IaC) Lab |
| Career Alignment | Cloud Engineer, DevOps Engineer, Cloud Administrator, Platform Engineer |
| Major Skills | CI/CD, Git, Build, Testing, Artifacts, Deployment, Security, Rollback |
| Deliverable | CI/CD Pipeline + Deployment Strategy + Pipeline Runbook |
π’ Scenario
Section titled βπ’ ScenarioβYour organization operates a cloud-hosted application.
Developers currently deploy updates manually.
Developer Laptop βApplication Code βManual Build βManual Testing βManual Upload βCloud ServerRecently, several problems occurred:
Developer A βForgot Testing βProduction FailureAnother deployment:
Developer B βBuilt Different Version βConfiguration ProblemAnother:
Developer C βDeployed Directly βNo ApprovalManagement wants a standardized process:
Developer βGit Repository βCI Pipeline βAutomated Testing βSecurity Checks βApproved Artifact βCD Pipeline βProductionYour mission is to design and implement this workflow.
π― Lab Objectives
Section titled βπ― Lab ObjectivesβBy completing this lab, you should be able to:
-
explain CI/CD
-
distinguish CI from CD
-
understand source control workflows
-
understand pipeline triggers
-
understand pipeline stages
-
understand build processes
-
automate testing
-
understand artifacts
-
understand artifact repositories
-
integrate security validation
-
manage pipeline secrets
-
understand service identities
-
implement least privilege
-
understand environment promotion
-
understand deployment approvals
-
compare deployment strategies
-
understand rolling deployments
-
understand blue-green deployments
-
understand canary deployments
-
understand rollback
-
understand Infrastructure as Code integration
-
monitor pipeline executions
-
troubleshoot pipeline failures
-
build a CI/CD operational runbook
01 β Understand CI/CD
Section titled β01 β Understand CI/CDβCI/CD commonly refers to:
Continuous Integration
+
Continuous Delivery / DeploymentThe objective is to create a repeatable software delivery workflow.
Developer Change βAutomated Pipeline βValidated Release02 β Understand Continuous Integration
Section titled β02 β Understand Continuous IntegrationβContinuous Integration focuses on frequently integrating code changes into a shared repository.
A typical workflow:
Developer βCode Change βCommit βRepository βAutomated Build βAutomated TestsCI helps identify problems earlier.
03 β Understand Continuous Delivery
Section titled β03 β Understand Continuous DeliveryβContinuous Delivery means changes are kept in a deployable state.
Code βBuild βTest βPackage βReady for Deployment βApproval βProductionProduction deployment may still require manual approval.
04 β Understand Continuous Deployment
Section titled β04 β Understand Continuous DeploymentβContinuous Deployment goes further.
Code βBuild βTest βValidation βAutomatic Production DeploymentIf all required checks pass, the change may be deployed automatically.
05 β CI vs Continuous Delivery vs Continuous Deployment
Section titled β05 β CI vs Continuous Delivery vs Continuous Deploymentβ| Model | Primary Purpose |
|---|---|
| Continuous Integration | Integrate and validate changes |
| Continuous Delivery | Keep releases deployable |
| Continuous Deployment | Automatically deploy validated changes |
Remember:
CI=Integrate + Validate
CD=Deliver / Deploy06 β Understand the Complete Pipeline
Section titled β06 β Understand the Complete PipelineβA mature pipeline may contain:
SOURCE βBUILD βTEST βSECURITY βPACKAGE βARTIFACT βDEPLOY βVERIFY βMONITOR07 β Understand Source Control
Section titled β07 β Understand Source ControlβThe pipeline begins with:
Source Code RepositoryVersion control provides:
-
history
-
collaboration
-
traceability
-
branching
-
rollback capability
-
code review
08 β Create the Lab Repository
Section titled β08 β Create the Lab RepositoryβCreate:
cloudplus-cicd-lab/Example:
mkdir cloudplus-cicd-labcd cloudplus-cicd-lab
git initCheck:
git status09 β Create a Simple Application
Section titled β09 β Create a Simple ApplicationβCreate:
app/Example structure:
cloudplus-cicd-lab/ββββ app/β βββ app.pyββββ tests/ββββ infrastructure/ββββ pipeline/ββββ README.mdββββ .gitignore10 β Create a Simple Python Application
Section titled β10 β Create a Simple Python ApplicationβCreate:
def cloud_status(): return "Cloud application healthy"
if __name__ == "__main__": print(cloud_status())Run:
python app/app.pyExpected:
Cloud application healthy11 β Understand the Build Stage
Section titled β11 β Understand the Build StageβThe build stage transforms source code into something deployable.
Depending on the application, this might include:
Compile
Install Dependencies
Package
Create Container Image
Generate Deployment Package12 β Build Stage Architecture
Section titled β12 β Build Stage ArchitectureβSource Code βBuild Environment βDependencies βBuild βDeployable Output13 β Understand Reproducible Builds
Section titled β13 β Understand Reproducible BuildsβA good pipeline should produce predictable results.
Avoid:
Developer Laptop A βBuild Works
Developer Laptop B βDifferent Dependencies βBuild FailsPrefer:
Controlled Build Environment βDefined Dependencies βRepeatable Build14 β Define Dependencies
Section titled β14 β Define DependenciesβFor Python, you might use:
requirements.txtOther ecosystems may use their own dependency-management files.
The principle is:
Dependencies should be defined rather than remembered manually.
15 β Understand Automated Testing
Section titled β15 β Understand Automated TestingβAfter building:
Build βAutomated TestsTests may include:
Unit Tests
Integration Tests
Functional Tests
Security Tests
Infrastructure Tests16 β Create a Basic Test
Section titled β16 β Create a Basic TestβCreate:
from app.app import cloud_status
def test_cloud_status(): assert cloud_status() == "Cloud application healthy"17 β Run the Test
Section titled β17 β Run the TestβIf using pytest:
pytestExpected:
PASSThe pipeline should stop when critical tests fail.
18 β Understand Pipeline Gates
Section titled β18 β Understand Pipeline GatesβA gate controls whether the pipeline can proceed.
Build βTest βPASS? / \YES NO β βNext StopStage19 β Why Pipeline Gates Matter
Section titled β19 β Why Pipeline Gates MatterβWithout gates:
Failed Test βIgnore βProductionWith gates:
Failed Test βPipeline Stops βEngineer Investigates20 β Understand Pipeline Triggers
Section titled β20 β Understand Pipeline TriggersβA pipeline can begin because of:
Commit
Pull Request
Merge
Schedule
Manual Trigger
Release Tag21 β Commit-Based Trigger
Section titled β21 β Commit-Based TriggerβExample:
Developer βgit push βRepository βPipeline Trigger22 β Pull Request Workflow
Section titled β22 β Pull Request WorkflowβA safer workflow:
Developer Branch βPull Request βBuild βTests βSecurity Checks βReview βMerge23 β Understand Branching
Section titled β23 β Understand BranchingβExample:
main | +---- feature-login | +---- feature-monitoring | +---- bugfix-networkDevelopers work independently and merge reviewed changes.
24 β Protect Important Branches
Section titled β24 β Protect Important BranchesβProduction-related branches may require:
Pull Request
Review
Passing Tests
Security Checks
ApprovalAvoid:
Developer βDirect Push βProductionwhere stronger controls are required.
25 β Understand Pipeline Configuration
Section titled β25 β Understand Pipeline ConfigurationβMany CI/CD systems define pipelines as code.
Conceptually:
stages: - build - test - security - deployExact syntax depends on the platform.
26 β Why Pipeline as Code Matters
Section titled β26 β Why Pipeline as Code MattersβPipeline definitions can be:
Version Controlled
Reviewed
Repeated
Audited
Tested27 β Build the CI Pipeline
Section titled β27 β Build the CI PipelineβYour initial pipeline should follow:
SOURCE βBUILD βTESTDocument each stage.
28 β Build Stage
Section titled β28 β Build StageβExample concept:
build: steps: - install dependencies - validate application - package application29 β Test Stage
Section titled β29 β Test StageβConceptually:
test: steps: - run unit tests - generate test results30 β Introduce a Test Failure
Section titled β30 β Introduce a Test FailureβTemporarily modify your test:
assert cloud_status() == "Wrong result"Run the test.
Expected:
FAIL31 β Observe Pipeline Behavior
Section titled β31 β Observe Pipeline BehaviorβThe correct behavior is:
Build βTest βFAIL βPipeline StopsRestore the correct test afterward.
32 β Understand Build Artifacts
Section titled β32 β Understand Build ArtifactsβAn artifact is an output produced by the pipeline.
Examples:
Application Package
Binary
Container Image
ZIP Archive
Deployment Template33 β Artifact Workflow
Section titled β33 β Artifact WorkflowβSource βBuild βArtifact βRepository βDeployment34 β Why Artifacts Matter
Section titled β34 β Why Artifacts MatterβYou should ideally deploy:
Same Tested Artifactrather than rebuilding differently for every environment.
35 β Understand Artifact Repositories
Section titled β35 β Understand Artifact RepositoriesβOrganizations may store artifacts in:
Package Repository
Container Registry
Object Storage
Artifact Management Platform36 β Understand Artifact Versioning
Section titled β36 β Understand Artifact VersioningβAvoid:
application-latestas the only identifier.
Prefer traceable versions such as:
application-1.0.1
application-1.0.2
application-1.0.337 β Connect Artifact to Source
Section titled β37 β Connect Artifact to SourceβA release should ideally be traceable:
Artifact βPipeline Run βCommit βDeveloper Change38 β Understand Security in CI/CD
Section titled β38 β Understand Security in CI/CDβA pipeline can become a powerful attack path.
Developer βRepository βPipeline βCloud Credentials βProductionTherefore pipeline security is critical.
39 β Add Security Validation
Section titled β39 β Add Security ValidationβA pipeline may perform:
Source Code Scan
Dependency Scan
Secret Scan
IaC Scan
Container Scan
Policy Validation40 β Build the Secure Pipeline
Section titled β40 β Build the Secure PipelineβSOURCE βBUILD βTEST βSECURITY SCAN βPACKAGE βDEPLOY41 β Understand Static Analysis
Section titled β41 β Understand Static AnalysisβStatic analysis evaluates code without executing the complete application.
It may identify:
Unsafe Code Patterns
Potential Vulnerabilities
Coding Issues42 β Understand Dependency Scanning
Section titled β42 β Understand Dependency ScanningβApplications depend on third-party packages.
Application βDependency A βDependency BA vulnerable dependency can introduce risk.
43 β Understand Secret Scanning
Section titled β43 β Understand Secret ScanningβRepositories should be checked for:
Passwords
API Keys
Access Tokens
Private Keys
Credentials44 β Never Store Pipeline Secrets in Code
Section titled β44 β Never Store Pipeline Secrets in CodeβPoor:
password: MySecretPasswordBetter:
Pipeline βSecret Store βRuntime Credential45 β Understand Pipeline Secrets
Section titled β45 β Understand Pipeline SecretsβExamples include:
Cloud Credentials
API Tokens
Registry Credentials
Signing Keys
Database CredentialsThey require controlled storage and access.
46 β Prefer Short-Lived Credentials
Section titled β46 β Prefer Short-Lived CredentialsβWhere supported:
Pipeline βFederated / Managed Identity βTemporary Credential βCloud APIis generally preferable to long-lived static credentials.
47 β Apply Least Privilege
Section titled β47 β Apply Least PrivilegeβIf the pipeline only deploys:
Web Applicationit should not automatically have permission to:
Delete Databases
Modify IAM Administrators
Change Billing
Delete Entire Networks48 β Separate Pipeline Identities
Section titled β48 β Separate Pipeline IdentitiesβConsider:
Development Pipeline Identity
Testing Pipeline Identity
Production Pipeline Identitywith appropriate permissions.
49 β Understand Supply Chain Security
Section titled β49 β Understand Supply Chain SecurityβCI/CD is part of the software supply chain.
Potential targets include:
Source Repository
Dependencies
Build System
Pipeline
Artifacts
Container Registry
Deployment Credentials50 β Protect the Source Repository
Section titled β50 β Protect the Source RepositoryβApply controls such as:
MFA
Least Privilege
Branch Protection
Code Review
Audit Logging51 β Protect Pipeline Configuration
Section titled β51 β Protect Pipeline ConfigurationβAn attacker who modifies:
Pipeline Definitionmay potentially change:
Build Behavior
Security Checks
Deployment Destination
Credential UsageTherefore pipeline code should be reviewed.
52 β Protect Build Environments
Section titled β52 β Protect Build EnvironmentsβBuild environments may process:
Source Code
Dependencies
Credentials
ArtifactsUse trusted and controlled environments.
53 β Protect Artifacts
Section titled β53 β Protect ArtifactsβArtifacts should maintain:
Integrity
Traceability
Controlled Access54 β Understand Artifact Integrity
Section titled β54 β Understand Artifact IntegrityβConceptually:
Source βBuild βArtifact βIntegrity Verification βDeploymentChecksums or signatures may be used depending on the environment.
55 β Understand Environment Promotion
Section titled β55 β Understand Environment PromotionβA release may move through:
Development βTesting βStaging βProduction56 β Why Promote the Same Artifact?
Section titled β56 β Why Promote the Same Artifact?βPoor:
Dev βBuild A
Test βBuild B
Production βBuild CBetter:
Build Once βArtifact βDev βTest βStaging βProduction57 β Environment-Specific Configuration
Section titled β57 β Environment-Specific ConfigurationβThe application artifact may remain the same while configuration differs.
Examples:
Database Endpoint
Environment Name
Scaling Settings
Feature Configuration58 β Keep Secrets Out of Artifacts
Section titled β58 β Keep Secrets Out of ArtifactsβDo not package:
Production Password
Private Key
API Secretinside the application artifact.
59 β Understand Deployment Approvals
Section titled β59 β Understand Deployment ApprovalsβProduction deployment may require:
Automated Checks βHuman Approval βDeploymentThis is common when changes have significant business impact.
60 β Build an Approval Gate
Section titled β60 β Build an Approval GateβStaging βValidation βPASS βApproval βProduction61 β Understand Separation of Duties
Section titled β61 β Understand Separation of DutiesβExample:
Developer βWrites Code
Reviewer βApproves Change
Operations βApproves Production Deployment62 β Understand Deployment Strategies
Section titled β62 β Understand Deployment StrategiesβMajor strategies include:
Recreate
Rolling
Blue-Green
Canary63 β Recreate Deployment
Section titled β63 β Recreate DeploymentβVersion 1 βSTOP βVersion 2 βSTARTSimple, but it may create downtime.
64 β Rolling Deployment
Section titled β64 β Rolling DeploymentβV1 V1 V1 V1
β Replace Gradually β
V2 V1 V1 V1
V2 V2 V1 V1
V2 V2 V2 V1
V2 V2 V2 V2Benefits may include reduced downtime.
65 β Rolling Deployment Risk
Section titled β65 β Rolling Deployment RiskβDuring deployment:
Version 1+Version 2may operate simultaneously.
The application must support this where required.
66 β Blue-Green Deployment
Section titled β66 β Blue-Green DeploymentβMaintain two environments:
BLUEVersion 1Production Traffic
GREENVersion 2New ReleaseAfter validation:
Traffic βGREEN67 β Blue-Green Architecture
Section titled β67 β Blue-Green Architectureβ Users | v Load Balancer | Traffic Switch / \ v v BLUE GREEN V1 V268 β Blue-Green Advantage
Section titled β68 β Blue-Green AdvantageβRollback may be faster:
GREEN Fails βTraffic βBLUEprovided the previous environment remains compatible and available.
69 β Canary Deployment
Section titled β69 β Canary DeploymentβDeploy the new version to a small percentage of users.
Users | +---- 95% β Version 1 | +---- 5% β Version 2Observe the new version before increasing traffic.
70 β Canary Progression
Section titled β70 β Canary Progressionβ5% β10% β25% β50% β100%only if validation remains successful.
71 β Monitor Canary Health
Section titled β71 β Monitor Canary HealthβTrack:
Error Rate
Latency
Availability
Application Logs
Resource Usage
Business Metrics72 β Compare Deployment Strategies
Section titled β72 β Compare Deployment Strategiesβ| Strategy | Downtime | Rollback | Complexity |
|---|---|---|---|
| Recreate | Possible | Moderate | Low |
| Rolling | Low | Moderate | Medium |
| Blue-Green | Low | Fast | Higher |
| Canary | Low | Controlled | Higher |
73 β Understand Rollback
Section titled β73 β Understand RollbackβA pipeline needs a defined response to failed deployment.
Deploy V2 βValidation βFAIL βRollback βV174 β Define Rollback Conditions
Section titled β74 β Define Rollback ConditionsβExamples:
Health Check Failure
Error Rate Increase
Application Unavailable
Critical Test Failure
Performance Degradation75 β Rollback Is More Than Application Code
Section titled β75 β Rollback Is More Than Application CodeβConsider:
Application
Infrastructure
Database Schema
Configuration
DataA previous application version may not work with a new database schema.
76 β Understand Forward Fix
Section titled β76 β Understand Forward FixβSometimes rollback is unsafe or impossible.
Another option:
Failure βFix βNew Version βDeployThis is a:
Forward fix.
77 β Build the Deployment Decision
Section titled β77 β Build the Deployment DecisionβDeployment Failure βCan Safely Roll Back? / \ YES NO β βRollback Forward Fix78 β Understand Database Changes
Section titled β78 β Understand Database ChangesβDatabase changes require special care.
Application V1 βDatabase Schema V1New deployment:
Application V2 βDatabase Schema V2Rollback could fail if schema changes are incompatible.
79 β Use Backward-Compatible Changes
Section titled β79 β Use Backward-Compatible ChangesβWhere possible, design changes that allow:
V1+V2to work during transition.
This is especially useful for rolling and canary deployments.
80 β Integrate Infrastructure as Code
Section titled β80 β Integrate Infrastructure as CodeβFrom Lab 23:
Infrastructure Code βValidation βPlan βDeploymentThis can become part of the pipeline.
81 β IaC CI/CD Workflow
Section titled β81 β IaC CI/CD WorkflowβGit Commit βIaC Validation βSecurity Scan βPlan βReview βApproval βApply82 β Separate Application and Infrastructure Changes
Section titled β82 β Separate Application and Infrastructure ChangesβYour repository might contain:
cloudplus-cicd-lab/ββββ app/βββ tests/βββ infrastructure/βββ pipeline/βββ README.md83 β Validate IaC Automatically
Section titled β83 β Validate IaC AutomaticallyβPipeline stage:
Infrastructure βFormat Check βValidate βSecurity Scan βPlan84 β Do Not Automatically Apply Destructive IaC Changes
Section titled β84 β Do Not Automatically Apply Destructive IaC ChangesβIf the plan contains:
DELETE DATABASEthe pipeline should not blindly proceed.
Require appropriate review.
85 β Understand Pipeline Variables
Section titled β85 β Understand Pipeline VariablesβPipeline variables may contain:
Environment Name
Region
Artifact Version
Deployment Target86 β Separate Variables from Secrets
Section titled β86 β Separate Variables from SecretsβVARIABLE
REGION=region-aversus:
SECRET
DATABASE_PASSWORD=*****Secrets require stronger controls.
87 β Understand Environment Variables
Section titled β87 β Understand Environment VariablesβApplications may receive runtime configuration through environment variables.
Example:
APP_ENV=production
LOG_LEVEL=infoAvoid exposing sensitive values through logs.
88 β Understand Pipeline Logging
Section titled β88 β Understand Pipeline LoggingβPipeline logs help answer:
What Ran?
When?
Who Triggered It?
Which Commit?
Which Tests Passed?
Which Stage Failed?
What Was Deployed?89 β Do Not Log Secrets
Section titled β89 β Do Not Log SecretsβPoor:
Connecting with password:SuperSecret123Logs may be retained and accessible to many users.
90 β Monitor Pipeline Health
Section titled β90 β Monitor Pipeline HealthβUseful metrics include:
Pipeline Success Rate
Failure Rate
Build Duration
Test Duration
Deployment Frequency
Deployment Failures91 β Monitor Deployment Health
Section titled β91 β Monitor Deployment HealthβAfter deployment:
Pipeline βProduction βMonitoringCheck:
Availability
Latency
Error Rate
Resource Utilization
Logs
Health Checks92 β Build Post-Deployment Validation
Section titled β92 β Build Post-Deployment ValidationβDeploy βHealth Check βApplication Test βMetrics Review βPASS? / \YES NO β βDone Rollback / Investigate93 β Create a Smoke Test
Section titled β93 β Create a Smoke TestβA smoke test quickly validates critical functionality.
Examples:
Application Responds
Login Endpoint Works
Database Connection Works
Health Endpoint Returns Success94 β Understand Deployment Verification
Section titled β94 β Understand Deployment VerificationβNever assume:
Pipeline Green=Application HealthyThe pipeline must validate the deployed workload.
95 β Build the Full Pipeline
Section titled β95 β Build the Full Pipelineβ Developer | v Git Repository | v Pull Request | +---------+---------+ | | v v Build Review | v Test | v Security Scan | v Artifact | v Development | v Testing | v Staging | v Approval | v Production | v Validate | v Monitor96 β Introduce a Build Failure
Section titled β96 β Introduce a Build FailureβCreate an intentional syntax error in the disposable lab application.
Expected:
Build βFAIL βPipeline StopsRestore the application afterward.
97 β Introduce a Test Failure
Section titled β97 β Introduce a Test FailureβModify a test so it fails.
Expected:
Build βPASS βTest βFAIL βDeployment Blocked98 β Introduce a Security Failure
Section titled β98 β Introduce a Security FailureβIn a disposable local exercise, create an obvious placeholder pattern for your security-checking process.
For example:
EXAMPLE_SECRET_DO_NOT_USEVerify that your security stage identifies the test condition if configured to do so.
Remove the test value afterward.
99 β Test Deployment Failure
Section titled β99 β Test Deployment FailureβSimulate:
Deployment βApplication Health Check βFAILDocument what should happen next.
100 β Test Rollback
Section titled β100 β Test RollbackβVerify:
Failed Release βPrevious Known-Good Version βApplication HealthyDo this only in a disposable lab environment.
101 β Test Approval Controls
Section titled β101 β Test Approval ControlsβAttempt:
Staging βProductionwithout required approval.
Expected:
BLOCKED102 β Test Unauthorized Pipeline Access
Section titled β102 β Test Unauthorized Pipeline AccessβVerify that an unauthorized account cannot:
Modify Pipeline
Approve Production
Access Secrets
Deploy Production103 β Test Secret Protection
Section titled β103 β Test Secret ProtectionβCheck:
Repository
Pipeline Configuration
Logs
ArtifactsEnsure sensitive values are not exposed.
104 β Test Artifact Traceability
Section titled β104 β Test Artifact TraceabilityβSelect a deployed artifact.
Determine:
Artifact Version
Commit
Pipeline Run
Build Time
Deployment Environment105 β Test Environment Promotion
Section titled β105 β Test Environment PromotionβVerify that:
Same Artifactmoves through:
Development βTesting βStaging βProduction106 β Test Monitoring
Section titled β106 β Test MonitoringβAfter deployment, intentionally create a safe application error in the disposable environment.
Verify:
Error βLogs βMonitoring βAlert / Detection107 β Troubleshooting β Pipeline Does Not Start
Section titled β107 β Troubleshooting β Pipeline Does Not StartβCheck:
Trigger
Branch
Pipeline Configuration
Repository Permissions
Pipeline Service108 β Troubleshooting β Build Fails
Section titled β108 β Troubleshooting β Build FailsβCheck:
Dependencies
Runtime Version
Build Commands
Configuration
Build Logs109 β Troubleshooting β Tests Pass Locally but Fail in CI
Section titled β109 β Troubleshooting β Tests Pass Locally but Fail in CIβInvestigate:
Environment Differences
Dependency Versions
Environment Variables
Operating System
External Dependencies
Test Data110 β Troubleshooting β Pipeline Cannot Access Cloud
Section titled β110 β Troubleshooting β Pipeline Cannot Access CloudβCheck:
Pipeline Identity
Authentication
Authorization
Network Connectivity
Cloud Account
Token Expiration111 β Troubleshooting β Deployment Permission Denied
Section titled β111 β Troubleshooting β Deployment Permission DeniedβThink:
Authentication βSuccessful
Authorization βInsufficientReview the deployment identityβs permissions.
112 β Troubleshooting β Artifact Missing
Section titled β112 β Troubleshooting β Artifact MissingβCheck:
Build Stage
Artifact Path
Upload Step
Artifact Repository
Version113 β Troubleshooting β Wrong Version Deployed
Section titled β113 β Troubleshooting β Wrong Version DeployedβInvestigate:
Artifact Tag
Pipeline Variables
Deployment Configuration
Repository Commit
Environment Promotion114 β Troubleshooting β Secret Appears in Logs
Section titled β114 β Troubleshooting β Secret Appears in LogsβTreat this as possible credential exposure.
Stop Exposure βProtect Logs βRotate Credential βInvestigate Access βCorrect PipelineFollow organizational security procedures.
115 β Troubleshooting β Production Deployment Fails
Section titled β115 β Troubleshooting β Production Deployment FailsβReview:
Pipeline Logs
Application Logs
Infrastructure Logs
Health Checks
Monitoring
Recent ChangesThen determine:
Rollbackor:
Forward Fix116 β Troubleshooting β Rollback Fails
Section titled β116 β Troubleshooting β Rollback FailsβInvestigate:
Database Compatibility
Configuration Changes
Infrastructure Changes
Missing Previous Artifact
Dependencies117 β Troubleshooting β Pipeline Suddenly Becomes Slow
Section titled β117 β Troubleshooting β Pipeline Suddenly Becomes SlowβReview:
Build Duration
Dependency Downloads
Test Duration
Runner Capacity
Network Performance
External Services118 β Troubleshooting β Security Scan Produces Findings
Section titled β118 β Troubleshooting β Security Scan Produces FindingsβDo not automatically ignore them.
Determine:
Finding βSeverity βValidity βRisk βRemediation / Exception119 β Build Pipeline Governance
Section titled β119 β Build Pipeline GovernanceβDefine:
Pipeline Owner
Repository Owner
Production Approver
Security Reviewer
Artifact Retention
Secret Management
Change Control120 β Define Pipeline Access
Section titled β120 β Define Pipeline AccessβApply:
Least Privilege
Role-Based Access
MFA
Separation of Duties121 β Define Production Controls
Section titled β121 β Define Production ControlsβExample:
[ ] Protected production branch[ ] Required code review[ ] Automated tests required[ ] Security checks required[ ] Deployment approval required[ ] Dedicated production identity[ ] Rollback available[ ] Deployment monitoring enabled122 β Define Artifact Controls
Section titled β122 β Define Artifact ControlsβVerify:
[ ] Artifact versioned[ ] Artifact traceable to commit[ ] Access restricted[ ] Integrity protected[ ] Retention defined[ ] Production artifact identifiable123 β Define Secret Controls
Section titled β123 β Define Secret ControlsβVerify:
[ ] Secrets not stored in repository[ ] Secrets not embedded in artifacts[ ] Secrets masked in logs[ ] Access restricted[ ] Rotation process defined[ ] Short-lived credentials preferred124 β Define Pipeline Security Controls
Section titled β124 β Define Pipeline Security ControlsβVerify:
[ ] Repository protected[ ] Pipeline code reviewed[ ] Dependencies scanned[ ] Secrets scanned[ ] IaC scanned[ ] Build environment protected[ ] Artifacts protected[ ] Deployment identity restricted[ ] Production approvals enabled[ ] Audit logs retained125 β Define Pipeline Reliability Controls
Section titled β125 β Define Pipeline Reliability ControlsβVerify:
[ ] Build repeatable[ ] Tests automated[ ] Failed tests block deployment[ ] Deployment validated[ ] Rollback tested[ ] Pipeline failures alerted[ ] Artifacts versioned[ ] Deployment state observable126 β Build the CI/CD Findings Register
Section titled β126 β Build the CI/CD Findings Registerβ| Finding | Risk | Recommendation | Priority |
|---|---|---|---|
| Manual Deployment | Human Error | Implement CI/CD | High |
| No Automated Tests | Defects Reach Production | Add Test Stage | High |
| Secrets in Repository | Credential Exposure | Use Secret Management | Critical |
| Pipeline Uses Admin Rights | Excessive Privilege | Apply Least Privilege | Critical |
| Direct Production Push | Unreviewed Changes | Protect Branch | High |
| No Security Scanning | Vulnerability Risk | Add Security Gates | High |
| Artifact Not Versioned | Poor Traceability | Version Artifacts | Medium |
| No Approval Gate | Unauthorized Deployment | Add Production Approval | High |
| No Rollback | Extended Outage | Define Rollback | High |
| No Deployment Monitoring | Delayed Detection | Add Post-Deployment Monitoring | High |
127 β Build the CI/CD Runbook
Section titled β127 β Build the CI/CD RunbookβUse:
Runbook:Cloud CI/CD Pipeline
Application:
Repository:
Pipeline Platform:
Pipeline Owner:
Cloud Environment:
Source Branch:
Trigger:
Build Stage:
Test Stage:
Security Stage:
Artifact:
Artifact Repository:
Development Deployment:
Testing Deployment:
Staging Deployment:
Production Approval:
Production Deployment:
Deployment Strategy:
Deployment Identity:
Required Permissions:
Secrets:
Post-Deployment Validation:
Monitoring:
Rollback:
Failure Handling:
Escalation:
Audit Logging:128 β Create the Final Lab Report
Section titled β128 β Create the Final Lab ReportβUse:
Lab:Cloud CI/CD Pipeline Lab
Application:
Repository:
Pipeline Platform:
Cloud Platform:
Build Environment:
Pipeline Trigger:
Build Result:
Automated Tests:
Security Checks:
Artifact:
Artifact Version:
Artifact Repository:
Development Deployment:
Testing Deployment:
Staging Deployment:
Production Deployment:
Deployment Strategy:
Approval Controls:
Pipeline Identity:
Permissions:
Secret Management:
IaC Integration:
Post-Deployment Validation:
Monitoring:
Rollback Test:
Failure Tests:
Findings:
Recommendations:
Lessons Learned:π§ͺ Final Validation Checklist
Section titled βπ§ͺ Final Validation Checklistβ| Validation | Status |
|---|---|
| CI/CD understood | |
| Continuous Integration understood | |
| Continuous Delivery understood | |
| Continuous Deployment understood | |
| Repository created | |
| Application created | |
| Source control configured | |
| Build stage understood | |
| Automated test created | |
| Test failure validated | |
| Pipeline triggers understood | |
| Pull request workflow understood | |
| Branch protection understood | |
| Pipeline as Code understood | |
| Artifact created | |
| Artifact versioning understood | |
| Artifact traceability understood | |
| Security validation understood | |
| Dependency scanning understood | |
| Secret scanning understood | |
| Pipeline secrets protected | |
| Least privilege reviewed | |
| Pipeline identities understood | |
| Supply chain security understood | |
| Environment promotion understood | |
| Approval gate understood | |
| Deployment strategies compared | |
| Rolling deployment understood | |
| Blue-green deployment understood | |
| Canary deployment understood | |
| Rollback understood | |
| Database rollback considerations understood | |
| IaC integrated into pipeline | |
| Pipeline logging understood | |
| Pipeline monitoring understood | |
| Post-deployment validation understood | |
| Failure scenarios tested | |
| Pipeline governance reviewed | |
| CI/CD runbook created | |
| Findings documented |
π― Certification Connection
Section titled βπ― Certification ConnectionβA Cloud+ scenario may say:
Developers manually deploy application changes and production frequently receives untested releases.
Think:
CI/CD pipeline with automated testing and deployment gates.
Another:
A deployment pipeline has permanent administrator credentials.
Think:
Credential-management and least-privilege failure.
Another:
A new application version should initially receive only 5% of production traffic.
Think:
Canary deployment.
Another:
The organization wants the new environment fully deployed before switching production traffic.
Think:
Blue-green deployment.
Another:
Application instances should be updated gradually while keeping the service available.
Think:
Rolling deployment.
Another:
A critical automated test fails.
Think:
Stop the pipeline and block deployment.
Another:
Production receives a different build than the version tested in staging.
Think:
Artifact consistency and environment-promotion problem.
Another:
An application deployment succeeds but monitoring shows a significant increase in errors.
Think:
Post-deployment validation and rollback/forward-fix decision.
π€ Interview Questions
Section titled βπ€ Interview QuestionsβPractice without notes.
1. What is CI/CD?
Section titled β1. What is CI/CD?β2. What is Continuous Integration?
Section titled β2. What is Continuous Integration?β3. Continuous Delivery vs Continuous Deployment?
Section titled β3. Continuous Delivery vs Continuous Deployment?β4. What is a pipeline?
Section titled β4. What is a pipeline?β5. What is a pipeline trigger?
Section titled β5. What is a pipeline trigger?β6. Why is version control important?
Section titled β6. Why is version control important?β7. What is an automated build?
Section titled β7. What is an automated build?β8. Why automate testing?
Section titled β8. Why automate testing?β9. What is a pipeline gate?
Section titled β9. What is a pipeline gate?β10. What is a build artifact?
Section titled β10. What is a build artifact?β11. Why version artifacts?
Section titled β11. Why version artifacts?β12. Why promote the same artifact?
Section titled β12. Why promote the same artifact?β13. How should pipeline secrets be managed?
Section titled β13. How should pipeline secrets be managed?β14. Why should pipeline identities use least privilege?
Section titled β14. Why should pipeline identities use least privilege?β15. What is branch protection?
Section titled β15. What is branch protection?β16. What is rolling deployment?
Section titled β16. What is rolling deployment?β17. What is blue-green deployment?
Section titled β17. What is blue-green deployment?β18. What is canary deployment?
Section titled β18. What is canary deployment?β19. What is rollback?
Section titled β19. What is rollback?β20. When might a forward fix be preferable?
Section titled β20. When might a forward fix be preferable?β21. Why are database changes difficult to roll back?
Section titled β21. Why are database changes difficult to roll back?β22. How can IaC integrate with CI/CD?
Section titled β22. How can IaC integrate with CI/CD?β23. Why monitor pipelines?
Section titled β23. Why monitor pipelines?β24. Why perform post-deployment validation?
Section titled β24. Why perform post-deployment validation?β25. How would you secure a cloud CI/CD pipeline?
Section titled β25. How would you secure a cloud CI/CD pipeline?βπ¨ Scenario Interview Question 1
Section titled βπ¨ Scenario Interview Question 1βDevelopers can push directly to the production branch.
Improve:
Protected Branch βPull Request βTests βSecurity Checks βReview βMergeπ¨ Scenario Interview Question 2
Section titled βπ¨ Scenario Interview Question 2βThe pipeline contains a plaintext cloud administrator password.
Think:
Credential exposure.
Use an approved secrets-management or workload-identity mechanism.
π¨ Scenario Interview Question 3
Section titled βπ¨ Scenario Interview Question 3βVersion 2 should be tested against a small percentage of real production traffic.
Use:
Canary deployment.
π¨ Scenario Interview Question 4
Section titled βπ¨ Scenario Interview Question 4βThe company wants near-instant rollback by switching traffic to the previous environment.
Use:
Blue-green deployment.
π¨ Scenario Interview Question 5
Section titled βπ¨ Scenario Interview Question 5βA pipeline deploys successfully but the application health check fails.
Think:
Deployment βVerification βFailure βRollback / Forward Fixπ¨ Scenario Interview Question 6
Section titled βπ¨ Scenario Interview Question 6βThe same application is rebuilt separately for testing and production.
Risk:
Tested Buildβ Production BuildPrefer:
Build Once βVersioned Artifact βPromoteπ¨ Scenario Interview Question 7
Section titled βπ¨ Scenario Interview Question 7βThe CI/CD service can create and delete every resource in the cloud account.
Think:
Excessive permissions.
Apply least privilege to the pipeline identity.
π¨ Scenario Interview Question 8
Section titled βπ¨ Scenario Interview Question 8βAn engineer modifies the pipeline to skip security scanning.
Controls should include:
Pipeline as Code βCode Review βProtected Branch βAudit Loggingπ¨ Scenario Interview Question 9
Section titled βπ¨ Scenario Interview Question 9βProduction fails after a database schema migration, and the old application cannot use the new schema.
This demonstrates why rollback planning must consider:
Application+Database+Data Compatibilityπ¨ Scenario Interview Question 10
Section titled βπ¨ Scenario Interview Question 10βA pipeline passes every stage but nobody knows which commit produced the running production version.
Improve:
Artifact and deployment traceability.
π§ CI/CD Interview Framework
Section titled βπ§ CI/CD Interview FrameworkβRemember:
CODE βCOMMIT βREVIEW βBUILD βTEST βSECURITY βPACKAGE βARTIFACT βPROMOTE βAPPROVE βDEPLOY βVERIFY βMONITOR βROLLBACKπ¬ Interview Tip
Section titled βπ¬ Interview TipβAvoid:
βCI/CD automatically deploys applications.β
A stronger answer is:
βI would design the CI/CD workflow so every change is traceable from source control through build, automated testing, security validation, artifact creation, environment promotion, and deployment. I would protect production branches, use dedicated least-privilege pipeline identities, keep secrets outside source code, version artifacts, introduce appropriate approval gates, choose a deployment strategy based on availability requirements, perform post-deployment health validation, monitor the release, and maintain a tested rollback or forward-fix procedure.β
That demonstrates Cloud Engineer and DevOps operational thinking.
π Portfolio Deliverables
Section titled βπ Portfolio DeliverablesβKeep sanitized versions of:
1. CI/CD Pipeline Definition
Section titled β1. CI/CD Pipeline DefinitionβShow:
Build
Test
Security
Package
Deploy2. Application Repository
Section titled β2. Application RepositoryβInclude:
app/
tests/
infrastructure/
pipeline/
README.md3. Pipeline Architecture Diagram
Section titled β3. Pipeline Architecture DiagramβShow:
Developer βRepository βCI/CD βArtifact βCloud4. Pipeline Security Review
Section titled β4. Pipeline Security ReviewβDocument:
Repository Security
Secrets
Identity
Permissions
Artifacts
Approvals
Logging5. Deployment Strategy
Section titled β5. Deployment StrategyβDocument whether you selected:
Rolling
Blue-Green
Canaryand explain why.
6. Rollback Plan
Section titled β6. Rollback PlanβInclude:
Failure Criteria
Previous Version
Application Recovery
Infrastructure Recovery
Database Considerations
Validation7. CI/CD Runbook
Section titled β7. CI/CD RunbookβDocument the complete operational workflow.
π Resume Examples
Section titled βπ Resume ExamplesβInstead of:
Worked with CI/CD.
Use:
Built cloud CI/CD workflows integrating source control, automated builds, testing, security validation, versioned artifacts, environment promotion, deployment approvals, and post-deployment verification.
Or:
Implemented secure CI/CD practices using protected source-control workflows, least-privilege deployment identities, centralized secrets management, automated security gates, artifact traceability, and monitored production deployments.
Or:
Designed cloud application delivery workflows supporting rolling, blue-green, and canary deployment strategies with automated health validation and rollback procedures.
β Job-Readiness Check
Section titled ββ Job-Readiness CheckβYou should now be able to:
-
explain CI/CD
-
distinguish CI, Continuous Delivery, and Continuous Deployment
-
understand source control workflows
-
explain pipeline triggers
-
build basic pipeline stages
-
automate testing
-
understand pipeline gates
-
explain artifacts
-
version artifacts
-
understand artifact repositories
-
integrate security validation
-
protect pipeline secrets
-
apply least privilege
-
understand pipeline identities
-
explain supply chain security
-
promote releases across environments
-
use deployment approvals
-
compare deployment strategies
-
explain rolling deployments
-
explain blue-green deployments
-
explain canary deployments
-
design rollback procedures
-
understand forward fixes
-
account for database changes
-
integrate IaC with CI/CD
-
understand pipeline logging
-
monitor deployments
-
perform post-deployment validation
-
troubleshoot pipeline failures
-
build CI/CD runbooks
π Mission Complete
Section titled βπ Mission CompleteβYou have progressed from:
Developer βManual Build βManual Testing βManual Deploymentto:
Code βVersion Control βAutomated Build βAutomated Testing βSecurity Validation βVersioned Artifact βControlled Deployment βVerification βMonitoringYou now understand an important cloud engineering principle:
A successful CI/CD pipeline is not simply a deployment script. It is a controlled software-delivery system that makes changes repeatable, testable, traceable, secure, and recoverable.
π Whatβs Next?
Section titled βπ Whatβs Next?βYou have now built the core operational chain:
Cloud Operations βAutomation βInfrastructure as Code βCI/CDThe next step is to bring these capabilities together in a practical troubleshooting exercise.
You will investigate cloud problems across:
-
compute
-
networking
-
storage
-
identity
-
application connectivity
-
monitoring
-
logging
-
resource utilization
-
deployment failures
-
configuration issues
-
service dependencies
-
root-cause analysis
-
remediation
-
validation
-
incident documentation
You will work through problems systematically:
SYMPTOM βSCOPE βEVIDENCE βHYPOTHESIS βTEST βROOT CAUSE βREMEDIATE βVALIDATE βDOCUMENTβ‘οΈ Next: Lab 25 β Cloud Troubleshooting Lab