Module 5: CI/CD Pipelines
Automate your software delivery process with Continuous Integration and Continuous Deployment pipelines.
What is CI/CD?
CI/CD stands for Continuous Integration and Continuous Deployment. Think of it like an assembly line in a car factory. Instead of building each car by hand (slow and error-prone), the assembly line automates repetitive tasks, runs quality checks at each stage, and delivers finished cars faster and more reliably.
In software development, CI/CD automates the process of testing your code, building your application, and deploying it to production. Every time you push code changes, the pipeline automatically runs tests, checks for errors, and deploys your app if everything passes.
🏭 Real-World Analogy:
Imagine you're running a pizza restaurant:
- Without CI/CD: Each pizza is made completely by hand, one at a time. The chef makes the dough, adds toppings, bakes it, and delivers it. If something goes wrong, you only find out when the customer complains.
- With CI/CD: You have a pizza assembly line. Dough is prepared automatically, toppings are added at stations, quality checks happen at each step, and pizzas are delivered hot. If something's wrong (like burnt crust), the system catches it before it reaches the customer.
CI vs CD: What's the Difference?
🔄 Continuous Integration (CI)
CI is about automatically testing and validating code changes as soon as they're pushed to the repository.
- ✓ Automatically run tests on every commit
- ✓ Check code quality and style
- ✓ Build the application
- ✓ Catch bugs early before they reach production
🚀 Continuous Deployment (CD)
CD automatically deploys your application to production after all tests pass.
- ✓ Automatically deploy to staging/production
- ✓ Roll back if deployment fails
- ✓ Deploy multiple times per day
- ✓ Reduce manual deployment errors
GitHub Actions
GitHub Actions is a CI/CD platform built into GitHub. It lets you automate workflows directly in your repository. Think of it as hiring a robot assistant that watches your code repository and automatically runs tasks whenever you push changes.
Basic Workflow Structure
A GitHub Actions workflow is defined in a YAML file. Here's a simple example that runs tests on every push:
# .github/workflows/ci.yml
name: CI Pipeline
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
jobs:
test:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v3
- name: Setup Node.js
uses: actions/setup-node@v3
with:
node-version: '18'
- name: Install dependencies
run: npm install
- name: Run tests
run: npm test
- name: Build application
run: npm run build
📖 Understanding the Workflow:
- on: Triggers the workflow when code is pushed to main or a pull request is opened
- jobs: Defines what tasks to run (in this case, a "test" job)
- runs-on: Specifies the operating system (Ubuntu Linux)
- steps: Individual tasks that run in sequence
Complete CI/CD Pipeline with Deployment
Here's a more advanced workflow that tests, builds, and deploys your application:
# .github/workflows/deploy.yml
name: Deploy to Production
on:
push:
branches: [ main ]
jobs:
build-and-deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Run tests
run: |
npm install
npm test
- name: Build Docker image
run: docker build -t myapp:latest .
- name: Push to Docker Hub
run: |
echo ${DOCKER_PASSWORD} | docker login -u ${DOCKER_USERNAME} --password-stdin
docker push myapp:latest
- name: Deploy to server
run: |
ssh user@server 'docker pull myapp:latest'
ssh user@server 'docker-compose up -d'
GitLab CI/CD
GitLab CI/CD is similar to GitHub Actions but built into GitLab. It uses a file called .gitlab-ci.ymlto define your pipeline. GitLab CI is known for its powerful features and built-in container registry.
Basic GitLab Pipeline
# .gitlab-ci.yml
stages:
- test
- build
- deploy
test_job:
stage: test
image: node:18
script:
- npm install
- npm test
build_job:
stage: build
image: docker:latest
services:
- docker:dind
script:
- docker build -t myapp .
- docker push myapp
deploy_job:
stage: deploy
script:
- kubectl apply -f deployment.yaml
only:
- main
💡 Key Differences from GitHub Actions:
- • Uses stages to organize jobs (test, build, deploy)
- • Jobs in the same stage run in parallel
- • Built-in Docker support with docker:dind (Docker-in-Docker)
- • only: keyword controls when jobs run (similar to GitHub's "on")
Jenkins Pipeline
Jenkins is one of the oldest and most popular CI/CD tools. It's self-hosted (you run it on your own server) and highly customizable with thousands of plugins. Think of Jenkins as the Swiss Army knife of CI/CD - it can do almost anything, but requires more setup.
Declarative Pipeline
Jenkins uses a file called Jenkinsfile written in Groovy. Here's a declarative pipeline:
// Jenkinsfile
pipeline {
agent any
stages {
stage('Checkout') {
steps {
git 'https://github.com/user/repo.git'
}
}
stage('Test') {
steps {
sh 'npm install'
sh 'npm test'
}
}
stage('Build') {
steps {
sh 'npm run build'
}
}
stage('Deploy') {
when {
branch 'main'
}
steps {
sh './deploy.sh'
}
}
}
post {
success {
echo 'Pipeline succeeded!'
}
failure {
echo 'Pipeline failed!'
}
}
}
📖 Jenkins Concepts:
- agent: Where the pipeline runs (any available agent)
- stages: Major phases of your pipeline
- steps: Individual commands within a stage
- when: Conditional execution (only run on certain branches)
- post: Actions to run after pipeline completes
Automated Testing in CI/CD
Testing is the heart of CI/CD. Without automated tests, you can't confidently deploy code automatically. Think of tests as quality inspectors on an assembly line - they catch defects before products reach customers.
Types of Tests in CI/CD
🧪 Unit Tests
Test individual functions in isolation
Fast, run on every commit
🔗 Integration Tests
Test how components work together
Slower, run before deployment
🌐 End-to-End Tests
Test complete user workflows
Slowest, run before production
Example: Testing in GitHub Actions
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-node@v3
# Unit tests
- name: Run unit tests
run: npm run test:unit
# Integration tests
- name: Run integration tests
run: npm run test:integration
# Code coverage
- name: Generate coverage report
run: npm run test:coverage
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v3
Deployment Strategies
How you deploy your application matters. Different strategies offer different trade-offs between speed, safety, and complexity. Let's explore the most common approaches.
🔵 Blue-Green Deployment
Run two identical production environments: Blue (current) and Green (new). Deploy to Green, test it, then switch traffic from Blue to Green. If something breaks, instantly switch back to Blue.
Pros: Instant rollback, zero downtime
Cons: Requires double the infrastructure
Use when: You need guaranteed rollback capability
🐤 Canary Deployment
Deploy the new version to a small percentage of users (5-10%) first. Monitor for errors. If everything looks good, gradually increase traffic to the new version. Like sending a canary into a coal mine to test for danger.
Pros: Limits blast radius of bugs, gradual rollout
Cons: More complex to implement
Use when: You want to test in production with minimal risk
🔄 Rolling Deployment
Update servers one at a time (or in small batches). Server 1 gets the new version, then Server 2, then Server 3, etc. Like updating a fleet of delivery trucks one at a time while others keep delivering.
Pros: No extra infrastructure needed, zero downtime
Cons: Slower rollback, mixed versions during deployment
Use when: You have multiple servers and want simple deployments
🛠️ Hands-On Project: Build a Complete CI/CD Pipeline
Let's build a real CI/CD pipeline for a Node.js application using GitHub Actions. This pipeline will test, build, and deploy your app automatically.
Project Requirements:
- ✓ Run tests on every push
- ✓ Build Docker image if tests pass
- ✓ Deploy to staging on push to develop branch
- ✓ Deploy to production on push to main branch
- ✓ Send notifications on success/failure
Step 1: Create the Workflow File
Create .github/workflows/main.yml in your repository:
name: CI/CD Pipeline
on:
push:
branches: [ main, develop ]
pull_request:
branches: [ main ]
env:
NODE_VERSION: '18'
DOCKER_IMAGE: myapp
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Setup Node.js
uses: actions/setup-node@v3
with:
node-version: ${{ env.NODE_VERSION }}
- name: Install dependencies
run: npm ci
- name: Run linter
run: npm run lint
- name: Run tests
run: npm test
build:
needs: test
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Build Docker image
run: docker build -t ${{ env.DOCKER_IMAGE }} .
- name: Login to Docker Hub
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
- name: Push image
run: docker push ${{ env.DOCKER_IMAGE }}
deploy-staging:
needs: build
if: github.ref == 'refs/heads/develop'
runs-on: ubuntu-latest
steps:
- name: Deploy to staging
run: echo "Deploying to staging..."
deploy-production:
needs: build
if: github.ref == 'refs/heads/main'
runs-on: ubuntu-latest
steps:
- name: Deploy to production
run: echo "Deploying to production..."
💡 Key Features:
- • needs: Makes build wait for tests to pass
- • if: Conditional deployment based on branch
- • secrets: Securely store credentials (set in GitHub repo settings)
- • env: Environment variables for reusability
Best Practices
✅ Do
- ✓ Keep pipelines fast (under 10 minutes)
- ✓ Run tests in parallel when possible
- ✓ Use caching for dependencies
- ✓ Fail fast - run quick tests first
- ✓ Monitor pipeline metrics
- ✓ Use secrets for sensitive data
- ✓ Version your pipeline configuration
❌ Don't
- ✗ Hardcode credentials in workflows
- ✗ Skip tests to deploy faster
- ✗ Deploy directly to production without staging
- ✗ Ignore failed builds
- ✗ Make pipelines too complex
- ✗ Deploy on Fridays (seriously!)
- ✗ Forget to test rollback procedures
📚 Module Summary
You've learned how to automate your software delivery with CI/CD pipelines:
- ✓ Understanding CI/CD concepts and benefits
- ✓ Building pipelines with GitHub Actions, GitLab CI, and Jenkins
- ✓ Implementing automated testing strategies
- ✓ Choosing the right deployment strategy
- ✓ Following CI/CD best practices
Next up: Learn Infrastructure as Code to manage your infrastructure with the same automation principles!