Building a CI/CD Pipeline From Scratch: A Practical Guide for Developers
What CI/CD Actually Does (Plain English)
Pipeline Architecture
Setting Up Your First Pipeline with GitHub Actions
Minimal CI Pipeline (Node.js Example)
Adding CD: Deploy to a Server
Docker-Based Deploy (More Portable)
Environment Separation
Caching Dependencies
Matrix Testing: Test Across Multiple Versions
Secrets Management
When to Use CI/CD
When NOT to Use (or Keep It Simple)
Common Mistakes
Full Pipeline at a Glance
Practical Takeaways Originally inspired by Akoode's CI/CD pipeline guide — rewritten here with more depth, code, and less hand-waving. I've seen teams spend hours manually running tests, zipping build artifacts, SSHing into servers, and crossing fingers before every deploy. CI/CD pipelines exist to kill that workflow. This guide skips the theory lecture and gets into how to actually build one. We'll use GitHub Actions as the CI/CD platform — it's free for public repos, tightly integrated with GitHub, and requires zero external infrastructure to get started. The pipeline is just a sequence of automated steps triggered by a git event. git push / PR open
│▼┌─────────────┐│ Trigger │ ← GitHub webhook fires└─────┬───────┘│▼┌─────────────┐│ Build │ ← Install deps, compile, bundle└─────┬───────┘│▼┌─────────────┐│ Test │ ← Unit, integration, lint└─────┬───────┘│▼┌─────────────┐│ Deploy │ ← Push to staging/prod└─────────────┘Each stage is a job. Jobs run on runners (GitHub-hosted VMs or your own). They can run in parallel or sequentially with dependencies between them. Create this file in your repo: .github/workflows/ci-cd.yml That's it. Push this file, and every PR gets auto-tested. No server, no webhook config. After CI passes, deploy to production. Here we'll SSH into a VPS and pull + restart: Store your SSH key and server IP in GitHub Secrets (Settings → Secrets and variables → Actions). Never hardcode credentials in the YAML. If you're deploying containers: Using the commit SHA as the image tag gives you a clean audit trail — every deploy is traceable to a specific commit. Don't deploy everything to production. Use branch-based environment targeting: Pair with GitHub Environments (Settings → Environments) to add manual approval gates before production: GitHub will pause and require an approver before proceeding. Useful for regulated teams or high-stakes deploys. Don't reinstall node_modules from scratch on every run. Cache it: This alone can cut pipeline runtime by 60–70% on most projects. Need to support Node 18 and 20? Don't write two jobs: GitHub runs these in parallel — fast and zero duplication. ✅ Any team with more than one developer✅ Frequent deploys (more than once a week)✅ You have a test suite (even a small one)✅ Multiple environments (dev, staging, prod)✅ Open source projects where contributors submit PRs ❌ Solo hobby project with no test suite — a basic deploy script is fine❌ Legacy monolith where builds take 45 minutes — fix the build first❌ Highly regulated environments where automated prod deploys are prohibited — use CD to staging only, with manual prod promotion 1. Not pinning action versions 2. Running everything on every pushUse path filters to skip unnecessary runs: 3. Storing secrets in env filesDon't commit .env.production to the repo. Use GitHub Secrets + a secrets manager (HashiCorp Vault, AWS Secrets Manager) for anything sensitive. 4. No rollback plan
Tag your Docker images with the git SHA. If prod breaks, you can redeploy the previous image in 30 seconds. The goal isn't a perfect pipeline on day one. It's getting something automated, then adding stages as your confidence and test coverage grow. Templates let you quickly answer FAQs or store snippets for re-use. Hide child comments as well For further actions, you may consider blocking this person and/or reporting abuse
name: CI on: push: branches: [main] pull_request: branches: [main] jobs: build-and-test: runs-on: ubuntu-latest steps: - name: Checkout code uses: actions/checkout@v4 - name: Set up Node.js uses: actions/setup-node@v4 with: node-version: '20' cache: 'npm' - name: Install dependencies run: npm ci - name: Run linter run: npm run lint - name: Run tests run: npm test - name: Build run: npm run build
name: CI on: push: branches: [main] pull_request: branches: [main] jobs: build-and-test: runs-on: ubuntu-latest steps: - name: Checkout code uses: actions/checkout@v4 - name: Set up Node.js uses: actions/setup-node@v4 with: node-version: '20' cache: 'npm' - name: Install dependencies run: npm ci - name: Run linter run: npm run lint - name: Run tests run: npm test - name: Build run: npm run build
name: CI on: push: branches: [main] pull_request: branches: [main] jobs: build-and-test: runs-on: ubuntu-latest steps: - name: Checkout code uses: actions/checkout@v4 - name: Set up Node.js uses: actions/setup-node@v4 with: node-version: '20' cache: 'npm' - name: Install dependencies run: npm ci - name: Run linter run: npm run lint - name: Run tests run: npm test - name: Build run: npm run build
deploy: needs: build-and-test # only runs if CI passes runs-on: ubuntu-latest if: github.ref == 'refs/heads/main' # only on main branch steps: - name: Deploy via SSH uses: appleboy/[email protected] with: host: ${{ secrets.DEPLOY_HOST }} username: ${{ secrets.DEPLOY_USER }} key: ${{ secrets.DEPLOY_SSH_KEY }} script: | cd /var/www/myapp git pull origin main npm ci --omit=dev pm2 restart myapp
deploy: needs: build-and-test # only runs if CI passes runs-on: ubuntu-latest if: github.ref == 'refs/heads/main' # only on main branch steps: - name: Deploy via SSH uses: appleboy/[email protected] with: host: ${{ secrets.DEPLOY_HOST }} username: ${{ secrets.DEPLOY_USER }} key: ${{ secrets.DEPLOY_SSH_KEY }} script: | cd /var/www/myapp git pull origin main npm ci --omit=dev pm2 restart myapp
deploy: needs: build-and-test # only runs if CI passes runs-on: ubuntu-latest if: github.ref == 'refs/heads/main' # only on main branch steps: - name: Deploy via SSH uses: appleboy/[email protected] with: host: ${{ secrets.DEPLOY_HOST }} username: ${{ secrets.DEPLOY_USER }} key: ${{ secrets.DEPLOY_SSH_KEY }} script: | cd /var/www/myapp git pull origin main npm ci --omit=dev pm2 restart myapp
build-and-push: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Log in to Docker Hub uses: docker/login-action@v3 with: username: ${{ secrets.DOCKER_USERNAME }} password: ${{ secrets.DOCKER_TOKEN }} - name: Build and push Docker image uses: docker/build-push-action@v5 with: context: . push: true tags: yourusername/myapp:${{ github.sha }}
build-and-push: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Log in to Docker Hub uses: docker/login-action@v3 with: username: ${{ secrets.DOCKER_USERNAME }} password: ${{ secrets.DOCKER_TOKEN }} - name: Build and push Docker image uses: docker/build-push-action@v5 with: context: . push: true tags: yourusername/myapp:${{ github.sha }}
build-and-push: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Log in to Docker Hub uses: docker/login-action@v3 with: username: ${{ secrets.DOCKER_USERNAME }} password: ${{ secrets.DOCKER_TOKEN }} - name: Build and push Docker image uses: docker/build-push-action@v5 with: context: . push: true tags: yourusername/myapp:${{ github.sha }}
on: push: branches: - main # → production - staging # → staging env - 'feat/**' # → preview envs (optional)
on: push: branches: - main # → production - staging # → staging env - 'feat/**' # → preview envs (optional)
on: push: branches: - main # → production - staging # → staging env - 'feat/**' # → preview envs (optional)
deploy-prod: environment: name: production url: https://myapp.com
deploy-prod: environment: name: production url: https://myapp.com
deploy-prod: environment: name: production url: https://myapp.com
- uses: actions/setup-node@v4 with: node-version: '20' cache: 'npm' # ← this line handles caching automatically
- uses: actions/setup-node@v4 with: node-version: '20' cache: 'npm' # ← this line handles caching automatically
- uses: actions/setup-node@v4 with: node-version: '20' cache: 'npm' # ← this line handles caching automatically
- uses: actions/setup-python@v5 with: python-version: '3.12' cache: 'pip'
- uses: actions/setup-python@v5 with: python-version: '3.12' cache: 'pip'
- uses: actions/setup-python@v5 with: python-version: '3.12' cache: 'pip'
test: runs-on: ubuntu-latest strategy: matrix: node-version: [18, 20, 22] steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 with: node-version: ${{ matrix.node-version }} - run: npm ci && npm test
test: runs-on: ubuntu-latest strategy: matrix: node-version: [18, 20, 22] steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 with: node-version: ${{ matrix.node-version }} - run: npm ci && npm test
test: runs-on: ubuntu-latest strategy: matrix: node-version: [18, 20, 22] steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 with: node-version: ${{ matrix.node-version }} - run: npm ci && npm test
- name: Deploy env: API_KEY: ${{ secrets.PROD_API_KEY }} run: ./deploy.sh
- name: Deploy env: API_KEY: ${{ secrets.PROD_API_KEY }} run: ./deploy.sh
- name: Deploy env: API_KEY: ${{ secrets.PROD_API_KEY }} run: ./deploy.sh
# Bad — can break silently when the action updates
uses: actions/checkout@main # Good — locked to a specific version
uses: actions/checkout@v4
# Bad — can break silently when the action updates
uses: actions/checkout@main # Good — locked to a specific version
uses: actions/checkout@v4
# Bad — can break silently when the action updates
uses: actions/checkout@main # Good — locked to a specific version
uses: actions/checkout@v4
on: push: paths: - 'src/**' - 'package.json'
on: push: paths: - 'src/**' - 'package.json'
on: push: paths: - 'src/**' - 'package.json'
name: CI/CD Pipeline on: push: branches: [main] pull_request: branches: [main] jobs: test: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 with: node-version: '20' cache: 'npm' - run: npm ci - run: npm run lint - run: npm test deploy: needs: test runs-on: ubuntu-latest if: github.ref == 'refs/heads/main' environment: production steps: - uses: actions/checkout@v4 - name: Build Docker image run: docker build -t myapp:${{ github.sha }} . - name: Push to registry run: | echo ${{ secrets.DOCKER_TOKEN }} | docker login -u ${{ secrets.DOCKER_USERNAME }} --password-stdin docker push myapp:${{ github.sha }} - name: Deploy uses: appleboy/[email protected] with: host: ${{ secrets.DEPLOY_HOST }} username: ${{ secrets.DEPLOY_USER }} key: ${{ secrets.DEPLOY_SSH_KEY }} script: | docker pull myapp:${{ github.sha }} docker stop myapp || true docker run -d --name myapp -p 3000:3000 myapp:${{ github.sha }}
name: CI/CD Pipeline on: push: branches: [main] pull_request: branches: [main] jobs: test: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 with: node-version: '20' cache: 'npm' - run: npm ci - run: npm run lint - run: npm test deploy: needs: test runs-on: ubuntu-latest if: github.ref == 'refs/heads/main' environment: production steps: - uses: actions/checkout@v4 - name: Build Docker image run: docker build -t myapp:${{ github.sha }} . - name: Push to registry run: | echo ${{ secrets.DOCKER_TOKEN }} | docker login -u ${{ secrets.DOCKER_USERNAME }} --password-stdin docker push myapp:${{ github.sha }} - name: Deploy uses: appleboy/[email protected] with: host: ${{ secrets.DEPLOY_HOST }} username: ${{ secrets.DEPLOY_USER }} key: ${{ secrets.DEPLOY_SSH_KEY }} script: | docker pull myapp:${{ github.sha }} docker stop myapp || true docker run -d --name myapp -p 3000:3000 myapp:${{ github.sha }}
name: CI/CD Pipeline on: push: branches: [main] pull_request: branches: [main] jobs: test: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 with: node-version: '20' cache: 'npm' - run: npm ci - run: npm run lint - run: npm test deploy: needs: test runs-on: ubuntu-latest if: github.ref == 'refs/heads/main' environment: production steps: - uses: actions/checkout@v4 - name: Build Docker image run: docker build -t myapp:${{ github.sha }} . - name: Push to registry run: | echo ${{ secrets.DOCKER_TOKEN }} | docker login -u ${{ secrets.DOCKER_USERNAME }} --password-stdin docker push myapp:${{ github.sha }} - name: Deploy uses: appleboy/[email protected] with: host: ${{ secrets.DEPLOY_HOST }} username: ${{ secrets.DEPLOY_USER }} key: ${{ secrets.DEPLOY_SSH_KEY }} script: | docker pull myapp:${{ github.sha }} docker stop myapp || true docker run -d --name myapp -p 3000:3000 myapp:${{ github.sha }} - CI (Continuous Integration): Every time code is pushed or a PR is opened, automatically run your build and tests. Catch breakage early, not in prod.
- CD (Continuous Delivery/Deployment): After CI passes, automatically ship the artifact to staging or production — no human clicking "deploy" required. - Store secrets in GitHub Secrets, not in .env files committed to the repo
- Use environment-scoped secrets for prod vs staging differences
- Rotate secrets regularly (SSH keys, API tokens)
- Never echo secrets in run steps — they'll be masked in logs, but it's still bad practice - Start small: even a single npm test in CI adds real value
- needs: keyword is your sequencing primitive — use it
- Branch protection rules + required CI checks = no broken code on main
- Commit SHA tagging on Docker images = instant rollback capability
- Cache dependencies — it's free performance
- Use GitHub Environments for approval gates before prod
