15 KiB
15 KiB
Setup CI/CD Pipeline Command
You are helping the user set up a CI/CD pipeline for automated testing, building, and deployment following Sngular's DevOps best practices.
Instructions
-
Determine the platform:
- GitHub Actions
- GitLab CI
- Jenkins
- CircleCI
- Azure DevOps
- Bitbucket Pipelines
-
Identify application type:
- Node.js/TypeScript application
- Python application
- Go application
- Frontend application (React, Vue, Next.js)
- Full-stack application
- Monorepo with multiple services
-
Ask about pipeline requirements:
- Linting and code quality checks
- Unit and integration tests
- Build and compile steps
- Docker image building
- Deployment targets (staging, production)
- Security scanning
- Performance testing
-
Determine trigger events:
- Push to main/master
- Pull requests
- Tag/release creation
- Scheduled runs
- Manual triggers
GitHub Actions Workflows
Basic CI Pipeline
# .github/workflows/ci.yml
name: CI
on:
push:
branches: [ main, develop ]
pull_request:
branches: [ main, develop ]
jobs:
lint:
name: Lint
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '20'
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Run linter
run: npm run lint
- name: Check formatting
run: npm run format:check
test:
name: Test
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [18, 20]
steps:
- uses: actions/checkout@v4
- name: Setup Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node-version }}
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Run tests
run: npm test -- --coverage
- name: Upload coverage
uses: codecov/codecov-action@v3
with:
files: ./coverage/coverage-final.json
flags: unittests
build:
name: Build
runs-on: ubuntu-latest
needs: [lint, test]
steps:
- uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '20'
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Build application
run: npm run build
- name: Upload build artifacts
uses: actions/upload-artifact@v4
with:
name: build
path: dist/
CI/CD with Docker
# .github/workflows/ci-cd.yml
name: CI/CD
on:
push:
branches: [ main ]
tags: [ 'v*' ]
pull_request:
branches: [ main ]
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}
jobs:
test:
name: Test
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '20'
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Run tests
run: npm test
security-scan:
name: Security Scan
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Run Trivy vulnerability scanner
uses: aquasecurity/trivy-action@master
with:
scan-type: 'fs'
scan-ref: '.'
format: 'sarif'
output: 'trivy-results.sarif'
- name: Upload Trivy results to GitHub Security
uses: github/codeql-action/upload-sarif@v2
with:
sarif_file: 'trivy-results.sarif'
build-and-push:
name: Build and Push Docker Image
runs-on: ubuntu-latest
needs: [test, security-scan]
if: github.event_name != 'pull_request'
permissions:
contents: read
packages: write
steps:
- uses: actions/checkout@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Log in to Container Registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tags: |
type=ref,event=branch
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=sha
- name: Build and push
uses: docker/build-push-action@v5
with:
context: .
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
deploy-staging:
name: Deploy to Staging
runs-on: ubuntu-latest
needs: build-and-push
if: github.ref == 'refs/heads/main'
environment:
name: staging
url: https://staging.example.com
steps:
- name: Deploy to staging
run: |
echo "Deploying to staging environment"
# Add deployment commands here
deploy-production:
name: Deploy to Production
runs-on: ubuntu-latest
needs: build-and-push
if: startsWith(github.ref, 'refs/tags/v')
environment:
name: production
url: https://example.com
steps:
- name: Deploy to production
run: |
echo "Deploying to production environment"
# Add deployment commands here
Monorepo Pipeline
# .github/workflows/monorepo-ci.yml
name: Monorepo CI
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
jobs:
detect-changes:
name: Detect Changes
runs-on: ubuntu-latest
outputs:
frontend: ${{ steps.filter.outputs.frontend }}
backend: ${{ steps.filter.outputs.backend }}
steps:
- uses: actions/checkout@v4
- uses: dorny/paths-filter@v2
id: filter
with:
filters: |
frontend:
- 'apps/frontend/**'
- 'packages/ui/**'
backend:
- 'apps/backend/**'
- 'packages/api/**'
test-frontend:
name: Test Frontend
runs-on: ubuntu-latest
needs: detect-changes
if: needs.detect-changes.outputs.frontend == 'true'
defaults:
run:
working-directory: apps/frontend
steps:
- uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '20'
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Run tests
run: npm test
- name: Build
run: npm run build
test-backend:
name: Test Backend
runs-on: ubuntu-latest
needs: detect-changes
if: needs.detect-changes.outputs.backend == 'true'
defaults:
run:
working-directory: apps/backend
services:
postgres:
image: postgres:16
env:
POSTGRES_PASSWORD: postgres
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
ports:
- 5432:5432
steps:
- uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '20'
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Run migrations
run: npm run migrate
env:
DATABASE_URL: postgresql://postgres:postgres@localhost:5432/test
- name: Run tests
run: npm test
env:
DATABASE_URL: postgresql://postgres:postgres@localhost:5432/test
- name: Build
run: npm run build
GitLab CI Pipeline
# .gitlab-ci.yml
stages:
- lint
- test
- build
- deploy
variables:
DOCKER_DRIVER: overlay2
DOCKER_TLS_CERTDIR: "/certs"
# Templates
.node_template: &node_template
image: node:20-alpine
cache:
key: ${CI_COMMIT_REF_SLUG}
paths:
- node_modules/
before_script:
- npm ci
lint:
<<: *node_template
stage: lint
script:
- npm run lint
- npm run format:check
test:unit:
<<: *node_template
stage: test
script:
- npm test -- --coverage
coverage: '/All files[^|]*\|[^|]*\s+([\d\.]+)/'
artifacts:
when: always
reports:
junit: junit.xml
coverage_report:
coverage_format: cobertura
path: coverage/cobertura-coverage.xml
test:e2e:
<<: *node_template
stage: test
services:
- postgres:16-alpine
variables:
POSTGRES_DB: testdb
POSTGRES_USER: testuser
POSTGRES_PASSWORD: testpass
DATABASE_URL: postgresql://testuser:testpass@postgres:5432/testdb
script:
- npm run test:e2e
build:
stage: build
image: docker:24
services:
- docker:24-dind
before_script:
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
script:
- docker build -t $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA .
- docker build -t $CI_REGISTRY_IMAGE:latest .
- docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA
- docker push $CI_REGISTRY_IMAGE:latest
only:
- main
- tags
deploy:staging:
stage: deploy
image: alpine:latest
before_script:
- apk add --no-cache curl
script:
- echo "Deploying to staging"
- curl -X POST $STAGING_WEBHOOK_URL
environment:
name: staging
url: https://staging.example.com
only:
- main
deploy:production:
stage: deploy
image: alpine:latest
before_script:
- apk add --no-cache curl
script:
- echo "Deploying to production"
- curl -X POST $PRODUCTION_WEBHOOK_URL
environment:
name: production
url: https://example.com
when: manual
only:
- tags
Jenkins Pipeline
// Jenkinsfile
pipeline {
agent any
environment {
NODE_VERSION = '20'
DOCKER_REGISTRY = 'registry.example.com'
IMAGE_NAME = 'myapp'
}
stages {
stage('Checkout') {
steps {
checkout scm
}
}
stage('Install Dependencies') {
agent {
docker {
image "node:${NODE_VERSION}-alpine"
reuseNode true
}
}
steps {
sh 'npm ci'
}
}
stage('Lint') {
agent {
docker {
image "node:${NODE_VERSION}-alpine"
reuseNode true
}
}
steps {
sh 'npm run lint'
}
}
stage('Test') {
agent {
docker {
image "node:${NODE_VERSION}-alpine"
reuseNode true
}
}
steps {
sh 'npm test -- --coverage'
}
post {
always {
junit 'junit.xml'
publishHTML([
allowMissing: false,
alwaysLinkToLastBuild: true,
keepAll: true,
reportDir: 'coverage',
reportFiles: 'index.html',
reportName: 'Coverage Report'
])
}
}
}
stage('Build') {
agent {
docker {
image "node:${NODE_VERSION}-alpine"
reuseNode true
}
}
steps {
sh 'npm run build'
}
}
stage('Docker Build') {
when {
branch 'main'
}
steps {
script {
docker.build("${DOCKER_REGISTRY}/${IMAGE_NAME}:${BUILD_NUMBER}")
docker.build("${DOCKER_REGISTRY}/${IMAGE_NAME}:latest")
}
}
}
stage('Docker Push') {
when {
branch 'main'
}
steps {
script {
docker.withRegistry("https://${DOCKER_REGISTRY}", 'docker-credentials') {
docker.image("${DOCKER_REGISTRY}/${IMAGE_NAME}:${BUILD_NUMBER}").push()
docker.image("${DOCKER_REGISTRY}/${IMAGE_NAME}:latest").push()
}
}
}
}
stage('Deploy to Staging') {
when {
branch 'main'
}
steps {
sh """
kubectl set image deployment/myapp \
myapp=${DOCKER_REGISTRY}/${IMAGE_NAME}:${BUILD_NUMBER} \
--namespace=staging
"""
}
}
stage('Deploy to Production') {
when {
tag pattern: "v\\d+\\.\\d+\\.\\d+", comparator: "REGEXP"
}
steps {
input message: 'Deploy to production?', ok: 'Deploy'
sh """
kubectl set image deployment/myapp \
myapp=${DOCKER_REGISTRY}/${IMAGE_NAME}:${BUILD_NUMBER} \
--namespace=production
"""
}
}
}
post {
always {
cleanWs()
}
success {
echo 'Pipeline succeeded!'
}
failure {
echo 'Pipeline failed!'
// Send notification
}
}
}
Best Practices
1. Caching Dependencies
# GitHub Actions
- name: Cache dependencies
uses: actions/cache@v3
with:
path: ~/.npm
key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
restore-keys: |
${{ runner.os }}-node-
2. Matrix Builds
# Test multiple versions
strategy:
matrix:
node-version: [18, 20, 21]
os: [ubuntu-latest, windows-latest, macos-latest]
3. Conditional Execution
# Only run on specific branches
if: github.ref == 'refs/heads/main'
# Only run for PRs
if: github.event_name == 'pull_request'
# Only run for tags
if: startsWith(github.ref, 'refs/tags/')
4. Secrets Management
# Use secrets from repository settings
env:
DATABASE_URL: ${{ secrets.DATABASE_URL }}
API_KEY: ${{ secrets.API_KEY }}
5. Parallel Jobs
# Jobs run in parallel by default
jobs:
lint:
# ...
test:
# ...
security-scan:
# ...
6. Job Dependencies
jobs:
test:
# ...
build:
needs: test # Wait for test to complete
# ...
deploy:
needs: [test, build] # Wait for multiple jobs
# ...
Security Best Practices
- Store secrets in CI platform's secret management
- Use minimal permissions for CI tokens
- Scan dependencies for vulnerabilities
- Scan Docker images for security issues
- Don't log sensitive information
- Use branch protection rules
- Require status checks before merging
- Enable signed commits
Monitoring and Notifications
Slack Notifications (GitHub Actions)
- name: Slack Notification
uses: 8398a7/action-slack@v3
if: always()
with:
status: ${{ job.status }}
text: 'CI Pipeline ${{ job.status }}'
webhook_url: ${{ secrets.SLACK_WEBHOOK }}
Ask the user: "What CI/CD platform would you like to use?"