612 lines
12 KiB
Markdown
612 lines
12 KiB
Markdown
# CI/CD Security
|
|
|
|
Comprehensive guide to securing CI/CD pipelines, secrets management, and supply chain security.
|
|
|
|
## Table of Contents
|
|
|
|
- [Secrets Management](#secrets-management)
|
|
- [OIDC Authentication](#oidc-authentication)
|
|
- [Supply Chain Security](#supply-chain-security)
|
|
- [Access Control](#access-control)
|
|
- [Secure Pipeline Patterns](#secure-pipeline-patterns)
|
|
- [Vulnerability Scanning](#vulnerability-scanning)
|
|
|
|
---
|
|
|
|
## Secrets Management
|
|
|
|
### Never Commit Secrets
|
|
|
|
**Prevention methods:**
|
|
- Use `.gitignore` for sensitive files
|
|
- Enable pre-commit hooks (git-secrets, gitleaks)
|
|
- Use secret scanning (GitHub, GitLab)
|
|
|
|
**If secrets are exposed:**
|
|
1. Rotate compromised credentials immediately
|
|
2. Remove from git history: `git filter-repo` or BFG Repo-Cleaner
|
|
3. Audit access logs for unauthorized usage
|
|
|
|
### Platform Secret Stores
|
|
|
|
**GitHub Secrets:**
|
|
```yaml
|
|
# Repository, Environment, or Organization secrets
|
|
steps:
|
|
- name: Deploy
|
|
env:
|
|
API_KEY: ${{ secrets.API_KEY }}
|
|
DB_PASSWORD: ${{ secrets.DB_PASSWORD }}
|
|
run: ./deploy.sh
|
|
```
|
|
|
|
**Secret hierarchy:**
|
|
1. Environment secrets (highest priority)
|
|
2. Repository secrets
|
|
3. Organization secrets (lowest priority)
|
|
|
|
**GitLab CI/CD Variables:**
|
|
```yaml
|
|
# Project > Settings > CI/CD > Variables
|
|
deploy:
|
|
script:
|
|
- echo $API_KEY
|
|
- deploy --token $DEPLOY_TOKEN
|
|
variables:
|
|
ENVIRONMENT: "production" # Non-secret variable
|
|
```
|
|
|
|
**Variable types:**
|
|
- **Protected:** Only available on protected branches
|
|
- **Masked:** Hidden in job logs
|
|
- **Environment scope:** Limit to specific environments
|
|
|
|
### External Secret Management
|
|
|
|
**HashiCorp Vault:**
|
|
```yaml
|
|
# GitHub Actions
|
|
- uses: hashicorp/vault-action@v3
|
|
with:
|
|
url: https://vault.example.com
|
|
method: jwt
|
|
role: cicd-role
|
|
secrets: |
|
|
secret/data/app api_key | API_KEY ;
|
|
secret/data/db password | DB_PASSWORD
|
|
```
|
|
|
|
**AWS Secrets Manager:**
|
|
```yaml
|
|
- name: Get secrets
|
|
run: |
|
|
SECRET=$(aws secretsmanager get-secret-value \
|
|
--secret-id prod/api/key \
|
|
--query SecretString --output text)
|
|
echo "::add-mask::$SECRET"
|
|
echo "API_KEY=$SECRET" >> $GITHUB_ENV
|
|
```
|
|
|
|
**Azure Key Vault:**
|
|
```yaml
|
|
- uses: Azure/get-keyvault-secrets@v1
|
|
with:
|
|
keyvault: "my-keyvault"
|
|
secrets: 'api-key, db-password'
|
|
```
|
|
|
|
### Secret Rotation
|
|
|
|
**Implement rotation policies:**
|
|
```yaml
|
|
check-secret-age:
|
|
steps:
|
|
- name: Check secret age
|
|
run: |
|
|
CREATED=$(aws secretsmanager describe-secret \
|
|
--secret-id myapp/api-key \
|
|
--query 'CreatedDate' --output text)
|
|
AGE=$(( ($(date +%s) - $(date -d "$CREATED" +%s)) / 86400 ))
|
|
if [ $AGE -gt 90 ]; then
|
|
echo "Secret is $AGE days old, rotation required"
|
|
exit 1
|
|
fi
|
|
```
|
|
|
|
**Best practices:**
|
|
- Rotate secrets every 90 days
|
|
- Use short-lived credentials when possible
|
|
- Audit secret access logs
|
|
- Automate rotation where possible
|
|
|
|
---
|
|
|
|
## OIDC Authentication
|
|
|
|
### Why OIDC?
|
|
|
|
**Benefits over static credentials:**
|
|
- No long-lived secrets in CI/CD
|
|
- Automatic token expiration
|
|
- Fine-grained permissions
|
|
- Audit trail of authentication
|
|
|
|
### GitHub Actions OIDC
|
|
|
|
**AWS example:**
|
|
```yaml
|
|
permissions:
|
|
id-token: write # Required for OIDC
|
|
contents: read
|
|
|
|
jobs:
|
|
deploy:
|
|
steps:
|
|
- uses: aws-actions/configure-aws-credentials@v4
|
|
with:
|
|
role-to-assume: arn:aws:iam::123456789:role/GitHubActionsRole
|
|
aws-region: us-east-1
|
|
|
|
- run: aws s3 sync dist/ s3://my-bucket
|
|
```
|
|
|
|
**AWS IAM Trust Policy:**
|
|
```json
|
|
{
|
|
"Version": "2012-10-17",
|
|
"Statement": [{
|
|
"Effect": "Allow",
|
|
"Principal": {
|
|
"Federated": "arn:aws:iam::123456789:oidc-provider/token.actions.githubusercontent.com"
|
|
},
|
|
"Action": "sts:AssumeRoleWithWebIdentity",
|
|
"Condition": {
|
|
"StringEquals": {
|
|
"token.actions.githubusercontent.com:aud": "sts.amazonaws.com",
|
|
"token.actions.githubusercontent.com:sub": "repo:owner/repo:ref:refs/heads/main"
|
|
}
|
|
}
|
|
}]
|
|
}
|
|
```
|
|
|
|
**GCP example:**
|
|
```yaml
|
|
- uses: google-github-actions/auth@v2
|
|
with:
|
|
workload_identity_provider: 'projects/123/locations/global/workloadIdentityPools/github/providers/github-provider'
|
|
service_account: 'github-actions@project.iam.gserviceaccount.com'
|
|
|
|
- run: gcloud storage cp dist/* gs://my-bucket
|
|
```
|
|
|
|
**Azure example:**
|
|
```yaml
|
|
- uses: azure/login@v2
|
|
with:
|
|
client-id: ${{ secrets.AZURE_CLIENT_ID }}
|
|
tenant-id: ${{ secrets.AZURE_TENANT_ID }}
|
|
subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
|
|
|
|
- run: az storage blob upload-batch -d mycontainer -s dist/
|
|
```
|
|
|
|
### GitLab OIDC
|
|
|
|
**Configure ID token:**
|
|
```yaml
|
|
deploy:
|
|
id_tokens:
|
|
GITLAB_OIDC_TOKEN:
|
|
aud: https://aws.amazonaws.com
|
|
script:
|
|
- |
|
|
CREDENTIALS=$(aws sts assume-role-with-web-identity \
|
|
--role-arn $AWS_ROLE_ARN \
|
|
--role-session-name gitlab-ci \
|
|
--web-identity-token $GITLAB_OIDC_TOKEN \
|
|
--duration-seconds 3600)
|
|
```
|
|
|
|
**Vault integration:**
|
|
```yaml
|
|
deploy:
|
|
id_tokens:
|
|
VAULT_ID_TOKEN:
|
|
aud: https://vault.example.com
|
|
before_script:
|
|
- export VAULT_TOKEN=$(vault write -field=token auth/jwt/login role=cicd-role jwt=$VAULT_ID_TOKEN)
|
|
```
|
|
|
|
---
|
|
|
|
## Supply Chain Security
|
|
|
|
### Dependency Verification
|
|
|
|
**Lock files:**
|
|
- Always commit lock files
|
|
- Use `npm ci`, not `npm install`
|
|
- Enable `--frozen-lockfile` (Yarn) or `--frozen-lockfile` (pnpm)
|
|
|
|
**Checksum verification:**
|
|
```yaml
|
|
- name: Verify dependencies
|
|
run: |
|
|
npm ci --audit=true
|
|
npx lockfile-lint --path package-lock.json --validate-https
|
|
```
|
|
|
|
**SBOM generation:**
|
|
```yaml
|
|
- name: Generate SBOM
|
|
run: |
|
|
syft dir:. -o spdx-json > sbom.json
|
|
|
|
- uses: actions/upload-artifact@v4
|
|
with:
|
|
name: sbom
|
|
path: sbom.json
|
|
```
|
|
|
|
### Action/Workflow Security
|
|
|
|
**Pin to commit SHA (GitHub):**
|
|
```yaml
|
|
# Bad - mutable tag
|
|
- uses: actions/checkout@v4
|
|
|
|
# Better - specific version
|
|
- uses: actions/checkout@v4.1.0
|
|
|
|
# Best - pinned to SHA
|
|
- uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.0
|
|
```
|
|
|
|
**Verify action sources:**
|
|
- Only use actions from trusted sources
|
|
- Review action code before first use
|
|
- Monitor Dependabot alerts for actions
|
|
- Use verified creators when possible
|
|
|
|
**GitLab include verification:**
|
|
```yaml
|
|
include:
|
|
- project: 'security/ci-templates'
|
|
ref: 'v2.1.0' # Pin to specific version
|
|
file: '/security-scan.yml'
|
|
```
|
|
|
|
### Container Image Security
|
|
|
|
**Use specific tags:**
|
|
```yaml
|
|
# Bad
|
|
image: node:latest
|
|
|
|
# Good
|
|
image: node:20.11.0-alpine
|
|
|
|
# Best
|
|
image: node:20.11.0-alpine@sha256:abc123...
|
|
```
|
|
|
|
**Minimal base images:**
|
|
```dockerfile
|
|
# Prefer distroless or alpine
|
|
FROM gcr.io/distroless/node20-debian12
|
|
|
|
# Or alpine
|
|
FROM node:20-alpine
|
|
```
|
|
|
|
**Image scanning:**
|
|
```yaml
|
|
- name: Build image
|
|
run: docker build -t myapp:${{ github.sha }} .
|
|
|
|
- name: Scan image
|
|
run: |
|
|
trivy image --severity HIGH,CRITICAL myapp:${{ github.sha }}
|
|
grype myapp:${{ github.sha }}
|
|
```
|
|
|
|
### Code Signing
|
|
|
|
**Sign commits:**
|
|
```bash
|
|
git config --global user.signingkey <key-id>
|
|
git config --global commit.gpgsign true
|
|
```
|
|
|
|
**Verify signed commits (GitHub):**
|
|
```yaml
|
|
- name: Verify signatures
|
|
run: |
|
|
git verify-commit HEAD || exit 1
|
|
```
|
|
|
|
**Sign artifacts:**
|
|
```yaml
|
|
- name: Sign release
|
|
run: |
|
|
cosign sign myregistry/myapp:${{ github.sha }}
|
|
```
|
|
|
|
---
|
|
|
|
## Access Control
|
|
|
|
### Principle of Least Privilege
|
|
|
|
**GitHub permissions:**
|
|
```yaml
|
|
# Minimal permissions
|
|
permissions:
|
|
contents: read # Only read code
|
|
pull-requests: write # Comment on PRs
|
|
|
|
jobs:
|
|
deploy:
|
|
permissions:
|
|
contents: read
|
|
id-token: write # For OIDC
|
|
```
|
|
|
|
**GitLab protected branches:**
|
|
- Configure in Settings > Repository > Protected branches
|
|
- Restrict who can push and merge
|
|
- Require approval before merge
|
|
|
|
### Branch Protection
|
|
|
|
**GitHub branch protection rules:**
|
|
- Require pull request reviews
|
|
- Require status checks to pass
|
|
- Require signed commits
|
|
- Require linear history
|
|
- Include administrators
|
|
- Restrict who can push
|
|
|
|
**GitLab merge request approval rules:**
|
|
```yaml
|
|
# .gitlab/CODEOWNERS
|
|
* @senior-devs
|
|
/infra/ @devops-team
|
|
/security/ @security-team
|
|
```
|
|
|
|
### Environment Protection
|
|
|
|
**GitHub environment rules:**
|
|
- Required reviewers (up to 6)
|
|
- Wait timer before deployment
|
|
- Deployment branches (limit to specific branches)
|
|
- Custom deployment protection rules
|
|
|
|
**GitLab deployment protection:**
|
|
```yaml
|
|
production:
|
|
environment:
|
|
name: production
|
|
rules:
|
|
- if: '$CI_COMMIT_BRANCH == "main"'
|
|
when: manual # Require manual trigger
|
|
only:
|
|
variables:
|
|
- $APPROVED == "true"
|
|
```
|
|
|
|
### Audit Logging
|
|
|
|
**Enable audit logs:**
|
|
- GitHub: Enterprise > Settings > Audit log
|
|
- GitLab: Admin Area > Monitoring > Audit Events
|
|
|
|
**Monitor for:**
|
|
- Secret access
|
|
- Permission changes
|
|
- Workflow modifications
|
|
- Deployment approvals
|
|
|
|
---
|
|
|
|
## Secure Pipeline Patterns
|
|
|
|
### Isolate Untrusted Code
|
|
|
|
**Separate test from deploy:**
|
|
```yaml
|
|
test:
|
|
# Runs on PRs from forks
|
|
permissions:
|
|
contents: read
|
|
pull-requests: write
|
|
|
|
deploy:
|
|
if: github.event_name == 'push' # Not on PR
|
|
permissions:
|
|
contents: read
|
|
id-token: write
|
|
```
|
|
|
|
**GitLab fork protection:**
|
|
```yaml
|
|
deploy:
|
|
rules:
|
|
- if: '$CI_PROJECT_PATH == "myorg/myrepo"' # Only from main repo
|
|
- if: '$CI_COMMIT_BRANCH == "main"'
|
|
```
|
|
|
|
### Sanitize Inputs
|
|
|
|
**Avoid command injection:**
|
|
```yaml
|
|
# Bad - command injection risk
|
|
- run: echo "Title: ${{ github.event.issue.title }}"
|
|
|
|
# Good - use environment variable
|
|
- env:
|
|
TITLE: ${{ github.event.issue.title }}
|
|
run: echo "Title: $TITLE"
|
|
```
|
|
|
|
**Validate inputs:**
|
|
```yaml
|
|
- name: Validate version
|
|
run: |
|
|
if [[ ! "${{ inputs.version }}" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
|
|
echo "Invalid version format"
|
|
exit 1
|
|
fi
|
|
```
|
|
|
|
### Network Restrictions
|
|
|
|
**Limit egress:**
|
|
```yaml
|
|
# GitHub Actions with StepSecurity
|
|
- uses: step-security/harden-runner@v2
|
|
with:
|
|
egress-policy: block
|
|
allowed-endpoints: |
|
|
api.github.com:443
|
|
npmjs.org:443
|
|
```
|
|
|
|
**GitLab network policy:**
|
|
```yaml
|
|
# Kubernetes NetworkPolicy for GitLab Runner pods
|
|
apiVersion: networking.k8s.io/v1
|
|
kind: NetworkPolicy
|
|
metadata:
|
|
name: gitlab-runner-policy
|
|
spec:
|
|
podSelector:
|
|
matchLabels:
|
|
app: gitlab-runner
|
|
policyTypes:
|
|
- Egress
|
|
egress:
|
|
- to:
|
|
- namespaceSelector: {}
|
|
ports:
|
|
- protocol: TCP
|
|
port: 443
|
|
```
|
|
|
|
---
|
|
|
|
## Vulnerability Scanning
|
|
|
|
### Dependency Scanning
|
|
|
|
**npm audit:**
|
|
```yaml
|
|
- run: npm audit --audit-level=high
|
|
```
|
|
|
|
**Snyk:**
|
|
```yaml
|
|
- uses: snyk/actions/node@master
|
|
env:
|
|
SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}
|
|
with:
|
|
args: --severity-threshold=high
|
|
```
|
|
|
|
**GitLab Dependency Scanning:**
|
|
```yaml
|
|
include:
|
|
- template: Security/Dependency-Scanning.gitlab-ci.yml
|
|
```
|
|
|
|
### Static Application Security Testing (SAST)
|
|
|
|
**CodeQL (GitHub):**
|
|
```yaml
|
|
- uses: github/codeql-action/init@v3
|
|
with:
|
|
languages: javascript, python
|
|
|
|
- uses: github/codeql-action/autobuild@v3
|
|
|
|
- uses: github/codeql-action/analyze@v3
|
|
```
|
|
|
|
**SonarQube:**
|
|
```yaml
|
|
- uses: sonarsource/sonarqube-scan-action@master
|
|
env:
|
|
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
|
|
```
|
|
|
|
### Container Scanning
|
|
|
|
**Trivy:**
|
|
```yaml
|
|
- run: |
|
|
docker build -t myapp .
|
|
trivy image --severity HIGH,CRITICAL --exit-code 1 myapp
|
|
```
|
|
|
|
**Grype:**
|
|
```yaml
|
|
- uses: anchore/scan-action@v3
|
|
with:
|
|
image: myapp:latest
|
|
fail-build: true
|
|
severity-cutoff: high
|
|
```
|
|
|
|
### Dynamic Application Security Testing (DAST)
|
|
|
|
**OWASP ZAP:**
|
|
```yaml
|
|
dast:
|
|
stage: test
|
|
image: owasp/zap2docker-stable
|
|
script:
|
|
- zap-baseline.py -t https://staging.example.com -r report.html
|
|
artifacts:
|
|
paths:
|
|
- report.html
|
|
```
|
|
|
|
---
|
|
|
|
## Security Checklist
|
|
|
|
### Repository Level
|
|
- [ ] Enable branch protection
|
|
- [ ] Require code review
|
|
- [ ] Enable secret scanning
|
|
- [ ] Configure CODEOWNERS
|
|
- [ ] Enable signed commits
|
|
- [ ] Audit third-party integrations
|
|
|
|
### Pipeline Level
|
|
- [ ] Use OIDC instead of static credentials
|
|
- [ ] Pin actions/includes to specific versions
|
|
- [ ] Minimize permissions
|
|
- [ ] Sanitize user inputs
|
|
- [ ] Enable vulnerability scanning
|
|
- [ ] Separate test from deploy workflows
|
|
- [ ] Add security gates
|
|
|
|
### Secrets Management
|
|
- [ ] Use platform secret stores
|
|
- [ ] Enable secret masking
|
|
- [ ] Rotate secrets regularly
|
|
- [ ] Use short-lived credentials
|
|
- [ ] Audit secret access
|
|
- [ ] Never log secrets
|
|
|
|
### Monitoring & Response
|
|
- [ ] Enable audit logging
|
|
- [ ] Monitor for security alerts
|
|
- [ ] Set up incident response plan
|
|
- [ ] Regular security reviews
|
|
- [ ] Dependency update automation
|
|
- [ ] Security training for team
|