zhongwei/gh-bandofai-puerto-plugins-engineering
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial commit

Browse Source
This commit is contained in:
Zhongwei Li
2025-11-29 17:59:49 +08:00
commit 6e1bba5e72
16 changed files with 7270 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

25
.claude-plugin/plugin.json Normal file
View File

@@ -0,0 +1,25 @@
{
"name": "engineering",
"description": "Full-stack engineering team with 7 specialized roles covering frontend, backend, mobile, data, DevOps, and QA",
"version": "1.0.0",
"author": {
"name": "Puerto"
},
"skills": [
"./skills/frontend-development",
"./skills/backend-architecture",
"./skills/mobile-development",
"./skills/devops-practices",
"./skills/data-engineering",
"./skills/testing-practices"
],
"agents": [
"./agents/engineering-lead.md",
"./agents/frontend-engineer.md",
"./agents/backend-engineer.md",
"./agents/devops-engineer.md",
"./agents/mobile-engineer.md",
"./agents/data-engineer.md",
"./agents/qa-engineer.md"
]
}

3
README.md Normal file
View File

@@ -0,0 +1,3 @@
# engineering
Full-stack engineering team with 7 specialized roles covering frontend, backend, mobile, data, DevOps, and QA

37
agents/backend-engineer.md Normal file
View File

@@ -0,0 +1,37 @@
# Backend Engineer
PROACTIVELY use for backend development including APIs, services, databases, and server-side logic. Handles data models, business logic, authentication, and integration with external services.
**Core Capabilities:**
- API design and implementation (REST, GraphQL, gRPC)
- Database design and optimization (SQL, NoSQL)
- Authentication and authorization (JWT, OAuth, sessions)
- Business logic and domain modeling
- Microservices architecture
- Message queues and event-driven systems
- Caching strategies (Redis, Memcached)
- Backend testing (unit, integration, API tests)
- ORM/query builders (Prisma, TypeORM, Sequelize)
- Server frameworks (Express, Fastify, NestJS, Django, Flask)
**When to Use:**
- Building REST or GraphQL APIs
- Database schema design
- Authentication systems
- Business logic implementation
- Data migration and seeding
- API documentation (OpenAPI/Swagger)
- Backend performance optimization
- Integration with third-party services
**Tools Available:** Read, Write, Edit, Bash, Grep, Glob
**Skills:** backend-architecture, testing-practices
**Examples:**
- "Create a REST API for user management with authentication"
- "Design database schema for e-commerce application"
- "Implement OAuth 2.0 authentication flow"
- "Add caching layer with Redis for product catalog"
- "Write integration tests for payment API"
- "Optimize N+1 query problems in ORM"

37
agents/data-engineer.md Normal file
View File

@@ -0,0 +1,37 @@
# Data Engineer
PROACTIVELY use for data pipelines, data warehousing, ETL/ELT processes, and ML infrastructure. Handles data architecture, processing workflows, and analytics infrastructure.
**Core Capabilities:**
- Data pipeline development (Airflow, Prefect, Dagster)
- ETL/ELT processes and data transformation
- Data warehousing (Snowflake, BigQuery, Redshift)
- Stream processing (Kafka, Flink, Spark Streaming)
- Batch processing (Spark, Hadoop)
- Data modeling (dimensional modeling, data vault)
- ML pipeline infrastructure (MLOps)
- Data quality and validation
- Data governance and lineage
- SQL optimization and query performance
**When to Use:**
- Building data pipelines
- ETL/ELT development
- Data warehouse design
- Real-time data processing
- ML infrastructure setup
- Data quality implementation
- Analytics infrastructure
- Data migration projects
**Tools Available:** Read, Write, Edit, Bash, Grep, Glob
**Skills:** data-engineering, backend-architecture
**Examples:**
- "Create Airflow DAG for daily ETL pipeline"
- "Design dimensional model for analytics warehouse"
- "Build real-time streaming pipeline with Kafka and Spark"
- "Implement data quality checks with Great Expectations"
- "Set up MLOps pipeline for model training and deployment"
- "Optimize SQL queries for large-scale data processing"

37
agents/devops-engineer.md Normal file
View File

@@ -0,0 +1,37 @@
# DevOps Engineer
PROACTIVELY use for infrastructure, CI/CD pipelines, deployment, monitoring, and reliability engineering. Handles cloud infrastructure, containerization, automation, and observability.
**Core Capabilities:**
- CI/CD pipeline design and implementation
- Infrastructure as Code (Terraform, CloudFormation, Pulumi)
- Container orchestration (Kubernetes, Docker Swarm)
- Cloud platforms (AWS, Azure, GCP)
- Monitoring and observability (Prometheus, Grafana, ELK, Datadog)
- Deployment strategies (blue-green, canary, rolling)
- Configuration management (Ansible, Chef, Puppet)
- GitOps workflows (ArgoCD, Flux)
- Security and compliance (secrets management, RBAC)
- Performance tuning and cost optimization
**When to Use:**
- Setting up CI/CD pipelines
- Infrastructure provisioning
- Kubernetes deployment and management
- Monitoring and alerting setup
- Deployment automation
- Cloud resource optimization
- Security hardening
- Disaster recovery planning
**Tools Available:** Read, Write, Edit, Bash, Grep, Glob
**Skills:** devops-practices, backend-architecture
**Examples:**
- "Create GitHub Actions CI/CD pipeline for Node.js app"
- "Set up Kubernetes cluster with Terraform on AWS"
- "Implement blue-green deployment strategy"
- "Configure Prometheus and Grafana monitoring"
- "Optimize cloud costs and right-size resources"
- "Set up secrets management with Vault"

32
agents/engineering-lead.md Normal file
View File

@@ -0,0 +1,32 @@
# Engineering Lead
PROACTIVELY use for system architecture, technical strategy, and engineering leadership decisions. Provides architectural guidance, makes technology choices, and coordinates across the engineering team.
**Core Capabilities:**
- System architecture and design
- Technology stack decisions
- Technical roadmap planning
- Engineering team coordination
- Code review and quality standards
- Performance and scalability strategy
- Technical debt management
- Cross-team integration planning
**When to Use:**
- Designing overall system architecture
- Making critical technology decisions
- Planning technical initiatives
- Resolving architectural conflicts
- Establishing engineering standards
- Scaling and performance planning
**Tools Available:** Read, Write, Edit, Bash, Grep, Glob
**Skills:** backend-architecture, devops-practices, testing-practices
**Examples:**
- "Design the architecture for a microservices-based application"
- "Evaluate technology stack options for our new project"
- "Create technical roadmap for Q1"
- "Review system design for scalability concerns"
- "Establish code quality and testing standards"

34
agents/frontend-engineer.md Normal file
View File

@@ -0,0 +1,34 @@
# Frontend Engineer
PROACTIVELY use for all frontend development tasks including React/Vue/Svelte components, state management, responsive styling, and accessibility implementation.
**Core Capabilities:**
- Component development with TypeScript/JavaScript
- State management (Redux, Zustand, Context API, Recoil)
- Responsive design and styling (CSS, Tailwind, styled-components, CSS Modules)
- Accessibility (WCAG 2.1 compliance, ARIA, keyboard navigation)
- Frontend testing (unit, integration, E2E)
- Performance optimization (code splitting, lazy loading, memoization)
- Build tooling and configuration (Vite, Webpack, esbuild)
- Modern framework expertise (React, Vue, Svelte, Next.js, Nuxt)
**When to Use:**
- Building user interfaces and components
- Implementing design systems
- State management setup
- Frontend testing
- Performance optimization
- Accessibility improvements
- Responsive layout implementation
**Tools Available:** Read, Write, Edit, Bash, Grep, Glob
**Skills:** frontend-development, testing-practices
**Examples:**
- "Create a responsive navbar component with accessibility"
- "Implement Redux state management for user authentication"
- "Optimize React app performance and reduce bundle size"
- "Add E2E tests for checkout flow with Playwright"
- "Build a design system with reusable components"
- "Implement dark mode with CSS variables and context"

37
agents/mobile-engineer.md Normal file
View File

@@ -0,0 +1,37 @@
# Mobile Engineer
PROACTIVELY use for iOS and Android mobile application development. Handles native and cross-platform development, mobile UI/UX, performance optimization, and app store deployment.
**Core Capabilities:**
- iOS development (Swift, SwiftUI, UIKit)
- Android development (Kotlin, Jetpack Compose)
- Cross-platform frameworks (React Native, Flutter)
- Mobile UI/UX patterns (navigation, gestures, animations)
- Native APIs (camera, location, notifications, biometrics)
- Mobile state management
- Offline-first architecture and data sync
- Mobile testing (unit, UI, integration)
- App store optimization and deployment
- Mobile performance optimization
- Push notifications and deep linking
**When to Use:**
- Building iOS or Android apps
- Cross-platform mobile development
- Mobile UI implementation
- Native feature integration
- App store submission
- Mobile performance optimization
- Offline data handling
**Tools Available:** Read, Write, Edit, Bash, Grep, Glob
**Skills:** mobile-development, testing-practices
**Examples:**
- "Create a React Native app with navigation and authentication"
- "Build SwiftUI interface with MVVM architecture"
- "Implement offline-first data sync for mobile app"
- "Add biometric authentication to iOS app"
- "Optimize React Native app performance and bundle size"
- "Set up push notifications with Firebase Cloud Messaging"

38
agents/qa-engineer.md Normal file
View File

@@ -0,0 +1,38 @@
# QA Engineer
PROACTIVELY use for quality assurance, testing strategy, test automation, and quality standards. Ensures software quality through comprehensive testing approaches.
**Core Capabilities:**
- Test strategy and planning
- Unit testing (Jest, Vitest, pytest, JUnit)
- Integration testing
- End-to-end testing (Playwright, Cypress, Selenium)
- API testing (Postman, REST Assured, Supertest)
- Performance testing (k6, JMeter, Lighthouse)
- Security testing basics
- Test automation frameworks
- CI/CD test integration
- Bug tracking and reporting
- Quality metrics and reporting
**When to Use:**
- Creating test strategies
- Writing automated tests
- Setting up testing infrastructure
- Performance and load testing
- API contract testing
- Security testing basics
- Test coverage improvement
- Quality assurance processes
**Tools Available:** Read, Write, Edit, Bash, Grep, Glob
**Skills:** testing-practices, frontend-development, backend-architecture
**Examples:**
- "Create comprehensive E2E test suite with Playwright"
- "Set up API integration tests with contract testing"
- "Implement performance testing for critical user flows"
- "Add visual regression testing with Percy"
- "Create test automation framework for microservices"
- "Set up load testing with k6 and analyze results"

93
plugin.lock.json Normal file
View File

@@ -0,0 +1,93 @@
{
"$schema": "internal://schemas/plugin.lock.v1.json",
"pluginId": "gh:bandofai/puerto:plugins/engineering",
"normalized": {
"repo": null,
"ref": "refs/tags/v20251128.0",
"commit": "35a07338567ed1cbef35339857a849113622820a",
"treeHash": "631977effc5347f58c298b33f9389b70cfe6f1673d6b4077e54ff0014bbdf912",
"generatedAt": "2025-11-28T10:14:10.242057Z",
"toolVersion": "publish_plugins.py@0.2.0"
},
"origin": {
"remote": "git@github.com:zhongweili/42plugin-data.git",
"branch": "master",
"commit": "aa1497ed0949fd50e99e70d6324a29c5b34f9390",
"repoRoot": "/Users/zhongweili/projects/openmind/42plugin-data"
},
"manifest": {
"name": "engineering",
"description": "Full-stack engineering team with 7 specialized roles covering frontend, backend, mobile, data, DevOps, and QA",
"version": "1.0.0"
},
"content": {
"files": [
{
"path": "README.md",
"sha256": "fbe05c9e1329242286bf9eae5175af3891d99f15d3ebb81211db531bf910fd15"
},
{
"path": "agents/qa-engineer.md",
"sha256": "9fcab9ed7b2c81ab1ab5d65c0430140669c65e21b6ff75376a53af82d0c5b0d4"
},
{
"path": "agents/devops-engineer.md",
"sha256": "8acbd318ce04ccea775a2c0fdc5058d1d1a5c0ad05a383a449b3718599ef8d0f"
},
{
"path": "agents/engineering-lead.md",
"sha256": "f30948b380eecccaf262705b0e75fc06ebacb4155baf6725e4b5530c4dc69954"
},
{
"path": "agents/frontend-engineer.md",
"sha256": "2e6b3017e5ee3628171b1b9dbca78b9ffd21eb0d495fc2b7854de7b178ebd462"
},
{
"path": "agents/data-engineer.md",
"sha256": "69a52f422887d5ad77fc5f13f88178c8cd7b82a965e80dafa6a7b7996f6ce0ed"
},
{
"path": "agents/backend-engineer.md",
"sha256": "f839e0cba7b47e6ba652ea9e1b15251991e7a3aecd21a8d1428fbdb4901828dc"
},
{
"path": "agents/mobile-engineer.md",
"sha256": "a07586c148a9079dae9f8255c8d9ecea3c9e7195038648205eb739d59d6feb8e"
},
{
"path": ".claude-plugin/plugin.json",
"sha256": "aa6416b4d50a75ce2f1dc4061a8c2bc057712c26e0896208a523b230748a3140"
},
{
"path": "skills/mobile-development/SKILL.md",
"sha256": "b1dece61758d7f64cc807fdfb5a915f50bad8528b42d3939e3a23ae2a559b26f"
},
{
"path": "skills/devops-practices/SKILL.md",
"sha256": "9a02485662de025e1cfbec1cce556ea45f7a856e633c06d56696ab4e95e87ea5"
},
{
"path": "skills/frontend-development/SKILL.md",
"sha256": "00ad00000b099e6d6e6250cf98edefec5ee6aebc88183235a2e9e48c7becea11"
},
{
"path": "skills/data-engineering/SKILL.md",
"sha256": "d446abf6926708f43c2d0f264ece72f5223a08c003eec8361d45452ae1d0294d"
},
{
"path": "skills/backend-architecture/SKILL.md",
"sha256": "433aab533ae7e913fe0b8398eae40c210069a15176833145c32cd348e2db2e42"
},
{
"path": "skills/testing-practices/SKILL.md",
"sha256": "b5aa62892f0818d47c0ac78c6f174c4f37660add8bc3b8e114f504491a61aebf"
}
],
"dirSha256": "631977effc5347f58c298b33f9389b70cfe6f1673d6b4077e54ff0014bbdf912"
},
"security": {
"scannedAt": null,
"scannerVersion": null,
"flags": []
}
}

814
skills/backend-architecture/SKILL.md Normal file
View File

@@ -0,0 +1,814 @@
# Backend Architecture
**Comprehensive patterns for APIs, databases, and server-side systems**
Consolidated from:
- backend-architect skills
- api-developer skills
- database-architect skills
---
# Schema Design Skill
Expert patterns for relational database schema design, normalization, and constraint management.
## Core Principles
### 1. Normalization Levels
**1NF (First Normal Form)**:
- Atomic values only (no arrays, no comma-separated lists)
- Each column contains single value
- No repeating groups
**2NF (Second Normal Form)**:
- Must be in 1NF
- No partial dependencies on composite primary keys
- Every non-key column depends on the entire primary key
**3NF (Third Normal Form)**:
- Must be in 2NF
- No transitive dependencies
- Every non-key column depends only on the primary key
**BCNF (Boyce-Codd Normal Form)**:
- Must be in 3NF
- Every determinant is a candidate key
**Strategic Denormalization**:
- Only denormalize with performance data justification
- Document the trade-off
- Consider materialized views instead
- Plan for data consistency maintenance
### 2. Primary Key Selection
**UUID (Recommended for distributed systems)**:
```sql
id UUID PRIMARY KEY DEFAULT uuid_generate_v4()
```
- Pros: Globally unique, no coordination needed, harder to enumerate
- Cons: Larger storage (16 bytes), random order (index fragmentation)
**Auto-increment Integer**:
```sql
id SERIAL PRIMARY KEY -- PostgreSQL
id INT AUTO_INCREMENT PRIMARY KEY -- MySQL
id INTEGER PRIMARY KEY AUTOINCREMENT -- SQLite
```
- Pros: Small storage (4-8 bytes), sequential (better index performance)
- Cons: Coordination needed, easy to enumerate, not globally unique
**Composite Keys** (for junction tables):
```sql
PRIMARY KEY (user_id, role_id)
```
### 3. Foreign Key Constraints
**Always define foreign keys** for referential integrity:
```sql
CONSTRAINT fk_orders_customer
FOREIGN KEY (customer_id)
REFERENCES customers(id)
ON DELETE CASCADE -- or RESTRICT, SET NULL
ON UPDATE CASCADE
```
**ON DELETE options**:
- `CASCADE`: Delete child rows when parent deleted
- `RESTRICT`: Prevent delete if children exist
- `SET NULL`: Set foreign key to NULL
- `NO ACTION`: Similar to RESTRICT (database-specific)
### 4. Check Constraints
**Use check constraints for business rules**:
```sql
-- Email format validation
CONSTRAINT email_format CHECK (
email ~* '^[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}$'
)
-- Positive values
CONSTRAINT total_positive CHECK (total >= 0)
-- Enum-like values
CONSTRAINT valid_status CHECK (
status IN ('pending', 'processing', 'completed', 'cancelled')
)
-- Date ranges
CONSTRAINT valid_date_range CHECK (end_date > start_date)
```
### 5. Index Strategy
**Index Types and When to Use**:
**B-Tree (Default)**:
- WHERE clauses: `WHERE status = 'active'`
- ORDER BY: `ORDER BY created_at DESC`
- Range queries: `WHERE price BETWEEN 10 AND 100`
- Joins: Foreign key columns
**GIN (PostgreSQL - Generalized Inverted Index)**:
- JSONB columns: `WHERE data @> '{"key": "value"}'`
- Arrays: `WHERE tags @> ARRAY['postgresql']`
- Full-text search: `WHERE to_tsvector(text) @@ to_tsquery('search')`
**GiST (PostgreSQL - Generalized Search Tree)**:
- Geometric data: `WHERE location && box '((0,0),(1,1))'`
- Full-text search: Alternative to GIN
- Range types: `WHERE daterange && '[2025-01-01, 2025-12-31]'`
**Hash (Limited use)**:
- Equality only: `WHERE id = 123`
- Not recommended (B-tree usually better)
**Composite Index Column Order**:
```sql
-- Rule: Most selective column first, or most commonly filtered
CREATE INDEX idx_orders_status_created ON orders(status, created_at DESC);
-- Works for:
-- WHERE status = 'pending' ✅
-- WHERE status = 'pending' AND created_at > NOW() - INTERVAL '7 days' ✅
-- WHERE status = 'pending' ORDER BY created_at DESC ✅
-- Does NOT work for:
-- WHERE created_at > NOW() - INTERVAL '7 days' ❌ (doesn't start with status)
```
## Schema Patterns
### Pattern 1: Soft Delete
```sql
CREATE TABLE users (
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
email VARCHAR(255) NOT NULL,
name VARCHAR(255) NOT NULL,
deleted_at TIMESTAMP NULL,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);
-- Partial unique index (only for non-deleted rows)
CREATE UNIQUE INDEX idx_users_email_active
ON users(email)
WHERE deleted_at IS NULL;
-- Query pattern: Always filter deleted
SELECT * FROM users WHERE deleted_at IS NULL;
```
### Pattern 2: Audit Trail
```sql
CREATE TABLE users (
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
email VARCHAR(255) NOT NULL,
name VARCHAR(255) NOT NULL,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
created_by UUID REFERENCES users(id),
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
updated_by UUID REFERENCES users(id)
);
-- Separate audit log table for full history
CREATE TABLE users_audit (
audit_id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
user_id UUID NOT NULL,
operation VARCHAR(10) NOT NULL, -- INSERT, UPDATE, DELETE
old_values JSONB,
new_values JSONB,
changed_by UUID REFERENCES users(id),
changed_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);
```
### Pattern 3: Many-to-Many with Metadata
```sql
-- Junction table with additional attributes
CREATE TABLE user_roles (
user_id UUID REFERENCES users(id) ON DELETE CASCADE,
role_id UUID REFERENCES roles(id) ON DELETE CASCADE,
granted_by UUID REFERENCES users(id),
granted_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
expires_at TIMESTAMP,
PRIMARY KEY (user_id, role_id)
);
CREATE INDEX idx_user_roles_user ON user_roles(user_id);
CREATE INDEX idx_user_roles_role ON user_roles(role_id);
CREATE INDEX idx_user_roles_expires ON user_roles(expires_at)
WHERE expires_at IS NOT NULL;
```
### Pattern 4: Hierarchical Data (Adjacency List)
```sql
CREATE TABLE categories (
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
name VARCHAR(255) NOT NULL,
parent_id UUID REFERENCES categories(id),
path TEXT, -- Materialized path: /electronics/computers/laptops
level INT, -- Denormalized for performance
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX idx_categories_parent ON categories(parent_id);
CREATE INDEX idx_categories_path ON categories(path);
```
### Pattern 5: Polymorphic Associations (Avoid if Possible)
**❌ Problematic Approach**:
```sql
-- Weak referential integrity
CREATE TABLE comments (
id UUID PRIMARY KEY,
content TEXT NOT NULL,
commentable_type VARCHAR(50), -- 'Post' or 'Photo'
commentable_id UUID, -- No real foreign key!
created_at TIMESTAMP
);
```
**✅ Better Approach (Exclusive Arcs)**:
```sql
CREATE TABLE comments (
id UUID PRIMARY KEY,
content TEXT NOT NULL,
post_id UUID REFERENCES posts(id) ON DELETE CASCADE,
photo_id UUID REFERENCES photos(id) ON DELETE CASCADE,
created_at TIMESTAMP,
-- Exactly one must be set
CONSTRAINT one_commentable CHECK (
(post_id IS NOT NULL AND photo_id IS NULL) OR
(post_id IS NULL AND photo_id IS NOT NULL)
)
);
CREATE INDEX idx_comments_post ON comments(post_id);
CREATE INDEX idx_comments_photo ON comments(photo_id);
```
## Naming Conventions
**Tables**: Plural nouns, lowercase, underscores
```
users, orders, order_items, user_preferences
```
**Columns**: Singular nouns, lowercase, underscores
```
id, email, first_name, created_at, customer_id
```
**Primary Keys**: Always `id`
```
id UUID PRIMARY KEY
```
**Foreign Keys**: `{referenced_table_singular}_id`
```
customer_id, product_id, user_id
```
**Indexes**: `idx_{table}_{column(s)}[_{condition}]`
```
idx_users_email
idx_orders_customer_id
idx_orders_status_created
idx_users_email_active (partial index)
```
**Constraints**: `{type}_{table}_{description}`
```
pk_users (primary key)
fk_orders_customer (foreign key)
uq_users_email (unique)
ck_orders_total_positive (check)
```
## Common Anti-Patterns to Avoid
**❌ Generic JSON Columns (EAV Pattern)**:
```sql
-- Bad: No schema, no constraints, no indexes
CREATE TABLE entities (
id UUID PRIMARY KEY,
type VARCHAR(50),
attributes JSONB
);
```
**❌ Comma-Separated Lists**:
```sql
-- Bad: Violates 1NF, can't join efficiently
CREATE TABLE users (
id UUID PRIMARY KEY,
tags TEXT -- 'javascript,python,sql'
);
```
**✅ Use junction table instead**:
```sql
CREATE TABLE user_tags (
user_id UUID REFERENCES users(id),
tag_id UUID REFERENCES tags(id),
PRIMARY KEY (user_id, tag_id)
);
```
**❌ Nullable Boolean Columns**:
```sql
-- Bad: Three states (true, false, null) - ambiguous
is_active BOOLEAN NULL
```
**✅ Be explicit**:
```sql
-- Good: Two clear states
is_active BOOLEAN NOT NULL DEFAULT true
```
## ER Diagram Notation (Mermaid)
```mermaid
erDiagram
CUSTOMER ||--o{ ORDER : places
CUSTOMER {
uuid id PK "Primary Key"
string email UK "Unique Key"
string name
timestamp created_at
}
ORDER ||--|{ LINE_ITEM : contains
ORDER {
uuid id PK
uuid customer_id FK
decimal total
string status
}
PRODUCT ||--o{ LINE_ITEM : "ordered in"
LINE_ITEM {
uuid id PK
uuid order_id FK
uuid product_id FK
int quantity
decimal unit_price
}
```
**Cardinality Symbols**:
- `||--||` : One to exactly one
- `||--o|` : One to zero or one
- `||--o{` : One to zero or more
- `}|--|{` : One or more to one or more
## Database-Specific Best Practices
### PostgreSQL
```sql
-- Enable UUID extension
CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
-- Use JSONB (not JSON) for better performance
metadata JSONB
-- Use array types when appropriate
tags TEXT[]
-- Use full-text search
CREATE INDEX idx_products_search ON products
USING GIN (to_tsvector('english', name || ' ' || description));
-- Use enums for fixed sets
CREATE TYPE order_status AS ENUM ('pending', 'processing', 'completed', 'cancelled');
```
### MySQL
```sql
-- Use InnoDB engine (default in 8.0+)
ENGINE=InnoDB
-- Use UTF8MB4 for full Unicode support (including emoji)
DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci
-- Use generated columns for computed values
price_with_tax DECIMAL(10,2) GENERATED ALWAYS AS (price * 1.20) STORED
-- Partition large tables
PARTITION BY RANGE (YEAR(created_at)) (
PARTITION p2023 VALUES LESS THAN (2024),
PARTITION p2024 VALUES LESS THAN (2025),
PARTITION p2025 VALUES LESS THAN (2026)
);
```
### SQLite
```sql
-- Use STRICT tables for type enforcement (3.37+)
CREATE TABLE users (
id INTEGER PRIMARY KEY,
email TEXT NOT NULL,
age INTEGER NOT NULL
) STRICT;
-- Use WITHOUT ROWID for space efficiency
CREATE TABLE user_settings (
user_id INTEGER PRIMARY KEY,
theme TEXT NOT NULL,
locale TEXT NOT NULL
) WITHOUT ROWID;
-- Use triggers for complex constraints
CREATE TRIGGER check_age_before_insert
BEFORE INSERT ON users
FOR EACH ROW
WHEN NEW.age < 18
BEGIN
SELECT RAISE(ABORT, 'Users must be 18 or older');
END;
```
## Quality Checklist
**Schema Completeness**:
- [ ] All tables have primary keys
- [ ] All relationships have foreign keys
- [ ] Appropriate NOT NULL constraints
- [ ] Check constraints for business rules
- [ ] Default values where appropriate
- [ ] Created_at/updated_at timestamps
**Normalization**:
- [ ] Schema is at least 3NF
- [ ] No repeating groups
- [ ] No partial dependencies
- [ ] No transitive dependencies
- [ ] Denormalization justified and documented
**Performance**:
- [ ] Indexes on all foreign keys
- [ ] Indexes on commonly filtered columns
- [ ] Composite indexes for multi-column queries
- [ ] Covering indexes for frequent queries
- [ ] Partial indexes where appropriate
**Maintainability**:
- [ ] Consistent naming conventions
- [ ] Clear table and column names
- [ ] Comments on complex structures
- [ ] ER diagram provided
- [ ] Design decisions documented
---
## MCP-Enhanced Schema Design
### PostgreSQL MCP for Schema Validation
When PostgreSQL MCP is available, validate schema designs directly against production databases:
```typescript
// Runtime detection - no configuration needed
const hasPostgres = typeof mcp__postgres__query !== 'undefined';
if (hasPostgres) {
console.log("✓ Using PostgreSQL MCP for schema design validation");
// Validate schema against existing database
const schemaCheck = await mcp__postgres__query({
sql: `
SELECT
table_name,
column_name,
data_type,
is_nullable,
column_default,
character_maximum_length
FROM information_schema.columns
WHERE table_schema = 'public'
AND table_name IN ('users', 'orders', 'products')
ORDER BY table_name, ordinal_position
`
});
console.log(`✓ Retrieved schema for ${schemaCheck.rows.length} columns`);
// Check constraints
const constraints = await mcp__postgres__query({
sql: `
SELECT
tc.table_name,
tc.constraint_name,
tc.constraint_type,
kcu.column_name,
ccu.table_name AS foreign_table_name,
ccu.column_name AS foreign_column_name
FROM information_schema.table_constraints tc
LEFT JOIN information_schema.key_column_usage kcu
ON tc.constraint_name = kcu.constraint_name
LEFT JOIN information_schema.constraint_column_usage ccu
ON tc.constraint_name = ccu.constraint_name
WHERE tc.table_schema = 'public'
ORDER BY tc.table_name, tc.constraint_type
`
});
console.log(`✓ Found ${constraints.rows.length} constraints`);
// Validate foreign key relationships
const orphanedRecords = await mcp__postgres__query({
sql: `
SELECT
'orders' as table_name,
COUNT(*) as orphaned_count
FROM orders o
LEFT JOIN users u ON o.user_id = u.id
WHERE u.id IS NULL
`
});
if (orphanedRecords.rows[0].orphaned_count > 0) {
console.log(`⚠️ Found ${orphanedRecords.rows[0].orphaned_count} orphaned records`);
} else {
console.log("✓ All foreign key relationships valid");
}
// Test DDL before executing
const ddlTest = await mcp__postgres__query({
sql: `
BEGIN;
-- Test adding new column
ALTER TABLE users ADD COLUMN test_column VARCHAR(100);
-- Check table size after change
SELECT pg_size_pretty(pg_relation_size('users')) as table_size;
ROLLBACK;
`
});
console.log(`✓ DDL validated (would not break existing data)`);
} else {
console.log("ℹ️ PostgreSQL MCP not available");
console.log(" Install for schema validation:");
console.log(" npm install -g @modelcontextprotocol/server-postgres");
}
```
### Benefits Comparison
| Aspect | With PostgreSQL MCP | Without MCP (Traditional) |
|--------|-------------------|--------------------------|
| **Schema Exploration** | Query information_schema instantly | Request schema dump → wait |
| **Constraint Validation** | Check FK relationships on real data | Assume constraints work |
| **DDL Testing** | Test ALTER statements with ROLLBACK | Deploy and hope |
| **Data Distribution** | Analyze with pg_stats | Guess cardinality |
| **Impact Analysis** | Query actual table sizes | Estimate impact |
| **Normalization Check** | Find duplicates in production | Theoretical analysis |
| **Migration Safety** | Test on production replica | Cross fingers |
**When to use PostgreSQL MCP:**
- Designing schema for existing database
- Validating normalization against real data
- Testing DDL changes before deployment
- Analyzing data distribution for index design
- Finding schema anomalies
- Planning migrations
- Reverse engineering existing schemas
**When traditional approach needed:**
- Greenfield database design
- Designing for future data
- Theoretical schema modeling
- No database access
### Real-World Example: Adding User Preferences
**With PostgreSQL MCP (15 minutes):**
```typescript
// 1. Analyze current users table
const currentSchema = await mcp__postgres__query({
sql: `
SELECT
column_name,
data_type,
is_nullable
FROM information_schema.columns
WHERE table_name = 'users'
ORDER BY ordinal_position
`
});
console.log(`✓ Users table has ${currentSchema.rows.length} columns`);
// 2. Check for existing preference data
const preferencesCheck = await mcp__postgres__query({
sql: `
SELECT
COUNT(DISTINCT user_id) as users_with_prefs,
COUNT(*) as total_prefs,
AVG(array_length(string_to_array(preferences, ','), 1)) as avg_prefs_per_user
FROM user_metadata
WHERE preferences IS NOT NULL
`
});
console.log(`✓ ${preferencesCheck.rows[0].users_with_prefs} users have preferences`);
// 3. Design decision: Separate table vs JSONB column
const tableSize = await mcp__postgres__query({
sql: `
SELECT
pg_size_pretty(pg_relation_size('users')) as current_size,
pg_size_pretty(pg_relation_size('users') * 1.2) as estimated_with_jsonb
`
});
console.log(`✓ Adding JSONB column would increase size to ${tableSize.rows[0].estimated_with_jsonb}`);
// 4. Test the migration (with ROLLBACK)
const migrationTest = await mcp__postgres__query({
sql: `
BEGIN;
-- Add preferences column
ALTER TABLE users ADD COLUMN preferences JSONB DEFAULT '{}'::jsonb;
-- Add GIN index for JSONB queries
CREATE INDEX idx_users_preferences ON users USING GIN (preferences);
-- Test query performance
EXPLAIN ANALYZE
SELECT * FROM users
WHERE preferences @> '{"theme": "dark"}'::jsonb;
ROLLBACK;
`
});
console.log("✓ Migration tested successfully");
// Decision: Use JSONB column (flexible, good performance with GIN index)
```
**Without MCP (2 hours):**
1. Request schema documentation (15 min wait)
2. Analyze schema manually (20 min)
3. Make design decision based on assumptions (15 min)
4. Write migration script (15 min)
5. Deploy to test database (10 min)
6. Load test data (20 min)
7. Test queries (10 min)
8. Find issues (15 min)
9. Revise and redeploy (15 min)
### Schema Validation Patterns
```typescript
// Comprehensive schema validation
async function validateSchema() {
const hasPostgres = typeof mcp__postgres__query !== 'undefined';
if (hasPostgres) {
// 1. Check for missing indexes on foreign keys
const missingIndexes = await mcp__postgres__query({
sql: `
SELECT
tc.table_name,
kcu.column_name
FROM information_schema.table_constraints tc
JOIN information_schema.key_column_usage kcu
ON tc.constraint_name = kcu.constraint_name
WHERE tc.constraint_type = 'FOREIGN KEY'
AND NOT EXISTS (
SELECT 1
FROM pg_indexes
WHERE tablename = tc.table_name
AND indexdef LIKE '%' || kcu.column_name || '%'
)
`
});
if (missingIndexes.rows.length > 0) {
console.log("⚠️ Missing indexes on foreign keys:");
missingIndexes.rows.forEach(row => {
console.log(` ${row.table_name}.${row.column_name}`);
});
}
// 2. Check for columns that should be NOT NULL
const nullableColumns = await mcp__postgres__query({
sql: `
SELECT
table_name,
column_name,
COUNT(*) FILTER (WHERE value IS NULL) as null_count,
COUNT(*) as total_count
FROM (
SELECT 'users' as table_name, 'email' as column_name, email as value FROM users
UNION ALL
SELECT 'orders' as table_name, 'user_id' as column_name, user_id::text as value FROM orders
) data
GROUP BY table_name, column_name
HAVING COUNT(*) FILTER (WHERE value IS NULL) = 0
AND EXISTS (
SELECT 1
FROM information_schema.columns
WHERE table_name = data.table_name
AND column_name = data.column_name
AND is_nullable = 'YES'
)
`
});
if (nullableColumns.rows.length > 0) {
console.log("⚠️ Columns that could be NOT NULL:");
nullableColumns.rows.forEach(row => {
console.log(` ${row.table_name}.${row.column_name} (0 NULLs in ${row.total_count} rows)`);
});
}
// 3. Check for denormalization opportunities
const duplicateData = await mcp__postgres__query({
sql: `
SELECT
user_id,
email,
COUNT(*) as duplicate_count
FROM users
GROUP BY user_id, email
HAVING COUNT(*) > 1
`
});
return {
missingIndexes: missingIndexes.rows,
nullableColumns: nullableColumns.rows,
duplicates: duplicateData.rows
};
}
}
```
### PostgreSQL MCP Installation
```bash
# Install PostgreSQL MCP
npm install -g @modelcontextprotocol/server-postgres
# Configure for schema design
{
"mcpServers": {
"postgres": {
"command": "npx",
"args": ["-y", "@modelcontextprotocol/server-postgres"],
"env": {
"POSTGRES_CONNECTION_STRING": "postgresql://schema_designer:pass@db.company.com:5432/production"
}
}
}
}
```
Once installed, all agents reading this skill automatically validate schemas against live databases.
### Schema Design Workflow with MCP
1. **Explore Existing Schema**: Query information_schema
2. **Analyze Data Distribution**: Use pg_stats
3. **Check Constraints**: Validate FK relationships
4. **Test DDL Changes**: Use BEGIN...ROLLBACK
5. **Estimate Impact**: Query table/index sizes
6. **Validate Normalization**: Find duplicates
7. **Plan Indexes**: Analyze query patterns
8. **Generate Migration**: Create safe DDL scripts
---
**Version**: 1.0
**Last Updated**: January 2025
**MCP Enhancement**: PostgreSQL for data-driven schema design
**Best Practices**: Industry-proven schema design patterns

1120
skills/data-engineering/SKILL.md Normal file
View File

File diff suppressed because it is too large Load Diff

881
skills/devops-practices/SKILL.md Normal file
View File

@@ -0,0 +1,881 @@
# DevOps Practices
**CI/CD, Infrastructure, Deployment, and Monitoring**
Consolidated from:
- devops-engineer skills
- cloud-architect skills
- site-reliability-engineer skills
- release-manager skills
# CI/CD Patterns Skill
**Expert-level CI/CD pipeline design patterns and best practices**
## Core Principles
1. **Pipeline as Code**: All pipeline configuration in version control
2. **Fast Feedback**: Fail fast, provide clear error messages
3. **Build Once**: Build artifacts once, deploy everywhere
4. **Idempotent**: Running twice produces same result
5. **Secure by Default**: Security scanning integrated, not optional
## Multi-Stage Pipeline Pattern
```
┌─────────┐ ┌──────┐ ┌──────────┐ ┌────────┐ ┌────────┐
│ Build │──>│ Test │──>│ Security │──>│ Deploy │──>│ Verify │
└─────────┘ └──────┘ └──────────┘ └────────┘ └────────┘
Fast Medium Slow Manual Quick
(<2 min) (<5 min) (<10 min) (Approval) (<2 min)
```
### Stage Ordering
1. **Build**: Compile code, create artifacts (fast fail)
2. **Test**: Unit → Integration → E2E (fastest first)
3. **Security**: SAST → Dependency scan → Container scan
4. **Deploy**: Dev → Staging → Prod (progressive)
5. **Verify**: Smoke tests, health checks
## Optimization Strategies
### 1. Caching
```yaml
# Cache dependencies
cache:
key: ${CI_COMMIT_REF_SLUG}
paths:
- node_modules/
- .pip/
- .m2/
- .gradle/
```
**Impact**: 50-80% faster builds
### 2. Parallelization
```yaml
# Run tests in parallel
test:
parallel: 4
script:
- npm test -- --shard=${CI_NODE_INDEX}/${CI_NODE_TOTAL}
```
**Impact**: 4x faster test execution
### 3. Conditional Execution
```yaml
# Skip unnecessary steps
deploy:
only:
- main
- /^release-.*$/
changes:
- src/**
- Dockerfile
```
**Impact**: Reduce unnecessary runs by 70%
### 4. Docker Layer Caching
```dockerfile
# Multi-stage build
FROM node:18 AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production
COPY . .
RUN npm run build
FROM node:18-alpine
COPY --from=builder /app/dist /app/dist
COPY --from=builder /app/node_modules /app/node_modules
```
**Impact**: 10x faster Docker builds
## Security Scanning Integration
### SAST (Static Application Security Testing)
```yaml
sast:
stage: security
image: returntocorp/semgrep
script:
- semgrep --config=auto --json --output=sast-report.json .
artifacts:
reports:
sast: sast-report.json
```
**Tools**:
- **Semgrep**: Fast, customizable (free)
- **SonarQube**: Comprehensive code quality
- **CodeQL**: GitHub's semantic analysis
### Dependency Scanning
```yaml
dependency_scan:
stage: security
script:
- npm audit --audit-level=high
- snyk test --severity-threshold=high
allow_failure: false # Fail on critical vulnerabilities
```
**Tools**:
- **Snyk**: Comprehensive, auto-fix (free tier)
- **Dependabot**: GitHub native
- **npm audit**: Built-in Node.js
- **safety**: Python packages
### Container Scanning
```yaml
container_scan:
stage: security
image: aquasec/trivy
script:
- trivy image --severity HIGH,CRITICAL myapp:${CI_COMMIT_SHA}
```
**Tools**:
- **Trivy**: Fast, accurate (free)
- **Clair**: CoreOS project
- **Anchore**: Policy-based
### Secret Detection
```yaml
secrets_scan:
stage: security
image: zricethezav/gitleaks
script:
- gitleaks detect --source . --verbose
```
**Tools**:
- **Gitleaks**: Fast, configurable
- **TruffleHog**: High accuracy
- **git-secrets**: AWS focus
## Testing Strategies
### Test Pyramid
```
/\
/ \ E2E Tests (5%)
/____\ Slow, brittle
/ \
/ Integration \ (15%)
/________________\
/ \
/ Unit Tests (80%) \ Fast, reliable
/______________________\
```
### Test Execution Order
1. **Linting**: Fastest, catches syntax errors
2. **Unit tests**: Fast, isolated
3. **Integration tests**: Medium, database/API
4. **E2E tests**: Slow, full system
### Coverage Requirements
```yaml
test:
script:
- npm test -- --coverage --coverageThreshold='{"global":{"branches":80,"functions":80,"lines":80}}'
```
**Thresholds**:
- **Unit**: ≥80% coverage (enforce)
- **Integration**: ≥60% coverage (goal)
- **E2E**: Critical paths only
## Artifact Management
### Build Artifacts
```yaml
build:
script:
- npm run build
artifacts:
name: "build-${CI_COMMIT_SHA}"
paths:
- dist/
expire_in: 1 week
```
### Docker Images
```yaml
build_image:
script:
- docker build -t ${REGISTRY}/${IMAGE}:${CI_COMMIT_SHA} .
- docker tag ${REGISTRY}/${IMAGE}:${CI_COMMIT_SHA} ${REGISTRY}/${IMAGE}:latest
- docker push ${REGISTRY}/${IMAGE}:${CI_COMMIT_SHA}
- docker push ${REGISTRY}/${IMAGE}:latest
```
**Tagging Strategy**:
- **Commit SHA**: Immutable, traceable
- **Semantic version**: v1.2.3 (releases)
- **Branch name**: develop, staging
- **latest**: Most recent (use with caution)
## Deployment Patterns
### Environment Progression
```
Commit → Dev (auto) → Staging (auto) → Prod (manual)
```
### Deployment with Approval
```yaml
deploy_prod:
stage: deploy
environment:
name: production
url: https://app.example.com
when: manual # Require manual trigger
only:
- main
script:
- ./deploy.sh production
```
### Deployment with Verification
```yaml
deploy:
script:
- ./deploy.sh
- |
# Wait for deployment
for i in {1..30}; do
if curl -f https://app.example.com/health; then
echo "Deployment successful!"
exit 0
fi
sleep 10
done
echo "Deployment failed!"
exit 1
```
### Rollback on Failure
```yaml
deploy:
script:
- ./deploy.sh || (./rollback.sh && exit 1)
```
## Notification Patterns
### Slack Notifications
```yaml
notify_slack:
stage: .post
when: on_failure
script:
- |
curl -X POST -H 'Content-type: application/json' \
--data "{
\"text\": \"Pipeline failed for ${CI_PROJECT_NAME} on ${CI_COMMIT_BRANCH}\",
\"attachments\": [{
\"color\": \"danger\",
\"fields\": [{
\"title\": \"Commit\",
\"value\": \"${CI_COMMIT_SHORT_SHA}: ${CI_COMMIT_MESSAGE}\"
}, {
\"title\": \"Author\",
\"value\": \"${CI_COMMIT_AUTHOR}\"
}, {
\"title\": \"Pipeline\",
\"value\": \"${CI_PIPELINE_URL}\"
}]
}]
}" \
${SLACK_WEBHOOK_URL}
```
### Email on Production Deploy
```yaml
notify_email:
stage: .post
only:
- main
script:
- |
echo "Deployed ${CI_COMMIT_SHORT_SHA} to production" | \
mail -s "Production Deployment" team@example.com
```
## Branch Protection
### Required Checks
```yaml
# .github/workflows/required-checks.yml
name: Required Checks
on: [pull_request]
jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: npm run lint
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: npm test
security:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: npm audit
```
### GitHub Branch Protection Rules
- Require pull request reviews (1-2 reviewers)
- Require status checks to pass
- Require branches to be up to date
- Include administrators
- Restrict force pushes
## Common Patterns by Platform
### GitHub Actions
```yaml
name: CI/CD
on:
push:
branches: [main, develop]
pull_request:
branches: [main]
env:
NODE_VERSION: 18
REGISTRY: ghcr.io
jobs:
build-and-test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: ${{ env.NODE_VERSION }}
cache: 'npm'
- run: npm ci
- run: npm run build
- run: npm test -- --coverage
- uses: codecov/codecov-action@v3
```
### GitLab CI
```yaml
stages:
- build
- test
- security
- deploy
variables:
DOCKER_DRIVER: overlay2
SECURE_ANALYZERS_PREFIX: "registry.gitlab.com/security-products"
build:
stage: build
script:
- npm ci
- npm run build
artifacts:
paths:
- dist/
cache:
key: ${CI_COMMIT_REF_SLUG}
paths:
- node_modules/
test:
stage: test
script:
- npm test -- --coverage
coverage: '/All files[^|]*\|[^|]*\s+([\d\.]+)/'
```
### Jenkins
```groovy
pipeline {
agent any
environment {
NODE_VERSION = '18'
REGISTRY = 'registry.example.com'
}
stages {
stage('Build') {
steps {
sh 'npm ci'
sh 'npm run build'
}
}
stage('Test') {
parallel {
stage('Unit') {
steps {
sh 'npm test'
}
}
stage('Lint') {
steps {
sh 'npm run lint'
}
}
}
}
stage('Security') {
steps {
sh 'npm audit'
sh 'snyk test'
}
}
stage('Deploy') {
when {
branch 'main'
}
steps {
sh './deploy.sh'
}
}
}
post {
always {
junit 'reports/**/*.xml'
publishHTML([
reportDir: 'coverage',
reportFiles: 'index.html',
reportName: 'Coverage'
])
}
failure {
emailext(
subject: "Build Failed: ${env.JOB_NAME}",
body: "Check ${env.BUILD_URL}",
to: "${env.CHANGE_AUTHOR_EMAIL}"
)
}
}
}
```
## Cost Optimization
### GitHub Actions
- Use caching (50% faster, free)
- Use matrix builds sparingly
- Self-hosted runners for private repos
- **Cost**: $0.008/minute (Linux)
### GitLab CI
- Use shared runners (free tier: 400 minutes/month)
- Cache dependencies
- Limit parallel jobs
- **Cost**: Free tier available, $19/user/month Pro
### Jenkins
- Use spot instances for agents
- Shut down idle agents
- Containerized agents
- **Cost**: Infrastructure only
## Troubleshooting
### Slow Builds
1. Profile pipeline (which stage is slow?)
2. Add caching for dependencies
3. Parallelize independent jobs
4. Optimize Docker layers
5. Use smaller base images
### Flaky Tests
1. Identify flaky tests (run 100x)
2. Add explicit waits (not sleep)
3. Mock external dependencies
4. Isolate test data
5. Retry failed tests (max 3x)
### Failed Deployments
1. Check deployment logs
2. Verify health checks
3. Check resource constraints
4. Validate configuration
5. Rollback if needed
## Best Practices Summary
✅ **DO**:
- Keep pipelines fast (<10 min total)
- Fail fast (lint first, slow tests last)
- Cache dependencies
- Use semantic versioning
- Scan for vulnerabilities
- Require manual approval for prod
- Send notifications on failure
- Monitor pipeline performance
❌ **DON'T**:
- Hardcode secrets (use secrets management)
- Skip tests in CI
- Deploy without verification
- Use latest tag in prod
- Ignore security warnings
- Run unnecessary jobs
- Leave old artifacts
## Quick Reference
| Task | GitHub Actions | GitLab CI | Jenkins |
|------|----------------|-----------|---------|
| **Syntax** | YAML | YAML | Groovy |
| **Caching** | `cache:` key | `cache:` section | Pipeline plugin |
| **Artifacts** | `actions/upload-artifact` | `artifacts:` section | `archiveArtifacts` |
| **Secrets** | Repository secrets | CI/CD variables | Credentials plugin |
| **Matrix** | `strategy: matrix:` | `parallel:` | `matrix {}` |
| **Conditions** | `if:` | `only:` / `except:` | `when {}` |
---
## 🚀 MCP Integration: GitHub + Context7 for CI/CD Automation
### Runtime Detection & Usage
The skill automatically detects available MCPs for CI/CD workflow enhancement:
```typescript
const hasGitHub = typeof mcp__github__create_or_update_file !== 'undefined';
const hasContext7 = typeof mcp__context7__get_library_docs !== 'undefined';
if (hasGitHub && hasContext7) {
// Get latest CI/CD framework documentation
const githubActionsDocs = await mcp__context7__get_library_docs({
context7CompatibleLibraryID: "/actions/toolkit",
topic: "GitHub Actions workflow syntax caching artifacts",
tokens: 3000
});
// Create optimized workflow directly in repository
await mcp__github__create_or_update_file({
owner: "myorg",
repo: "myapp",
path: ".github/workflows/ci.yml",
content: generatedWorkflow,
message: "Add optimized CI/CD pipeline with caching"
});
} else {
console.log("ℹ️ GitHub/Context7 MCP not available");
console.log(" GitHub: npx @modelcontextprotocol/create-server github");
console.log(" Context7: npm install -g @context7/mcp-server");
}
```
### Real-World Workflow Examples
**Example 1: Multi-Stage Pipeline Generation with Best Practices**
```typescript
// Without MCP: Manual workflow writing (3 hours)
// 1. Read GitHub Actions docs
// 2. Research caching strategies
// 3. Write YAML from scratch
// 4. Test and debug
// 5. Optimize
// With GitHub + Context7 MCP: AI-assisted generation (15 minutes)
const actionsDocs = await mcp__context7__get_library_docs({
context7CompatibleLibraryID: "/actions/toolkit",
topic: "caching dependencies matrix builds artifacts security scanning",
tokens: 4000
});
const securityDocs = await mcp__context7__get_library_docs({
context7CompatibleLibraryID: "/returntocorp/semgrep",
topic: "CI integration security scanning",
tokens: 2500
});
// Generate optimized workflow
const workflow = generateGitHubActionsWorkflow({
language: "node",
stages: ["build", "test", "security", "deploy"],
patterns: actionsDocs,
securityScan: securityDocs
});
// Deploy directly to repository
await mcp__github__create_or_update_file({
owner: "myorg",
repo: "myapp",
path: ".github/workflows/ci.yml",
content: workflow,
message: "feat: add optimized CI/CD pipeline
- Multi-stage build with caching
- Parallel test execution
- Security scanning (SAST + dependency)
- Conditional deployment"
});
// ✅ 12x faster pipeline creation
// ✅ Latest best practices applied
// ✅ Automatic repository integration
```
**Example 2: GitLab CI to GitHub Actions Migration**
```typescript
// Analyze existing GitLab CI configuration
const gitlabConfig = await mcp__github__get_file_contents({
owner: "myorg",
repo: "legacy-app",
path: ".gitlab-ci.yml"
});
// Get GitHub Actions patterns
const migrationDocs = await mcp__context7__get_library_docs({
context7CompatibleLibraryID: "/actions/toolkit",
topic: "GitLab CI migration GitHub Actions equivalents",
tokens: 3500
});
// Convert GitLab CI → GitHub Actions
const convertedWorkflow = convertGitLabToGitHubActions({
gitlabConfig: gitlabConfig.content,
patterns: migrationDocs
});
// Create PR with converted workflow
await mcp__github__create_pull_request({
owner: "myorg",
repo: "legacy-app",
title: "Migrate from GitLab CI to GitHub Actions",
body: `## Migration Summary
- Converted all stages to GitHub Actions jobs
- Preserved caching strategy
- Maintained deployment logic
- Added security scanning
## Changes
- \`.gitlab-ci.yml\` → \`.github/workflows/ci.yml\`
- Updated cache paths for GitHub Actions
- Converted variables to GitHub secrets
`,
head: "feat/github-actions-migration",
base: "main",
files: [
{ path: ".github/workflows/ci.yml", content: convertedWorkflow }
]
});
// ✅ Migration (1 hour vs 1 day)
// ✅ Automatic PR creation
// ✅ Best practices applied
```
**Example 3: CI/CD Performance Optimization**
```typescript
// Analyze current pipeline performance
const workflows = await mcp__github__list_workflows({
owner: "myorg",
repo: "myapp"
});
const runs = await mcp__github__list_workflow_runs({
owner: "myorg",
repo: "myapp",
workflow_id: workflows[0].id,
per_page: 100
});
// Get optimization patterns
const optimizationDocs = await mcp__context7__get_library_docs({
context7CompatibleLibraryID: "/actions/toolkit",
topic: "workflow optimization caching parallelization",
tokens: 3500
});
// Analyze bottlenecks
const analysis = analyzeWorkflowPerformance(runs);
// Results: Test stage takes 15 min (80% of total time)
// Optimize with parallelization
const optimizedWorkflow = await optimizeWorkflow({
currentWorkflow: workflow,
bottlenecks: analysis.bottlenecks,
patterns: optimizationDocs,
strategies: ["parallelize-tests", "cache-dependencies", "matrix-builds"]
});
// Deploy optimized workflow
await mcp__github__create_or_update_file({
owner: "myorg",
repo: "myapp",
path: ".github/workflows/ci.yml",
content: optimizedWorkflow,
message: "perf: optimize CI pipeline
- Parallelize tests across 4 runners
- Add dependency caching
- Use matrix strategy for multi-version testing
Reduces pipeline time: 18 min → 5 min (72% faster)"
});
// ✅ Pipeline time: 18 min → 5 min (72% faster)
// ✅ Cost reduction: 4x less compute time
// ✅ Faster feedback for developers
```
**Example 4: Automated Security Scanning Integration**
```typescript
// Get security tool documentation
const securityTools = [
"/returntocorp/semgrep",
"/aquasecurity/trivy",
"/zricethezav/gitleaks"
];
const securityDocs = await Promise.all(
securityTools.map(tool =>
mcp__context7__get_library_docs({
context7CompatibleLibraryID: tool,
topic: "CI integration security scanning",
tokens: 2500
})
)
);
// Generate comprehensive security workflow
const securityWorkflow = generateSecurityWorkflow({
sast: securityDocs[0], // Semgrep
container: securityDocs[1], // Trivy
secrets: securityDocs[2] // Gitleaks
});
// Add to repository
await mcp__github__create_or_update_file({
owner: "myorg",
repo: "myapp",
path: ".github/workflows/security.yml",
content: securityWorkflow,
message: "security: add comprehensive security scanning
- SAST: Semgrep for code analysis
- Container: Trivy for image scanning
- Secrets: Gitleaks for credential detection
Fails pipeline on HIGH/CRITICAL vulnerabilities"
});
// ✅ Comprehensive security (30 min vs 4 hours)
// ✅ Multiple scan types integrated
// ✅ Production-ready thresholds
```
### Available Library IDs for CI/CD
**GitHub Actions**:
- `/actions/toolkit` - GitHub Actions core
- `/actions/cache` - Caching action
- `/actions/upload-artifact` - Artifact uploads
- `/actions/download-artifact` - Artifact downloads
**GitLab CI**:
- `/gitlab-org/gitlab` - GitLab CI/CD
- `/gitlab-org/gitlab-runner` - GitLab Runner
**Jenkins**:
- `/jenkinsci/jenkins` - Jenkins core
- `/jenkinsci/pipeline-plugin` - Pipeline as Code
**CI/CD Tools**:
- `/circleci/circleci-docs` - CircleCI
- `/travis-ci/travis-ci` - Travis CI
- `/drone/drone` - Drone CI
**Security Scanning**:
- `/returntocorp/semgrep` - SAST
- `/aquasecurity/trivy` - Container scanning
- `/zricethezav/gitleaks` - Secret detection
- `/snyk/cli` - Dependency scanning
**Build Tools**:
- `/docker/build-push-action` - Docker builds
- `/docker/metadata-action` - Docker metadata
### Benefits Comparison
| Task | Without MCP | With GitHub + Context7 MCP | Time Saved |
|------|-------------|---------------------------|------------|
| New pipeline creation | 3 hours (manual docs + trial/error) | 15 min (AI-assisted) | 92% faster |
| GitLab → GitHub migration | 1 day (conversion + testing) | 1 hour (automated) | 88% faster |
| Pipeline optimization | 4 hours (profiling + research) | 30 min (analysis + apply) | 87% faster |
| Security integration | 4 hours (tool research + setup) | 30 min (multi-tool setup) | 87% faster |
| Branch protection setup | 30 min (manual clicking) | 2 min (API automation) | 93% faster |
### When to Use GitHub + Context7 MCP
**Ideal for**:
- ✅ Creating new CI/CD pipelines from scratch
- ✅ Migrating between CI/CD platforms
- ✅ Optimizing existing pipeline performance
- ✅ Integrating security scanning tools
- ✅ Setting up branch protection rules
- ✅ Automating PR workflows
- ✅ Multi-repo pipeline standardization
**Not needed for**:
- ❌ Simple one-stage builds
- ❌ Basic linting workflows
- ❌ Trivial pipeline modifications
### Installation
```bash
# Install GitHub MCP
npx @modelcontextprotocol/create-server github
# Install Context7 MCP
npm install -g @context7/mcp-server
# Configure in Claude Code MCP settings
# Both servers will auto-detect and enable integration
```
### Security Best Practices
When using GitHub + Context7 MCP for CI/CD:
- Never commit secrets to workflow files (use GitHub Secrets)
- Validate all security scanner configurations
- Use pinned versions for actions (not `@main`)
- Review generated workflows before merging
- Enable branch protection on main branches
- Require status checks before merging
- Use least privilege for CI/CD service accounts
---
**Version**: 1.0.0
**Last Updated**: 2025-01-20
**Patterns**: 20+
**Best Practices**: Production-tested

3660
skills/frontend-development/SKILL.md Normal file
View File

File diff suppressed because it is too large Load Diff

215
skills/mobile-development/SKILL.md Normal file
View File

@@ -0,0 +1,215 @@
# Mobile Development
**iOS, Android, React Native, and Flutter development patterns**
## Platform-Specific Development
### iOS (Swift/SwiftUI)
#### Project Structure
```
MyApp/
├── Models/
├── Views/
├── ViewModels/
├── Services/
└── Resources/
```
#### SwiftUI Patterns
```swift
struct ContentView: View {
@StateObject private var viewModel = ViewModel()
var body: some View {
NavigationView {
List(viewModel.items) { item in
ItemRow(item: item)
}
.navigationTitle("Items")
}
.task {
await viewModel.loadItems()
}
}
}
```
### Android (Kotlin/Jetpack Compose)
#### Project Structure
```
app/
├── data/
├── domain/
├── presentation/
│ ├── ui/
│ └── viewmodels/
└── di/
```
#### Jetpack Compose Patterns
```kotlin
@Composable
fun ItemList(viewModel: ItemViewModel = hiltViewModel()) {
val items by viewModel.items.collectAsState()
LazyColumn {
items(items) { item ->
ItemCard(item = item)
}
}
}
```
## Cross-Platform Development
### React Native
#### Component Structure
```typescript
import { StyleSheet, View, Text } from 'react-native'
const MyComponent: React.FC<Props> = ({ title }) => {
return (
<View style={styles.container}>
<Text style={styles.title}>{title}</Text>
</View>
)
}
const styles = StyleSheet.create({
container: {
flex: 1,
padding: 16,
},
title: {
fontSize: 24,
fontWeight: 'bold',
},
})
```
#### Navigation
```typescript
import { NavigationContainer } from '@react-navigation/native'
import { createNativeStackNavigator } from '@react-navigation/native-stack'
const Stack = createNativeStackNavigator()
export const App = () => (
<NavigationContainer>
<Stack.Navigator>
<Stack.Screen name="Home" component={HomeScreen} />
<Stack.Screen name="Details" component={DetailsScreen} />
</Stack.Navigator>
</NavigationContainer>
)
```
### Flutter
#### Widget Structure
```dart
class MyWidget extends StatelessWidget {
final String title;
const MyWidget({Key? key, required this.title}) : super(key: key);
@override
Widget build(BuildContext context) {
return Container(
padding: EdgeInsets.all(16),
child: Text(
title,
style: Theme.of(context).textTheme.headline4,
),
);
}
}
```
## Mobile Best Practices
### Performance
- Lazy loading
- Image optimization
- List virtualization
- Memory management
- Battery optimization
### Offline Support
- Local data persistence
- Sync strategies
- Conflict resolution
- Cache management
### Security
- Secure storage (Keychain/Keystore)
- Certificate pinning
- Code obfuscation
- Biometric authentication
### App Store Guidelines
- iOS App Store Review Guidelines
- Google Play Store policies
- Privacy policies
- Content ratings
## Testing
### Unit Tests
- Business logic
- ViewModels
- Utility functions
### Widget/Component Tests
- UI components
- User interactions
- State changes
### Integration Tests
- E2E flows
- API integration
- Navigation flows
### Platform Testing
- iOS Simulator
- Android Emulator
- Physical devices
- Cloud device farms (Firebase Test Lab, BrowserStack)
## Deployment
### iOS
- TestFlight beta testing
- App Store Connect
- Provisioning profiles
- Code signing
### Android
- Internal testing track
- Google Play Console
- App signing
- Release management
## Common Patterns
### State Management
- Redux/MobX (React Native)
- Provider/Riverpod (Flutter)
- MVVM (iOS/Android)
- BLoC (Flutter)
### API Integration
- REST APIs
- GraphQL
- WebSockets
- Offline-first sync
### Push Notifications
- Firebase Cloud Messaging
- Apple Push Notification Service
- Local notifications
- Deep linking

207
skills/testing-practices/SKILL.md Normal file
View File

@@ -0,0 +1,207 @@
# Testing Practices
**Comprehensive testing strategies across the stack**
## Testing Pyramid
### Unit Tests (70%)
- Test individual functions/methods
- Fast execution
- No external dependencies
- High coverage of business logic
### Integration Tests (20%)
- Test component interactions
- Database queries
- API endpoints
- Service integrations
### E2E Tests (10%)
- Test user workflows
- Full stack testing
- Browser automation
- Critical paths only
## Frontend Testing
### Component Testing (React)
```tsx
import { render, screen, fireEvent } from '@testing-library/react'
import { Button } from './Button'
describe('Button', () => {
it('calls onClick when clicked', () => {
const onClick = jest.fn()
render(<Button onClick={onClick}>Click me</Button>)
fireEvent.click(screen.getByText('Click me'))
expect(onClick).toHaveBeenCalledTimes(1)
})
})
```
### State Management Testing
```tsx
import { renderHook, act } from '@testing-library/react-hooks'
import { useCounter } from './useCounter'
it('increments counter', () => {
const { result } = renderHook(() => useCounter())
act(() => {
result.current.increment()
})
expect(result.current.count).toBe(1)
})
```
### E2E Testing (Playwright)
```typescript
import { test, expect } from '@playwright/test'
test('user can log in', async ({ page }) => {
await page.goto('/login')
await page.fill('[name=email]', 'user@example.com')
await page.fill('[name=password]', 'password123')
await page.click('button[type=submit]')
await expect(page).toHaveURL('/dashboard')
await expect(page.locator('h1')).toContainText('Welcome')
})
```
## Backend Testing
### API Testing
```typescript
import request from 'supertest'
import { app } from './app'
describe('POST /api/users', () => {
it('creates a new user', async () => {
const res = await request(app)
.post('/api/users')
.send({
name: 'John Doe',
email: 'john@example.com'
})
.expect(201)
expect(res.body).toHaveProperty('id')
expect(res.body.email).toBe('john@example.com')
})
})
```
### Database Testing
```typescript
beforeEach(async () => {
await db.migrate.rollback()
await db.migrate.latest()
await db.seed.run()
})
afterEach(async () => {
await db.destroy()
})
```
## Test Best Practices
### AAA Pattern
```typescript
it('example test', () => {
// Arrange
const input = { ... }
// Act
const result = functionUnderTest(input)
// Assert
expect(result).toBe(expected)
})
```
### Test Isolation
- Each test should be independent
- No shared state between tests
- Clean up after each test
- Use beforeEach/afterEach appropriately
### Meaningful Test Names
```typescript
// ❌ BAD
it('works', () => { ... })
// ✅ GOOD
it('returns 404 when user not found', () => { ... })
```
### Mocking
```typescript
jest.mock('./api')
it('handles API errors', async () => {
api.fetchUser.mockRejectedValue(new Error('Network error'))
// Test error handling
})
```
## Performance Testing
### Load Testing
- Apache JMeter
- k6
- Artillery
### Stress Testing
- Identify breaking points
- Test recovery
- Monitor resource usage
## Test Coverage
### Coverage Metrics
- Line coverage: 80%+ target
- Branch coverage: 75%+ target
- Function coverage: 90%+ target
### Coverage Tools
- Jest (--coverage)
- Istanbul/nyc
- Codecov/Coveralls
## CI/CD Integration
### GitHub Actions
```yaml
name: Tests
on: [push, pull_request]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- run: npm install
- run: npm test
- run: npm run test:e2e
```
### Test Stages
1. Lint and type check
2. Unit tests
3. Integration tests
4. Build verification
5. E2E tests (on staging)
## Debugging Tests
- Use `test.only` to isolate
- Add console.logs strategically
- Use debugger statements
- Check test environment setup
- Verify mock implementations