zhongwei/gh-emillindfors-claude-marketplace-plugins-rust-lambda
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial commit

Browse Source
This commit is contained in:
Zhongwei Li
2025-11-29 18:25:52 +08:00
commit e840c60a06
19 changed files with 6852 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

477
commands/lambda-advanced.md Normal file
View File

@@ -0,0 +1,477 @@
---
description: Advanced Lambda topics including extensions, container images, and local development
---
You are helping the user with advanced Rust Lambda topics including custom extensions, container images, and enhanced local development.
## Your Task
Guide the user through advanced Lambda patterns and deployment options.
## Lambda Extensions
Extensions run alongside your function to provide observability, security, or governance capabilities.
### When to Build Extensions
- Custom monitoring/observability
- Secret rotation
- Configuration management
- Security scanning
- Custom logging
### Creating a Rust Extension
Add to `Cargo.toml`:
```toml
[dependencies]
lambda-extension = "0.13"
tokio = { version = "1", features = ["macros"] }
tracing = "0.1"
```
Basic extension:
```rust
use lambda_extension::{service_fn, Error, LambdaEvent, NextEvent};
use tracing::info;
async fn handler(event: LambdaEvent) -> Result<(), Error> {
match event.next {
NextEvent::Shutdown(_e) => {
info!("Shutting down extension");
}
NextEvent::Invoke(_e) => {
info!("Function invoked");
// Collect telemetry, logs, etc.
}
}
Ok(())
}
#[tokio::main]
async fn main() -> Result<(), Error> {
tracing_subscriber::fmt().init();
let extension_name = "my-rust-extension";
lambda_extension::run(service_fn(handler)).await
}
```
### Deploy Extension as Layer
```bash
# Build extension
cargo lambda build --release --extension
# Create layer
aws lambda publish-layer-version \
--layer-name my-rust-extension \
--zip-file fileb://target/lambda/extensions/my-extension.zip \
--compatible-runtimes provided.al2023 \
--compatible-architectures arm64
# Add to function
cargo lambda deploy \
--layers arn:aws:lambda:region:account:layer:my-rust-extension:1
```
### Logging Extension Example
```rust
use lambda_extension::{service_fn, Error, LambdaLog, LambdaLogRecord};
use std::fs::OpenOptions;
use std::io::Write;
async fn handler(logs: Vec<LambdaLog>) -> Result<(), Error> {
let mut file = OpenOptions::new()
.create(true)
.append(true)
.open("/tmp/extension-logs.txt")?;
for log in logs {
match log.record {
LambdaLogRecord::Function(record) => {
writeln!(file, "[FUNCTION] {}", record)?;
}
LambdaLogRecord::Extension(record) => {
writeln!(file, "[EXTENSION] {}", record)?;
}
_ => {}
}
}
Ok(())
}
```
## Container Images
Deploy Lambda as container image instead of ZIP (max 10GB vs 250MB).
### When to Use Containers
**Use containers when**:
- Large dependencies (>250MB uncompressed)
- Custom system libraries
- Complex build process
- Team familiar with Docker
- Need exact runtime control
**Use ZIP when**:
- Simple deployment
- Fast iteration
- Smaller functions
- Standard dependencies
### Dockerfile for Rust Lambda
```dockerfile
FROM public.ecr.aws/lambda/provided:al2023-arm64
# Install Rust
RUN yum install -y gcc && \
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y && \
source $HOME/.cargo/env
# Copy source
WORKDIR /var/task
COPY Cargo.toml Cargo.lock ./
COPY src ./src
# Build
RUN source $HOME/.cargo/env && \
cargo build --release && \
cp target/release/bootstrap ${LAMBDA_RUNTIME_DIR}/bootstrap
CMD ["bootstrap"]
```
### Multi-stage Build (Smaller Image)
```dockerfile
# Build stage
FROM rust:1.75-slim as builder
WORKDIR /app
COPY Cargo.toml Cargo.lock ./
COPY src ./src
RUN cargo build --release --target x86_64-unknown-linux-musl
# Runtime stage
FROM public.ecr.aws/lambda/provided:al2023
COPY --from=builder /app/target/x86_64-unknown-linux-musl/release/bootstrap \
${LAMBDA_RUNTIME_DIR}/bootstrap
CMD ["bootstrap"]
```
### Build and Deploy Container
```bash
# Build image
docker build -t my-rust-lambda .
# Tag for ECR
docker tag my-rust-lambda:latest \
123456789012.dkr.ecr.us-east-1.amazonaws.com/my-rust-lambda:latest
# Login to ECR
aws ecr get-login-password --region us-east-1 | \
docker login --username AWS --password-stdin \
123456789012.dkr.ecr.us-east-1.amazonaws.com
# Push
docker push 123456789012.dkr.ecr.us-east-1.amazonaws.com/my-rust-lambda:latest
# Create/update Lambda
aws lambda create-function \
--function-name my-rust-lambda \
--package-type Image \
--code ImageUri=123456789012.dkr.ecr.us-east-1.amazonaws.com/my-rust-lambda:latest \
--role arn:aws:iam::123456789012:role/lambda-role
```
## Local Development
### Option 1: cargo-lambda watch (Recommended)
```bash
# Start local Lambda emulator
cargo lambda watch
# Invoke in another terminal
cargo lambda invoke --data-ascii '{"test": "data"}'
# With specific event file
cargo lambda invoke --data-file events/api-gateway.json
```
### Option 2: LocalStack (Full AWS Emulation)
```bash
# Install LocalStack
pip install localstack
# Start LocalStack
localstack start
# Deploy to LocalStack
samlocal deploy
# Or with cargo-lambda
cargo lambda build --release
aws --endpoint-url=http://localhost:4566 lambda create-function \
--function-name my-function \
--runtime provided.al2023 \
--role arn:aws:iam::000000000000:role/lambda-role \
--handler bootstrap \
--zip-file fileb://target/lambda/bootstrap.zip
```
### Option 3: SAM Local
```bash
# template.yaml
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Resources:
MyFunction:
Type: AWS::Serverless::Function
Properties:
CodeUri: .
Handler: bootstrap
Runtime: provided.al2023
# Start local API
sam local start-api
# Invoke function
sam local invoke MyFunction -e events/test.json
```
## Lambda Layers (Note: Not Recommended for Rust)
**AWS Recommendation**: Don't use layers for Rust dependencies.
**Why**: Rust compiles to a single static binary. All dependencies are included at compile time.
**Exception**: Use layers for:
- Lambda Extensions
- Shared native libraries (rare)
- Non-Rust resources (config files, ML models)
## VPC Configuration
Connect Lambda to VPC for private resource access.
```bash
cargo lambda deploy \
--subnet-ids subnet-12345 subnet-67890 \
--security-group-ids sg-12345
```
**Performance impact**:
- Cold start: +10-15 seconds (Hyperplane ENI creation)
- Warm start: No impact
**Mitigation**:
- Use multiple subnets/AZs
- Keep functions warm
- Consider NAT Gateway for internet access
## Reserved Concurrency
Limit concurrent executions:
```bash
aws lambda put-function-concurrency \
--function-name my-function \
--reserved-concurrent-executions 10
```
**Use cases**:
- Protect downstream resources
- Cost control
- Predictable scaling
## Asynchronous Invocation
### Configure Destinations
```bash
# On success, send to SQS
aws lambda put-function-event-invoke-config \
--function-name my-function \
--destination-config '{
"OnSuccess": {
"Destination": "arn:aws:sqs:us-east-1:123:success-queue"
},
"OnFailure": {
"Destination": "arn:aws:sns:us-east-1:123:failure-topic"
}
}'
```
### Dead Letter Queue
```rust
// Lambda automatically retries failed async invocations
// Configure DLQ for ultimate failures
#[tokio::main]
async fn main() -> Result<(), Error> {
run(service_fn(function_handler)).await
}
async fn function_handler(event: LambdaEvent<Request>) -> Result<Response, Error> {
// If this fails after retries, goes to DLQ
process_event(&event.payload).await?;
Ok(Response::success())
}
```
## Event Source Mappings
### SQS with Batch Processing
```rust
use aws_lambda_events::event::sqs::SqsEvent;
async fn handler(event: LambdaEvent<SqsEvent>) -> Result<(), Error> {
// Process batch concurrently
let futures = event.payload.records
.into_iter()
.map(|record| async move {
let body: Message = serde_json::from_str(&record.body?)?;
process_message(body).await
});
futures::future::try_join_all(futures).await?;
Ok(())
}
```
Configure batch size:
```bash
aws lambda create-event-source-mapping \
--function-name my-function \
--event-source-arn arn:aws:sqs:us-east-1:123:my-queue \
--batch-size 10 \
--maximum-batching-window-in-seconds 5
```
## Advanced Error Handling
### Partial Batch Responses (SQS)
```rust
use lambda_runtime::{LambdaEvent, Error};
use aws_lambda_events::event::sqs::{SqsEvent, SqsBatchResponse};
async fn handler(event: LambdaEvent<SqsEvent>) -> Result<SqsBatchResponse, Error> {
let mut failed_ids = Vec::new();
for record in event.payload.records {
match process_record(&record).await {
Ok(_) => {},
Err(e) => {
tracing::error!("Failed to process: {}", e);
if let Some(msg_id) = record.message_id {
failed_ids.push(msg_id);
}
}
}
}
Ok(SqsBatchResponse {
batch_item_failures: failed_ids
.into_iter()
.map(|id| sqs::SqsBatchItemFailure { item_identifier: id })
.collect(),
})
}
```
## Multi-Region Deployment
```bash
# Deploy to multiple regions
for region in us-east-1 us-west-2 eu-west-1; do
echo "Deploying to $region"
cargo lambda deploy --region $region my-function
done
```
## Blue/Green Deployments
```bash
# Create alias
aws lambda create-alias \
--function-name my-function \
--name production \
--function-version 1
# Gradual rollout
aws lambda update-alias \
--function-name my-function \
--name production \
--routing-config '{"AdditionalVersionWeights": {"2": 0.1}}'
# Full cutover
aws lambda update-alias \
--function-name my-function \
--name production \
--function-version 2
```
## Testing Strategies
### Integration Tests with LocalStack
```rust
#[cfg(test)]
mod tests {
use super::*;
#[tokio::test]
async fn test_with_localstack() {
// Set LocalStack endpoint
std::env::set_var("AWS_ENDPOINT_URL", "http://localhost:4566");
let event = create_test_event();
let response = function_handler(event).await.unwrap();
assert_eq!(response.status, "success");
}
}
```
### Load Testing
```bash
# Artillery config (artillery.yml)
config:
target: "https://function-url.lambda-url.us-east-1.on.aws"
phases:
- duration: 60
arrivalRate: 10
name: "Warm up"
- duration: 300
arrivalRate: 100
name: "Load test"
scenarios:
- flow:
- post:
url: "/"
json:
test: "data"
# Run
artillery run artillery.yml
```
Guide the user through these advanced topics based on their specific needs and architecture requirements.

240
commands/lambda-build.md Normal file
View File

@@ -0,0 +1,240 @@
---
description: Build Rust Lambda function for AWS deployment with optimizations
---
You are helping the user build their Rust Lambda function for AWS deployment.
## Your Task
Guide the user through building their Lambda function with appropriate optimizations:
1. **Verify project setup**:
- Check that Cargo.toml has release profile optimizations
- Verify lambda_runtime dependency is present
- Confirm project compiles: `cargo check`
2. **Choose architecture**:
Ask the user which architecture to target:
- **x86_64** (default): Compatible with most existing infrastructure
- **ARM64** (Graviton2): 20% better price/performance, often faster cold starts
- **Both**: Build for both architectures
3. **Build command**:
**For x86_64**:
```bash
cargo lambda build --release
```
**For ARM64** (recommended):
```bash
cargo lambda build --release --arm64
```
**For both**:
```bash
cargo lambda build --release
cargo lambda build --release --arm64
```
**With zip output** (for manual deployment):
```bash
cargo lambda build --release --output-format zip
```
4. **Verify build**:
- Check binary size: `ls -lh target/lambda/*/bootstrap`
- Typical sizes:
- Small function: 1-3 MB
- With AWS SDK: 5-10 MB
- Large dependencies: 10-20 MB
- If too large, suggest optimizations (see below)
5. **Build output location**:
- x86_64: `target/lambda/<function-name>/bootstrap`
- ARM64: `target/lambda/<function-name>/bootstrap` (when building with --arm64)
- Zip: `target/lambda/<function-name>.zip`
## Release Profile Optimization
Ensure Cargo.toml has optimal release profile:
```toml
[profile.release]
opt-level = 'z' # Optimize for size (or 3 for speed)
lto = true # Link-time optimization
codegen-units = 1 # Better optimization (slower compile)
strip = true # Remove debug symbols
panic = 'abort' # Smaller panic handling
```
### Optimization Tradeoffs
**For smaller binary (faster cold start)**:
```toml
opt-level = 'z'
```
**For faster execution**:
```toml
opt-level = 3
```
## Size Optimization Tips
If the binary is too large:
1. **Check dependencies**:
```bash
cargo tree
```
Look for unnecessary or duplicate dependencies
2. **Use feature flags**:
```toml
# Only enable needed features
tokio = { version = "1", features = ["macros", "rt"] }
# Instead of:
# tokio = { version = "1", features = ["full"] }
```
3. **Audit with cargo-bloat**:
```bash
cargo install cargo-bloat
cargo bloat --release -n 20
```
4. **Consider lighter alternatives**:
- Use `ureq` instead of `reqwest` for simple HTTP
- Use `rustls` instead of `native-tls`
- Minimize AWS SDK crates
5. **Remove unused code**:
- Ensure `strip = true` in profile
- Use `cargo-unused-features` to find unused features
## Cross-Compilation Requirements
cargo-lambda uses Zig for cross-compilation. If you encounter issues:
1. **Install Zig**:
```bash
# macOS
brew install zig
# Linux (download from ziglang.org)
wget https://ziglang.org/download/0.11.0/zig-linux-x86_64-0.11.0.tar.xz
tar xf zig-linux-x86_64-0.11.0.tar.xz
export PATH=$PATH:$PWD/zig-linux-x86_64-0.11.0
```
2. **Verify Zig**:
```bash
zig version
```
## Build Flags
Additional useful flags:
```bash
# Build specific binary in workspace
cargo lambda build --release --bin my-function
# Build all binaries
cargo lambda build --release --all
# Build with compiler flags
cargo lambda build --release -- -C target-cpu=native
# Verbose output
cargo lambda build --release --verbose
```
## Testing the Build
After building, test locally:
```bash
# Start local Lambda runtime
cargo lambda watch
# In another terminal, invoke the function
cargo lambda invoke --data-ascii '{"test": "data"}'
```
## Build Performance
Speed up builds:
1. **Use sccache**:
```bash
cargo install sccache
export RUSTC_WRAPPER=sccache
```
2. **Parallel compilation** (already enabled by default)
3. **Incremental compilation** (for development):
```toml
[profile.dev]
incremental = true
```
## Architecture Decision Guide
**Choose x86_64 when**:
- Need compatibility with existing x86 infrastructure
- Using dependencies that don't support ARM64
- Already have x86 configuration/scripts
**Choose ARM64 when**:
- Want better price/performance (20% savings)
- Need faster execution
- Want potentially faster cold starts
- Starting fresh project (recommended)
**Build both when**:
- Want to test both architectures
- Supporting multiple deployment targets
- Migrating from x86 to ARM
## Common Build Issues
### Issue: "Zig not found"
**Solution**: Install Zig (see above)
### Issue: "Cannot find -lssl"
**Solution**: Install OpenSSL development files
```bash
# Ubuntu/Debian
sudo apt-get install libssl-dev pkg-config
# macOS
brew install openssl
```
### Issue: "Binary too large" (>50MB uncompressed)
**Solution**:
- Review dependencies with `cargo tree`
- Enable all size optimizations
- Consider splitting into multiple functions
### Issue: Build succeeds but Lambda fails
**Solution**:
- Ensure building for correct architecture
- Test locally with `cargo lambda watch`
- Check CloudWatch logs for specific errors
## Next Steps
After successful build:
1. Test locally: `/lambda-invoke` or `cargo lambda watch`
2. Deploy: Use `/lambda-deploy`
3. Set up CI/CD: Use `/lambda-github-actions`
Report the build results including:
- Binary size
- Architecture
- Build time
- Any warnings or suggestions

267
commands/lambda-cost.md Normal file
View File

@@ -0,0 +1,267 @@
---
description: Deep dive into Lambda cost optimization strategies for Rust functions
---
You are helping the user optimize the cost of their Rust Lambda functions.
## Your Task
Guide the user through advanced cost optimization techniques using AWS Lambda Power Tuning, memory configuration, and Rust-specific optimizations.
## Lambda Pricing Model
**Cost = (Requests × $0.20 per 1M) + (GB-seconds × $0.0000166667)**
- **Requests**: $0.20 per 1 million requests
- **Duration**: Charged per GB-second
- 1 GB-second = 1 GB memory × 1 second execution
- ARM64 (Graviton2): 20% cheaper than x86_64
**Example**:
- 512 MB, 100ms execution, 1M requests/month
- Duration: 0.5 GB × 0.1s × 1M = 50,000 GB-seconds
- Cost: $0.20 + (50,000 × $0.0000166667) = $0.20 + $0.83 = $1.03
## Memory vs CPU Allocation
Lambda allocates CPU proportional to memory:
- **128 MB**: 0.08 vCPU
- **512 MB**: 0.33 vCPU
- **1024 MB**: 0.58 vCPU
- **1769 MB**: 1.00 vCPU (full core)
- **3008 MB**: 1.77 vCPU
- **10240 MB**: 6.00 vCPU
**Key insight**: More memory = more CPU = faster execution = potentially lower cost
## AWS Lambda Power Tuning Tool
Automatically finds optimal memory configuration.
### Setup
```bash
# Deploy Power Tuning (one-time)
git clone https://github.com/alexcasalboni/aws-lambda-power-tuning
cd aws-lambda-power-tuning
sam deploy --guided
# Run power tuning
aws stepfunctions start-execution \
--state-machine-arn arn:aws:states:REGION:ACCOUNT:stateMachine:powerTuningStateMachine \
--input '{
"lambdaARN": "arn:aws:lambda:REGION:ACCOUNT:function:my-rust-function",
"powerValues": [128, 256, 512, 1024, 1536, 2048, 3008],
"num": 10,
"payload": "{\"test\": \"data\"}",
"parallelInvocation": true,
"strategy": "cost"
}'
```
**Strategies**:
- `cost`: Minimize cost
- `speed`: Minimize duration
- `balanced`: Balance cost and speed
## Rust-Specific Optimizations
### 1. Binary Size Reduction
**Cargo.toml**:
```toml
[profile.release]
opt-level = 'z' # Optimize for size
lto = true # Link-time optimization
codegen-units = 1 # Single codegen unit
strip = true # Strip symbols
panic = 'abort' # Smaller panic handler
[profile.release.package."*"]
opt-level = 'z' # Optimize dependencies too
```
**Result**: 3-5x smaller binary = faster cold starts = lower duration
### 2. Use ARM64 (Graviton2)
```bash
cargo lambda build --release --arm64
cargo lambda deploy --arch arm64
```
**Savings**: 20% lower cost for same performance
### 3. Dependency Optimization
```bash
# Analyze binary size
cargo install cargo-bloat
cargo bloat --release -n 20
# Find unused features
cargo install cargo-unused-features
cargo unused-features
# Remove unused dependencies
cargo install cargo-udeps
cargo +nightly udeps
```
**Example**:
```toml
# ❌ Full tokio (heavy)
tokio = { version = "1", features = ["full"] }
# ✅ Only needed features (light)
tokio = { version = "1", features = ["macros", "rt"] }
```
### 4. Use Lightweight Alternatives
- `ureq` instead of `reqwest` for simple HTTP
- `rustls` instead of `native-tls`
- `simdjson` instead of `serde_json` for large JSON
- Avoid `regex` for simple string operations
## Memory Configuration Strategies
### Strategy 1: Start Low, Test Up
```bash
# Test different memory sizes
for mem in 128 256 512 1024 2048; do
echo "Testing ${mem}MB"
cargo lambda deploy --memory $mem
# Run load test
# Measure duration and cost
done
```
### Strategy 2: Monitor CloudWatch Metrics
```bash
# Get duration statistics
aws cloudwatch get-metric-statistics \
--namespace AWS/Lambda \
--metric-name Duration \
--dimensions Name=FunctionName,Value=my-function \
--start-time 2025-01-01T00:00:00Z \
--end-time 2025-01-02T00:00:00Z \
--period 3600 \
--statistics Average,Maximum
# Get memory usage
aws cloudwatch get-metric-statistics \
--namespace AWS/Lambda \
--metric-name MemoryUtilization \
--dimensions Name=FunctionName,Value=my-function \
--start-time 2025-01-01T00:00:00Z \
--end-time 2025-01-02T00:00:00Z \
--period 3600 \
--statistics Maximum
```
### Strategy 3: Right-size Based on Workload
**IO-intensive** (API calls, DB queries):
- Start: 512 MB
- Sweet spot: Usually 512-1024 MB
- Reason: Limited by network, not CPU
**Compute-intensive** (data processing):
- Start: 1024 MB
- Sweet spot: Usually 1769-3008 MB
- Reason: More CPU = faster = lower total cost
**Mixed workload**:
- Start: 1024 MB
- Test: 1024, 1769, 2048 MB
- Use Power Tuning tool
## Cost Optimization Checklist
- [ ] Use ARM64 architecture (20% savings)
- [ ] Optimize binary size (faster cold starts)
- [ ] Remove unused dependencies
- [ ] Use lightweight alternatives
- [ ] Run AWS Lambda Power Tuning
- [ ] Right-size memory based on workload
- [ ] Set appropriate timeout (not too high)
- [ ] Reduce cold starts (keep functions warm if needed)
- [ ] Use reserved concurrency for predictable workloads
- [ ] Batch requests when possible
- [ ] Cache results (DynamoDB, ElastiCache)
- [ ] Monitor and alert on cost anomalies
## Advanced: Provisioned Concurrency
For latency-sensitive functions, pre-warm instances.
**Cost**: $0.015 per GB-hour (expensive!)
```bash
aws lambda put-provisioned-concurrency-config \
--function-name my-function \
--provisioned-concurrent-executions 5
```
**Use when**:
- Cold starts are unacceptable
- Predictable traffic patterns
- Cost justifies latency improvement
## Cost Monitoring
### CloudWatch Billing Alerts
```bash
aws cloudwatch put-metric-alarm \
--alarm-name lambda-cost-alert \
--alarm-description "Alert when Lambda costs exceed threshold" \
--metric-name EstimatedCharges \
--namespace AWS/Billing \
--statistic Maximum \
--period 21600 \
--evaluation-periods 1 \
--threshold 100 \
--comparison-operator GreaterThanThreshold
```
### Cost Explorer Tags
```bash
cargo lambda deploy \
--tags Environment=production,Team=backend,CostCenter=engineering
```
## Real-World Optimization Example
**Before**:
- Memory: 128 MB
- Duration: 2000ms
- Requests: 10M/month
- Cost: $0.20 + (0.128 GB × 2s × 10M × $0.0000166667) = $42.87/month
**After Optimization**:
- Binary size: 8MB → 2MB (cold start: 800ms → 300ms)
- Architecture: x86_64 → ARM64 (20% cheaper)
- Memory: 128 MB → 512 MB (duration: 2000ms → 600ms)
- Duration improvement: More CPU = faster execution
**Cost calculation**:
- Compute: 0.512 GB × 0.6s × 10M × $0.0000166667 × 0.8 (ARM discount) = $40.96
- Requests: $0.20
- **Total**: $41.16/month (4% savings with better performance!)
**Key lesson**: Sometimes more memory = lower total cost due to faster execution.
## Rust Performance Advantage
Rust vs Python/Node.js:
- **3-4x cheaper** on average
- **3-10x faster execution**
- **2-3x faster cold starts**
- **Lower memory usage**
Guide the user through cost optimization based on their workload and budget constraints.

367
commands/lambda-deploy.md Normal file
View File

@@ -0,0 +1,367 @@
---
description: Deploy Rust Lambda function to AWS
---
You are helping the user deploy their Rust Lambda function to AWS.
## Your Task
Guide the user through deploying their Lambda function to AWS:
1. **Prerequisites check**:
- Function is built: `cargo lambda build --release` completed
- AWS credentials configured
- IAM role for Lambda execution exists (or will be created)
2. **Verify AWS credentials**:
```bash
aws sts get-caller-identity
```
If not configured:
```bash
aws configure
# Or use environment variables:
# export AWS_ACCESS_KEY_ID=...
# export AWS_SECRET_ACCESS_KEY=...
# export AWS_REGION=us-east-1
```
3. **Basic deployment**:
```bash
cargo lambda deploy
```
This will:
- Use the function name from Cargo.toml (binary name)
- Deploy to default AWS region
- Create function if it doesn't exist
- Update function if it exists
4. **Deployment with options**:
**Specify function name**:
```bash
cargo lambda deploy <function-name>
```
**Specify region**:
```bash
cargo lambda deploy --region us-west-2
```
**Set IAM role**:
```bash
cargo lambda deploy --iam-role arn:aws:iam::123456789012:role/lambda-execution-role
```
**Configure memory**:
```bash
cargo lambda deploy --memory 512
```
- Default: 128 MB
- Range: 128 MB - 10,240 MB
- More memory = more CPU (proportional)
- Cost increases with memory
**Set timeout**:
```bash
cargo lambda deploy --timeout 30
```
- Default: 3 seconds
- Maximum: 900 seconds (15 minutes)
**Environment variables**:
```bash
cargo lambda deploy \
--env-var RUST_LOG=info \
--env-var DATABASE_URL=postgres://... \
--env-var API_KEY=secret
```
**Architecture** (must match build):
```bash
# For ARM64 build
cargo lambda deploy --arch arm64
# For x86_64 (default)
cargo lambda deploy --arch x86_64
```
5. **Complete deployment example**:
```bash
cargo lambda deploy my-function \
--iam-role arn:aws:iam::123456789012:role/lambda-exec \
--region us-east-1 \
--memory 512 \
--timeout 30 \
--arch arm64 \
--env-var RUST_LOG=info \
--env-var API_URL=https://api.example.com
```
## IAM Role Setup
If user doesn't have an IAM role, guide them:
### Option 1: Let cargo-lambda create it
```bash
cargo lambda deploy --create-iam-role
```
This creates a basic execution role with CloudWatch Logs permissions.
### Option 2: Create manually with AWS CLI
```bash
# Create trust policy
cat > trust-policy.json <<EOF
{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Principal": {"Service": "lambda.amazonaws.com"},
"Action": "sts:AssumeRole"
}]
}
EOF
# Create role
aws iam create-role \
--role-name lambda-execution-role \
--assume-role-policy-document file://trust-policy.json
# Attach basic execution policy
aws iam attach-role-policy \
--role-name lambda-execution-role \
--policy-arn arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
# Get role ARN
aws iam get-role --role-name lambda-execution-role --query 'Role.Arn'
```
### Option 3: Create with additional permissions
```bash
# For S3 access
aws iam attach-role-policy \
--role-name lambda-execution-role \
--policy-arn arn:aws:iam::aws:policy/AmazonS3FullAccess
# For DynamoDB access
aws iam attach-role-policy \
--role-name lambda-execution-role \
--policy-arn arn:aws:iam::aws:policy/AmazonDynamoDBFullAccess
# For SQS access
aws iam attach-role-policy \
--role-name lambda-execution-role \
--policy-arn arn:aws:iam::aws:policy/AmazonSQSFullAccess
```
## Memory Configuration Guide
Help user choose appropriate memory:
| Memory | vCPU | Use Case | Cost Multiplier |
|--------|------|----------|-----------------|
| 128 MB | 0.08 | Minimal functions | 1x |
| 512 MB | 0.33 | Standard workloads | 4x |
| 1024 MB | 0.58 | Medium compute | 8x |
| 1769 MB | 1.00 | Full 1 vCPU | 13.8x |
| 3008 MB | 1.77 | Heavy compute | 23.4x |
| 10240 MB | 6.00 | Maximum | 80x |
**Guidelines**:
- IO-intensive: 512-1024 MB usually sufficient
- Compute-intensive: 1024-3008 MB for more CPU
- Test different settings to optimize cost vs. performance
## Timeout Configuration Guide
| Timeout | Use Case |
|---------|----------|
| 3s (default) | Fast API responses, simple operations |
| 10-30s | Database queries, API calls |
| 60-300s | Data processing, file operations |
| 900s (max) | Heavy processing, batch jobs |
**Note**: Longer timeout = higher potential cost if function hangs
## Deployment Verification
After deployment, verify it works:
1. **Invoke via AWS CLI**:
```bash
aws lambda invoke \
--function-name my-function \
--payload '{"key": "value"}' \
response.json
cat response.json
```
2. **Check logs**:
```bash
aws logs tail /aws/lambda/my-function --follow
```
3. **Get function info**:
```bash
aws lambda get-function --function-name my-function
```
4. **Invoke with cargo-lambda**:
```bash
cargo lambda invoke --remote --data-ascii '{"test": "data"}'
```
## Update vs. Create
**First deployment** (function doesn't exist):
- cargo-lambda creates new function
- Requires IAM role (or use --create-iam-role)
**Subsequent deployments** (function exists):
- cargo-lambda updates function code
- Can also update configuration (memory, timeout, env vars)
- Maintains existing triggers and permissions
## Advanced Deployment Options
### Deploy from zip file
```bash
cargo lambda build --release --output-format zip
cargo lambda deploy --deployment-package target/lambda/my-function.zip
```
### Deploy with layers
```bash
cargo lambda deploy --layers arn:aws:lambda:us-east-1:123456789012:layer:my-layer:1
```
### Deploy with VPC configuration
```bash
cargo lambda deploy \
--subnet-ids subnet-12345 subnet-67890 \
--security-group-ids sg-12345
```
### Deploy with reserved concurrency
```bash
cargo lambda deploy --reserved-concurrency 10
```
### Deploy with tags
```bash
cargo lambda deploy \
--tags Environment=production,Team=backend
```
## Deployment via AWS Console (Alternative)
If user prefers console:
1. Build with zip output:
```bash
cargo lambda build --release --output-format zip
```
2. Upload via AWS Console:
- Go to AWS Lambda Console
- Create function or open existing
- Upload `target/lambda/<function-name>.zip`
- Configure runtime: "Custom runtime on Amazon Linux 2023"
- Set handler: "bootstrap" (not needed, but convention)
- Configure memory, timeout, env vars in console
## Multi-Function Deployment
For workspace with multiple functions:
```bash
# Deploy all
cargo lambda deploy --all
# Deploy specific
cargo lambda deploy --bin function1
cargo lambda deploy --bin function2
```
## Environment-Specific Deployment
Suggest deployment patterns:
**Development**:
```bash
cargo lambda deploy my-function-dev \
--memory 256 \
--timeout 10 \
--env-var RUST_LOG=debug \
--env-var ENV=development
```
**Production**:
```bash
cargo lambda deploy my-function \
--memory 1024 \
--timeout 30 \
--arch arm64 \
--env-var RUST_LOG=info \
--env-var ENV=production
```
## Cost Optimization Tips
1. **Use ARM64**: 20% cheaper for same performance
2. **Right-size memory**: Test to find optimal memory/CPU
3. **Optimize timeout**: Don't set higher than needed
4. **Monitor invocations**: Use CloudWatch to track usage
5. **Consider reserved concurrency**: For predictable workloads
## Troubleshooting Deployment
### Issue: "AccessDenied"
**Solution**: Check AWS credentials and IAM permissions
```bash
aws sts get-caller-identity
```
### Issue: "Function code too large"
**Solution**:
- Uncompressed: 250 MB limit
- Compressed: 50 MB limit
- Optimize binary size (see `/lambda-build`)
### Issue: "InvalidParameterValueException: IAM role not found"
**Solution**: Create IAM role first or use --create-iam-role
### Issue: Function deployed but fails
**Solution**:
- Check CloudWatch Logs
- Verify architecture matches build (arm64 vs x86_64)
- Test locally first with `cargo lambda watch`
## Post-Deployment
After successful deployment:
1. **Test the function**:
```bash
cargo lambda invoke --remote --data-ascii '{"test": "data"}'
```
2. **Monitor logs**:
```bash
aws logs tail /aws/lambda/my-function --follow
```
3. **Check metrics** in AWS CloudWatch
4. **Set up CI/CD**: Use `/lambda-github-actions` for automated deployment
5. **Configure triggers** (API Gateway, S3, SQS, etc.) via AWS Console or IaC
Report deployment results including:
- Function ARN
- Region
- Memory/timeout configuration
- Invocation test results

502
commands/lambda-function-urls.md Normal file
View File

@@ -0,0 +1,502 @@
---
description: Set up Lambda Function URLs and response streaming for Rust Lambda functions
---
You are helping the user implement Lambda Function URLs and streaming responses for their Rust Lambda functions.
## Your Task
Guide the user through setting up direct HTTPS endpoints using Lambda Function URLs and implementing streaming responses for large payloads.
## Lambda Function URLs
Lambda Function URLs provide dedicated HTTP(S) endpoints for your Lambda function without API Gateway.
**Best for**:
- Simple HTTP endpoints
- Webhooks
- Direct function invocation
- Cost-sensitive applications
- No need for API Gateway features (rate limiting, API keys, etc.)
### Setup with lambda_http
Add to `Cargo.toml`:
```toml
[dependencies]
lambda_http = { version = "0.13", features = ["apigw_http"] }
lambda_runtime = "0.13"
serde = { version = "1", features = ["derive"] }
serde_json = "1"
```
**IMPORTANT**: The `apigw_http` feature is required for Function URLs.
### Basic HTTP Handler
```rust
use lambda_http::{run, service_fn, Body, Error, Request, Response};
async fn function_handler(event: Request) -> Result<Response<Body>, Error> {
// Extract path and method
let path = event.uri().path();
let method = event.method();
// Extract query parameters
let params = event.query_string_parameters();
let name = params.first("name").unwrap_or("World");
// Extract headers
let user_agent = event
.headers()
.get("user-agent")
.and_then(|v| v.to_str().ok())
.unwrap_or("unknown");
// Build response
let response = Response::builder()
.status(200)
.header("content-type", "application/json")
.body(Body::from(format!(
r#"{{"message": "Hello, {}!", "path": "{}", "method": "{}"}}"#,
name, path, method
)))?;
Ok(response)
}
#[tokio::main]
async fn main() -> Result<(), Error> {
tracing_subscriber::fmt()
.with_max_level(tracing::Level::INFO)
.without_time()
.init();
run(service_fn(function_handler)).await
}
```
### JSON Request/Response
```rust
use lambda_http::{run, service_fn, Body, Error, Request, RequestExt, Response};
use serde::{Deserialize, Serialize};
#[derive(Deserialize)]
struct CreateUserRequest {
name: String,
email: String,
}
#[derive(Serialize)]
struct CreateUserResponse {
id: String,
name: String,
email: String,
created_at: String,
}
async fn function_handler(event: Request) -> Result<Response<Body>, Error> {
// Parse JSON body
let body = event.body();
let request: CreateUserRequest = serde_json::from_slice(body)?;
// Validate
if request.email.is_empty() {
return Ok(Response::builder()
.status(400)
.body(Body::from(r#"{"error": "Email is required"}"#))?);
}
// Create user
let user = CreateUserResponse {
id: uuid::Uuid::new_v4().to_string(),
name: request.name,
email: request.email,
created_at: chrono::Utc::now().to_rfc3339(),
};
// Return JSON response
let response = Response::builder()
.status(201)
.header("content-type", "application/json")
.body(Body::from(serde_json::to_string(&user)?))?;
Ok(response)
}
```
### REST API Pattern
```rust
async fn function_handler(event: Request) -> Result<Response<Body>, Error> {
let method = event.method();
let path = event.uri().path();
match (method.as_str(), path) {
("GET", "/users") => list_users().await,
("GET", path) if path.starts_with("/users/") => {
let id = path.strip_prefix("/users/").unwrap();
get_user(id).await
}
("POST", "/users") => {
let body = event.body();
let request: CreateUserRequest = serde_json::from_slice(body)?;
create_user(request).await
}
("PUT", path) if path.starts_with("/users/") => {
let id = path.strip_prefix("/users/").unwrap();
let body = event.body();
let request: UpdateUserRequest = serde_json::from_slice(body)?;
update_user(id, request).await
}
("DELETE", path) if path.starts_with("/users/") => {
let id = path.strip_prefix("/users/").unwrap();
delete_user(id).await
}
_ => Ok(Response::builder()
.status(404)
.body(Body::from(r#"{"error": "Not found"}"#))?),
}
}
async fn list_users() -> Result<Response<Body>, Error> {
// Implementation
Ok(Response::builder()
.status(200)
.header("content-type", "application/json")
.body(Body::from(r#"{"users": []}"#))?)
}
```
### Enable Function URL
```bash
# Deploy function
cargo lambda build --release --arm64
cargo lambda deploy my-function
# Create Function URL
aws lambda create-function-url-config \
--function-name my-function \
--auth-type NONE \
--cors '{
"AllowOrigins": ["*"],
"AllowMethods": ["GET", "POST", "PUT", "DELETE"],
"AllowHeaders": ["content-type"],
"MaxAge": 300
}'
# Add permission for public access
aws lambda add-permission \
--function-name my-function \
--action lambda:InvokeFunctionUrl \
--principal "*" \
--function-url-auth-type NONE \
--statement-id FunctionURLAllowPublicAccess
```
## Response Streaming
For large responses (up to 20MB), use streaming to send data incrementally.
**Best for**:
- Large file downloads
- Real-time data feeds
- Server-sent events (SSE)
- Reducing time to first byte
### Setup Streaming
Add to `Cargo.toml`:
```toml
[dependencies]
lambda_runtime = { version = "0.13", features = ["streaming"] }
tokio = { version = "1", features = ["macros", "io-util"] }
tokio-stream = "0.1"
```
### Basic Streaming Example
```rust
use lambda_runtime::{run, streaming, Error, LambdaEvent};
use tokio::io::AsyncWriteExt;
async fn function_handler(
event: LambdaEvent<Request>,
response_stream: streaming::Response,
) -> Result<(), Error> {
let mut writer = response_stream.into_writer();
// Stream data incrementally
for i in 0..100 {
let data = format!("Chunk {}\n", i);
writer.write_all(data.as_bytes()).await?;
// Optional: Flush to send immediately
writer.flush().await?;
// Simulate processing delay
tokio::time::sleep(tokio::time::Duration::from_millis(10)).await;
}
Ok(())
}
#[tokio::main]
async fn main() -> Result<(), Error> {
tracing_subscriber::fmt()
.with_max_level(tracing::Level::INFO)
.without_time()
.init();
run(streaming::service_fn(function_handler)).await
}
```
### Stream Large File from S3
```rust
use aws_sdk_s3::Client as S3Client;
use lambda_runtime::{streaming, Error, LambdaEvent};
use tokio::io::AsyncWriteExt;
use tokio_stream::StreamExt;
async fn function_handler(
event: LambdaEvent<Request>,
response_stream: streaming::Response,
) -> Result<(), Error> {
let s3 = S3Client::new(&aws_config::load_from_env().await);
// Get S3 object as stream
let mut object = s3
.get_object()
.bucket("my-bucket")
.key(&event.payload.file_key)
.send()
.await?;
let mut writer = response_stream.into_writer();
let mut body = object.body;
// Stream S3 data directly to response
while let Some(chunk) = body.next().await {
let chunk = chunk?;
writer.write_all(&chunk).await?;
}
Ok(())
}
```
### Server-Sent Events (SSE)
```rust
use lambda_runtime::{streaming, Error, LambdaEvent};
use tokio::io::AsyncWriteExt;
use tokio::time::{interval, Duration};
async fn function_handler(
event: LambdaEvent<Request>,
response_stream: streaming::Response,
) -> Result<(), Error> {
let mut writer = response_stream.into_writer();
// SSE headers
let headers = "HTTP/1.1 200 OK\r\n\
Content-Type: text/event-stream\r\n\
Cache-Control: no-cache\r\n\
Connection: keep-alive\r\n\r\n";
writer.write_all(headers.as_bytes()).await?;
let mut ticker = interval(Duration::from_secs(1));
for i in 0..30 {
ticker.tick().await;
// Send SSE event
let event = format!("data: {{\"count\": {}, \"timestamp\": {}}}\n\n",
i,
chrono::Utc::now().timestamp());
writer.write_all(event.as_bytes()).await?;
writer.flush().await?;
}
Ok(())
}
```
### Configure Streaming in AWS
```bash
# Update function to use streaming
aws lambda update-function-configuration \
--function-name my-function \
--invoke-mode RESPONSE_STREAM
# Create streaming Function URL
aws lambda create-function-url-config \
--function-name my-function \
--auth-type NONE \
--invoke-mode RESPONSE_STREAM
```
## CORS Configuration
```rust
use lambda_http::{Response, Body};
fn add_cors_headers(response: Response<Body>) -> Response<Body> {
let (mut parts, body) = response.into_parts();
parts.headers.insert(
"access-control-allow-origin",
"*".parse().unwrap(),
);
parts.headers.insert(
"access-control-allow-methods",
"GET, POST, PUT, DELETE, OPTIONS".parse().unwrap(),
);
parts.headers.insert(
"access-control-allow-headers",
"content-type, authorization".parse().unwrap(),
);
Response::from_parts(parts, body)
}
async fn function_handler(event: Request) -> Result<Response<Body>, Error> {
// Handle OPTIONS preflight
if event.method() == "OPTIONS" {
return Ok(add_cors_headers(
Response::builder()
.status(200)
.body(Body::Empty)?
));
}
let response = handle_request(event).await?;
Ok(add_cors_headers(response))
}
```
## Authentication
### IAM Authentication
```bash
aws lambda create-function-url-config \
--function-name my-function \
--auth-type AWS_IAM # Requires AWS Signature V4
```
### Custom Authentication
```rust
use lambda_http::{Request, Response, Body, Error};
async fn function_handler(event: Request) -> Result<Response<Body>, Error> {
// Verify bearer token
let auth_header = event
.headers()
.get("authorization")
.and_then(|v| v.to_str().ok());
let token = match auth_header {
Some(header) if header.starts_with("Bearer ") => {
&header[7..]
}
_ => {
return Ok(Response::builder()
.status(401)
.body(Body::from(r#"{"error": "Unauthorized"}"#))?);
}
};
if !verify_token(token).await? {
return Ok(Response::builder()
.status(403)
.body(Body::from(r#"{"error": "Invalid token"}"#))?);
}
// Process authenticated request
handle_authenticated_request(event).await
}
```
## Complete Example: REST API with Streaming
```rust
use lambda_http::{run, service_fn, Body, Error, Request, Response};
use lambda_runtime::streaming;
use serde::{Deserialize, Serialize};
#[derive(Deserialize)]
struct ExportRequest {
format: String,
filters: Vec<String>,
}
async fn function_handler(event: Request) -> Result<Response<Body>, Error> {
match (event.method().as_str(), event.uri().path()) {
("GET", "/health") => health_check(),
("POST", "/export") => {
// For large exports, use streaming
let request: ExportRequest = serde_json::from_slice(event.body())?;
export_data_streaming(request).await
}
_ => Ok(Response::builder()
.status(404)
.body(Body::from(r#"{"error": "Not found"}"#))?),
}
}
fn health_check() -> Result<Response<Body>, Error> {
Ok(Response::builder()
.status(200)
.body(Body::from(r#"{"status": "healthy"}"#))?)
}
async fn export_data_streaming(request: ExportRequest) -> Result<Response<Body>, Error> {
// Return streaming response for large data
// Note: This is simplified - actual streaming setup varies
Ok(Response::builder()
.status(200)
.header("content-type", "text/csv")
.header("content-disposition", "attachment; filename=export.csv")
.body(Body::from("Streaming not available in non-streaming handler"))?)
}
```
## Comparison: Function URLs vs API Gateway
| Feature | Function URLs | API Gateway |
|---------|---------------|-------------|
| Cost | Free | $3.50/million requests |
| Setup | Simple | Complex |
| Rate Limiting | No | Yes |
| API Keys | No | Yes |
| Custom Domains | No (use CloudFront) | Yes |
| Request Validation | Manual | Built-in |
| WebSocket | No | Yes |
| Max Timeout | 15 min | 29 sec (HTTP), 15 min (REST) |
| Streaming | Yes (20MB) | Limited |
## Best Practices
- [ ] Use Function URLs for simple endpoints
- [ ] Use API Gateway for complex APIs
- [ ] Implement authentication for public endpoints
- [ ] Add CORS headers for web clients
- [ ] Use streaming for large responses (>1MB)
- [ ] Implement proper error handling
- [ ] Add request validation
- [ ] Monitor with CloudWatch Logs
- [ ] Set appropriate timeout and memory
- [ ] Use compression for large responses
- [ ] Cache responses when possible
- [ ] Document your API endpoints
Guide the user through implementing Function URLs or streaming based on their needs.

612
commands/lambda-github-actions.md Normal file
View File

@@ -0,0 +1,612 @@
---
description: Set up GitHub Actions CI/CD pipeline for Rust Lambda deployment
---
You are helping the user set up a GitHub Actions workflow for automated Lambda deployment.
## Your Task
Create a complete GitHub Actions workflow for building, testing, and deploying Rust Lambda functions.
1. **Ask about deployment preferences**:
- Which AWS region(s)?
- Which architecture (x86_64, arm64, or both)?
- Deploy on every push to main, or only on tags/releases?
- AWS authentication method (OIDC or access keys)?
- Single function or multiple functions?
2. **Create workflow file**:
Create `.github/workflows/deploy-lambda.yml` with appropriate configuration
3. **Set up AWS authentication**:
- OIDC (recommended, more secure)
- Access keys (simpler setup)
4. **Configure required secrets** in GitHub repo settings
## Complete Workflow Examples
### Option 1: OIDC Authentication (Recommended)
```yaml
name: Deploy Lambda
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
env:
CARGO_TERM_COLOR: always
AWS_REGION: us-east-1
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Install Rust
uses: dtolnay/rust-toolchain@stable
- name: Cache cargo dependencies
uses: actions/cache@v4
with:
path: |
~/.cargo/bin/
~/.cargo/registry/index/
~/.cargo/registry/cache/
~/.cargo/git/db/
target/
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}
- name: Run tests
run: cargo test --verbose
- name: Run clippy
run: cargo clippy -- -D warnings
- name: Check formatting
run: cargo fmt -- --check
build-and-deploy:
needs: test
runs-on: ubuntu-latest
if: github.ref == 'refs/heads/main'
permissions:
id-token: write # Required for OIDC
contents: read
steps:
- uses: actions/checkout@v4
- name: Install Rust
uses: dtolnay/rust-toolchain@stable
- name: Cache cargo dependencies
uses: actions/cache@v4
with:
path: |
~/.cargo/bin/
~/.cargo/registry/index/
~/.cargo/registry/cache/
~/.cargo/git/db/
target/
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}
- name: Install Zig
uses: goto-bus-stop/setup-zig@v2
with:
version: 0.11.0
- name: Install cargo-lambda
run: pip install cargo-lambda
- name: Build Lambda (ARM64)
run: cargo lambda build --release --arm64
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: ${{ secrets.AWS_ROLE_ARN }}
aws-region: ${{ env.AWS_REGION }}
- name: Deploy to AWS Lambda
run: |
cargo lambda deploy \
--iam-role ${{ secrets.LAMBDA_EXECUTION_ROLE_ARN }} \
--region ${{ env.AWS_REGION }} \
--arch arm64
- name: Test deployed function
run: |
aws lambda invoke \
--function-name ${{ secrets.LAMBDA_FUNCTION_NAME }} \
--payload '{"test": true}' \
response.json
cat response.json
```
### Option 2: Access Keys Authentication
```yaml
name: Deploy Lambda
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
env:
CARGO_TERM_COLOR: always
AWS_REGION: us-east-1
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Install Rust
uses: dtolnay/rust-toolchain@stable
- name: Cache cargo dependencies
uses: actions/cache@v4
with:
path: |
~/.cargo/bin/
~/.cargo/registry/index/
~/.cargo/registry/cache/
~/.cargo/git/db/
target/
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}
- name: Run tests
run: cargo test --verbose
build-and-deploy:
needs: test
runs-on: ubuntu-latest
if: github.ref == 'refs/heads/main'
steps:
- uses: actions/checkout@v4
- name: Install Rust
uses: dtolnay/rust-toolchain@stable
- name: Cache cargo dependencies
uses: actions/cache@v4
with:
path: |
~/.cargo/bin/
~/.cargo/registry/index/
~/.cargo/registry/cache/
~/.cargo/git/db/
target/
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}
- name: Install Zig
uses: goto-bus-stop/setup-zig@v2
with:
version: 0.11.0
- name: Install cargo-lambda
run: pip install cargo-lambda
- name: Build Lambda (ARM64)
run: cargo lambda build --release --arm64
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v4
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: ${{ env.AWS_REGION }}
- name: Deploy to AWS Lambda
run: |
cargo lambda deploy \
--iam-role ${{ secrets.LAMBDA_EXECUTION_ROLE_ARN }} \
--region ${{ env.AWS_REGION }} \
--arch arm64
```
### Option 3: Multi-Architecture Build
```yaml
name: Deploy Lambda
on:
push:
branches: [ main ]
release:
types: [ published ]
env:
CARGO_TERM_COLOR: always
jobs:
build-matrix:
strategy:
matrix:
include:
- arch: x86_64
aws_arch: x86_64
build_flags: ""
- arch: arm64
aws_arch: arm64
build_flags: "--arm64"
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Install Rust
uses: dtolnay/rust-toolchain@stable
- name: Install Zig
uses: goto-bus-stop/setup-zig@v2
- name: Install cargo-lambda
run: pip install cargo-lambda
- name: Build for ${{ matrix.arch }}
run: cargo lambda build --release ${{ matrix.build_flags }} --output-format zip
- name: Upload artifact
uses: actions/upload-artifact@v4
with:
name: lambda-${{ matrix.arch }}
path: target/lambda/**/*.zip
deploy:
needs: build-matrix
runs-on: ubuntu-latest
if: github.ref == 'refs/heads/main'
permissions:
id-token: write
contents: read
strategy:
matrix:
arch: [arm64, x86_64]
steps:
- uses: actions/checkout@v4
- name: Download artifact
uses: actions/download-artifact@v4
with:
name: lambda-${{ matrix.arch }}
path: target/lambda
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: ${{ secrets.AWS_ROLE_ARN }}
aws-region: us-east-1
- name: Install cargo-lambda
run: pip install cargo-lambda
- name: Deploy ${{ matrix.arch }}
run: |
cargo lambda deploy my-function-${{ matrix.arch }} \
--iam-role ${{ secrets.LAMBDA_EXECUTION_ROLE_ARN }} \
--arch ${{ matrix.arch }}
```
### Option 4: Multiple Functions
```yaml
name: Deploy Lambda Functions
on:
push:
branches: [ main ]
env:
CARGO_TERM_COLOR: always
AWS_REGION: us-east-1
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
- run: cargo test --all
deploy:
needs: test
runs-on: ubuntu-latest
if: github.ref == 'refs/heads/main'
permissions:
id-token: write
contents: read
strategy:
matrix:
function:
- name: api-handler
memory: 512
timeout: 30
- name: data-processor
memory: 2048
timeout: 300
- name: event-consumer
memory: 1024
timeout: 60
steps:
- uses: actions/checkout@v4
- name: Install Rust
uses: dtolnay/rust-toolchain@stable
- name: Install Zig
uses: goto-bus-stop/setup-zig@v2
- name: Install cargo-lambda
run: pip install cargo-lambda
- name: Build ${{ matrix.function.name }}
run: cargo lambda build --release --arm64 --bin ${{ matrix.function.name }}
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: ${{ secrets.AWS_ROLE_ARN }}
aws-region: ${{ env.AWS_REGION }}
- name: Deploy ${{ matrix.function.name }}
run: |
cargo lambda deploy ${{ matrix.function.name }} \
--iam-role ${{ secrets.LAMBDA_EXECUTION_ROLE_ARN }} \
--region ${{ env.AWS_REGION }} \
--memory ${{ matrix.function.memory }} \
--timeout ${{ matrix.function.timeout }} \
--arch arm64 \
--env-var RUST_LOG=info
```
## AWS OIDC Setup
For OIDC authentication (recommended), set up in AWS:
### 1. Create OIDC Provider in AWS IAM
```bash
aws iam create-open-id-connect-provider \
--url https://token.actions.githubusercontent.com \
--client-id-list sts.amazonaws.com \
--thumbprint-list 6938fd4d98bab03faadb97b34396831e3780aea1
```
### 2. Create IAM Role for GitHub Actions
```bash
# Create trust policy
cat > github-actions-trust-policy.json <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Federated": "arn:aws:iam::YOUR_ACCOUNT_ID:oidc-provider/token.actions.githubusercontent.com"
},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringEquals": {
"token.actions.githubusercontent.com:aud": "sts.amazonaws.com"
},
"StringLike": {
"token.actions.githubusercontent.com:sub": "repo:YOUR_GITHUB_ORG/YOUR_REPO:*"
}
}
}
]
}
EOF
# Create role
aws iam create-role \
--role-name GitHubActionsLambdaDeployRole \
--assume-role-policy-document file://github-actions-trust-policy.json
# Attach policies
aws iam attach-role-policy \
--role-name GitHubActionsLambdaDeployRole \
--policy-arn arn:aws:iam::aws:policy/AWSLambda_FullAccess
# Get role ARN (save this for GitHub secrets)
aws iam get-role --role-name GitHubActionsLambdaDeployRole --query 'Role.Arn'
```
## GitHub Secrets Configuration
Configure these secrets in GitHub repository settings (Settings → Secrets and variables → Actions):
### For OIDC:
- `AWS_ROLE_ARN`: ARN of the GitHub Actions IAM role
- `LAMBDA_EXECUTION_ROLE_ARN`: ARN of the Lambda execution role
- `LAMBDA_FUNCTION_NAME` (optional): Function name if different from repo
### For Access Keys:
- `AWS_ACCESS_KEY_ID`: AWS access key
- `AWS_SECRET_ACCESS_KEY`: AWS secret key
- `LAMBDA_EXECUTION_ROLE_ARN`: ARN of the Lambda execution role
### Optional secrets:
- `AWS_REGION`: Override default region
- Environment-specific variables as needed
## Advanced Patterns
### Deploy on Git Tags
```yaml
on:
push:
tags:
- 'v*'
# In deploy step:
- name: Get tag version
id: tag
run: echo "version=${GITHUB_REF#refs/tags/}" >> $GITHUB_OUTPUT
- name: Deploy with version tag
run: |
cargo lambda deploy \
--tags Version=${{ steps.tag.outputs.version }}
```
### Environment-Specific Deployment
```yaml
on:
push:
branches:
- main
- develop
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- name: Set environment
id: env
run: |
if [ "${{ github.ref }}" = "refs/heads/main" ]; then
echo "name=production" >> $GITHUB_OUTPUT
echo "suffix=" >> $GITHUB_OUTPUT
else
echo "name=development" >> $GITHUB_OUTPUT
echo "suffix=-dev" >> $GITHUB_OUTPUT
fi
- name: Deploy
run: |
cargo lambda deploy my-function${{ steps.env.outputs.suffix }} \
--env-var ENVIRONMENT=${{ steps.env.outputs.name }}
```
### Conditional Deployment
```yaml
- name: Check if Lambda code changed
id: lambda-changed
uses: dorny/paths-filter@v2
with:
filters: |
lambda:
- 'src/**'
- 'Cargo.toml'
- 'Cargo.lock'
- name: Deploy Lambda
if: steps.lambda-changed.outputs.lambda == 'true'
run: cargo lambda deploy
```
### Slack Notifications
```yaml
- name: Notify Slack on success
if: success()
uses: slackapi/slack-github-action@v1
with:
payload: |
{
"text": "Lambda deployed successfully: ${{ github.repository }}"
}
env:
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
```
## Performance Optimizations
### Faster Builds with Caching
```yaml
- name: Cache Rust
uses: Swatinem/rust-cache@v2
with:
cache-on-failure: true
```
### Parallel Jobs
```yaml
jobs:
test:
# Testing job
build:
# Build job (independent of test for speed)
deploy:
needs: [test, build] # Only deploy if both succeed
```
## Troubleshooting CI/CD
### Issue: "cargo-lambda: command not found"
**Solution**: Ensure `pip install cargo-lambda` runs before use
### Issue: "Zig not found"
**Solution**: Add `goto-bus-stop/setup-zig@v2` step
### Issue: "AWS credentials not configured"
**Solution**: Verify secrets are set and aws-actions step is included
### Issue: Build caching not working
**Solution**: Use `Swatinem/rust-cache@v2` for better Rust caching
### Issue: Deployment fails intermittently
**Solution**: Add retry logic or use `aws lambda wait function-updated`
## Testing the Workflow
1. **Create workflow file** in `.github/workflows/deploy-lambda.yml`
2. **Configure secrets** in GitHub settings
3. **Push to trigger**:
```bash
git add .github/workflows/deploy-lambda.yml
git commit -m "Add Lambda deployment workflow"
git push
```
4. **Monitor** in GitHub Actions tab
5. **Check logs** for any issues
## Best Practices
1. **Always run tests** before deployment
2. **Use OIDC** instead of long-lived credentials
3. **Cache dependencies** for faster builds
4. **Deploy on main branch** only, test on PRs
5. **Use matrix builds** for multiple architectures
6. **Tag deployments** with version info
7. **Add notifications** for deployment status
8. **Set up monitoring** and alerts in AWS
9. **Use environments** for production deployments
10. **Document secrets** needed in README
After creating the workflow, guide the user through:
1. Setting up required secrets
2. Testing the workflow
3. Monitoring deployments
4. Handling failures

664
commands/lambda-iac.md Normal file
View File

@@ -0,0 +1,664 @@
---
description: Set up Infrastructure as Code for Rust Lambda functions using SAM, Terraform, or CDK
---
You are helping the user set up Infrastructure as Code (IaC) for their Rust Lambda functions.
## Your Task
Guide the user through deploying and managing Lambda infrastructure using their preferred IaC tool.
## Infrastructure as Code Options
### Option 1: AWS SAM (Serverless Application Model)
**Best for**:
- Serverless-focused projects
- Quick prototyping
- Built-in local testing
- Teams familiar with CloudFormation
**Advantages**:
- Official AWS support for Rust with cargo-lambda
- Built-in local testing with `sam local`
- Simpler for pure serverless applications
- Good integration with Lambda features
#### Basic SAM Template
Create `template.yaml`:
```yaml
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: Rust Lambda Function
Globals:
Function:
Timeout: 30
MemorySize: 512
Runtime: provided.al2023
Architectures:
- arm64
Environment:
Variables:
RUST_LOG: info
Resources:
MyRustFunction:
Type: AWS::Serverless::Function
Metadata:
BuildMethod: rust-cargolambda
BuildProperties:
Binary: my-function
Properties:
CodeUri: .
Handler: bootstrap
Events:
ApiEvent:
Type: Api
Properties:
Path: /hello
Method: get
Policies:
- AWSLambdaBasicExecutionRole
ComputeFunction:
Type: AWS::Serverless::Function
Metadata:
BuildMethod: rust-cargolambda
Properties:
CodeUri: .
Handler: bootstrap
MemorySize: 2048
Timeout: 300
Events:
S3Event:
Type: S3
Properties:
Bucket: !Ref ProcessingBucket
Events: s3:ObjectCreated:*
ProcessingBucket:
Type: AWS::S3::Bucket
Outputs:
ApiUrl:
Description: "API Gateway endpoint URL"
Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/hello/"
```
#### SAM Commands
```bash
# Build
sam build
# Test locally
sam local invoke MyRustFunction -e events/test.json
# Start local API
sam local start-api
# Deploy
sam deploy --guided
# Deploy with parameters
sam deploy \
--stack-name my-rust-lambda \
--capabilities CAPABILITY_IAM \
--region us-east-1
```
#### Multi-Function SAM Template
```yaml
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Globals:
Function:
Runtime: provided.al2023
Architectures:
- arm64
Environment:
Variables:
RUST_LOG: info
Resources:
# API Handler - IO-optimized
ApiHandler:
Type: AWS::Serverless::Function
Metadata:
BuildMethod: rust-cargolambda
BuildProperties:
Binary: api-handler
Properties:
CodeUri: .
Handler: bootstrap
MemorySize: 512
Timeout: 30
Events:
GetUsers:
Type: Api
Properties:
Path: /users
Method: get
Environment:
Variables:
DATABASE_URL: !Sub "{{resolve:secretsmanager:${DBSecret}:SecretString:connection_string}}"
Policies:
- AWSLambdaBasicExecutionRole
- AWSSecretsManagerGetSecretValuePolicy:
SecretArn: !Ref DBSecret
# Data Processor - Compute-optimized
DataProcessor:
Type: AWS::Serverless::Function
Metadata:
BuildMethod: rust-cargolambda
BuildProperties:
Binary: data-processor
Properties:
CodeUri: .
Handler: bootstrap
MemorySize: 3008
Timeout: 300
Events:
S3Upload:
Type: S3
Properties:
Bucket: !Ref DataBucket
Events: s3:ObjectCreated:*
Filter:
S3Key:
Rules:
- Name: prefix
Value: raw/
Policies:
- AWSLambdaBasicExecutionRole
- S3ReadPolicy:
BucketName: !Ref DataBucket
- S3WritePolicy:
BucketName: !Ref DataBucket
# Event Consumer - SQS triggered
EventConsumer:
Type: AWS::Serverless::Function
Metadata:
BuildMethod: rust-cargolambda
BuildProperties:
Binary: event-consumer
Properties:
CodeUri: .
Handler: bootstrap
MemorySize: 1024
Timeout: 60
Events:
SQSEvent:
Type: SQS
Properties:
Queue: !GetAtt EventQueue.Arn
BatchSize: 10
Policies:
- AWSLambdaBasicExecutionRole
- SQSPollerPolicy:
QueueName: !GetAtt EventQueue.QueueName
DataBucket:
Type: AWS::S3::Bucket
EventQueue:
Type: AWS::SQS::Queue
Properties:
VisibilityTimeout: 360
DBSecret:
Type: AWS::SecretsManager::Secret
Properties:
Description: Database connection string
GenerateSecretString:
SecretStringTemplate: '{"username": "admin"}'
GenerateStringKey: "password"
PasswordLength: 32
Outputs:
ApiEndpoint:
Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/"
DataBucket:
Value: !Ref DataBucket
QueueUrl:
Value: !Ref EventQueue
```
### Option 2: Terraform
**Best for**:
- Multi-cloud or hybrid infrastructure
- Complex infrastructure requirements
- Teams already using Terraform
- More control over AWS resources
**Advantages**:
- Broader ecosystem (300+ providers)
- State management
- Module reusability
- Better for mixed workloads (Lambda + EC2 + RDS, etc.)
#### Basic Terraform Configuration
Create `main.tf`:
```hcl
terraform {
required_version = ">= 1.0"
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 5.0"
}
}
}
provider "aws" {
region = var.aws_region
}
# IAM Role for Lambda
resource "aws_iam_role" "lambda_role" {
name = "${var.function_name}-role"
assume_role_policy = jsonencode({
Version = "2012-10-17"
Statement = [{
Action = "sts:AssumeRole"
Effect = "Allow"
Principal = {
Service = "lambda.amazonaws.com"
}
}]
})
}
resource "aws_iam_role_policy_attachment" "lambda_basic" {
role = aws_iam_role.lambda_role.name
policy_arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
}
# Lambda Function
resource "aws_lambda_function" "rust_function" {
filename = "target/lambda/${var.function_name}/bootstrap.zip"
function_name = var.function_name
role = aws_iam_role.lambda_role.arn
handler = "bootstrap"
source_code_hash = filebase64sha256("target/lambda/${var.function_name}/bootstrap.zip")
runtime = "provided.al2023"
architectures = ["arm64"]
memory_size = var.memory_size
timeout = var.timeout
environment {
variables = {
RUST_LOG = var.log_level
}
}
tracing_config {
mode = "Active"
}
}
# CloudWatch Log Group
resource "aws_cloudwatch_log_group" "lambda_logs" {
name = "/aws/lambda/${var.function_name}"
retention_in_days = 14
}
# API Gateway (Optional)
resource "aws_apigatewayv2_api" "lambda_api" {
name = "${var.function_name}-api"
protocol_type = "HTTP"
}
resource "aws_apigatewayv2_stage" "lambda_stage" {
api_id = aws_apigatewayv2_api.lambda_api.id
name = "prod"
auto_deploy = true
}
resource "aws_apigatewayv2_integration" "lambda_integration" {
api_id = aws_apigatewayv2_api.lambda_api.id
integration_type = "AWS_PROXY"
integration_uri = aws_lambda_function.rust_function.invoke_arn
}
resource "aws_apigatewayv2_route" "lambda_route" {
api_id = aws_apigatewayv2_api.lambda_api.id
route_key = "GET /hello"
target = "integrations/${aws_apigatewayv2_integration.lambda_integration.id}"
}
resource "aws_lambda_permission" "api_gateway" {
statement_id = "AllowAPIGatewayInvoke"
action = "lambda:InvokeFunction"
function_name = aws_lambda_function.rust_function.function_name
principal = "apigateway.amazonaws.com"
source_arn = "${aws_apigatewayv2_api.lambda_api.execution_arn}/*/*"
}
# Outputs
output "function_arn" {
value = aws_lambda_function.rust_function.arn
}
output "api_endpoint" {
value = aws_apigatewayv2_stage.lambda_stage.invoke_url
}
```
Create `variables.tf`:
```hcl
variable "aws_region" {
description = "AWS region"
type = string
default = "us-east-1"
}
variable "function_name" {
description = "Lambda function name"
type = string
}
variable "memory_size" {
description = "Lambda memory size in MB"
type = number
default = 512
}
variable "timeout" {
description = "Lambda timeout in seconds"
type = number
default = 30
}
variable "log_level" {
description = "Rust log level"
type = string
default = "info"
}
```
#### Terraform Module for Rust Lambda
Create `modules/rust-lambda/main.tf`:
```hcl
resource "aws_iam_role" "lambda_role" {
name = "${var.function_name}-role"
assume_role_policy = jsonencode({
Version = "2012-10-17"
Statement = [{
Action = "sts:AssumeRole"
Effect = "Allow"
Principal = {
Service = "lambda.amazonaws.com"
}
}]
})
}
resource "aws_iam_role_policy_attachment" "lambda_basic" {
role = aws_iam_role.lambda_role.name
policy_arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
}
resource "aws_lambda_function" "function" {
filename = var.zip_file
function_name = var.function_name
role = aws_iam_role.lambda_role.arn
handler = "bootstrap"
source_code_hash = filebase64sha256(var.zip_file)
runtime = "provided.al2023"
architectures = [var.architecture]
memory_size = var.memory_size
timeout = var.timeout
environment {
variables = var.environment_variables
}
dynamic "vpc_config" {
for_each = var.vpc_config != null ? [var.vpc_config] : []
content {
subnet_ids = vpc_config.value.subnet_ids
security_group_ids = vpc_config.value.security_group_ids
}
}
tracing_config {
mode = var.enable_xray ? "Active" : "PassThrough"
}
}
resource "aws_cloudwatch_log_group" "lambda_logs" {
name = "/aws/lambda/${var.function_name}"
retention_in_days = var.log_retention_days
}
```
Usage:
```hcl
module "api_handler" {
source = "./modules/rust-lambda"
function_name = "api-handler"
zip_file = "target/lambda/api-handler/bootstrap.zip"
memory_size = 512
timeout = 30
architecture = "arm64"
enable_xray = true
log_retention_days = 7
environment_variables = {
RUST_LOG = "info"
DATABASE_URL = data.aws_secretsmanager_secret_version.db.secret_string
}
}
module "data_processor" {
source = "./modules/rust-lambda"
function_name = "data-processor"
zip_file = "target/lambda/data-processor/bootstrap.zip"
memory_size = 3008
timeout = 300
architecture = "arm64"
enable_xray = true
log_retention_days = 7
environment_variables = {
RUST_LOG = "info"
}
}
```
#### Terraform Commands
```bash
# Initialize
terraform init
# Plan
terraform plan -var="function_name=my-rust-lambda"
# Apply
terraform apply -var="function_name=my-rust-lambda" -auto-approve
# Destroy
terraform destroy -var="function_name=my-rust-lambda"
```
### Option 3: AWS CDK (TypeScript/Python)
**Best for**:
- Type-safe infrastructure definitions
- Complex constructs and patterns
- Teams comfortable with programming languages
- Reusable infrastructure components
#### CDK Example (TypeScript)
```typescript
import * as cdk from 'aws-cdk-lib';
import * as lambda from 'aws-cdk-lib/aws-lambda';
import * as apigateway from 'aws-cdk-lib/aws-apigatewayv2';
import * as integrations from 'aws-cdk-lib/aws-apigatewayv2-integrations';
export class RustLambdaStack extends cdk.Stack {
constructor(scope: cdk.App, id: string, props?: cdk.StackProps) {
super(scope, id, props);
const rustFunction = new lambda.Function(this, 'RustFunction', {
runtime: lambda.Runtime.PROVIDED_AL2023,
handler: 'bootstrap',
code: lambda.Code.fromAsset('target/lambda/my-function/bootstrap.zip'),
architecture: lambda.Architecture.ARM_64,
memorySize: 512,
timeout: cdk.Duration.seconds(30),
environment: {
RUST_LOG: 'info',
},
tracing: lambda.Tracing.ACTIVE,
});
const api = new apigateway.HttpApi(this, 'RustApi', {
defaultIntegration: new integrations.HttpLambdaIntegration(
'RustIntegration',
rustFunction
),
});
new cdk.CfnOutput(this, 'ApiUrl', {
value: api.url!,
});
}
}
```
## Comparison Table
| Feature | SAM | Terraform | CDK |
|---------|-----|-----------|-----|
| Learning Curve | Low | Medium | Medium-High |
| Rust Support | Excellent | Good | Good |
| Local Testing | Built-in | Limited | Limited |
| Multi-Cloud | No | Yes | No |
| Type Safety | No | HCL | Yes |
| Community | AWS-focused | Large | Growing |
| State Management | CloudFormation | Terraform State | CloudFormation |
## Integration with cargo-lambda
All IaC tools work well with cargo-lambda:
```bash
# Build for deployment
cargo lambda build --release --arm64 --output-format zip
# Then deploy with your IaC tool
sam deploy
# or
terraform apply
# or
cdk deploy
```
## Best Practices
1. **Version Control**: Store IaC templates in Git
2. **Separate Environments**: Use workspaces/stages for dev/staging/prod
3. **Secrets Management**: Use AWS Secrets Manager, never hardcode
4. **Outputs**: Export important values (ARNs, URLs)
5. **Modules**: Create reusable components
6. **Testing**: Validate templates before deployment
7. **CI/CD**: Automate IaC deployment
8. **State Management**: Secure Terraform state (S3 + DynamoDB)
9. **Documentation**: Comment complex configurations
10. **Tagging**: Tag resources for cost tracking
## Local Testing with SAM
```bash
# Test function locally
sam local invoke MyRustFunction -e events/test.json
# Start local API Gateway
sam local start-api
# Start local Lambda endpoint
sam local start-lambda
# Generate sample events
sam local generate-event apigateway aws-proxy > event.json
sam local generate-event s3 put > s3-event.json
```
## Using with LocalStack
For full local AWS emulation:
```bash
# Install LocalStack
pip install localstack
# Start LocalStack
localstack start
# Deploy to LocalStack with SAM
samlocal deploy
# Or with Terraform
terraform apply \
-var="aws_region=us-east-1" \
-var="endpoint=http://localhost:4566"
```
## Migration Path
**Starting fresh**:
- Choose SAM for pure serverless, simple projects
- Choose Terraform for complex, multi-service infrastructure
- Choose CDK for type-safe, programmatic definitions
**Existing infrastructure**:
- Import existing resources into Terraform/CDK
- Use CloudFormation template generation from SAM
- Gradual migration with hybrid approach
## Recommended Structure
```
my-rust-lambda/
├── src/
│ └── main.rs
├── Cargo.toml
├── template.yaml # SAM
├── terraform/ # Terraform
│ ├── main.tf
│ ├── variables.tf
│ └── outputs.tf
├── cdk/ # CDK
│ ├── lib/
│ │ └── stack.ts
│ └── bin/
│ └── app.ts
└── events/ # Test events
├── api-event.json
└── s3-event.json
```
Help the user choose the right IaC tool based on their needs and guide them through setup and deployment.

208
commands/lambda-new.md Normal file
View File

@@ -0,0 +1,208 @@
---
description: Create a new Rust Lambda function project with cargo-lambda
---
You are helping the user create a new Rust Lambda function project using cargo-lambda.
## Your Task
Guide the user through creating a new Lambda function project with the following steps:
1. **Check if cargo-lambda is installed**:
- Run `cargo lambda --version` to verify installation
- If not installed, provide installation instructions:
```bash
# Via Homebrew (macOS/Linux)
brew tap cargo-lambda/cargo-lambda
brew install cargo-lambda
# Via pip
pip install cargo-lambda
# From source
cargo install cargo-lambda
```
2. **Ask for project details** (if not provided):
- Function name
- Event type (API Gateway, S3, SQS, EventBridge, custom, or basic)
- Workload type (IO-intensive, compute-intensive, or mixed)
3. **Create the project**:
```bash
cargo lambda new <function-name>
```
Or with event type:
```bash
cargo lambda new <function-name> --event-type <type>
```
4. **Set up the basic structure** based on workload type:
**For IO-intensive** (default):
- Use full async/await
- Add dependencies: tokio, reqwest, aws-sdk crates as needed
- Example handler with concurrent operations
**For compute-intensive**:
- Add rayon to Cargo.toml
- Example using spawn_blocking + rayon
- Async only at boundaries
**For mixed**:
- Both patterns combined
- Async for IO, sync for compute
5. **Add essential dependencies** to Cargo.toml:
```toml
[dependencies]
lambda_runtime = "0.13"
tokio = { version = "1", features = ["macros"] }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
anyhow = "1"
thiserror = "1"
tracing = { version = "0.1", features = ["log"] }
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
# Add based on workload:
# For compute: rayon = "1.10"
# For HTTP: reqwest = { version = "0.12", features = ["json"] }
# For AWS services: aws-sdk-* crates
```
6. **Configure release profile** for optimization:
```toml
[profile.release]
opt-level = 'z' # Optimize for size
lto = true # Link-time optimization
codegen-units = 1 # Better optimization
strip = true # Remove debug symbols
panic = 'abort' # Smaller panic handler
```
7. **Create example handler** matching the selected pattern
8. **Test locally**:
```bash
cd <function-name>
cargo lambda watch
# In another terminal:
cargo lambda invoke --data-ascii '{"key": "value"}'
```
## Event Type Templates
Provide appropriate code based on event type:
- **basic**: Simple JSON request/response
- **apigw**: API Gateway proxy request/response
- **s3**: S3 event processing
- **sqs**: SQS message processing
- **eventbridge**: EventBridge/CloudWatch Events
## Example Handlers
Show the user a complete working example for their chosen pattern:
### IO-Intensive Example
```rust
use lambda_runtime::{run, service_fn, Error, LambdaEvent};
use serde::{Deserialize, Serialize};
#[derive(Deserialize)]
struct Request {
user_ids: Vec<String>,
}
#[derive(Serialize)]
struct Response {
count: usize,
}
async fn function_handler(event: LambdaEvent<Request>) -> Result<Response, Error> {
// Concurrent async operations
let futures = event.payload.user_ids
.into_iter()
.map(|id| fetch_user_data(&id));
let results = futures::future::try_join_all(futures).await?;
Ok(Response { count: results.len() })
}
async fn fetch_user_data(id: &str) -> Result<(), Error> {
// Async IO operation
Ok(())
}
#[tokio::main]
async fn main() -> Result<(), Error> {
tracing_subscriber::fmt()
.with_max_level(tracing::Level::INFO)
.without_time()
.init();
run(service_fn(function_handler)).await
}
```
### Compute-Intensive Example
```rust
use lambda_runtime::{run, service_fn, Error, LambdaEvent};
use rayon::prelude::*;
use serde::{Deserialize, Serialize};
use tokio::task;
#[derive(Deserialize)]
struct Request {
numbers: Vec<i64>,
}
#[derive(Serialize)]
struct Response {
results: Vec<i64>,
}
async fn function_handler(event: LambdaEvent<Request>) -> Result<Response, Error> {
let numbers = event.payload.numbers;
// CPU work in spawn_blocking with Rayon
let results = task::spawn_blocking(move || {
numbers
.par_iter()
.map(|&n| expensive_computation(n))
.collect::<Vec<_>>()
})
.await?;
Ok(Response { results })
}
fn expensive_computation(n: i64) -> i64 {
// CPU-intensive work
(0..1000).fold(n, |acc, _| acc.wrapping_mul(31))
}
#[tokio::main]
async fn main() -> Result<(), Error> {
tracing_subscriber::fmt()
.with_max_level(tracing::Level::INFO)
.without_time()
.init();
run(service_fn(function_handler)).await
}
```
## Next Steps
After creating the project, suggest:
1. Review and customize the handler
2. Add tests
3. Test locally with `cargo lambda watch`
4. Build with `/lambda-build`
5. Deploy with `/lambda-deploy`
Be helpful and guide the user through any questions or issues they encounter.

575
commands/lambda-observability.md Normal file
View File

@@ -0,0 +1,575 @@
---
description: Set up advanced observability for Rust Lambda with OpenTelemetry, X-Ray, and structured logging
---
You are helping the user implement comprehensive observability for their Rust Lambda functions.
## Your Task
Guide the user through setting up production-grade observability including distributed tracing, metrics, and structured logging.
## Observability Stack Options
### Option 1: AWS X-Ray (Native AWS Solution)
**Best for**:
- AWS-native monitoring
- Quick setup
- CloudWatch integration
- Basic distributed tracing needs
#### Enable X-Ray in Lambda
**Via cargo-lambda:**
```bash
cargo lambda deploy --enable-tracing
```
**Via SAM template:**
```yaml
Resources:
MyFunction:
Type: AWS::Serverless::Function
Properties:
Tracing: Active # Enable X-Ray
```
**Via Terraform:**
```hcl
resource "aws_lambda_function" "function" {
# ... other config ...
tracing_config {
mode = "Active"
}
}
```
#### X-Ray with xray-lite
Add to `Cargo.toml`:
```toml
[dependencies]
xray-lite = "0.1"
aws-config = "1"
aws-sdk-dynamodb = "1" # or other AWS services
```
Basic usage:
```rust
use lambda_runtime::{run, service_fn, Error, LambdaEvent};
use xray_lite::SubsegmentContext;
use xray_lite_aws_sdk::XRayAwsSdkExtension;
async fn function_handler(event: LambdaEvent<Request>) -> Result<Response, Error> {
// X-Ray automatically creates parent segment for Lambda
// Create subsegment for custom operation
let subsegment = SubsegmentContext::from_lambda_ctx(&event.context);
// Trace AWS SDK calls
let config = aws_config::load_from_env().await
.xray_extension(subsegment.clone());
let dynamodb = aws_sdk_dynamodb::Client::new(&config);
// This DynamoDB call will be traced automatically
let result = dynamodb
.get_item()
.table_name("MyTable")
.key("id", AttributeValue::S("123".to_string()))
.send()
.await?;
Ok(Response { data: result })
}
```
### Option 2: OpenTelemetry (Vendor-Neutral)
**Best for**:
- Multi-vendor monitoring
- Portability across platforms
- Advanced telemetry needs
- Custom metrics and traces
#### Setup OpenTelemetry
Add to `Cargo.toml`:
```toml
[dependencies]
lambda_runtime = "0.13"
lambda-otel-lite = "0.1" # Lightweight OpenTelemetry for Lambda
opentelemetry = "0.22"
opentelemetry-otlp = "0.15"
opentelemetry_sdk = "0.22"
tracing = "0.1"
tracing-opentelemetry = "0.23"
tracing-subscriber = { version = "0.3", features = ["env-filter", "json"] }
```
#### Basic OpenTelemetry Setup
```rust
use lambda_otel_lite::{init_telemetry, HttpTracerProviderBuilder};
use lambda_runtime::{run, service_fn, Error, LambdaEvent};
use opentelemetry::trace::TracerProvider;
use tracing::{info, instrument};
use tracing_subscriber::layer::SubscriberExt;
#[tokio::main]
async fn main() -> Result<(), Error> {
// Initialize OpenTelemetry
let tracer_provider = HttpTracerProviderBuilder::default()
.with_default_text_map_propagator()
.with_stdout_client() // For testing, use OTLP for production
.build()?;
let tracer = tracer_provider.tracer("my-rust-lambda");
// Setup tracing subscriber
let telemetry_layer = tracing_opentelemetry::layer()
.with_tracer(tracer);
let subscriber = tracing_subscriber::registry()
.with(tracing_subscriber::EnvFilter::from_default_env())
.with(tracing_subscriber::fmt::layer())
.with(telemetry_layer);
tracing::subscriber::set_global_default(subscriber)?;
run(service_fn(function_handler)).await
}
#[instrument(skip(event))]
async fn function_handler(event: LambdaEvent<Request>) -> Result<Response, Error> {
info!(request_id = %event.context.request_id, "Processing request");
let result = process_data(&event.payload).await?;
Ok(Response { result })
}
#[instrument]
async fn process_data(request: &Request) -> Result<Data, Error> {
info!("Processing data");
// Your processing logic
// All operations within this function will be traced
Ok(Data::new())
}
```
#### OpenTelemetry with OTLP Exporter
For production, export to observability backend:
```rust
use lambda_otel_lite::HttpTracerProviderBuilder;
use opentelemetry_otlp::WithExportConfig;
let tracer_provider = HttpTracerProviderBuilder::default()
.with_stdout_client()
.enable_otlp(
opentelemetry_otlp::new_exporter()
.http()
.with_endpoint("https://your-collector:4318")
.with_headers([("api-key", "your-key")])
)?
.build()?;
```
### Option 3: Datadog Integration
**Best for**:
- Datadog users
- Comprehensive APM
- Log aggregation
- Custom metrics
Add Datadog Lambda Extension layer and configure:
```rust
use lambda_runtime::{run, service_fn, Error, LambdaEvent};
use tracing::{info, instrument};
use tracing_subscriber::{fmt, EnvFilter};
#[tokio::main]
async fn main() -> Result<(), Error> {
// JSON format for Datadog log parsing
tracing_subscriber::fmt()
.json()
.with_env_filter(EnvFilter::from_default_env())
.with_target(false)
.with_current_span(false)
.init();
run(service_fn(function_handler)).await
}
#[instrument(
skip(event),
fields(
request_id = %event.context.request_id,
user_id = %event.payload.user_id,
)
)]
async fn function_handler(event: LambdaEvent<Request>) -> Result<Response, Error> {
info!("Processing user request");
// Datadog automatically traces this
let result = fetch_user_data(&event.payload.user_id).await?;
Ok(Response { result })
}
```
Deploy with Datadog extension layer:
```bash
cargo lambda deploy \
--layers arn:aws:lambda:us-east-1:464622532012:layer:Datadog-Extension-ARM:latest \
--env-var DD_API_KEY=your-api-key \
--env-var DD_SITE=datadoghq.com \
--env-var DD_SERVICE=my-rust-service \
--env-var DD_ENV=production
```
## Structured Logging Best Practices
### Using tracing with Spans
```rust
use tracing::{info, warn, error, debug, span, Level};
async fn function_handler(event: LambdaEvent<Request>) -> Result<Response, Error> {
let span = span!(
Level::INFO,
"process_request",
request_id = %event.context.request_id,
user_id = %event.payload.user_id,
);
let _enter = span.enter();
info!("Starting request processing");
match process_user(&event.payload.user_id).await {
Ok(user) => {
info!(user_name = %user.name, "User processed successfully");
Ok(Response { user })
}
Err(e) => {
error!(error = %e, "Failed to process user");
Err(e)
}
}
}
#[instrument(skip(db), fields(user_id = %user_id))]
async fn process_user(user_id: &str) -> Result<User, Error> {
debug!("Fetching user from database");
let user = fetch_from_db(user_id).await?;
info!(email = %user.email, "User fetched");
Ok(user)
}
```
### JSON Structured Logging
```rust
use tracing_subscriber::{fmt, EnvFilter, layer::SubscriberExt};
use serde_json::json;
#[tokio::main]
async fn main() -> Result<(), Error> {
// JSON output for CloudWatch Insights
tracing_subscriber::fmt()
.json()
.with_env_filter(EnvFilter::from_default_env())
.with_current_span(true)
.with_span_list(true)
.with_target(false)
.without_time() // CloudWatch adds timestamp
.init();
run(service_fn(function_handler)).await
}
// Logs will be structured JSON:
// {"level":"info","message":"Processing request","request_id":"abc123","user_id":"user456"}
```
### Custom Metrics with OpenTelemetry
```rust
use opentelemetry::metrics::{Counter, Histogram};
use opentelemetry::KeyValue;
struct Metrics {
request_counter: Counter<u64>,
duration_histogram: Histogram<f64>,
}
async fn function_handler(event: LambdaEvent<Request>) -> Result<Response, Error> {
let start = std::time::Instant::now();
// Increment counter
metrics.request_counter.add(
1,
&[
KeyValue::new("function", "my-lambda"),
KeyValue::new("region", "us-east-1"),
],
);
let result = process_request(&event.payload).await?;
// Record duration
let duration = start.elapsed().as_secs_f64();
metrics.duration_histogram.record(
duration,
&[KeyValue::new("status", "success")],
);
Ok(result)
}
```
## CloudWatch Logs Insights Queries
With structured logging, you can query efficiently:
```
# Find errors for specific user
fields @timestamp, message, error
| filter user_id = "user456"
| filter level = "error"
| sort @timestamp desc
# Calculate p95 latency
fields duration_ms
| stats percentile(duration_ms, 95) as p95_latency by bin(5m)
# Count requests by status
fields @timestamp
| filter message = "Request completed"
| stats count() by status
```
## Distributed Tracing Pattern
For microservices calling each other:
```rust
use opentelemetry::global;
use opentelemetry::trace::{Tracer, TracerProvider, SpanKind};
use opentelemetry_http::HeaderExtractor;
async fn function_handler(event: LambdaEvent<ApiGatewayRequest>) -> Result<Response, Error> {
let tracer = global::tracer("my-service");
// Extract trace context from incoming request
let parent_cx = global::get_text_map_propagator(|propagator| {
let headers = HeaderExtractor::new(&event.payload.headers);
propagator.extract(&headers)
});
// Create span with parent context
let span = tracer
.span_builder("handle_request")
.with_kind(SpanKind::Server)
.start_with_context(&tracer, &parent_cx);
let cx = opentelemetry::Context::current_with_span(span);
// Call downstream service with trace context
let client = reqwest::Client::new();
let response = client
.get("https://downstream-service.com/api")
.header("traceparent", extract_traceparent(&cx))
.send()
.await?;
Ok(Response { data: response.text().await? })
}
```
## AWS ADOT Lambda Layer
For automatic instrumentation (limited Rust support):
```bash
# Add ADOT layer (note: Rust needs manual instrumentation)
cargo lambda deploy \
--layers arn:aws:lambda:us-east-1:901920570463:layer:aws-otel-collector-arm64-ver-0-90-1:1 \
--env-var AWS_LAMBDA_EXEC_WRAPPER=/opt/otel-instrument \
--env-var OPENTELEMETRY_COLLECTOR_CONFIG_FILE=/var/task/collector.yaml
```
## Cold Start Monitoring
Track cold start vs warm start:
```rust
use std::sync::atomic::{AtomicBool, Ordering};
static COLD_START: AtomicBool = AtomicBool::new(true);
async fn function_handler(event: LambdaEvent<Request>) -> Result<Response, Error> {
let is_cold_start = COLD_START.swap(false, Ordering::Relaxed);
info!(
cold_start = is_cold_start,
"Lambda invocation"
);
// Process request...
Ok(Response {})
}
```
## Error Tracking
### Capturing Error Context
```rust
use tracing::error;
use thiserror::Error;
#[derive(Error, Debug)]
enum LambdaError {
#[error("Database error: {0}")]
Database(#[from] sqlx::Error),
#[error("External API error: {status}, {message}")]
ExternalApi { status: u16, message: String },
}
async fn function_handler(event: LambdaEvent<Request>) -> Result<Response, Error> {
match process_request(&event.payload).await {
Ok(result) => {
info!("Request processed successfully");
Ok(Response { result })
}
Err(e) => {
error!(
error = %e,
error_type = std::any::type_name_of_val(&e),
request_id = %event.context.request_id,
"Request failed"
);
// Optionally send to error tracking service
send_to_sentry(&e, &event.context).await;
Err(e.into())
}
}
}
```
## Performance Monitoring
### Measure Operation Duration
```rust
use std::time::Instant;
use tracing::info;
#[instrument]
async fn expensive_operation() -> Result<Data, Error> {
let start = Instant::now();
let result = do_work().await?;
let duration = start.elapsed();
info!(duration_ms = duration.as_millis(), "Operation completed");
Ok(result)
}
```
### Automatic Instrumentation
```rust
use tracing::instrument;
// Automatically creates span and logs entry/exit
#[instrument(
skip(db), // Don't log entire db object
fields(
user_id = %user_id,
operation = "fetch_user"
),
err // Log errors automatically
)]
async fn fetch_user(db: &Database, user_id: &str) -> Result<User, Error> {
db.get_user(user_id).await
}
```
## Observability Checklist
- [ ] Enable X-Ray or OpenTelemetry tracing
- [ ] Use structured logging (JSON format)
- [ ] Add span instrumentation to key functions
- [ ] Track cold vs warm starts
- [ ] Monitor error rates and types
- [ ] Measure operation durations
- [ ] Set up CloudWatch Logs Insights queries
- [ ] Configure alerts for errors and latency
- [ ] Track custom business metrics
- [ ] Propagate trace context across services
- [ ] Set appropriate log retention
- [ ] Use log levels correctly (debug, info, warn, error)
## Recommended Stack
**For AWS-only**:
- X-Ray for tracing
- CloudWatch Logs with structured JSON
- CloudWatch Insights for queries
- xray-lite for Rust integration
**For multi-cloud/vendor-neutral**:
- OpenTelemetry for tracing
- OTLP exporter to your backend
- lambda-otel-lite for Lambda optimization
- tracing crate for structured logging
**For Datadog users**:
- Datadog Lambda Extension
- DD_TRACE_ENABLED for automatic tracing
- JSON structured logging
- Custom metrics via DogStatsD
## Dependencies
```toml
[dependencies]
# Basic tracing
tracing = { version = "0.1", features = ["log"] }
tracing-subscriber = { version = "0.3", features = ["env-filter", "json"] }
# X-Ray
xray-lite = "0.1"
xray-lite-aws-sdk = "0.1"
# OpenTelemetry
lambda-otel-lite = "0.1"
opentelemetry = "0.22"
opentelemetry-otlp = "0.15"
opentelemetry_sdk = "0.22"
tracing-opentelemetry = "0.23"
# AWS SDK (for tracing AWS calls)
aws-config = "1"
aws-sdk-dynamodb = "1" # or other services
```
Guide the user through setting up observability appropriate for their needs and monitoring backend.

669
commands/lambda-optimize-compute.md Normal file
View File

@@ -0,0 +1,669 @@
---
description: Optimize Rust Lambda function for compute-intensive workloads using Rayon and spawn_blocking
---
You are helping the user optimize their Lambda function for compute-intensive workloads.
## Your Task
Guide the user to optimize their Lambda for CPU-intensive operations using synchronous parallel processing with Rayon and spawn_blocking.
## Compute-Intensive Characteristics
Functions that:
- Process large datasets
- Perform mathematical computations
- Transform/parse data
- Image/video processing
- Compression/decompression
- Encryption/hashing
- Machine learning inference
**Goal**: Maximize CPU utilization without blocking async runtime
## Key Principle: Async at Boundaries, Sync in Middle
```
Input (async) → CPU Work (sync) → Output (async)
```
```rust
async fn function_handler(event: LambdaEvent<Request>) -> Result<Response, Error> {
// Phase 1: Async input (if needed)
let data = fetch_input_data().await?;
// Phase 2: Sync compute with spawn_blocking + Rayon
let results = tokio::task::spawn_blocking(move || {
data.par_iter()
.map(|item| expensive_computation(item))
.collect::<Vec<_>>()
})
.await?;
// Phase 3: Async output (if needed)
upload_results(&results).await?;
Ok(Response { results })
}
```
## Core Pattern: spawn_blocking + Rayon
### Why This Pattern?
1. **spawn_blocking**: Moves work off async runtime to blocking thread pool
2. **Rayon**: Efficiently parallelizes CPU work across available cores
3. **Together**: Best CPU utilization without blocking async operations
### Basic Pattern
```rust
use tokio::task;
use rayon::prelude::*;
async fn function_handler(event: LambdaEvent<Request>) -> Result<Response, Error> {
let numbers = event.payload.numbers;
// Move CPU work to blocking thread pool
let results = task::spawn_blocking(move || {
// Use Rayon for parallel computation
numbers
.par_iter()
.map(|&n| cpu_intensive_work(n))
.collect::<Vec<_>>()
})
.await?;
Ok(Response { results })
}
// Pure CPU work - synchronous, no async
fn cpu_intensive_work(n: i64) -> i64 {
// Heavy computation here
(0..10000).fold(n, |acc, _| {
acc.wrapping_mul(31).wrapping_add(17)
})
}
```
## Complete Compute-Optimized Example
```rust
use lambda_runtime::{run, service_fn, Error, LambdaEvent};
use rayon::prelude::*;
use serde::{Deserialize, Serialize};
use tokio::task;
use tracing::{info, instrument};
#[derive(Deserialize)]
struct Request {
data: Vec<DataPoint>,
algorithm: String,
}
#[derive(Serialize)]
struct Response {
processed: Vec<ProcessedData>,
stats: Statistics,
}
#[derive(Debug, Clone, Deserialize)]
struct DataPoint {
values: Vec<f64>,
metadata: String,
}
#[derive(Debug, Serialize)]
struct ProcessedData {
result: f64,
classification: String,
}
#[derive(Debug, Serialize)]
struct Statistics {
count: usize,
mean: f64,
std_dev: f64,
}
async fn function_handler(event: LambdaEvent<Request>) -> Result<Response, Error> {
let data = event.payload.data;
let algorithm = event.payload.algorithm;
info!("Processing {} data points with {}", data.len(), algorithm);
// CPU-intensive work in spawn_blocking
let results = task::spawn_blocking(move || {
process_data_parallel(data, &algorithm)
})
.await??;
Ok(results)
}
// All CPU work happens here - synchronous and parallel
fn process_data_parallel(data: Vec<DataPoint>, algorithm: &str) -> Result<Response, Error> {
// Parallel processing with Rayon
let processed: Vec<ProcessedData> = data
.par_iter()
.map(|point| {
let result = match algorithm {
"standard" => compute_standard(&point.values),
"advanced" => compute_advanced(&point.values),
_ => compute_standard(&point.values),
};
let classification = classify_result(result);
ProcessedData { result, classification }
})
.collect();
// Compute statistics
let stats = compute_statistics(&processed);
Ok(Response { processed, stats })
}
// Pure computation - no IO, no async
fn compute_standard(values: &[f64]) -> f64 {
// CPU-intensive mathematical computation
let sum: f64 = values.iter().sum();
let mean = sum / values.len() as f64;
values.iter()
.map(|&x| (x - mean).powi(2))
.sum::<f64>()
.sqrt()
}
fn compute_advanced(values: &[f64]) -> f64 {
// Even more intensive computation
let mut result = 0.0;
for &v in values {
for i in 0..1000 {
result += v * (i as f64).sin();
}
}
result
}
fn classify_result(value: f64) -> String {
match value {
x if x < 0.0 => "low".to_string(),
x if x < 10.0 => "medium".to_string(),
_ => "high".to_string(),
}
}
fn compute_statistics(processed: &[ProcessedData]) -> Statistics {
let count = processed.len();
let mean = processed.iter().map(|p| p.result).sum::<f64>() / count as f64;
let variance = processed
.iter()
.map(|p| (p.result - mean).powi(2))
.sum::<f64>() / count as f64;
let std_dev = variance.sqrt();
Statistics { count, mean, std_dev }
}
#[tokio::main]
async fn main() -> Result<(), Error> {
tracing_subscriber::fmt()
.with_max_level(tracing::Level::INFO)
.without_time()
.init();
run(service_fn(function_handler)).await
}
```
## Advanced Rayon Patterns
### Custom Thread Pool
```rust
use rayon::ThreadPoolBuilder;
use std::sync::OnceLock;
static THREAD_POOL: OnceLock<rayon::ThreadPool> = OnceLock::new();
fn get_thread_pool() -> &'static rayon::ThreadPool {
THREAD_POOL.get_or_init(|| {
ThreadPoolBuilder::new()
.num_threads(num_cpus::get())
.build()
.unwrap()
})
}
async fn function_handler(event: LambdaEvent<Request>) -> Result<Response, Error> {
let data = event.payload.data;
let results = task::spawn_blocking(move || {
let pool = get_thread_pool();
pool.install(|| {
data.par_iter()
.map(|item| process(item))
.collect::<Vec<_>>()
})
})
.await?;
Ok(Response { results })
}
```
### Parallel Fold (Reduce)
```rust
fn parallel_sum(numbers: Vec<i64>) -> i64 {
numbers
.par_iter()
.fold(|| 0i64, |acc, &x| acc + expensive_transform(x))
.reduce(|| 0, |a, b| a + b)
}
fn expensive_transform(n: i64) -> i64 {
// CPU work
(0..1000).fold(n, |acc, _| acc.wrapping_mul(31))
}
```
### Parallel Chunks
```rust
use rayon::prelude::*;
fn process_in_chunks(data: Vec<u8>) -> Vec<Vec<u8>> {
data.par_chunks(1024) // Process in 1KB chunks
.map(|chunk| {
// Expensive processing per chunk
compress_chunk(chunk)
})
.collect()
}
fn compress_chunk(chunk: &[u8]) -> Vec<u8> {
// CPU-intensive compression
chunk.to_vec() // Placeholder
}
```
### Parallel Chain
```rust
fn multi_stage_processing(data: Vec<DataPoint>) -> Vec<Output> {
data.par_iter()
.filter(|point| point.is_valid())
.map(|point| normalize(point))
.map(|normalized| transform(normalized))
.filter(|result| result.score > 0.5)
.collect()
}
```
## Mixed IO + Compute Pattern
For functions that do both IO and compute:
```rust
async fn function_handler(event: LambdaEvent<Request>) -> Result<Response, Error> {
// Phase 1: Async IO - Download data
let raw_data = download_from_s3(&event.payload.bucket, &event.payload.key).await?;
// Phase 2: Sync compute - Process data
let processed = task::spawn_blocking(move || {
process_data_parallel(raw_data)
})
.await??;
// Phase 3: Async IO - Upload results
upload_to_s3(&event.payload.output_bucket, &processed).await?;
Ok(Response { success: true })
}
fn process_data_parallel(data: Vec<u8>) -> Result<Vec<ProcessedChunk>, Error> {
// Parse and process in parallel
let chunks: Vec<Vec<u8>> = data
.chunks(1024)
.map(|c| c.to_vec())
.collect();
let results = chunks
.par_iter()
.map(|chunk| {
// CPU-intensive per chunk
parse_and_transform(chunk)
})
.collect::<Result<Vec<_>, _>>()?;
Ok(results)
}
```
## Image Processing Example
```rust
async fn function_handler(event: LambdaEvent<S3Event>) -> Result<(), Error> {
for record in event.payload.records {
let bucket = record.s3.bucket.name.unwrap();
let key = record.s3.object.key.unwrap();
// Async: Download image
let image_data = download_from_s3(&bucket, &key).await?;
// Sync: Process image with Rayon
let processed = task::spawn_blocking(move || {
process_image_parallel(image_data)
})
.await??;
// Async: Upload result
let output_key = format!("processed/{}", key);
upload_to_s3(&bucket, &output_key, &processed).await?;
}
Ok(())
}
fn process_image_parallel(image_data: Vec<u8>) -> Result<Vec<u8>, Error> {
// Parse image
let img = parse_image(&image_data)?;
let (width, height) = img.dimensions();
// Process rows in parallel
let rows: Vec<Vec<Pixel>> = (0..height)
.into_par_iter()
.map(|y| {
(0..width)
.map(|x| {
let pixel = img.get_pixel(x, y);
apply_filter(pixel) // CPU-intensive
})
.collect()
})
.collect();
// Flatten and encode
encode_image(rows)
}
```
## Data Transformation Example
```rust
#[derive(Deserialize)]
struct CsvRow {
id: String,
values: Vec<f64>,
}
async fn function_handler(event: LambdaEvent<Request>) -> Result<Response, Error> {
// Async: Download CSV from S3
let csv_data = download_csv(&event.payload.s3_key).await?;
// Sync: Parse and transform with Rayon
let transformed = task::spawn_blocking(move || {
parse_and_transform_csv(csv_data)
})
.await??;
// Async: Write to database
write_to_database(&transformed).await?;
Ok(Response { rows_processed: transformed.len() })
}
fn parse_and_transform_csv(csv_data: String) -> Result<Vec<TransformedRow>, Error> {
let rows: Vec<CsvRow> = csv_data
.lines()
.skip(1) // Skip header
.map(|line| parse_csv_line(line))
.collect::<Result<Vec<_>, _>>()?;
// Parallel transformation
let transformed = rows
.par_iter()
.map(|row| {
// CPU-intensive transformation
TransformedRow {
id: row.id.clone(),
mean: calculate_mean(&row.values),
median: calculate_median(&row.values),
std_dev: calculate_std_dev(&row.values),
outliers: detect_outliers(&row.values),
}
})
.collect();
Ok(transformed)
}
```
## Error Handling
```rust
use thiserror::Error;
#[derive(Error, Debug)]
enum ComputeError {
#[error("Invalid input: {0}")]
InvalidInput(String),
#[error("Computation failed: {0}")]
ComputationFailed(String),
#[error("Task join error: {0}")]
JoinError(#[from] tokio::task::JoinError),
}
async fn function_handler(event: LambdaEvent<Request>) -> Result<Response, Error> {
let data = event.payload.data;
if data.is_empty() {
return Err(ComputeError::InvalidInput("Empty data".to_string()).into());
}
let results = task::spawn_blocking(move || {
process_with_validation(data)
})
.await??; // Handle both JoinError and computation errors
Ok(Response { results })
}
fn process_with_validation(data: Vec<DataPoint>) -> Result<Vec<Output>, ComputeError> {
let results: Result<Vec<_>, _> = data
.par_iter()
.map(|point| {
if !point.is_valid() {
return Err(ComputeError::InvalidInput(
format!("Invalid point: {:?}", point)
));
}
process_point(point)
.map_err(|e| ComputeError::ComputationFailed(e.to_string()))
})
.collect();
results
}
```
## Memory Configuration
For compute-intensive Lambda, more memory = more CPU:
```bash
# More memory for more CPU power
cargo lambda deploy my-function --memory 3008
# Lambda vCPU allocation:
# 1769 MB = 1 full vCPU
# 3008 MB = ~1.77 vCPU
# 10240 MB = 6 vCPU
```
**Recommendation**: Test different memory settings to find optimal cost/performance
## Performance Optimization Checklist
- [ ] Use `tokio::task::spawn_blocking` for CPU work
- [ ] Use Rayon `.par_iter()` for parallel processing
- [ ] Keep async only at IO boundaries
- [ ] Avoid async/await inside CPU-intensive functions
- [ ] Use appropriate Lambda memory (more memory = more CPU)
- [ ] Minimize data copying (use references where possible)
- [ ] Profile to find hot paths
- [ ] Consider chunking for very large datasets
- [ ] Use `par_chunks` for better cache locality
- [ ] Test with realistic data sizes
- [ ] Monitor CPU utilization in CloudWatch
## Dependencies
Add to `Cargo.toml`:
```toml
[dependencies]
lambda_runtime = "0.13"
tokio = { version = "1", features = ["macros", "rt-multi-thread"] }
rayon = "1.10"
# Serialization
serde = { version = "1", features = ["derive"] }
serde_json = "1"
# Error handling
anyhow = "1"
thiserror = "1"
# Tracing
tracing = "0.1"
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
# Optional: CPU count
num_cpus = "1"
# For image processing (example)
# image = "0.24"
# For CSV processing (example)
# csv = "1"
```
## Testing Performance
```rust
#[cfg(test)]
mod tests {
use super::*;
#[tokio::test]
async fn test_parallel_performance() {
let data = vec![DataPoint::new(); 1000];
let start = std::time::Instant::now();
let results = task::spawn_blocking(move || {
process_data_parallel(data)
})
.await
.unwrap();
let duration = start.elapsed();
println!("Processed {} items in {:?}", results.len(), duration);
// Verify parallelism is effective
assert!(duration.as_millis() < 5000);
}
#[test]
fn test_computation() {
let input = DataPoint::example();
let result = cpu_intensive_work(&input);
assert!(result.is_valid());
}
}
```
## Benchmarking
Use Criterion for benchmarking:
```rust
// benches/compute_bench.rs
use criterion::{black_box, criterion_group, criterion_main, Criterion};
fn benchmark_computation(c: &mut Criterion) {
let data = vec![1i64; 10000];
c.bench_function("sequential", |b| {
b.iter(|| {
data.iter()
.map(|&n| black_box(expensive_computation(n)))
.collect::<Vec<_>>()
})
});
c.bench_function("parallel", |b| {
b.iter(|| {
data.par_iter()
.map(|&n| black_box(expensive_computation(n)))
.collect::<Vec<_>>()
})
});
}
criterion_group!(benches, benchmark_computation);
criterion_main!(benches);
```
Run with:
```bash
cargo bench
```
## Monitoring
Add instrumentation:
```rust
use tracing::{info, instrument};
#[instrument(skip(data))]
fn process_data_parallel(data: Vec<DataPoint>) -> Result<Vec<Output>, Error> {
let start = std::time::Instant::now();
let count = data.len();
let results = data
.par_iter()
.map(|point| process_point(point))
.collect::<Result<Vec<_>, _>>()?;
let duration = start.elapsed();
info!(
count,
duration_ms = duration.as_millis(),
throughput = count as f64 / duration.as_secs_f64(),
"Processing complete"
);
Ok(results)
}
```
After optimization, verify:
- CPU utilization is high (check CloudWatch)
- Execution time scales with data size
- Memory usage is within limits
- Cost is optimized for workload

599
commands/lambda-optimize-io.md Normal file
View File

@@ -0,0 +1,599 @@
---
description: Optimize Rust Lambda function for IO-intensive workloads with async patterns
---
You are helping the user optimize their Lambda function for IO-intensive workloads.
## Your Task
Guide the user to optimize their Lambda for maximum IO performance using async/await patterns.
## IO-Intensive Characteristics
Functions that:
- Make multiple HTTP/API requests
- Query databases
- Read/write from S3, DynamoDB
- Call external services
- Process message queues
- Send notifications
**Goal**: Maximize concurrency to reduce wall-clock time and cost
## Key Optimization Strategies
### 1. Concurrent Operations
Replace sequential operations with concurrent ones:
**❌ Sequential (Slow)**:
```rust
async fn function_handler(event: LambdaEvent<Request>) -> Result<Response, Error> {
// Each operation waits for previous one - slow!
let user = fetch_user().await?;
let posts = fetch_posts().await?;
let comments = fetch_comments().await?;
Ok(Response { user, posts, comments })
}
```
**✅ Concurrent (Fast)**:
```rust
use futures::future::try_join_all;
async fn function_handler(event: LambdaEvent<Request>) -> Result<Response, Error> {
// All operations run concurrently - fast!
let (user, posts, comments) = tokio::try_join!(
fetch_user(),
fetch_posts(),
fetch_comments(),
)?;
Ok(Response { user, posts, comments })
}
```
**Performance impact**: 3 sequential 100ms calls = 300ms. Concurrent = ~100ms.
### 2. Parallel Collection Processing
**❌ Sequential iteration**:
```rust
let mut results = Vec::new();
for id in user_ids {
let data = fetch_data(&id).await?;
results.push(data);
}
```
**✅ Concurrent iteration**:
```rust
use futures::future::try_join_all;
let futures = user_ids
.iter()
.map(|id| fetch_data(id));
let results = try_join_all(futures).await?;
```
**Alternative with buffer (limits concurrency)**:
```rust
use futures::stream::{self, StreamExt};
let results = stream::iter(user_ids)
.map(|id| fetch_data(&id))
.buffer_unordered(10) // Max 10 concurrent requests
.collect::<Vec<_>>()
.await;
```
### 3. Reuse Connections
**❌ Creating new client each time**:
```rust
async fn fetch_data(url: &str) -> Result<Data, Error> {
let client = reqwest::Client::new(); // New connection every call!
client.get(url).send().await?.json().await
}
```
**✅ Shared client with connection pooling**:
```rust
use std::sync::OnceLock;
use reqwest::Client;
// Initialized once per container
static HTTP_CLIENT: OnceLock<Client> = OnceLock::new();
fn get_client() -> &'static Client {
HTTP_CLIENT.get_or_init(|| {
Client::builder()
.timeout(Duration::from_secs(30))
.pool_max_idle_per_host(10)
.build()
.unwrap()
})
}
async fn fetch_data(url: &str) -> Result<Data, Error> {
let client = get_client(); // Reuses connections!
client.get(url).send().await?.json().await
}
```
### 4. Database Connection Pooling
**For PostgreSQL with sqlx**:
```rust
use std::sync::OnceLock;
use sqlx::{PgPool, postgres::PgPoolOptions};
static DB_POOL: OnceLock<PgPool> = OnceLock::new();
async fn get_pool() -> &'static PgPool {
DB_POOL.get_or_init(|| async {
PgPoolOptions::new()
.max_connections(5) // Limit connections
.connect(&std::env::var("DATABASE_URL").unwrap())
.await
.unwrap()
}).await
}
async fn function_handler(event: LambdaEvent<Request>) -> Result<Response, Error> {
let pool = get_pool().await;
// Use connection pool for queries
let user = sqlx::query_as!(User, "SELECT * FROM users WHERE id = $1", user_id)
.fetch_one(pool)
.await?;
Ok(Response { user })
}
```
**For DynamoDB**:
```rust
use std::sync::OnceLock;
use aws_sdk_dynamodb::Client;
static DYNAMODB_CLIENT: OnceLock<Client> = OnceLock::new();
async fn get_dynamodb() -> &'static Client {
DYNAMODB_CLIENT.get_or_init(|| async {
let config = aws_config::load_from_env().await;
Client::new(&config)
}).await
}
```
### 5. Batch Operations
When possible, use batch APIs:
**❌ Individual requests**:
```rust
for key in keys {
let item = dynamodb.get_item()
.table_name("MyTable")
.key("id", AttributeValue::S(key))
.send()
.await?;
}
```
**✅ Batch request**:
```rust
let batch_keys = keys
.iter()
.map(|key| {
[(
"id".to_string(),
AttributeValue::S(key.clone())
)].into()
})
.collect();
let response = dynamodb.batch_get_item()
.request_items("MyTable", KeysAndAttributes::builder()
.set_keys(Some(batch_keys))
.build()?)
.send()
.await?;
```
## Complete IO-Optimized Example
```rust
use lambda_runtime::{run, service_fn, Error, LambdaEvent};
use serde::{Deserialize, Serialize};
use std::sync::OnceLock;
use reqwest::Client;
use futures::future::try_join_all;
use tracing::info;
#[derive(Deserialize)]
struct Request {
user_ids: Vec<String>,
}
#[derive(Serialize)]
struct Response {
users: Vec<UserData>,
}
#[derive(Serialize)]
struct UserData {
user: User,
posts: Vec<Post>,
followers: usize,
}
// Shared HTTP client with connection pooling
static HTTP_CLIENT: OnceLock<Client> = OnceLock::new();
fn get_client() -> &'static Client {
HTTP_CLIENT.get_or_init(|| {
Client::builder()
.timeout(Duration::from_secs(30))
.pool_max_idle_per_host(10)
.tcp_keepalive(Duration::from_secs(60))
.build()
.unwrap()
})
}
async fn function_handler(event: LambdaEvent<Request>) -> Result<Response, Error> {
info!("Processing {} users", event.payload.user_ids.len());
// Process all users concurrently
let user_futures = event.payload.user_ids
.into_iter()
.map(|user_id| fetch_user_data(user_id));
let users = try_join_all(user_futures).await?;
Ok(Response { users })
}
async fn fetch_user_data(user_id: String) -> Result<UserData, Error> {
let client = get_client();
// Fetch all user data concurrently
let (user, posts, followers) = tokio::try_join!(
fetch_user(client, &user_id),
fetch_posts(client, &user_id),
fetch_follower_count(client, &user_id),
)?;
Ok(UserData { user, posts, followers })
}
async fn fetch_user(client: &Client, user_id: &str) -> Result<User, Error> {
let response = client
.get(format!("https://api.example.com/users/{}", user_id))
.send()
.await?
.json()
.await?;
Ok(response)
}
async fn fetch_posts(client: &Client, user_id: &str) -> Result<Vec<Post>, Error> {
let response = client
.get(format!("https://api.example.com/users/{}/posts", user_id))
.send()
.await?
.json()
.await?;
Ok(response)
}
async fn fetch_follower_count(client: &Client, user_id: &str) -> Result<usize, Error> {
let response: FollowerResponse = client
.get(format!("https://api.example.com/users/{}/followers", user_id))
.send()
.await?
.json()
.await?;
Ok(response.count)
}
#[tokio::main]
async fn main() -> Result<(), Error> {
tracing_subscriber::fmt()
.with_max_level(tracing::Level::INFO)
.without_time()
.init();
run(service_fn(function_handler)).await
}
```
## AWS SDK Optimization
### S3 Concurrent Operations
```rust
use aws_sdk_s3::Client;
use futures::future::try_join_all;
async fn download_multiple_files(
s3: &Client,
bucket: &str,
keys: Vec<String>,
) -> Result<Vec<Bytes>, Error> {
let futures = keys.iter().map(|key| async move {
s3.get_object()
.bucket(bucket)
.key(key)
.send()
.await?
.body
.collect()
.await
.map(|data| data.into_bytes())
});
try_join_all(futures).await
}
```
### DynamoDB Parallel Queries
```rust
async fn query_multiple_partitions(
dynamodb: &Client,
partition_keys: Vec<String>,
) -> Result<Vec<Item>, Error> {
let futures = partition_keys.iter().map(|pk| {
dynamodb
.query()
.table_name("MyTable")
.key_condition_expression("pk = :pk")
.expression_attribute_values(":pk", AttributeValue::S(pk.clone()))
.send()
});
let results = try_join_all(futures).await?;
let items = results
.into_iter()
.flat_map(|r| r.items.unwrap_or_default())
.collect();
Ok(items)
}
```
## Timeout and Retry Configuration
```rust
use reqwest::Client;
use std::time::Duration;
fn get_client() -> &'static Client {
HTTP_CLIENT.get_or_init(|| {
Client::builder()
.timeout(Duration::from_secs(30)) // Total request timeout
.connect_timeout(Duration::from_secs(10)) // Connection timeout
.pool_max_idle_per_host(10) // Connection pool size
.tcp_keepalive(Duration::from_secs(60)) // Keep connections alive
.build()
.unwrap()
})
}
```
With retries:
```rust
use tower::{ServiceBuilder, ServiceExt};
use tower::retry::RetryLayer;
// Add to dependencies: tower = { version = "0.4", features = ["retry"] }
async fn fetch_with_retry(url: &str) -> Result<Response, Error> {
let client = get_client();
for attempt in 1..=3 {
match client.get(url).send().await {
Ok(response) => return Ok(response),
Err(e) if attempt < 3 => {
tokio::time::sleep(Duration::from_millis(100 * attempt)).await;
continue;
}
Err(e) => return Err(e.into()),
}
}
unreachable!()
}
```
## Streaming Large Responses
For large files or responses:
```rust
use tokio::io::AsyncWriteExt;
use futures::StreamExt;
async fn download_large_file(s3: &Client, bucket: &str, key: &str) -> Result<(), Error> {
let mut object = s3
.get_object()
.bucket(bucket)
.key(key)
.send()
.await?;
// Stream to avoid loading entire file in memory
let mut body = object.body;
while let Some(chunk) = body.next().await {
let chunk = chunk?;
// Process chunk
process_chunk(&chunk).await?;
}
Ok(())
}
```
## Concurrency Limits
Control concurrency to avoid overwhelming external services:
```rust
use futures::stream::{self, StreamExt};
async fn process_with_limit(
items: Vec<Item>,
max_concurrent: usize,
) -> Result<Vec<Output>, Error> {
let results = stream::iter(items)
.map(|item| async move {
process_item(item).await
})
.buffer_unordered(max_concurrent) // Limit concurrent operations
.collect::<Vec<_>>()
.await;
results.into_iter().collect()
}
```
## Error Handling for Async Operations
```rust
use thiserror::Error;
#[derive(Error, Debug)]
enum LambdaError {
#[error("HTTP error: {0}")]
Http(#[from] reqwest::Error),
#[error("Database error: {0}")]
Database(String),
#[error("Timeout: operation took too long")]
Timeout,
}
async fn function_handler(event: LambdaEvent<Request>) -> Result<Response, Error> {
// Use timeout for async operations
let result = tokio::time::timeout(
Duration::from_secs(25), // Lambda timeout - buffer
process_request(event.payload)
)
.await
.map_err(|_| LambdaError::Timeout)??;
Ok(result)
}
```
## Performance Checklist
Apply these optimizations:
- [ ] Use `tokio::try_join!` for fixed concurrent operations
- [ ] Use `futures::future::try_join_all` for dynamic collections
- [ ] Initialize clients/pools once with `OnceLock`
- [ ] Configure connection pooling for HTTP clients
- [ ] Use batch APIs when available
- [ ] Set appropriate timeouts
- [ ] Add retries for transient failures
- [ ] Stream large responses
- [ ] Limit concurrency to avoid overwhelming services
- [ ] Use `buffer_unordered` for controlled parallelism
- [ ] Avoid blocking operations in async context
- [ ] Monitor cold start times
- [ ] Test with realistic event sizes
## Dependencies for IO Optimization
Add to `Cargo.toml`:
```toml
[dependencies]
lambda_runtime = "0.13"
tokio = { version = "1", features = ["macros", "rt-multi-thread", "time"] }
futures = "0.3"
# HTTP client
reqwest = { version = "0.12", features = ["json"] }
# AWS SDKs
aws-config = "1"
aws-sdk-s3 = "1"
aws-sdk-dynamodb = "1"
# Database (if needed)
sqlx = { version = "0.7", features = ["runtime-tokio-rustls", "postgres"] }
# Error handling
anyhow = "1"
thiserror = "1"
# Tracing
tracing = "0.1"
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
# Optional: retries
tower = { version = "0.4", features = ["retry", "timeout"] }
```
## Testing IO Performance
```rust
#[cfg(test)]
mod tests {
use super::*;
#[tokio::test]
async fn test_concurrent_performance() {
let start = std::time::Instant::now();
let results = fetch_multiple_users(vec!["1", "2", "3"]).await.unwrap();
let duration = start.elapsed();
// Should be ~100ms (concurrent), not ~300ms (sequential)
assert!(duration.as_millis() < 150);
assert_eq!(results.len(), 3);
}
}
```
## Monitoring
Add instrumentation to track IO performance:
```rust
use tracing::{info, instrument};
#[instrument(skip(client))]
async fn fetch_user(client: &Client, user_id: &str) -> Result<User, Error> {
let start = std::time::Instant::now();
let result = client
.get(format!("https://api.example.com/users/{}", user_id))
.send()
.await?
.json()
.await?;
info!(user_id, duration_ms = start.elapsed().as_millis(), "User fetched");
Ok(result)
}
```
After optimization, verify:
- Cold start time (should be minimal)
- Warm execution time (should be low due to concurrency)
- Memory usage (should be moderate)
- Error rates (should be low with retries)
- CloudWatch metrics show improved performance

668
commands/lambda-secrets.md Normal file
View File

@@ -0,0 +1,668 @@
---
description: Manage secrets and configuration for Rust Lambda functions using AWS Secrets Manager and Parameter Store
---
You are helping the user securely manage secrets and configuration for their Rust Lambda functions.
## Your Task
Guide the user through implementing secure secrets management using AWS Secrets Manager, Systems Manager Parameter Store, and the Parameters and Secrets Lambda Extension.
## Secrets Management Options
### Option 1: AWS Parameters and Secrets Lambda Extension (Recommended)
**Best for**:
- Production workloads
- Cost-conscious applications
- Low-latency requirements
- Local caching needs
**Advantages**:
- Cached locally (reduces latency and cost)
- No SDK calls needed
- Automatic refresh
- Works with both Secrets Manager and Parameter Store
#### Setup
1. **Add the extension layer** to your Lambda:
```bash
cargo lambda deploy \
--layers arn:aws:lambda:us-east-1:177933569100:layer:AWS-Parameters-and-Secrets-Lambda-Extension-Arm64:11
```
For x86_64:
```bash
cargo lambda deploy \
--layers arn:aws:lambda:us-east-1:177933569100:layer:AWS-Parameters-and-Secrets-Lambda-Extension:11
```
2. **Add IAM permissions**:
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"secretsmanager:GetSecretValue",
"ssm:GetParameter"
],
"Resource": [
"arn:aws:secretsmanager:us-east-1:123456789012:secret:my-secret-*",
"arn:aws:ssm:us-east-1:123456789012:parameter/myapp/*"
]
},
{
"Effect": "Allow",
"Action": "kms:Decrypt",
"Resource": "arn:aws:kms:us-east-1:123456789012:key/key-id"
}
]
}
```
3. **Use the Rust client**:
Add to `Cargo.toml`:
```toml
[dependencies]
aws-parameters-and-secrets-lambda = "0.1"
serde_json = "1"
```
Basic usage:
```rust
use aws_parameters_and_secrets_lambda::{Manager, ParameterError};
use lambda_runtime::{run, service_fn, Error, LambdaEvent};
use std::env;
async fn function_handler(event: LambdaEvent<Request>) -> Result<Response, Error> {
// Get secret from Secrets Manager
let manager = Manager::new();
let secret_value = manager
.get_secret("my-database-password")
.await?;
// Parse as JSON if needed
let db_config: DatabaseConfig = serde_json::from_str(&secret_value)?;
// Use the secret
let connection = connect_to_db(&db_config).await?;
Ok(Response { success: true })
}
#[derive(Deserialize)]
struct DatabaseConfig {
host: String,
port: u16,
username: String,
password: String,
database: String,
}
```
#### Get Parameter Store Values
```rust
use aws_parameters_and_secrets_lambda::Manager;
async fn get_config() -> Result<AppConfig, Error> {
let manager = Manager::new();
// Get simple parameter
let api_url = manager
.get_parameter("/myapp/api-url")
.await?;
// Get SecureString parameter (automatically decrypted)
let api_key = manager
.get_parameter("/myapp/api-key")
.await?;
Ok(AppConfig {
api_url,
api_key,
})
}
```
#### Caching and TTL
The extension caches secrets/parameters automatically. Configure TTL:
```bash
cargo lambda deploy \
--layers arn:aws:lambda:...:layer:AWS-Parameters-and-Secrets-Lambda-Extension-Arm64:11 \
--env-var PARAMETERS_SECRETS_EXTENSION_CACHE_ENABLED=true \
--env-var PARAMETERS_SECRETS_EXTENSION_CACHE_SIZE=1000 \
--env-var PARAMETERS_SECRETS_EXTENSION_MAX_CONNECTIONS=3
```
### Option 2: AWS SDK Direct Calls
**Best for**:
- Simple use cases
- One-time secret retrieval
- When extension layer isn't available
#### Using AWS SDK for Secrets Manager
Add to `Cargo.toml`:
```toml
[dependencies]
aws-config = "1"
aws-sdk-secretsmanager = "1"
```
Usage:
```rust
use aws_config::BehaviorVersion;
use aws_sdk_secretsmanager::Client as SecretsManagerClient;
use std::sync::OnceLock;
static SECRETS_CLIENT: OnceLock<SecretsManagerClient> = OnceLock::new();
async fn get_secrets_client() -> &'static SecretsManagerClient {
SECRETS_CLIENT.get_or_init(|| async {
let config = aws_config::load_defaults(BehaviorVersion::latest()).await;
SecretsManagerClient::new(&config)
}).await
}
async fn get_database_password() -> Result<String, Error> {
let client = get_secrets_client().await;
let response = client
.get_secret_value()
.secret_id("prod/database/password")
.send()
.await?;
Ok(response.secret_string().unwrap().to_string())
}
// For JSON secrets
async fn get_database_config() -> Result<DatabaseConfig, Error> {
let client = get_secrets_client().await;
let response = client
.get_secret_value()
.secret_id("prod/database/config")
.send()
.await?;
let secret_string = response.secret_string().unwrap();
let config: DatabaseConfig = serde_json::from_str(secret_string)?;
Ok(config)
}
```
#### Using AWS SDK for Parameter Store
Add to `Cargo.toml`:
```toml
[dependencies]
aws-config = "1"
aws-sdk-ssm = "1"
```
Usage:
```rust
use aws_sdk_ssm::Client as SsmClient;
use std::sync::OnceLock;
static SSM_CLIENT: OnceLock<SsmClient> = OnceLock::new();
async fn get_ssm_client() -> &'static SsmClient {
SSM_CLIENT.get_or_init(|| async {
let config = aws_config::load_defaults(BehaviorVersion::latest()).await;
SsmClient::new(&config)
}).await
}
async fn get_parameter(name: &str) -> Result<String, Error> {
let client = get_ssm_client().await;
let response = client
.get_parameter()
.name(name)
.with_decryption(true) // Decrypt SecureString
.send()
.await?;
Ok(response.parameter().unwrap().value().unwrap().to_string())
}
// Get multiple parameters
async fn get_parameters_by_path(path: &str) -> Result<HashMap<String, String>, Error> {
let client = get_ssm_client().await;
let mut parameters = HashMap::new();
let mut next_token = None;
loop {
let mut request = client
.get_parameters_by_path()
.path(path)
.with_decryption(true)
.recursive(true);
if let Some(token) = next_token {
request = request.next_token(token);
}
let response = request.send().await?;
for param in response.parameters() {
parameters.insert(
param.name().unwrap().to_string(),
param.value().unwrap().to_string(),
);
}
next_token = response.next_token().map(|s| s.to_string());
if next_token.is_none() {
break;
}
}
Ok(parameters)
}
```
### Option 3: Environment Variables (For Non-Sensitive Config)
**Best for**:
- Non-sensitive configuration
- Simple deployments
- Configuration that changes per environment
```rust
use std::env;
async fn function_handler(event: LambdaEvent<Request>) -> Result<Response, Error> {
let api_url = env::var("API_URL")
.expect("API_URL must be set");
let timeout_secs: u64 = env::var("TIMEOUT_SECONDS")
.unwrap_or_else(|_| "30".to_string())
.parse()
.expect("TIMEOUT_SECONDS must be a number");
// Use configuration
let client = build_client(&api_url, timeout_secs);
Ok(Response { })
}
```
Deploy with environment variables:
```bash
cargo lambda deploy \
--env-var API_URL=https://api.example.com \
--env-var TIMEOUT_SECONDS=30 \
--env-var ENVIRONMENT=production
```
## Best Practices
### 1. Initialize Secrets at Startup
```rust
use std::sync::OnceLock;
struct AppSecrets {
database_password: String,
api_key: String,
encryption_key: String,
}
static SECRETS: OnceLock<AppSecrets> = OnceLock::new();
async fn init_secrets() -> Result<&'static AppSecrets, Error> {
SECRETS.get_or_try_init(|| async {
let manager = Manager::new();
Ok(AppSecrets {
database_password: manager.get_secret("db-password").await?,
api_key: manager.get_parameter("/myapp/api-key").await?,
encryption_key: manager.get_secret("encryption-key").await?,
})
}).await
}
#[tokio::main]
async fn main() -> Result<(), Error> {
// Load secrets once at startup
init_secrets().await?;
run(service_fn(function_handler)).await
}
async fn function_handler(event: LambdaEvent<Request>) -> Result<Response, Error> {
// Access pre-loaded secrets
let secrets = SECRETS.get().unwrap();
let connection = connect_with_password(&secrets.database_password).await?;
Ok(Response {})
}
```
### 2. Separate Secrets by Environment
```
# Development
/dev/myapp/database/password
/dev/myapp/api-key
# Staging
/staging/myapp/database/password
/staging/myapp/api-key
# Production
/prod/myapp/database/password
/prod/myapp/api-key
```
Usage:
```rust
let env = std::env::var("ENVIRONMENT").unwrap_or_else(|_| "dev".to_string());
let param_name = format!("/{}/myapp/database/password", env);
let password = manager.get_parameter(&param_name).await?;
```
### 3. Handle Secret Rotation
```rust
use std::sync::RwLock;
use std::time::{Duration, Instant};
struct CachedSecret {
value: String,
last_updated: Instant,
ttl: Duration,
}
static SECRET_CACHE: OnceLock<RwLock<HashMap<String, CachedSecret>>> = OnceLock::new();
async fn get_secret_with_ttl(name: &str, ttl: Duration) -> Result<String, Error> {
let cache = SECRET_CACHE.get_or_init(|| RwLock::new(HashMap::new()));
// Check cache
{
let cache = cache.read().unwrap();
if let Some(cached) = cache.get(name) {
if cached.last_updated.elapsed() < cached.ttl {
return Ok(cached.value.clone());
}
}
}
// Fetch new value
let manager = Manager::new();
let value = manager.get_secret(name).await?;
// Update cache
{
let mut cache = cache.write().unwrap();
cache.insert(name.to_string(), CachedSecret {
value: value.clone(),
last_updated: Instant::now(),
ttl,
});
}
Ok(value)
}
```
### 4. Validate Secrets Format
```rust
use thiserror::Error;
#[derive(Error, Debug)]
enum SecretError {
#[error("Invalid secret format: {0}")]
InvalidFormat(String),
#[error("Missing required field: {0}")]
MissingField(String),
}
fn validate_database_config(config: &DatabaseConfig) -> Result<(), SecretError> {
if config.host.is_empty() {
return Err(SecretError::MissingField("host".to_string()));
}
if config.port == 0 {
return Err(SecretError::InvalidFormat("Port must be non-zero".to_string()));
}
if config.password.len() < 12 {
return Err(SecretError::InvalidFormat(
"Password must be at least 12 characters".to_string()
));
}
Ok(())
}
```
## Creating Secrets
### Via AWS CLI
**Secrets Manager**:
```bash
# Simple string secret
aws secretsmanager create-secret \
--name prod/database/password \
--secret-string "MySuperSecretPassword123!"
# JSON secret
aws secretsmanager create-secret \
--name prod/database/config \
--secret-string '{
"host": "db.example.com",
"port": 5432,
"username": "app_user",
"password": "MySuperSecretPassword123!",
"database": "myapp"
}'
```
**Parameter Store**:
```bash
# String parameter
aws ssm put-parameter \
--name /myapp/api-url \
--value "https://api.example.com" \
--type String
# SecureString parameter (encrypted)
aws ssm put-parameter \
--name /myapp/api-key \
--value "sk_live_abc123" \
--type SecureString
# With KMS key
aws ssm put-parameter \
--name /myapp/encryption-key \
--value "my-encryption-key" \
--type SecureString \
--key-id alias/myapp-key
```
### Via Terraform
**Secrets Manager**:
```hcl
resource "aws_secretsmanager_secret" "database_password" {
name = "prod/database/password"
description = "Database password for production"
}
resource "aws_secretsmanager_secret_version" "database_password" {
secret_id = aws_secretsmanager_secret.database_password.id
secret_string = var.database_password # From Terraform variables
}
# JSON secret
resource "aws_secretsmanager_secret" "database_config" {
name = "prod/database/config"
}
resource "aws_secretsmanager_secret_version" "database_config" {
secret_id = aws_secretsmanager_secret.database_config.id
secret_string = jsonencode({
host = "db.example.com"
port = 5432
username = "app_user"
password = var.database_password
database = "myapp"
})
}
```
**Parameter Store**:
```hcl
resource "aws_ssm_parameter" "api_url" {
name = "/myapp/api-url"
type = "String"
value = "https://api.example.com"
}
resource "aws_ssm_parameter" "api_key" {
name = "/myapp/api-key"
type = "SecureString"
value = var.api_key
}
```
## Secrets Manager vs Parameter Store
| Feature | Secrets Manager | Parameter Store |
|---------|----------------|-----------------|
| Cost | $0.40/secret/month + API calls | Free (Standard), $0.05/param/month (Advanced) |
| Max size | 65 KB | 4 KB (Standard), 8 KB (Advanced) |
| Rotation | Built-in | Manual |
| Versioning | Yes | Yes |
| Cross-account | Yes | Yes (Advanced) |
| Best for | Passwords, API keys | Configuration, non-rotated secrets |
## Complete Example
```rust
use aws_parameters_and_secrets_lambda::Manager;
use lambda_runtime::{run, service_fn, Error, LambdaEvent};
use serde::Deserialize;
use std::sync::OnceLock;
use tracing::info;
#[derive(Deserialize, Clone)]
struct AppConfig {
database: DatabaseConfig,
api_key: String,
feature_flags: FeatureFlags,
}
#[derive(Deserialize, Clone)]
struct DatabaseConfig {
host: String,
port: u16,
username: String,
password: String,
database: String,
}
#[derive(Deserialize, Clone)]
struct FeatureFlags {
new_feature_enabled: bool,
max_batch_size: usize,
}
static CONFIG: OnceLock<AppConfig> = OnceLock::new();
async fn load_config() -> Result<&'static AppConfig, Error> {
CONFIG.get_or_try_init(|| async {
let manager = Manager::new();
let env = std::env::var("ENVIRONMENT")?;
// Get database config from Secrets Manager
let db_secret = manager
.get_secret(&format!("{}/database/config", env))
.await?;
let database: DatabaseConfig = serde_json::from_str(&db_secret)?;
// Get API key from Parameter Store
let api_key = manager
.get_parameter(&format!("/{}/api-key", env))
.await?;
// Get feature flags from Parameter Store
let flags_json = manager
.get_parameter(&format!("/{}/feature-flags", env))
.await?;
let feature_flags: FeatureFlags = serde_json::from_str(&flags_json)?;
Ok(AppConfig {
database,
api_key,
feature_flags,
})
}).await
}
#[tokio::main]
async fn main() -> Result<(), Error> {
tracing_subscriber::fmt::init();
// Load configuration at startup
info!("Loading configuration...");
load_config().await?;
info!("Configuration loaded successfully");
run(service_fn(function_handler)).await
}
async fn function_handler(event: LambdaEvent<Request>) -> Result<Response, Error> {
let config = CONFIG.get().unwrap();
info!("Processing request with feature flags: {:?}", config.feature_flags);
// Use configuration
let db = connect_to_database(&config.database).await?;
let api_client = ApiClient::new(&config.api_key);
// Your business logic here
Ok(Response { success: true })
}
```
## Security Checklist
- [ ] Use Secrets Manager for sensitive data (passwords, keys)
- [ ] Use Parameter Store for configuration
- [ ] Never log secret values
- [ ] Use IAM policies to restrict access
- [ ] Enable encryption at rest (KMS)
- [ ] Use separate secrets per environment
- [ ] Implement secret rotation
- [ ] Validate secret format at startup
- [ ] Cache secrets to reduce API calls
- [ ] Use extension layer for production
- [ ] Set appropriate TTL for cached secrets
- [ ] Monitor secret access in CloudTrail
- [ ] Use least privilege IAM permissions
Guide the user through implementing secure secrets management appropriate for their needs.