Initial commit
This commit is contained in:
478
skills/tools-and-apis/policyengine-api-skill/SKILL.md
Normal file
478
skills/tools-and-apis/policyengine-api-skill/SKILL.md
Normal file
@@ -0,0 +1,478 @@
|
||||
---
|
||||
name: policyengine-api
|
||||
description: PolicyEngine API - Flask REST service powering policyengine.org and programmatic access
|
||||
---
|
||||
|
||||
# PolicyEngine API
|
||||
|
||||
The PolicyEngine API is a Flask-based REST service that provides tax and benefit calculations for the web app and programmatic users.
|
||||
|
||||
## For Users 👥
|
||||
|
||||
### What is the API?
|
||||
|
||||
When you use policyengine.org, the API processes your calculations on our servers.
|
||||
|
||||
**API base:** https://api.policyengine.org
|
||||
|
||||
**What it does:**
|
||||
- Runs tax and benefit calculations
|
||||
- Stores and retrieves policy reforms
|
||||
- Computes population-wide impacts
|
||||
- Serves parameter and variable metadata
|
||||
|
||||
### Public Access
|
||||
|
||||
The API is publicly accessible with rate limits:
|
||||
- **Unauthenticated:** 100 requests/minute
|
||||
- **Authenticated:** 1,000 requests/minute
|
||||
|
||||
**Try it:**
|
||||
```bash
|
||||
curl https://api.policyengine.org/us/policy/2
|
||||
```
|
||||
|
||||
### API Documentation
|
||||
|
||||
**OpenAPI spec:** https://api.policyengine.org/docs
|
||||
|
||||
**Interactive docs:** Swagger UI at API docs endpoint
|
||||
|
||||
## For Analysts 📊
|
||||
|
||||
### Using the API
|
||||
|
||||
**Option 1: Python client (recommended)**
|
||||
```python
|
||||
# Use the policyengine package
|
||||
# See policyengine-python-client-skill
|
||||
```
|
||||
|
||||
**Option 2: Direct API calls**
|
||||
```python
|
||||
import requests
|
||||
|
||||
# Calculate household impact
|
||||
response = requests.post(
|
||||
"https://api.policyengine.org/us/calculate",
|
||||
json={
|
||||
"household": household_situation,
|
||||
"policy_id": None # or reform_id
|
||||
}
|
||||
)
|
||||
result = response.json()
|
||||
```
|
||||
|
||||
### Key Endpoints
|
||||
|
||||
**Household calculations:**
|
||||
```
|
||||
POST /us/calculate
|
||||
POST /uk/calculate
|
||||
```
|
||||
|
||||
**Policy management:**
|
||||
```
|
||||
GET /us/policy/{policy_id}
|
||||
POST /us/policy
|
||||
```
|
||||
|
||||
**Economy impacts:**
|
||||
```
|
||||
GET /us/economy/{policy_id}/over/{baseline_policy_id}
|
||||
```
|
||||
|
||||
**Metadata:**
|
||||
```
|
||||
GET /us/parameters
|
||||
GET /us/variables
|
||||
GET /us/parameter/{parameter_name}
|
||||
GET /us/variable/{variable_name}
|
||||
```
|
||||
|
||||
### Rate Limits and Performance
|
||||
|
||||
**Rate limits:**
|
||||
- 100 req/min (unauthenticated)
|
||||
- 1,000 req/min (authenticated - contact team)
|
||||
|
||||
**Response times:**
|
||||
- Household calculation: ~200-500ms
|
||||
- Population impact: ~5-30 seconds
|
||||
- Cached results: <100ms
|
||||
|
||||
**Optimization:**
|
||||
- Use the same policy_id for multiple requests (caching)
|
||||
- Batch calculations when possible
|
||||
- Use webhooks for long-running jobs (population impacts)
|
||||
|
||||
## For Contributors 💻
|
||||
|
||||
### Repository
|
||||
|
||||
**Location:** PolicyEngine/policyengine-api
|
||||
|
||||
**Clone:**
|
||||
```bash
|
||||
git clone https://github.com/PolicyEngine/policyengine-api
|
||||
cd policyengine-api
|
||||
```
|
||||
|
||||
### Current Architecture
|
||||
|
||||
**To see current structure:**
|
||||
```bash
|
||||
tree policyengine_api/
|
||||
|
||||
# Key directories:
|
||||
ls policyengine_api/
|
||||
# - endpoints/ - HTTP endpoint handlers
|
||||
# - routes/ - Route registration
|
||||
# - services/ - Business logic
|
||||
# - compute_api/ - Calculation services
|
||||
# - economy_api/ - Economy impact calculations
|
||||
# - utils/ - Helpers (caching, validation)
|
||||
# - data/ - Static data
|
||||
```
|
||||
|
||||
### Current Implementation Patterns
|
||||
|
||||
**Reference endpoint (read this first):**
|
||||
```bash
|
||||
cat policyengine_api/endpoints/economy.py
|
||||
```
|
||||
|
||||
**This demonstrates:**
|
||||
- Standard endpoint structure
|
||||
- Request validation
|
||||
- Caching pattern
|
||||
- Error handling
|
||||
- Response formatting
|
||||
|
||||
**To find other endpoints:**
|
||||
```bash
|
||||
ls policyengine_api/endpoints/
|
||||
# - household.py
|
||||
# - policy.py
|
||||
# - economy.py
|
||||
# - metadata.py
|
||||
# - etc.
|
||||
```
|
||||
|
||||
### Standard Endpoint Pattern (Stable)
|
||||
|
||||
```python
|
||||
from flask import Blueprint, request, jsonify
|
||||
from policyengine_api.utils import cache
|
||||
|
||||
blueprint = Blueprint("my_endpoint", __name__)
|
||||
|
||||
@blueprint.route("/us/calculate", methods=["POST"])
|
||||
def calculate():
|
||||
"""Standard pattern: validate, cache-check, compute, cache, return."""
|
||||
try:
|
||||
# 1. Get and validate input
|
||||
data = request.json
|
||||
if not data:
|
||||
return jsonify({"error": "No data provided"}), 400
|
||||
|
||||
# 2. Generate cache key
|
||||
cache_key = f"calc_{hash(str(data))}"
|
||||
|
||||
# 3. Check cache
|
||||
cached = cache.get(cache_key)
|
||||
if cached:
|
||||
return jsonify(cached)
|
||||
|
||||
# 4. Compute
|
||||
result = perform_calculation(data)
|
||||
|
||||
# 5. Cache result
|
||||
cache.set(cache_key, result, expire=3600)
|
||||
|
||||
# 6. Return
|
||||
return jsonify(result)
|
||||
|
||||
except Exception as e:
|
||||
return jsonify({"error": str(e), "status": "error"}), 500
|
||||
```
|
||||
|
||||
**Current implementation details:**
|
||||
```bash
|
||||
# See actual endpoint for current pattern
|
||||
cat policyengine_api/endpoints/household.py
|
||||
```
|
||||
|
||||
### Caching Strategy
|
||||
|
||||
**To see current caching implementation:**
|
||||
```bash
|
||||
# Redis configuration
|
||||
cat policyengine_api/utils/cache.py
|
||||
|
||||
# Find cache usage
|
||||
grep -r "cache\." policyengine_api/endpoints/
|
||||
```
|
||||
|
||||
**Pattern:**
|
||||
- Redis for caching
|
||||
- Cache keys based on inputs
|
||||
- TTL varies by endpoint (1 hour to 1 day)
|
||||
- Clear cache on parameter changes
|
||||
|
||||
### Background Jobs
|
||||
|
||||
For long-running calculations (population impacts):
|
||||
|
||||
**To see current implementation:**
|
||||
```bash
|
||||
# RQ (Redis Queue) usage
|
||||
grep -r "@job" policyengine_api/
|
||||
|
||||
# Job patterns
|
||||
cat policyengine_api/economy_api/
|
||||
```
|
||||
|
||||
**Pattern:**
|
||||
- Use RQ for jobs > 5 seconds
|
||||
- Return job_id immediately
|
||||
- Poll for completion
|
||||
- Cache results
|
||||
|
||||
### Country Integration
|
||||
|
||||
**How API loads country packages:**
|
||||
```bash
|
||||
cat policyengine_api/country.py
|
||||
```
|
||||
|
||||
**Pattern:**
|
||||
- Dynamically imports country packages
|
||||
- Routes by country code (/us/, /uk/)
|
||||
- Manages multiple model versions
|
||||
|
||||
### Service Layer
|
||||
|
||||
**Business logic separated from endpoints:**
|
||||
```bash
|
||||
ls policyengine_api/services/
|
||||
```
|
||||
|
||||
**Pattern:**
|
||||
```python
|
||||
# endpoints/household.py
|
||||
from policyengine_api.services import household_service
|
||||
|
||||
@app.route("/us/calculate", methods=["POST"])
|
||||
def calculate():
|
||||
result = household_service.calculate(data)
|
||||
return jsonify(result)
|
||||
|
||||
# services/household_service.py
|
||||
def calculate(data):
|
||||
# Business logic here
|
||||
simulation = create_simulation(data)
|
||||
return simulation.calculate(...)
|
||||
```
|
||||
|
||||
### Testing
|
||||
|
||||
**To see current test patterns:**
|
||||
```bash
|
||||
ls tests/
|
||||
cat tests/test_household.py
|
||||
```
|
||||
|
||||
**Run tests:**
|
||||
```bash
|
||||
make test
|
||||
|
||||
# Specific test
|
||||
pytest tests/test_economy.py -v
|
||||
|
||||
# With coverage
|
||||
make test-coverage
|
||||
```
|
||||
|
||||
### Development Server
|
||||
|
||||
**Start locally:**
|
||||
```bash
|
||||
make debug
|
||||
```
|
||||
|
||||
**Test endpoint:**
|
||||
```bash
|
||||
curl http://localhost:5000/us/policy/2
|
||||
```
|
||||
|
||||
### Deployment
|
||||
|
||||
**To see deployment configuration:**
|
||||
```bash
|
||||
# Google Cloud Platform
|
||||
cat app.yaml # App Engine config
|
||||
cat cloudbuild.yaml # Cloud Build config
|
||||
|
||||
# Environment variables
|
||||
cat .env.example
|
||||
```
|
||||
|
||||
**Current deployment:**
|
||||
- Google App Engine
|
||||
- Cloud SQL (PostgreSQL)
|
||||
- Redis (caching)
|
||||
- Cloud Build (CI/CD)
|
||||
|
||||
### API Versions
|
||||
|
||||
**To see versioning strategy:**
|
||||
```bash
|
||||
grep -r "version" policyengine_api/
|
||||
```
|
||||
|
||||
**Current approach:**
|
||||
- API version in URLs (may add /v1/ prefix)
|
||||
- Country package versions independent
|
||||
- Breaking changes rare (backwards compatible)
|
||||
|
||||
## Architecture Diagrams
|
||||
|
||||
### Request Flow
|
||||
|
||||
```
|
||||
User/App → API Gateway → Flask App → Country Package → Core Engine
|
||||
↓
|
||||
Redis Cache
|
||||
↓
|
||||
Background Job (if needed)
|
||||
↓
|
||||
PostgreSQL (storage)
|
||||
```
|
||||
|
||||
### Dependencies
|
||||
|
||||
```
|
||||
policyengine-core
|
||||
↓
|
||||
policyengine-us, policyengine-uk, etc.
|
||||
↓
|
||||
policyengine-api (you are here)
|
||||
↓
|
||||
policyengine-app (consumes API)
|
||||
```
|
||||
|
||||
**To understand dependencies:**
|
||||
- See `policyengine-core-skill` for engine patterns
|
||||
- See `policyengine-us-skill` for country model usage
|
||||
- See `policyengine-app-skill` for how app calls API
|
||||
|
||||
## Common Development Tasks
|
||||
|
||||
### Task 1: Add New Endpoint
|
||||
|
||||
1. **Study reference implementation:**
|
||||
```bash
|
||||
cat policyengine_api/endpoints/economy.py
|
||||
```
|
||||
|
||||
2. **Create new endpoint file:**
|
||||
```python
|
||||
# policyengine_api/endpoints/my_endpoint.py
|
||||
# Follow the pattern from economy.py
|
||||
```
|
||||
|
||||
3. **Register route:**
|
||||
```bash
|
||||
# See route registration
|
||||
cat policyengine_api/routes/__init__.py
|
||||
```
|
||||
|
||||
4. **Add tests:**
|
||||
```bash
|
||||
# Follow test pattern
|
||||
cat tests/test_economy.py
|
||||
```
|
||||
|
||||
### Task 2: Modify Caching Behavior
|
||||
|
||||
**See current caching:**
|
||||
```bash
|
||||
cat policyengine_api/utils/cache.py
|
||||
```
|
||||
|
||||
**Common changes:**
|
||||
- Adjust TTL (time to live)
|
||||
- Change cache key generation
|
||||
- Add cache invalidation
|
||||
|
||||
### Task 3: Update Country Package Version
|
||||
|
||||
**To see how versions are managed:**
|
||||
```bash
|
||||
# Requirements
|
||||
cat requirements.txt | grep policyengine-
|
||||
|
||||
# Update and deploy
|
||||
# See deployment docs in README
|
||||
```
|
||||
|
||||
## Security and Best Practices
|
||||
|
||||
### Input Validation
|
||||
|
||||
**Always validate:**
|
||||
- Country code (us, uk, ca)
|
||||
- Policy ID format
|
||||
- Household structure
|
||||
- Parameter values
|
||||
|
||||
**See validation examples:**
|
||||
```bash
|
||||
grep -r "validate" policyengine_api/endpoints/
|
||||
```
|
||||
|
||||
### Error Handling
|
||||
|
||||
**Standard error response:**
|
||||
```python
|
||||
return jsonify({
|
||||
"error": "Error message",
|
||||
"details": additional_context,
|
||||
"status": "error"
|
||||
}), status_code
|
||||
```
|
||||
|
||||
**See error patterns:**
|
||||
```bash
|
||||
grep -A 5 "jsonify.*error" policyengine_api/endpoints/
|
||||
```
|
||||
|
||||
### Logging
|
||||
|
||||
**To see logging configuration:**
|
||||
```bash
|
||||
cat policyengine_api/gcp_logging.py
|
||||
```
|
||||
|
||||
**Pattern:**
|
||||
- Google Cloud Logging
|
||||
- Log all errors
|
||||
- Log slow queries (>1s)
|
||||
- Don't log sensitive data
|
||||
|
||||
## Related Skills
|
||||
|
||||
- **policyengine-python-client-skill** - Using the API
|
||||
- **policyengine-core-skill** - Understanding the engine
|
||||
- **policyengine-us-skill** - Country model integration
|
||||
- **policyengine-app-skill** - How app consumes API
|
||||
- **policyengine-standards-skill** - Code quality
|
||||
- **policyengine-writing-skill** - API documentation style
|
||||
|
||||
## Resources
|
||||
|
||||
**Repository:** https://github.com/PolicyEngine/policyengine-api
|
||||
**Live API:** https://api.policyengine.org
|
||||
**Documentation:** https://api.policyengine.org/docs
|
||||
**Status:** https://status.policyengine.org
|
||||
900
skills/tools-and-apis/policyengine-app-skill/SKILL.md
Normal file
900
skills/tools-and-apis/policyengine-app-skill/SKILL.md
Normal file
@@ -0,0 +1,900 @@
|
||||
---
|
||||
name: policyengine-app
|
||||
description: PolicyEngine React web application - the user interface at policyengine.org
|
||||
---
|
||||
|
||||
# PolicyEngine App
|
||||
|
||||
The PolicyEngine App is the React-based web application that users interact with at policyengine.org.
|
||||
|
||||
## For Users 👥
|
||||
|
||||
### What is the App?
|
||||
|
||||
The app at policyengine.org provides:
|
||||
- Interactive household calculator
|
||||
- Policy reform creator
|
||||
- Population impact analysis
|
||||
- Blog and research hub
|
||||
|
||||
**Access:** https://policyengine.org
|
||||
|
||||
### App Features
|
||||
|
||||
**Calculator:**
|
||||
- Enter household details
|
||||
- See tax and benefit calculations
|
||||
- Visualize marginal tax rates
|
||||
- Compare scenarios
|
||||
|
||||
**Policy designer:**
|
||||
- Browse all parameters
|
||||
- Create custom reforms
|
||||
- Share via URL
|
||||
- Download charts
|
||||
|
||||
**Research hub:**
|
||||
- Read policy analysis
|
||||
- Explore modeled programs
|
||||
- Access documentation
|
||||
|
||||
## For Analysts 📊
|
||||
|
||||
### Understanding App URLs
|
||||
|
||||
Reform URLs encode all policy changes in the query string, allowing sharing and reproducibility.
|
||||
|
||||
**Example URL:**
|
||||
```
|
||||
policyengine.org/us/policy?
|
||||
focus=policyOutput.policyBreakdown&
|
||||
reform=67696&
|
||||
region=enhanced_us&
|
||||
timePeriod=2025&
|
||||
baseline=2
|
||||
```
|
||||
|
||||
**Parameters:**
|
||||
- `focus` - Which section to display
|
||||
- `reform` - Reform ID from database
|
||||
- `region` - Geographic scope (enhanced_us, CA, congressional districts)
|
||||
- `timePeriod` - Year of analysis
|
||||
- `baseline` - Baseline policy ID
|
||||
|
||||
### Embedding PolicyEngine
|
||||
|
||||
**iFrame integration:**
|
||||
```html
|
||||
<iframe
|
||||
src="https://policyengine.org/us/household?embedded=true"
|
||||
width="100%"
|
||||
height="800">
|
||||
</iframe>
|
||||
```
|
||||
|
||||
**Parameter:**
|
||||
- `embedded=true` - Removes navigation, optimizes for embedding
|
||||
|
||||
### URL Structure
|
||||
|
||||
**Household calculator:**
|
||||
```
|
||||
/us/household?household=12345
|
||||
/uk/household?household=67890
|
||||
```
|
||||
|
||||
**Policy page:**
|
||||
```
|
||||
/us/policy?reform=12345
|
||||
/uk/policy?reform=67890
|
||||
```
|
||||
|
||||
**Research/blog:**
|
||||
```
|
||||
/us/research/article-slug
|
||||
/uk/research/article-slug
|
||||
```
|
||||
|
||||
## For Contributors 💻
|
||||
|
||||
### Repository
|
||||
|
||||
**Location:** PolicyEngine/policyengine-app
|
||||
|
||||
**Clone:**
|
||||
```bash
|
||||
git clone https://github.com/PolicyEngine/policyengine-app
|
||||
cd policyengine-app
|
||||
```
|
||||
|
||||
### Current Architecture
|
||||
|
||||
**To see current structure:**
|
||||
```bash
|
||||
tree src/ -L 2
|
||||
|
||||
# Key directories:
|
||||
ls src/
|
||||
# - pages/ - Page components
|
||||
# - applets/ - Reusable UI modules
|
||||
# - api/ - API integration
|
||||
# - controls/ - Form controls
|
||||
# - layout/ - Layout components
|
||||
# - posts/ - Blog posts
|
||||
# - routing/ - Routing configuration
|
||||
# - hooks/ - Custom React hooks
|
||||
# - data/ - Static data
|
||||
```
|
||||
|
||||
### Technology Stack
|
||||
|
||||
**Current dependencies:**
|
||||
```bash
|
||||
# See package.json for versions
|
||||
cat package.json
|
||||
|
||||
# Key dependencies:
|
||||
# - React 18
|
||||
# - React Router v6
|
||||
# - Plotly.js
|
||||
# - Ant Design
|
||||
# - axios
|
||||
```
|
||||
|
||||
### React Patterns (Critical)
|
||||
|
||||
**✅ Functional components only (no classes):**
|
||||
```javascript
|
||||
// CORRECT
|
||||
import { useState, useEffect } from "react";
|
||||
|
||||
export default function TaxCalculator({ income }) {
|
||||
const [tax, setTax] = useState(0);
|
||||
|
||||
useEffect(() => {
|
||||
calculateTax(income).then(setTax);
|
||||
}, [income]);
|
||||
|
||||
return <div>Tax: ${tax}</div>;
|
||||
}
|
||||
```
|
||||
|
||||
**❌ Class components forbidden:**
|
||||
```javascript
|
||||
// WRONG - Don't use class components
|
||||
class TaxCalculator extends Component {
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
**To find component examples:**
|
||||
```bash
|
||||
# Reference components
|
||||
ls src/pages/
|
||||
ls src/applets/
|
||||
|
||||
# See a complete page
|
||||
cat src/pages/HouseholdPage.jsx
|
||||
```
|
||||
|
||||
### State Management
|
||||
|
||||
**No global state (Redux, Context) - lift state up:**
|
||||
|
||||
```javascript
|
||||
// Parent manages state
|
||||
function PolicyPage() {
|
||||
const [reform, setReform] = useState({});
|
||||
const [impact, setImpact] = useState(null);
|
||||
|
||||
return (
|
||||
<>
|
||||
<PolicyEditor reform={reform} onChange={setReform} />
|
||||
<ImpactDisplay impact={impact} />
|
||||
</>
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
**To see state patterns:**
|
||||
```bash
|
||||
# Find useState usage
|
||||
grep -r "useState" src/pages/ | head -20
|
||||
```
|
||||
|
||||
### API Integration
|
||||
|
||||
**To see current API patterns:**
|
||||
```bash
|
||||
cat src/api/call.js # Base API caller
|
||||
cat src/api/variables.js # Variable metadata
|
||||
cat src/api/parameters.js # Parameter metadata
|
||||
```
|
||||
|
||||
**Standard pattern:**
|
||||
```javascript
|
||||
import { api call } from "api/call";
|
||||
|
||||
// Fetch data
|
||||
const result = await call(
|
||||
`/us/calculate`,
|
||||
{ household: householdData },
|
||||
"POST"
|
||||
);
|
||||
```
|
||||
|
||||
### Routing
|
||||
|
||||
**To see current routing:**
|
||||
```bash
|
||||
cat src/routing/routes.js
|
||||
|
||||
# Routes defined with React Router v6
|
||||
# See examples:
|
||||
grep -r "useNavigate" src/
|
||||
grep -r "useSearchParams" src/
|
||||
```
|
||||
|
||||
**URL parameters:**
|
||||
```javascript
|
||||
import { useSearchParams } from "react-router-dom";
|
||||
|
||||
const [searchParams, setSearchParams] = useSearchParams();
|
||||
|
||||
// Read
|
||||
const reformId = searchParams.get("reform");
|
||||
|
||||
// Update
|
||||
setSearchParams({ ...Object.fromEntries(searchParams), reform: newId });
|
||||
```
|
||||
|
||||
### Custom Hooks
|
||||
|
||||
**To see PolicyEngine-specific hooks:**
|
||||
```bash
|
||||
ls src/hooks/
|
||||
# - useCountryId.js - Current country
|
||||
# - useDisplayCategory.js
|
||||
# - etc.
|
||||
```
|
||||
|
||||
**Usage:**
|
||||
```javascript
|
||||
import { useCountryId } from "hooks/useCountryId";
|
||||
|
||||
function Component() {
|
||||
const [countryId, setCountryId] = useCountryId();
|
||||
// countryId = "us", "uk", or "ca"
|
||||
}
|
||||
```
|
||||
|
||||
### Charts and Visualization
|
||||
|
||||
**Plotly integration:**
|
||||
```bash
|
||||
# See chart components
|
||||
ls src/pages/policy/output/
|
||||
|
||||
# Reference implementation
|
||||
cat src/pages/policy/output/EconomyOutput.jsx
|
||||
```
|
||||
|
||||
**Standard Plotly pattern:**
|
||||
```javascript
|
||||
import Plot from "react-plotly.js";
|
||||
|
||||
const layout = {
|
||||
font: { family: "Roboto Serif" },
|
||||
plot_bgcolor: "white",
|
||||
// PolicyEngine branding
|
||||
};
|
||||
|
||||
<Plot
|
||||
data={traces}
|
||||
layout={layout}
|
||||
config={{ displayModeBar: false }}
|
||||
/>;
|
||||
```
|
||||
|
||||
### Blog Posts
|
||||
|
||||
**To see blog post structure:**
|
||||
```bash
|
||||
ls src/posts/articles/
|
||||
|
||||
# Read a recent post
|
||||
cat src/posts/articles/harris-eitc.md
|
||||
```
|
||||
|
||||
**Blog posts:**
|
||||
- Written in Markdown
|
||||
- Stored in `src/posts/articles/`
|
||||
- Include metadata (title, date, authors)
|
||||
- Follow policyengine-writing-skill style
|
||||
|
||||
**Adding a post:**
|
||||
```bash
|
||||
# Create new file
|
||||
# src/posts/articles/my-analysis.md
|
||||
|
||||
# Add to index (if needed)
|
||||
# See existing posts for format
|
||||
```
|
||||
|
||||
### Styling
|
||||
|
||||
**Current styling approach:**
|
||||
```bash
|
||||
# See style configuration
|
||||
ls src/style/
|
||||
|
||||
# Colors
|
||||
cat src/style/colors.js
|
||||
|
||||
# Ant Design theme
|
||||
cat src/style/theme.js
|
||||
```
|
||||
|
||||
**PolicyEngine colors:**
|
||||
- Teal: `#39C6C0` (primary accent)
|
||||
- Blue: `#2C6496` (charts, links)
|
||||
- Dark gray: `#616161` (text)
|
||||
|
||||
### Testing
|
||||
|
||||
**To see current tests:**
|
||||
```bash
|
||||
ls src/__tests__/
|
||||
|
||||
# Run tests
|
||||
make test
|
||||
|
||||
# Test pattern
|
||||
cat src/__tests__/example.test.js
|
||||
```
|
||||
|
||||
**Testing libraries:**
|
||||
- Jest (test runner)
|
||||
- React Testing Library (component testing)
|
||||
- User-centric testing (not implementation details)
|
||||
|
||||
### Development Server
|
||||
|
||||
**Start locally:**
|
||||
```bash
|
||||
make debug
|
||||
# Opens http://localhost:3000
|
||||
```
|
||||
|
||||
**Environment:**
|
||||
```bash
|
||||
# Environment variables
|
||||
cat .env.example
|
||||
|
||||
# Config
|
||||
ls src/config/
|
||||
```
|
||||
|
||||
### Building and Deployment
|
||||
|
||||
**Build:**
|
||||
```bash
|
||||
make build
|
||||
# Creates optimized production build
|
||||
```
|
||||
|
||||
**Deployment:**
|
||||
```bash
|
||||
# See deployment config
|
||||
cat netlify.toml # or appropriate hosting config
|
||||
```
|
||||
|
||||
## Component Patterns
|
||||
|
||||
### Standard Component Structure
|
||||
|
||||
**To see well-structured components:**
|
||||
```bash
|
||||
# Example page
|
||||
cat src/pages/HouseholdPage.jsx
|
||||
|
||||
# Example applet
|
||||
cat src/applets/PolicySearch.jsx
|
||||
```
|
||||
|
||||
**Pattern:**
|
||||
```javascript
|
||||
import { useState, useEffect } from "react";
|
||||
import { useSearchParams } from "react-router-dom";
|
||||
import { useCountryId } from "hooks/useCountryId";
|
||||
|
||||
export default function MyComponent({ prop1, prop2 }) {
|
||||
// 1. Hooks first
|
||||
const [state, setState] = useState(initialValue);
|
||||
const [countryId] = useCountryId();
|
||||
const [searchParams] = useSearchParams();
|
||||
|
||||
// 2. Effects
|
||||
useEffect(() => {
|
||||
// Side effects
|
||||
}, [dependencies]);
|
||||
|
||||
// 3. Event handlers
|
||||
const handleClick = () => {
|
||||
setState(newValue);
|
||||
};
|
||||
|
||||
// 4. Render
|
||||
return (
|
||||
<div>
|
||||
{/* JSX */}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
### Component Size Limit
|
||||
|
||||
**Keep components under 150 lines after formatting.**
|
||||
|
||||
**If component is too large:**
|
||||
1. Extract sub-components
|
||||
2. Move logic to custom hooks
|
||||
3. Split into multiple files
|
||||
|
||||
**To find large components:**
|
||||
```bash
|
||||
# Find files >150 lines
|
||||
find src/ -name "*.jsx" -exec wc -l {} \; | sort -rn | head -20
|
||||
```
|
||||
|
||||
### File Naming
|
||||
|
||||
**Components:** PascalCase.jsx
|
||||
- `HouseholdPage.jsx`
|
||||
- `PolicySearch.jsx`
|
||||
- `ImpactChart.jsx`
|
||||
|
||||
**Utilities:** camelCase.js
|
||||
- `formatCurrency.js`
|
||||
- `apiUtils.js`
|
||||
- `chartHelpers.js`
|
||||
|
||||
**Hooks:** camelCase.js with 'use' prefix
|
||||
- `useCountryId.js`
|
||||
- `usePolicy.js`
|
||||
|
||||
## Common Development Tasks
|
||||
|
||||
### Task 1: Add New Page
|
||||
|
||||
1. **See page structure:**
|
||||
```bash
|
||||
cat src/pages/HouseholdPage.jsx
|
||||
```
|
||||
|
||||
2. **Create new page:**
|
||||
```javascript
|
||||
// src/pages/MyNewPage.jsx
|
||||
export default function MyNewPage() {
|
||||
return <div>Content</div>;
|
||||
}
|
||||
```
|
||||
|
||||
3. **Add route:**
|
||||
```bash
|
||||
# See routing
|
||||
cat src/routing/routes.js
|
||||
|
||||
# Add your route following the pattern
|
||||
```
|
||||
|
||||
### Task 2: Add New Chart
|
||||
|
||||
1. **See chart examples:**
|
||||
```bash
|
||||
ls src/pages/policy/output/
|
||||
cat src/pages/policy/output/DistributionalImpact.jsx
|
||||
```
|
||||
|
||||
2. **Create chart component:**
|
||||
```javascript
|
||||
import Plot from "react-plotly.js";
|
||||
|
||||
export default function MyChart({ data }) {
|
||||
return (
|
||||
<Plot
|
||||
data={traces}
|
||||
layout={{
|
||||
font: { family: "Roboto Serif" },
|
||||
plot_bgcolor: "white"
|
||||
}}
|
||||
/>
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
### Task 3: Add Blog Post
|
||||
|
||||
1. **See post structure:**
|
||||
```bash
|
||||
cat src/posts/articles/harris-eitc.md
|
||||
```
|
||||
|
||||
2. **Create post:**
|
||||
```bash
|
||||
# Create markdown file
|
||||
# src/posts/articles/my-analysis.md
|
||||
|
||||
# Follow policyengine-writing-skill for style
|
||||
```
|
||||
|
||||
3. **Images:**
|
||||
```bash
|
||||
# Store in public/images/posts/
|
||||
# Reference in markdown
|
||||
```
|
||||
|
||||
## API Integration Patterns
|
||||
|
||||
### Fetching Data
|
||||
|
||||
**To see API call patterns:**
|
||||
```bash
|
||||
cat src/api/call.js
|
||||
```
|
||||
|
||||
**Standard pattern:**
|
||||
```javascript
|
||||
import { call } from "api/call";
|
||||
|
||||
const fetchData = async () => {
|
||||
const result = await call(
|
||||
`/us/calculate`,
|
||||
{ household: data },
|
||||
"POST"
|
||||
);
|
||||
return result;
|
||||
};
|
||||
```
|
||||
|
||||
### Loading States
|
||||
|
||||
**Pattern:**
|
||||
```javascript
|
||||
const [loading, setLoading] = useState(false);
|
||||
const [error, setError] = useState(null);
|
||||
const [data, setData] = useState(null);
|
||||
|
||||
useEffect(() => {
|
||||
setLoading(true);
|
||||
fetchData()
|
||||
.then(setData)
|
||||
.catch(setError)
|
||||
.finally(() => setLoading(false));
|
||||
}, [dependencies]);
|
||||
|
||||
if (loading) return <Spin />;
|
||||
if (error) return <Error message={error} />;
|
||||
return <Data data={data} />;
|
||||
```
|
||||
|
||||
## Performance Patterns
|
||||
|
||||
### Code Splitting
|
||||
|
||||
**To see code splitting:**
|
||||
```bash
|
||||
grep -r "React.lazy" src/
|
||||
```
|
||||
|
||||
**Pattern:**
|
||||
```javascript
|
||||
import { lazy, Suspense } from "react";
|
||||
|
||||
const HeavyComponent = lazy(() => import("./HeavyComponent"));
|
||||
|
||||
function Page() {
|
||||
return (
|
||||
<Suspense fallback={<Spin />}>
|
||||
<HeavyComponent />
|
||||
</Suspense>
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
### Memoization
|
||||
|
||||
**Use React.memo for expensive components:**
|
||||
```javascript
|
||||
import { memo } from "react";
|
||||
|
||||
const ExpensiveChart = memo(function ExpensiveChart({ data }) {
|
||||
// Only re-renders if data changes
|
||||
return <Plot data={data} />;
|
||||
});
|
||||
```
|
||||
|
||||
## Accessibility
|
||||
|
||||
**Requirements:**
|
||||
- Semantic HTML elements
|
||||
- ARIA labels for complex widgets
|
||||
- Keyboard navigation
|
||||
- Color contrast (WCAG AA)
|
||||
|
||||
**To see accessibility patterns:**
|
||||
```bash
|
||||
grep -r "aria-" src/
|
||||
grep -r "role=" src/
|
||||
```
|
||||
|
||||
## Country-Specific Features
|
||||
|
||||
### Country Switching
|
||||
|
||||
**To see country switching:**
|
||||
```bash
|
||||
cat src/hooks/useCountryId.js
|
||||
```
|
||||
|
||||
**Usage:**
|
||||
```javascript
|
||||
import { useCountryId } from "hooks/useCountryId";
|
||||
|
||||
function Component() {
|
||||
const [countryId] = useCountryId(); // "us", "uk", or "ca"
|
||||
|
||||
// Load country-specific data
|
||||
const data = countryId === "us" ? usData : ukData;
|
||||
}
|
||||
```
|
||||
|
||||
### Country-Specific Content
|
||||
|
||||
**Conditional rendering:**
|
||||
```javascript
|
||||
{countryId === "us" && <USSpecificComponent />}
|
||||
{countryId === "uk" && <UKSpecificComponent />}
|
||||
```
|
||||
|
||||
**To find country-specific code:**
|
||||
```bash
|
||||
grep -r "countryId ===" src/
|
||||
```
|
||||
|
||||
## Development Workflow
|
||||
|
||||
### Local Development
|
||||
|
||||
**Start dev server:**
|
||||
```bash
|
||||
make debug
|
||||
# App runs on http://localhost:3000
|
||||
# Connects to production API by default
|
||||
```
|
||||
|
||||
**Connect to local API:**
|
||||
```bash
|
||||
# See environment configuration
|
||||
cat src/config/environment.js
|
||||
|
||||
# Or set environment variable
|
||||
REACT_APP_API_URL=http://localhost:5000 make debug
|
||||
```
|
||||
|
||||
### Testing
|
||||
|
||||
**Run tests:**
|
||||
```bash
|
||||
make test
|
||||
```
|
||||
|
||||
**Watch mode:**
|
||||
```bash
|
||||
npm test -- --watch
|
||||
```
|
||||
|
||||
**Coverage:**
|
||||
```bash
|
||||
npm test -- --coverage
|
||||
```
|
||||
|
||||
### Linting and Formatting
|
||||
|
||||
**Format code (critical before committing):**
|
||||
```bash
|
||||
make format
|
||||
|
||||
# Or manually
|
||||
npm run lint -- --fix
|
||||
npx prettier --write .
|
||||
```
|
||||
|
||||
**Check linting (CI check):**
|
||||
```bash
|
||||
npm run lint -- --max-warnings=0
|
||||
```
|
||||
|
||||
## Current Implementation Reference
|
||||
|
||||
### Component Structure
|
||||
|
||||
**To see current page structure:**
|
||||
```bash
|
||||
ls src/pages/
|
||||
# - HouseholdPage.jsx
|
||||
# - PolicyPage.jsx
|
||||
# - HomePage.jsx
|
||||
# - etc.
|
||||
```
|
||||
|
||||
**To see a complete page:**
|
||||
```bash
|
||||
cat src/pages/PolicyPage.jsx
|
||||
```
|
||||
|
||||
### API Call Patterns
|
||||
|
||||
**To see current API integration:**
|
||||
```bash
|
||||
cat src/api/call.js # Base caller
|
||||
cat src/api/variables.js # Variable metadata fetching
|
||||
cat src/api/parameters.js # Parameter metadata fetching
|
||||
```
|
||||
|
||||
### Routing Configuration
|
||||
|
||||
**To see current routes:**
|
||||
```bash
|
||||
cat src/routing/routes.js
|
||||
```
|
||||
|
||||
### Form Controls
|
||||
|
||||
**To see PolicyEngine-specific form controls:**
|
||||
```bash
|
||||
ls src/controls/
|
||||
# - InputField.jsx
|
||||
# - SearchParamControl.jsx
|
||||
# - etc.
|
||||
```
|
||||
|
||||
### Chart Components
|
||||
|
||||
**To see chart implementations:**
|
||||
```bash
|
||||
ls src/pages/policy/output/
|
||||
# - BudgetaryImpact.jsx
|
||||
# - DistributionalImpact.jsx
|
||||
# - PovertyImpact.jsx
|
||||
# - etc.
|
||||
```
|
||||
|
||||
**Reference chart:**
|
||||
```bash
|
||||
cat src/pages/policy/output/DistributionalImpact.jsx
|
||||
```
|
||||
|
||||
## Multi-Repository Integration
|
||||
|
||||
### How App Relates to Other Repos
|
||||
|
||||
```
|
||||
policyengine-core (engine)
|
||||
↓
|
||||
policyengine-us, policyengine-uk (country models)
|
||||
↓
|
||||
policyengine-api (backend)
|
||||
↓
|
||||
policyengine-app (you are here)
|
||||
```
|
||||
|
||||
**Understanding the stack:**
|
||||
- See `policyengine-core-skill` for engine concepts
|
||||
- See `policyengine-us-skill` for what variables/parameters mean
|
||||
- See `policyengine-api-skill` for API endpoints the app calls
|
||||
|
||||
### Blog Posts Reference Country Models
|
||||
|
||||
**Blog posts often reference variables:**
|
||||
```bash
|
||||
# Posts reference variables like "income_tax", "ctc"
|
||||
# See policyengine-us-skill for variable definitions
|
||||
cat src/posts/articles/harris-eitc.md
|
||||
```
|
||||
|
||||
## Common Development Tasks
|
||||
|
||||
### Task 1: Add New Parameter to UI
|
||||
|
||||
1. **Understand parameter:**
|
||||
```bash
|
||||
# See parameter in country model
|
||||
cd ../policyengine-us
|
||||
cat policyengine_us/parameters/gov/irs/credits/ctc/amount/base_amount.yaml
|
||||
```
|
||||
|
||||
2. **Find similar parameter in app:**
|
||||
```bash
|
||||
cd ../policyengine-app
|
||||
grep -r "ctc.*amount" src/pages/policy/
|
||||
```
|
||||
|
||||
3. **Add UI control following pattern**
|
||||
|
||||
### Task 2: Add New Chart
|
||||
|
||||
1. **See existing charts:**
|
||||
```bash
|
||||
cat src/pages/policy/output/DistributionalImpact.jsx
|
||||
```
|
||||
|
||||
2. **Create new chart component**
|
||||
|
||||
3. **Add to policy output page**
|
||||
|
||||
### Task 3: Fix Bug in Calculator
|
||||
|
||||
1. **Find relevant component:**
|
||||
```bash
|
||||
# Search for the feature
|
||||
grep -r "keyword" src/pages/
|
||||
```
|
||||
|
||||
2. **Read component code**
|
||||
|
||||
3. **Make fix following React patterns**
|
||||
|
||||
4. **Test with dev server:**
|
||||
```bash
|
||||
make debug
|
||||
```
|
||||
|
||||
## Build and Deployment
|
||||
|
||||
**Production build:**
|
||||
```bash
|
||||
make build
|
||||
# Creates optimized bundle in build/
|
||||
```
|
||||
|
||||
**Deployment:**
|
||||
```bash
|
||||
# See deployment configuration
|
||||
cat netlify.toml # or appropriate config
|
||||
```
|
||||
|
||||
**Environment variables:**
|
||||
```bash
|
||||
# React env vars must have REACT_APP_ prefix
|
||||
REACT_APP_API_URL=https://api.policyengine.org
|
||||
|
||||
# Or use config file pattern (recommended)
|
||||
cat src/config/environment.js
|
||||
```
|
||||
|
||||
## Style Guide
|
||||
|
||||
**Follow policyengine-standards-skill for:**
|
||||
- ESLint configuration
|
||||
- Prettier formatting
|
||||
- Component size limits
|
||||
- File organization
|
||||
|
||||
**Follow policyengine-writing-skill for:**
|
||||
- Blog post content
|
||||
- Documentation
|
||||
- UI copy
|
||||
|
||||
## Resources
|
||||
|
||||
**Repository:** https://github.com/PolicyEngine/policyengine-app
|
||||
**Live app:** https://policyengine.org
|
||||
**Staging:** https://staging.policyengine.org (if applicable)
|
||||
|
||||
**Related skills:**
|
||||
- **policyengine-api-skill** - Understanding the backend
|
||||
- **policyengine-us-skill** - Understanding variables/parameters
|
||||
- **policyengine-writing-skill** - Blog post style
|
||||
- **policyengine-standards-skill** - Code quality
|
||||
487
skills/tools-and-apis/policyengine-core-skill/SKILL.md
Normal file
487
skills/tools-and-apis/policyengine-core-skill/SKILL.md
Normal file
@@ -0,0 +1,487 @@
|
||||
---
|
||||
name: policyengine-core
|
||||
description: PolicyEngine Core simulation engine - the foundation powering all PolicyEngine calculations
|
||||
---
|
||||
|
||||
# PolicyEngine Core
|
||||
|
||||
PolicyEngine Core is the microsimulation engine that powers all PolicyEngine calculations. It's a fork of OpenFisca-Core adapted for PolicyEngine's needs.
|
||||
|
||||
## For Users 👥
|
||||
|
||||
### What is Core?
|
||||
|
||||
When you use policyengine.org to calculate taxes or benefits, PolicyEngine Core is the "calculator" running behind the scenes.
|
||||
|
||||
**Core provides:**
|
||||
- The simulation engine that processes tax rules
|
||||
- Variable and parameter management
|
||||
- Entity relationships (person → family → household)
|
||||
- Period handling (2024, 2025, etc.)
|
||||
|
||||
You don't interact with Core directly - you use it through:
|
||||
- **Web app:** policyengine.org
|
||||
- **Python packages:** policyengine-us, policyengine-uk
|
||||
- **API:** api.policyengine.org
|
||||
|
||||
### Why Core Matters
|
||||
|
||||
Core ensures:
|
||||
- ✅ **Accuracy** - Calculations follow official rules exactly
|
||||
- ✅ **Consistency** - Same rules applied everywhere
|
||||
- ✅ **Transparency** - All rules traceable to legislation
|
||||
- ✅ **Performance** - Vectorized calculations for speed
|
||||
|
||||
## For Analysts 📊
|
||||
|
||||
### Understanding Core Concepts
|
||||
|
||||
When writing PolicyEngine code, you'll encounter Core concepts:
|
||||
|
||||
**Variables:**
|
||||
- Represent quantities (income_tax, ctc, snap, etc.)
|
||||
- Defined for specific entities (person, household, tax_unit)
|
||||
- Calculated from formulas or set directly
|
||||
|
||||
**Parameters:**
|
||||
- Policy rules that change over time (tax rates, benefit amounts)
|
||||
- Organized hierarchically (gov.irs.credits.ctc.amount.base_amount)
|
||||
- Stored in YAML files
|
||||
|
||||
**Entities:**
|
||||
- Person: Individual
|
||||
- Family: Family unit
|
||||
- Tax unit: Tax filing unit
|
||||
- Household: Physical household
|
||||
- Marital unit: Marital status grouping
|
||||
- SPM unit: Supplemental Poverty Measure unit
|
||||
|
||||
**Periods:**
|
||||
- Year: 2024, 2025, etc.
|
||||
- Month: 2024-01, 2024-02, etc.
|
||||
- Specific dates: 2024-06-15
|
||||
|
||||
### Core in Action
|
||||
|
||||
```python
|
||||
from policyengine_us import Simulation
|
||||
|
||||
# When you create a simulation
|
||||
sim = Simulation(situation=household)
|
||||
|
||||
# Core manages:
|
||||
# - Entity relationships
|
||||
# - Variable dependencies
|
||||
# - Parameter lookups
|
||||
# - Period conversions
|
||||
|
||||
# When you calculate
|
||||
result = sim.calculate("income_tax", 2024)
|
||||
|
||||
# Core:
|
||||
# 1. Checks if already calculated
|
||||
# 2. Identifies dependencies (income → AGI → taxable income → tax)
|
||||
# 3. Calculates dependencies first
|
||||
# 4. Applies formulas
|
||||
# 5. Returns result
|
||||
```
|
||||
|
||||
### Core vs Country Packages
|
||||
|
||||
**Core (policyengine-core):**
|
||||
- Generic simulation engine
|
||||
- No specific tax/benefit rules
|
||||
- Variable and parameter infrastructure
|
||||
|
||||
**Country packages (policyengine-us, etc.):**
|
||||
- Built on Core
|
||||
- Contain specific tax/benefit rules
|
||||
- Define variables and parameters for that country
|
||||
|
||||
**Relationship:**
|
||||
```
|
||||
policyengine-core (engine)
|
||||
↓ powers
|
||||
policyengine-us (US rules)
|
||||
↓ used by
|
||||
policyengine-api (REST API)
|
||||
↓ serves
|
||||
policyengine-app (web interface)
|
||||
```
|
||||
|
||||
## For Contributors 💻
|
||||
|
||||
### Repository
|
||||
|
||||
**Location:** PolicyEngine/policyengine-core
|
||||
**Origin:** Fork of OpenFisca-Core
|
||||
|
||||
**Clone:**
|
||||
```bash
|
||||
git clone https://github.com/PolicyEngine/policyengine-core
|
||||
```
|
||||
|
||||
### Current Architecture
|
||||
|
||||
**To see current structure:**
|
||||
```bash
|
||||
tree policyengine_core/
|
||||
|
||||
# Key directories:
|
||||
# - variables/ - Variable class and infrastructure
|
||||
# - parameters/ - Parameter class and infrastructure
|
||||
# - entities/ - Entity definitions
|
||||
# - simulations/ - Simulation class
|
||||
# - periods/ - Period handling
|
||||
# - reforms/ - Reform application
|
||||
```
|
||||
|
||||
**To understand a specific component:**
|
||||
```bash
|
||||
# Variable system
|
||||
cat policyengine_core/variables/variable.py
|
||||
|
||||
# Parameter system
|
||||
cat policyengine_core/parameters/parameter.py
|
||||
|
||||
# Simulation engine
|
||||
cat policyengine_core/simulations/simulation.py
|
||||
|
||||
# Entity system
|
||||
cat policyengine_core/entities/entity.py
|
||||
```
|
||||
|
||||
### Key Classes
|
||||
|
||||
**Variable:**
|
||||
```python
|
||||
# To see Variable class implementation
|
||||
cat policyengine_core/variables/variable.py
|
||||
|
||||
# Variables in country packages inherit from this:
|
||||
from policyengine_core.variables import Variable
|
||||
|
||||
class income_tax(Variable):
|
||||
value_type = float
|
||||
entity = Person
|
||||
label = "Income tax"
|
||||
definition_period = YEAR
|
||||
|
||||
def formula(person, period, parameters):
|
||||
# Vectorized formula
|
||||
return calculate_tax(...)
|
||||
```
|
||||
|
||||
**Simulation:**
|
||||
```python
|
||||
# To see Simulation class implementation
|
||||
cat policyengine_core/simulations/simulation.py
|
||||
|
||||
# Manages calculation graph and caching
|
||||
sim = Simulation(situation=situation)
|
||||
sim.calculate("variable", period)
|
||||
```
|
||||
|
||||
**Parameters:**
|
||||
```python
|
||||
# To see Parameter handling
|
||||
cat policyengine_core/parameters/parameter_node.py
|
||||
|
||||
# Access in formulas:
|
||||
parameters(period).gov.irs.credits.ctc.amount.base_amount
|
||||
```
|
||||
|
||||
### Vectorization (Critical!)
|
||||
|
||||
Core requires vectorized operations - no if-elif-else with arrays:
|
||||
|
||||
**❌ Wrong (scalar logic):**
|
||||
```python
|
||||
if age < 18:
|
||||
eligible = True
|
||||
else:
|
||||
eligible = False
|
||||
```
|
||||
|
||||
**✅ Correct (vectorized):**
|
||||
```python
|
||||
eligible = age < 18 # NumPy boolean array
|
||||
```
|
||||
|
||||
**Why:** Core processes many households simultaneously for performance.
|
||||
|
||||
**To see vectorization examples:**
|
||||
```bash
|
||||
# Search for where() usage (vectorized if-then-else)
|
||||
grep -r "np.where" policyengine_core/
|
||||
|
||||
# Find select() usage (vectorized case statements)
|
||||
grep -r "select" policyengine_core/
|
||||
```
|
||||
|
||||
### Formula Dependencies
|
||||
|
||||
Core automatically resolves variable dependencies:
|
||||
|
||||
```python
|
||||
class taxable_income(Variable):
|
||||
def formula(person, period, parameters):
|
||||
# Core automatically calculates these first:
|
||||
agi = person("adjusted_gross_income", period)
|
||||
deduction = person("standard_deduction", period)
|
||||
return agi - deduction
|
||||
|
||||
class income_tax(Variable):
|
||||
def formula(person, period, parameters):
|
||||
# Core knows to calculate taxable_income first
|
||||
taxable = person("taxable_income", period)
|
||||
return apply_brackets(taxable, ...)
|
||||
```
|
||||
|
||||
**To see dependency resolution:**
|
||||
```bash
|
||||
# Find trace functionality
|
||||
grep -r "trace" policyengine_core/simulations/
|
||||
|
||||
# Enable in your code:
|
||||
simulation.trace = True
|
||||
simulation.calculate("income_tax", 2024)
|
||||
```
|
||||
|
||||
### Period Handling
|
||||
|
||||
**To see period implementation:**
|
||||
```bash
|
||||
cat policyengine_core/periods/period.py
|
||||
|
||||
# Period types:
|
||||
# - YEAR: 2024
|
||||
# - MONTH: 2024-01
|
||||
# - ETERNITY: permanent values
|
||||
```
|
||||
|
||||
**Usage in variables:**
|
||||
```python
|
||||
# Annual variable
|
||||
definition_period = YEAR # Called with 2024
|
||||
|
||||
# Monthly variable
|
||||
definition_period = MONTH # Called with "2024-01"
|
||||
|
||||
# Convert periods
|
||||
yearly_value = person("monthly_income", period.this_year) * 12
|
||||
```
|
||||
|
||||
### Testing Core Changes
|
||||
|
||||
**To run Core tests:**
|
||||
```bash
|
||||
cd policyengine-core
|
||||
make test
|
||||
|
||||
# Specific test
|
||||
pytest tests/core/test_variables.py -v
|
||||
```
|
||||
|
||||
**To test in country package:**
|
||||
```bash
|
||||
# Changes to Core affect all country packages
|
||||
cd policyengine-us
|
||||
pip install -e ../policyengine-core # Local development install
|
||||
make test
|
||||
```
|
||||
|
||||
### Key Differences from OpenFisca
|
||||
|
||||
PolicyEngine Core differs from OpenFisca-Core:
|
||||
|
||||
**To see PolicyEngine changes:**
|
||||
```bash
|
||||
# Compare to OpenFisca
|
||||
# Core fork diverged to add:
|
||||
# - Enhanced performance
|
||||
# - Better error messages
|
||||
# - PolicyEngine-specific features
|
||||
|
||||
# See commit history for PolicyEngine changes
|
||||
git log --oneline
|
||||
```
|
||||
|
||||
## Core Development Workflow
|
||||
|
||||
### Making Changes to Core
|
||||
|
||||
1. **Clone repo:**
|
||||
```bash
|
||||
git clone https://github.com/PolicyEngine/policyengine-core
|
||||
```
|
||||
|
||||
2. **Install for development:**
|
||||
```bash
|
||||
make install
|
||||
```
|
||||
|
||||
3. **Make changes** to variable.py, simulation.py, etc.
|
||||
|
||||
4. **Test locally:**
|
||||
```bash
|
||||
make test
|
||||
```
|
||||
|
||||
5. **Test in country package:**
|
||||
```bash
|
||||
cd ../policyengine-us
|
||||
pip install -e ../policyengine-core
|
||||
make test
|
||||
```
|
||||
|
||||
6. **Format and commit:**
|
||||
```bash
|
||||
make format
|
||||
git commit -m "Description"
|
||||
```
|
||||
|
||||
### Understanding Impact
|
||||
|
||||
Changes to Core affect:
|
||||
- ✅ All country packages (US, UK, Canada, IL, NG)
|
||||
- ✅ The API
|
||||
- ✅ The web app
|
||||
- ✅ All analysis tools
|
||||
|
||||
**Critical:** Always test in multiple country packages before merging.
|
||||
|
||||
## Common Core Patterns
|
||||
|
||||
### Pattern 1: Adding a New Variable Type
|
||||
|
||||
**Current variable types:**
|
||||
```bash
|
||||
# See supported types
|
||||
grep "value_type" policyengine_core/variables/variable.py
|
||||
```
|
||||
|
||||
**Types:** int, float, bool, str, Enum, date
|
||||
|
||||
### Pattern 2: Custom Formulas
|
||||
|
||||
**Formula signature:**
|
||||
```python
|
||||
def formula(entity, period, parameters):
|
||||
# entity: Person, TaxUnit, Household, etc.
|
||||
# period: 2024, "2024-01", etc.
|
||||
# parameters: Parameter tree for period
|
||||
return calculated_value
|
||||
```
|
||||
|
||||
**To see formula examples:**
|
||||
```bash
|
||||
# Search country packages for formulas
|
||||
grep -A 10 "def formula" ../policyengine-us/policyengine_us/variables/ | head -50
|
||||
```
|
||||
|
||||
### Pattern 3: Parameter Access
|
||||
|
||||
**Accessing parameters in formulas:**
|
||||
```python
|
||||
# Navigate parameter tree
|
||||
param = parameters(period).gov.irs.credits.ctc.amount.base_amount
|
||||
|
||||
# Parameters automatically valid for period
|
||||
# No need to check dates manually
|
||||
```
|
||||
|
||||
**To see parameter structure:**
|
||||
```bash
|
||||
# Example from country package
|
||||
tree ../policyengine-us/policyengine_us/parameters/gov/
|
||||
```
|
||||
|
||||
## Advanced Topics
|
||||
|
||||
### Formula Caching
|
||||
|
||||
Core caches calculations automatically:
|
||||
```python
|
||||
# First call calculates
|
||||
tax1 = sim.calculate("income_tax", 2024)
|
||||
|
||||
# Second call returns cached value
|
||||
tax2 = sim.calculate("income_tax", 2024) # Instant
|
||||
```
|
||||
|
||||
### Neutralizing Variables
|
||||
|
||||
```python
|
||||
# Set variable to zero in reform
|
||||
reform = {
|
||||
"income_tax": {
|
||||
"2024-01-01.2100-12-31": 0
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Adding Variables
|
||||
|
||||
Country packages add variables by inheriting from Core's Variable class.
|
||||
|
||||
**See policyengine-us-skill for variable creation patterns.**
|
||||
|
||||
## Resources
|
||||
|
||||
**Repository:** https://github.com/PolicyEngine/policyengine-core
|
||||
|
||||
**Documentation:**
|
||||
- Core API docs (see README in repo)
|
||||
- OpenFisca docs (original): https://openfisca.org/doc/
|
||||
|
||||
**Related skills:**
|
||||
- **policyengine-us-skill** - Using Core through country packages
|
||||
- **policyengine-standards-skill** - Code quality standards
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
**Variable not found:**
|
||||
```python
|
||||
# Error: Variable 'income_tax' not found
|
||||
# Solution: Variable is defined in country package, not Core
|
||||
# Use policyengine-us, not policyengine-core directly
|
||||
```
|
||||
|
||||
**Scalar vs array operations:**
|
||||
```python
|
||||
# Error: truth value of array is ambiguous
|
||||
# Solution: Use np.where() instead of if-else
|
||||
# See vectorization section above
|
||||
```
|
||||
|
||||
**Period mismatch:**
|
||||
```python
|
||||
# Error: Cannot compute variable_name for period 2024-01
|
||||
# Solution: Check definition_period matches request
|
||||
# YEAR variables need YEAR periods (2024, not "2024-01")
|
||||
```
|
||||
|
||||
**To debug:**
|
||||
```python
|
||||
# Enable tracing
|
||||
sim.trace = True
|
||||
sim.calculate("variable", period)
|
||||
# See calculation dependency tree
|
||||
```
|
||||
|
||||
## Contributing to Core
|
||||
|
||||
**Before contributing:**
|
||||
1. Read Core README
|
||||
2. Understand OpenFisca architecture
|
||||
3. Test changes in multiple country packages
|
||||
4. Follow policyengine-standards-skill
|
||||
|
||||
**Development standards:**
|
||||
- Python 3.10-3.13
|
||||
- Black formatting (79-char)
|
||||
- Comprehensive tests
|
||||
- No breaking changes without discussion
|
||||
356
skills/tools-and-apis/policyengine-python-client-skill/SKILL.md
Normal file
356
skills/tools-and-apis/policyengine-python-client-skill/SKILL.md
Normal file
@@ -0,0 +1,356 @@
|
||||
---
|
||||
name: policyengine-python-client
|
||||
description: Using PolicyEngine programmatically via Python client or REST API
|
||||
---
|
||||
|
||||
# PolicyEngine Python Client
|
||||
|
||||
This skill covers programmatic access to PolicyEngine for analysts and researchers.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
# Install the Python client
|
||||
pip install policyengine
|
||||
|
||||
# Or for local development
|
||||
pip install policyengine-us # Just the US model (offline)
|
||||
```
|
||||
|
||||
## Quick Start: Python Client
|
||||
|
||||
```python
|
||||
from policyengine import Simulation
|
||||
|
||||
# Create a household
|
||||
household = {
|
||||
"people": {
|
||||
"you": {
|
||||
"age": {"2024": 30},
|
||||
"employment_income": {"2024": 50000}
|
||||
}
|
||||
},
|
||||
"households": {
|
||||
"your household": {
|
||||
"members": ["you"],
|
||||
"state_name": {"2024": "CA"}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
# Run simulation
|
||||
sim = Simulation(situation=household, country_id="us")
|
||||
income_tax = sim.calculate("income_tax", "2024")
|
||||
```
|
||||
|
||||
## For Users: Why Use Python?
|
||||
|
||||
**Web app limitations:**
|
||||
- ✅ Great for exploring policies interactively
|
||||
- ❌ Can't analyze many households at once
|
||||
- ❌ Can't automate repetitive analyses
|
||||
- ❌ Limited customization of charts
|
||||
|
||||
**Python benefits:**
|
||||
- ✅ Analyze thousands of households in batch
|
||||
- ✅ Automate regular policy analysis
|
||||
- ✅ Create custom visualizations
|
||||
- ✅ Integrate with other data sources
|
||||
- ✅ Reproducible research
|
||||
|
||||
## For Analysts: Common Workflows
|
||||
|
||||
### Workflow 1: Calculate Your Own Taxes
|
||||
|
||||
```python
|
||||
from policyengine import Simulation
|
||||
|
||||
# Your household (more complex than web app)
|
||||
household = {
|
||||
"people": {
|
||||
"you": {
|
||||
"age": {"2024": 35},
|
||||
"employment_income": {"2024": 75000},
|
||||
"qualified_dividend_income": {"2024": 5000},
|
||||
"charitable_cash_donations": {"2024": 3000}
|
||||
},
|
||||
"spouse": {
|
||||
"age": {"2024": 33},
|
||||
"employment_income": {"2024": 60000}
|
||||
},
|
||||
"child1": {"age": {"2024": 8}},
|
||||
"child2": {"age": {"2024": 5}}
|
||||
},
|
||||
# ... entities setup (see policyengine-us-skill)
|
||||
}
|
||||
|
||||
sim = Simulation(situation=household, country_id="us")
|
||||
|
||||
# Calculate specific values
|
||||
federal_income_tax = sim.calculate("income_tax", "2024")
|
||||
state_income_tax = sim.calculate("state_income_tax", "2024")
|
||||
ctc = sim.calculate("ctc", "2024")
|
||||
eitc = sim.calculate("eitc", "2024")
|
||||
|
||||
print(f"Federal income tax: ${federal_income_tax:,.0f}")
|
||||
print(f"State income tax: ${state_income_tax:,.0f}")
|
||||
print(f"Child Tax Credit: ${ctc:,.0f}")
|
||||
print(f"EITC: ${eitc:,.0f}")
|
||||
```
|
||||
|
||||
### Workflow 2: Analyze a Policy Reform
|
||||
|
||||
```python
|
||||
from policyengine import Simulation
|
||||
|
||||
# Define reform (increase CTC to $5,000)
|
||||
reform = {
|
||||
"gov.irs.credits.ctc.amount.base_amount": {
|
||||
"2024-01-01.2100-12-31": 5000
|
||||
}
|
||||
}
|
||||
|
||||
# Compare baseline vs reform
|
||||
household = create_household() # Your household definition
|
||||
|
||||
sim_baseline = Simulation(situation=household, country_id="us")
|
||||
sim_reform = Simulation(situation=household, country_id="us", reform=reform)
|
||||
|
||||
ctc_baseline = sim_baseline.calculate("ctc", "2024")
|
||||
ctc_reform = sim_reform.calculate("ctc", "2024")
|
||||
|
||||
print(f"CTC baseline: ${ctc_baseline:,.0f}")
|
||||
print(f"CTC reform: ${ctc_reform:,.0f}")
|
||||
print(f"Increase: ${ctc_reform - ctc_baseline:,.0f}")
|
||||
```
|
||||
|
||||
### Workflow 3: Batch Analysis
|
||||
|
||||
```python
|
||||
import pandas as pd
|
||||
from policyengine import Simulation
|
||||
|
||||
# Analyze multiple households
|
||||
households = [
|
||||
{"income": 30000, "children": 0},
|
||||
{"income": 50000, "children": 2},
|
||||
{"income": 100000, "children": 3},
|
||||
]
|
||||
|
||||
results = []
|
||||
for h in households:
|
||||
situation = create_household(income=h["income"], num_children=h["children"])
|
||||
sim = Simulation(situation=situation, country_id="us")
|
||||
|
||||
results.append({
|
||||
"income": h["income"],
|
||||
"children": h["children"],
|
||||
"income_tax": sim.calculate("income_tax", "2024"),
|
||||
"ctc": sim.calculate("ctc", "2024"),
|
||||
"eitc": sim.calculate("eitc", "2024")
|
||||
})
|
||||
|
||||
df = pd.DataFrame(results)
|
||||
print(df)
|
||||
```
|
||||
|
||||
## Using the REST API Directly
|
||||
|
||||
### Authentication
|
||||
|
||||
**Public access:**
|
||||
- 100 requests per minute (unauthenticated)
|
||||
- No API key needed for basic use
|
||||
|
||||
**Authenticated access:**
|
||||
- 1,000 requests per minute
|
||||
- Contact hello@policyengine.org for API key
|
||||
|
||||
### Key Endpoints
|
||||
|
||||
**Calculate household impact:**
|
||||
```python
|
||||
import requests
|
||||
|
||||
url = "https://api.policyengine.org/us/calculate"
|
||||
payload = {
|
||||
"household": household_dict,
|
||||
"policy_id": reform_id # or None for baseline
|
||||
}
|
||||
|
||||
response = requests.post(url, json=payload)
|
||||
result = response.json()
|
||||
```
|
||||
|
||||
**Get policy details:**
|
||||
```python
|
||||
# Get policy metadata
|
||||
response = requests.get("https://api.policyengine.org/us/policy/12345")
|
||||
policy = response.json()
|
||||
```
|
||||
|
||||
**Get parameter values:**
|
||||
```python
|
||||
# Get current parameter value
|
||||
response = requests.get(
|
||||
"https://api.policyengine.org/us/parameter/gov.irs.credits.ctc.amount.base_amount"
|
||||
)
|
||||
parameter = response.json()
|
||||
```
|
||||
|
||||
### For Full API Documentation
|
||||
|
||||
**OpenAPI spec:** https://api.policyengine.org/docs
|
||||
|
||||
**To explore:**
|
||||
```bash
|
||||
# View all endpoints
|
||||
curl https://api.policyengine.org/docs
|
||||
|
||||
# Test calculate endpoint
|
||||
curl -X POST https://api.policyengine.org/us/calculate \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"household": {...}}'
|
||||
```
|
||||
|
||||
## Limitations and Considerations
|
||||
|
||||
### Rate Limits
|
||||
|
||||
**Unauthenticated:**
|
||||
- 100 requests/minute
|
||||
- Good for exploratory analysis
|
||||
|
||||
**Authenticated:**
|
||||
- 1,000 requests/minute
|
||||
- Required for production use
|
||||
|
||||
### Data Privacy
|
||||
|
||||
- PolicyEngine does not store household data
|
||||
- All calculations happen server-side and are not logged
|
||||
- Reform URLs are public (don't include personal info in reforms)
|
||||
|
||||
### Performance
|
||||
|
||||
**API calls:**
|
||||
- Simple household: ~200-500ms
|
||||
- Population impact: ~5-30 seconds (varies by reform)
|
||||
- Use caching for repeated calculations
|
||||
|
||||
**Local simulation (policyengine-us):**
|
||||
- Faster for batch analysis
|
||||
- No rate limits
|
||||
- No network dependency
|
||||
- Limited to one country per package
|
||||
|
||||
## Choosing Local vs API
|
||||
|
||||
### Use Local (policyengine-us package)
|
||||
|
||||
**When:**
|
||||
- Batch analysis of many households
|
||||
- Need offline capability
|
||||
- Analyzing parameter sweeps (axes)
|
||||
- Development/testing
|
||||
|
||||
**Install:**
|
||||
```bash
|
||||
pip install policyengine-us # US only
|
||||
pip install policyengine-uk # UK only
|
||||
```
|
||||
|
||||
**Example:**
|
||||
```python
|
||||
from policyengine_us import Simulation
|
||||
|
||||
# Works offline
|
||||
sim = Simulation(situation=household)
|
||||
```
|
||||
|
||||
### Use API (policyengine or requests)
|
||||
|
||||
**When:**
|
||||
- Multi-country analysis
|
||||
- Using latest model version
|
||||
- Don't want to manage dependencies
|
||||
- Integration with web services
|
||||
|
||||
**Example:**
|
||||
```python
|
||||
import requests
|
||||
|
||||
# Requires internet
|
||||
response = requests.post("https://api.policyengine.org/us/calculate", ...)
|
||||
```
|
||||
|
||||
## For Contributors: Understanding the Client
|
||||
|
||||
**Repository:** PolicyEngine/policyengine.py
|
||||
|
||||
**To see implementation:**
|
||||
```bash
|
||||
# Clone the client
|
||||
git clone https://github.com/PolicyEngine/policyengine.py
|
||||
|
||||
# See the Simulation class
|
||||
cat policyengine/simulation.py
|
||||
|
||||
# See API integration
|
||||
cat policyengine/api.py
|
||||
```
|
||||
|
||||
**Architecture:**
|
||||
- `Simulation` class wraps API calls
|
||||
- `calculate()` method handles caching
|
||||
- Transparent fallback between API and local
|
||||
|
||||
## Advanced: Direct Country Package Usage
|
||||
|
||||
For maximum control and performance, use country packages directly:
|
||||
|
||||
```python
|
||||
from policyengine_us import Simulation
|
||||
|
||||
# Full control over situation structure
|
||||
situation = {
|
||||
# Complete situation dictionary
|
||||
# See policyengine-us-skill for patterns
|
||||
}
|
||||
|
||||
sim = Simulation(situation=situation)
|
||||
result = sim.calculate("variable_name", 2024)
|
||||
```
|
||||
|
||||
**Benefits:**
|
||||
- No API dependency
|
||||
- Faster (no network)
|
||||
- Full access to all variables
|
||||
- Use axes for parameter sweeps
|
||||
|
||||
**See policyengine-us-skill for detailed patterns.**
|
||||
|
||||
## Examples and Tutorials
|
||||
|
||||
**PolicyEngine documentation:**
|
||||
- US: https://policyengine.org/us/docs
|
||||
- UK: https://policyengine.org/uk/docs
|
||||
|
||||
**Example notebooks:**
|
||||
- Repository: PolicyEngine/analysis-notebooks
|
||||
- See policyengine-analysis-skill for analysis patterns
|
||||
|
||||
**Community examples:**
|
||||
- Blog posts: policyengine.org/us/research
|
||||
- GitHub discussions: github.com/PolicyEngine discussions
|
||||
|
||||
## Getting Help
|
||||
|
||||
**For usage questions:**
|
||||
- GitHub Discussions: https://github.com/PolicyEngine/policyengine-us/discussions
|
||||
|
||||
**For bugs:**
|
||||
- File issues in appropriate repo (policyengine-us, policyengine.py, etc.)
|
||||
|
||||
**For collaboration:**
|
||||
- Email: hello@policyengine.org
|
||||
Reference in New Issue
Block a user