Initial commit
This commit is contained in:
Zhongwei Li
545
docs/INTERNAL_FLOW_ANALYSIS.md
Normal file
545
docs/INTERNAL_FLOW_ANALYSIS.md
Normal file
@@ -0,0 +1,545 @@
|
||||
# Agent-Skill-Creator Internal Flow: What Happens "Under the Hood"
|
||||
|
||||
## 🎯 **Example Scenario**
|
||||
|
||||
**User Command:**
|
||||
```
|
||||
"I'd like to automate what is being explained and described in this article [financial data analysis article content]"
|
||||
```
|
||||
|
||||
## 🚀 **Complete Detailed Flow**
|
||||
|
||||
### **PHASE 0: Detection and Automatic Activation**
|
||||
|
||||
#### **0.1 User Intent Analysis**
|
||||
Claude Code analyzes the command and detects activation patterns:
|
||||
|
||||
```
|
||||
DETECTED PATTERNS:
|
||||
✅ "automate" → Workflow automation activation
|
||||
✅ "what is being explained" → External content processing
|
||||
✅ "in this article" → Transcribed/intent processing
|
||||
✅ Complete command → Activates Agent-Skill-Creator
|
||||
```
|
||||
|
||||
#### **0.2 Meta-Skill Loading**
|
||||
```python
|
||||
# Claude Code internal system
|
||||
if matches_pattern(user_input, SKILL_ACTIVATION_PATTERNS):
|
||||
load_skill("agent-creator-en-v2")
|
||||
activate_5_phase_process(user_input)
|
||||
```
|
||||
|
||||
**What happens:**
|
||||
- The agent-creator's `SKILL.md` is loaded into memory
|
||||
- The skill context is prepared
|
||||
- The 5 phases are initialized
|
||||
|
||||
---
|
||||
|
||||
### **PHASE 1: DISCOVERY - Research and Analysis**
|
||||
|
||||
#### **1.1 Article Content Processing**
|
||||
```python
|
||||
# Internal processing simulation
|
||||
def analyze_article_content(article_text):
|
||||
# Structured information extraction
|
||||
workflows = extract_workflows(article_text)
|
||||
tools_mentioned = identify_tools(article_text)
|
||||
data_sources = find_data_sources(article_text)
|
||||
complexity_assessment = estimate_complexity(article_text)
|
||||
|
||||
return {
|
||||
'workflows': workflows,
|
||||
'tools': tools_mentioned,
|
||||
'data_sources': data_sources,
|
||||
'complexity': complexity_assessment
|
||||
}
|
||||
```
|
||||
|
||||
**Practical Example - Financial Analysis Article:**
|
||||
```
|
||||
ANALYZED ARTICLE CONTENT:
|
||||
├─ Identified Workflows:
|
||||
│ ├─ "Download stock market data"
|
||||
│ ├─ "Calculate technical indicators"
|
||||
│ ├─ "Generate analysis charts"
|
||||
│ └─ "Create weekly report"
|
||||
├─ Mentioned Tools:
|
||||
│ ├─ "pandas library"
|
||||
│ ├─ "Alpha Vantage API"
|
||||
│ ├─ "Matplotlib for charts"
|
||||
│ └─ "Excel for reports"
|
||||
└─ Data Sources:
|
||||
├─ "Yahoo Finance API"
|
||||
├─ "Local CSV files"
|
||||
└─ "SQL database"
|
||||
```
|
||||
|
||||
#### **1.2 API and Tools Research**
|
||||
```bash
|
||||
# Automatic WebSearch performed by Claude
|
||||
WebSearch: "Best Python libraries for financial data analysis 2025"
|
||||
WebSearch: "Alpha Vantage API documentation Python integration"
|
||||
WebSearch: "Financial reporting automation tools Python"
|
||||
```
|
||||
|
||||
#### **1.3 AgentDB Enhancement (if available)**
|
||||
```python
|
||||
# Transparent AgentDB integration
|
||||
agentdb_insights = query_agentdb_for_patterns("financial_analysis")
|
||||
if agentdb_insights.success_rate > 0.8:
|
||||
apply_learned_patterns(agentdb_insights.patterns)
|
||||
```
|
||||
|
||||
#### **1.4 Technology Stack Decision**
|
||||
```
|
||||
TECHNICAL DECISION:
|
||||
✅ Python as primary language
|
||||
✅ pandas for data manipulation
|
||||
✅ Alpha Vantage for market data
|
||||
✅ Matplotlib/Seaborn for visualizations
|
||||
✅ ReportLab for PDF generation
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### **PHASE 2: DESIGN - Functionality Specification**
|
||||
|
||||
#### **2.1 Use Case Analysis**
|
||||
```python
|
||||
def define_use_cases(workflows_identified):
|
||||
use_cases = []
|
||||
for workflow in workflows_identified:
|
||||
use_case = {
|
||||
'name': workflow['title'],
|
||||
'description': workflow['description'],
|
||||
'inputs': workflow['required_inputs'],
|
||||
'outputs': workflow['expected_outputs'],
|
||||
'frequency': workflow['frequency'],
|
||||
'complexity': workflow['complexity_level']
|
||||
}
|
||||
use_cases.append(use_case)
|
||||
return use_cases
|
||||
```
|
||||
|
||||
**Defined Use Cases:**
|
||||
```
|
||||
USE CASE 1: Data Acquisition
|
||||
- Description: Download historical stock data
|
||||
- Input: List of tickers, period
|
||||
- Output: DataFrame with OHLCV data
|
||||
- Frequency: Daily
|
||||
|
||||
USE CASE 2: Technical Analysis
|
||||
- Description: Calculate technical indicators
|
||||
- Input: Price DataFrame
|
||||
- Output: DataFrame with indicators
|
||||
- Frequency: On demand
|
||||
|
||||
USE CASE 3: Report Generation
|
||||
- Description: Create PDF report
|
||||
- Input: Analysis results
|
||||
- Output: Formatted report
|
||||
- Frequency: Weekly
|
||||
```
|
||||
|
||||
#### **2.2 Methodology Definition**
|
||||
```python
|
||||
def specify_methodologies(use_cases):
|
||||
methodologies = {
|
||||
'data_validation': 'Data quality validation',
|
||||
'error_handling': 'Robust error handling',
|
||||
'caching_strategy': 'Data caching for performance',
|
||||
'logging': 'Detailed logging for debugging',
|
||||
'configuration': 'Flexible configuration via JSON'
|
||||
}
|
||||
return methodologies
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### **PHASE 3: ARCHITECTURE - Structural Decision**
|
||||
|
||||
#### **3.1 Complexity Analysis (DECISION_LOGIC.md applied)**
|
||||
```python
|
||||
# Automatic evaluation based on article content
|
||||
complexity_score = calculate_complexity({
|
||||
'number_of_workflows': 4, # Data + Analysis + Reports + Alerts
|
||||
'workflow_complexity': 'medium', # API calls + calculations + formatting
|
||||
'data_sources': 3, # Yahoo Finance + CSV + Database
|
||||
'estimated_code_lines': 2500, # Above Simple Skill threshold
|
||||
'domain_expertise': ['finance', 'data_science', 'reporting']
|
||||
})
|
||||
|
||||
# Architecture decision
|
||||
if complexity_score > SIMPLE_SKILL_THRESHOLD:
|
||||
architecture = "complex_skill_suite"
|
||||
else:
|
||||
architecture = "simple_skill"
|
||||
```
|
||||
|
||||
**In this example:**
|
||||
```
|
||||
ANALYSIS RESULT:
|
||||
✅ Multiple distinct workflows (4)
|
||||
✅ Medium-high complexity
|
||||
✅ Multiple data sources
|
||||
✅ Estimate > 2000 lines of code
|
||||
✅ Multiple domains of expertise
|
||||
|
||||
DECISION: Complex Skill Suite
|
||||
GENERATED NAME: financial-analysis-suite-cskill
|
||||
```
|
||||
|
||||
#### **3.2 Component Structure Definition**
|
||||
```python
|
||||
def design_component_skills(complexity_analysis):
|
||||
if complexity_analysis.architecture == "complex_skill_suite":
|
||||
components = {
|
||||
'data-acquisition': 'Handle data sourcing and validation',
|
||||
'technical-analysis': 'Calculate indicators and signals',
|
||||
'visualization': 'Create charts and graphs',
|
||||
'reporting': 'Generate professional reports'
|
||||
}
|
||||
return components
|
||||
```
|
||||
|
||||
#### **3.3 Performance and Cache Planning**
|
||||
```python
|
||||
performance_plan = {
|
||||
'data_cache': 'Cache market data for 1 day',
|
||||
'calculation_cache': 'Cache expensive calculations',
|
||||
'parallel_processing': 'Process multiple stocks concurrently',
|
||||
'batch_operations': 'Batch API calls when possible'
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### **PHASE 4: DETECTION - Keywords and Activation**
|
||||
|
||||
#### **4.1 Keyword Analysis**
|
||||
```python
|
||||
def determine_activation_keywords(workflows, tools):
|
||||
keywords = {
|
||||
'primary': [
|
||||
'financial analysis',
|
||||
'market data',
|
||||
'technical indicators',
|
||||
'investment reports'
|
||||
],
|
||||
'secondary': [
|
||||
'automate analysis',
|
||||
'generate charts',
|
||||
'calculate returns',
|
||||
'data extraction'
|
||||
],
|
||||
'domains': [
|
||||
'finance',
|
||||
'investments',
|
||||
'quantitative analysis',
|
||||
'stock market'
|
||||
]
|
||||
}
|
||||
return keywords
|
||||
```
|
||||
|
||||
#### **4.2 Precise Description Creation**
|
||||
```python
|
||||
def create_skill_descriptions(components):
|
||||
descriptions = {}
|
||||
for component_name, component_function in components.items():
|
||||
description = f"""
|
||||
Component skill for {component_function} in financial analysis.
|
||||
|
||||
When to use: When user mentions {determine_activation_keywords(component_name)}
|
||||
|
||||
Capabilities: {list_component_capabilities(component_name)}
|
||||
"""
|
||||
descriptions[component_name] = description
|
||||
return descriptions
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### **PHASE 5: IMPLEMENTATION - Code Creation**
|
||||
|
||||
#### **5.1 Directory Structure Creation**
|
||||
```bash
|
||||
# Automatically created by the system
|
||||
mkdir -p financial-analysis-suite/.claude-plugin
|
||||
mkdir -p financial-analysis-suite/data-acquisition/{scripts,references,assets}
|
||||
mkdir -p financial-analysis-suite/technical-analysis/{scripts,references,assets}
|
||||
mkdir -p financial-analysis-suite/visualization/{scripts,references,assets}
|
||||
mkdir -p financial-analysis-suite/reporting/{scripts,references,assets}
|
||||
mkdir -p financial-analysis-suite/shared/{utils,config,templates}
|
||||
```
|
||||
|
||||
#### **5.2 marketplace.json Generation**
|
||||
```json
|
||||
{
|
||||
"name": "financial-analysis-suite",
|
||||
"plugins": [
|
||||
{
|
||||
"name": "data-acquisition",
|
||||
"source": "./data-acquisition/",
|
||||
"skills": ["./SKILL.md"]
|
||||
},
|
||||
{
|
||||
"name": "technical-analysis",
|
||||
"source": "./technical-analysis/",
|
||||
"skills": ["./SKILL.md"]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
#### **5.3 SKILL.md Files Creation**
|
||||
For each component, the system generates:
|
||||
|
||||
```markdown
|
||||
---
|
||||
name: data-acquisition
|
||||
description: Component skill for acquiring financial market data from multiple sources including APIs, CSV files, and real-time feeds.
|
||||
---
|
||||
|
||||
# Financial Data Acquisition
|
||||
|
||||
This component skill handles all data acquisition needs for the financial analysis suite.
|
||||
|
||||
## When to Use This Component Skill
|
||||
Use this skill when you need to:
|
||||
- Download market data from APIs (Alpha Vantage, Yahoo Finance)
|
||||
- Import data from CSV/Excel files
|
||||
- Validate and clean financial data
|
||||
- Store data in standardized format
|
||||
```
|
||||
|
||||
#### **5.4 Python Scripts Generation**
|
||||
```python
|
||||
# data-acquisition/scripts/fetch_data.py
|
||||
import pandas as pd
|
||||
import yfinance as yf
|
||||
from datetime import datetime, timedelta
|
||||
|
||||
class FinancialDataFetcher:
|
||||
def __init__(self, config_file='config/data_sources.json'):
|
||||
self.config = self.load_config(config_file)
|
||||
|
||||
def fetch_stock_data(self, tickers, period='1y'):
|
||||
"""Fetch historical stock data for given tickers"""
|
||||
data = {}
|
||||
for ticker in tickers:
|
||||
try:
|
||||
stock = yf.Ticker(ticker)
|
||||
data[ticker] = stock.history(period=period)
|
||||
logger.info(f"Successfully fetched data for {ticker}")
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to fetch data for {ticker}: {e}")
|
||||
return data
|
||||
|
||||
def validate_data_quality(self, data):
|
||||
"""Validate the quality of fetched data"""
|
||||
validation_results = {}
|
||||
for ticker, df in data.items():
|
||||
results = {
|
||||
'missing_data_pct': df.isnull().sum().sum() / df.size,
|
||||
'date_range': f"{df.index.min()} to {df.index.max()}",
|
||||
'completeness': len(df) / 252 # Trading days
|
||||
}
|
||||
validation_results[ticker] = results
|
||||
return validation_results
|
||||
```
|
||||
|
||||
#### **5.5 Configuration Files Creation**
|
||||
```json
|
||||
// shared/config/data_sources.json
|
||||
{
|
||||
"api_keys": {
|
||||
"alpha_vantage": "YOUR_API_KEY_HERE",
|
||||
"quandl": "YOUR_QUANDL_KEY_HERE"
|
||||
},
|
||||
"data_sources": {
|
||||
"yahoo_finance": {
|
||||
"enabled": true,
|
||||
"rate_limit": 2000,
|
||||
"timeout": 30
|
||||
},
|
||||
"alpha_vantage": {
|
||||
"enabled": true,
|
||||
"rate_limit": 5,
|
||||
"timeout": 60
|
||||
}
|
||||
},
|
||||
"cache_settings": {
|
||||
"market_data_ttl": 86400,
|
||||
"calculations_ttl": 3600
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### **5.6 Documentation and References Generation**
|
||||
```markdown
|
||||
# data-acquisition/references/api_documentation.md
|
||||
|
||||
## Alpha Vantage API
|
||||
- **Endpoint**: TIME_SERIES_DAILY
|
||||
- **Rate Limit**: 5 calls/minute (free tier)
|
||||
- **Data Format**: JSON
|
||||
- **Authentication**: API Key in header
|
||||
|
||||
## Yahoo Finance (yfinance)
|
||||
- **Library**: yfinance
|
||||
- **Rate Limit**: None documented (be reasonable)
|
||||
- **Data Format**: pandas DataFrame
|
||||
- **Authentication**: None required
|
||||
```
|
||||
|
||||
#### **5.7 Main README Creation**
|
||||
```markdown
|
||||
# Financial Analysis Suite
|
||||
|
||||
Complete automated financial analysis system that processes market data, performs technical analysis, and generates professional investment reports.
|
||||
|
||||
## Quick Start
|
||||
1. Install dependencies: `pip install -r requirements.txt`
|
||||
2. Configure API keys in `shared/config/data_sources.json`
|
||||
3. Install as Claude plugin: `/plugin marketplace add ./`
|
||||
4. Use: "Analyze AAPL, MSFT, GOOG performance and create weekly report"
|
||||
|
||||
## Components
|
||||
- **Data Acquisition**: Automated market data collection
|
||||
- **Technical Analysis**: Indicator calculations and signal generation
|
||||
- **Visualization**: Chart creation and trend analysis
|
||||
- **Reporting**: Professional PDF report generation
|
||||
```
|
||||
|
||||
#### **5.8 Automatic Installation Test**
|
||||
```python
|
||||
# scripts/test_installation.py
|
||||
def test_suite_installation():
|
||||
"""Test that all components work correctly"""
|
||||
print("🧪 Testing Financial Analysis Suite installation...")
|
||||
|
||||
# Test imports
|
||||
try:
|
||||
import pandas as pd
|
||||
import yfinance as yf
|
||||
import matplotlib.pyplot as plt
|
||||
print("✅ All dependencies imported successfully")
|
||||
except ImportError as e:
|
||||
print(f"❌ Missing dependency: {e}")
|
||||
return False
|
||||
|
||||
# Test configuration
|
||||
try:
|
||||
with open('shared/config/data_sources.json') as f:
|
||||
config = json.load(f)
|
||||
print("✅ Configuration file loaded successfully")
|
||||
except FileNotFoundError:
|
||||
print("❌ Configuration file missing")
|
||||
return False
|
||||
|
||||
# Test basic functionality
|
||||
try:
|
||||
test_data = yf.download('AAPL', period='1mo')
|
||||
if not test_data.empty:
|
||||
print("✅ Basic data fetching works")
|
||||
else:
|
||||
print("❌ Data fetching failed")
|
||||
return False
|
||||
except Exception as e:
|
||||
print(f"❌ Basic functionality test failed: {e}")
|
||||
return False
|
||||
|
||||
print("🎉 All tests passed! Suite is ready to use.")
|
||||
return True
|
||||
|
||||
if __name__ == "__main__":
|
||||
test_suite_installation()
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🎯 **Final Result - What the User Receives**
|
||||
|
||||
After approximately **45-90 minutes** of autonomous processing, the user will have:
|
||||
|
||||
```
|
||||
financial-analysis-suite-cskill/
|
||||
├── .claude-plugin/
|
||||
│ └── marketplace.json ← Suite manifest
|
||||
├── data-acquisition-cskill/
|
||||
│ ├── SKILL.md ← Component skill 1
|
||||
│ ├── scripts/
|
||||
│ │ ├── fetch_data.py ← Functional code
|
||||
│ │ ├── validate_data.py ← Validation
|
||||
│ │ └── cache_manager.py ← Cache
|
||||
│ ├── references/
|
||||
│ │ └── api_documentation.md ← Documentation
|
||||
│ └── assets/
|
||||
├── technical-analysis-cskill/
|
||||
│ ├── SKILL.md ← Component skill 2
|
||||
│ ├── scripts/
|
||||
│ │ ├── indicators.py ← Technical calculations
|
||||
│ │ ├── signals.py ← Signal generation
|
||||
│ │ └── backtester.py ← Historical tests
|
||||
│ └── references/
|
||||
├── visualization-cskill/
|
||||
│ ├── SKILL.md ← Component skill 3
|
||||
│ └── scripts/chart_generator.py
|
||||
├── reporting-cskill/
|
||||
│ ├── SKILL.md ← Component skill 4
|
||||
│ └── scripts/report_generator.py
|
||||
├── shared/
|
||||
│ ├── utils/
|
||||
│ ├── config/
|
||||
│ └── templates/
|
||||
├── requirements.txt ← Python dependencies
|
||||
├── README.md ← User guide
|
||||
├── DECISIONS.md ← Decision explanations
|
||||
└── test_installation.py ← Automatic test
|
||||
```
|
||||
|
||||
**Note:** All components use the "-cskill" convention to identify that they were created by Agent-Skill-Creator.
|
||||
|
||||
## 🚀 **How to Use the Created Skill**
|
||||
|
||||
**Immediately after creation:**
|
||||
```bash
|
||||
# Install the suite
|
||||
cd financial-analysis-suite
|
||||
/plugin marketplace add ./
|
||||
|
||||
# Use the components
|
||||
"Analyze technical indicators for AAPL using the data acquisition and technical analysis components"
|
||||
|
||||
"Generate a comprehensive financial report for portfolio [MSFT, GOOGL, TSLA]"
|
||||
|
||||
"Compare performance of tech stocks using the analysis suite"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🧠 **Intelligence Behind the Process**
|
||||
|
||||
### **What Makes This Possible:**
|
||||
|
||||
1. **Semantic Understanding**: Claude understands the article's content, not just keywords
|
||||
2. **Structured Extraction**: Identifies workflows, tools, and patterns
|
||||
3. **Autonomous Decision-Making**: Chooses the appropriate architecture without human intervention
|
||||
4. **Functional Generation**: Creates code that actually works, not templates
|
||||
5. **Continuous Learning**: With AgentDB, improves with each creation
|
||||
|
||||
### **Differential Compared to Simple Approaches:**
|
||||
|
||||
| Simple Approach | Agent-Skill-Creator |
|
||||
|------------------|---------------------|
|
||||
| Generates templates | Creates functional code |
|
||||
| Requires programming | Fully autonomous |
|
||||
| No architecture decision | Architecture intelligence |
|
||||
| Basic documentation | Complete documentation |
|
||||
| Manual testing | Automatic testing |
|
||||
|
||||
**Agent-Skill-Creator transforms articles and descriptions into fully functional, production-ready Claude Code skills!** 🎉
|
||||
Reference in New Issue
Block a user