zhongwei/gh-francyjglisboa-agent-skill-creator
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial commit

Browse Source
This commit is contained in:
Zhongwei Li
2025-11-29 18:27:25 +08:00
commit e18b9b4fa8
77 changed files with 35441 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

806
references/multi-intent-detection.md Normal file
View File

@@ -0,0 +1,806 @@
# Multi-Intent Detection System v1.0
**Version:** 1.0
**Purpose:** Advanced detection and handling of complex user queries with multiple intentions
**Target:** Support complex queries with 95%+ intent accuracy and proper capability routing
---
## 🎯 **Overview**
Multi-Intent Detection extends the activation system to handle complex user queries that contain multiple intentions, requiring the skill to understand and prioritize different user goals within a single request.
### **Problem Solved**
**Before:** Skills could only handle single-intent queries, failing when users expressed multiple goals or complex requirements
**After:** Skills can detect, prioritize, and handle multiple intents within a single query, routing to appropriate capabilities
---
## 🧠 **Multi-Intent Architecture**
### **Intent Classification Hierarchy**
```
Primary Intent (Main Goal)
├── Secondary Intent 1 (Sub-goal)
├── Secondary Intent 2 (Additional requirement)
├── Tertiary Intent (Context/Modifier)
└── Meta Intent (How to present results)
```
### **Intent Types**
#### **1. Primary Intents**
The main action or goal the user wants to accomplish:
- `analyze` - Analyze data or information
- `create` - Create new content or agent
- `compare` - Compare multiple items
- `monitor` - Track or watch something
- `transform` - Convert or change format
#### **2. Secondary Intents**
Additional requirements or sub-goals:
- `and_visualize` - Also create visualization
- `and_save` - Also save results
- `and_explain` - Also provide explanation
- `and_compare` - Also do comparison
- `and_alert` - Also set up alerts
#### **3. Contextual Intents**
Modifiers that affect how results should be presented:
- `quick_summary` - Brief overview
- `detailed_analysis` - In-depth analysis
- `step_by_step` - Process explanation
- `real_time` - Live/current data
- `historical` - Historical data
#### **4. Meta Intents**
How the user wants to interact:
- `just_show_me` - Direct results
- `teach_me` - Educational approach
- `help_me_decide` - Decision support
- `automate_for_me` - Automation request
---
## 🔍 **Intent Detection Algorithms**
### **Multi-Intent Parser**
```python
def parse_multiple_intents(query, skill_capabilities):
"""Parse multiple intents from a complex user query"""
# Step 1: Identify primary intent
primary_intent = extract_primary_intent(query)
# Step 2: Identify secondary intents
secondary_intents = extract_secondary_intents(query)
# Step 3: Identify contextual modifiers
contextual_intents = extract_contextual_intents(query)
# Step 4: Identify meta intent
meta_intent = extract_meta_intent(query)
# Step 5: Validate against skill capabilities
validated_intents = validate_intents_against_capabilities(
primary_intent, secondary_intents, contextual_intents, skill_capabilities
)
return {
'primary_intent': validated_intents['primary'],
'secondary_intents': validated_intents['secondary'],
'contextual_intents': validated_intents['contextual'],
'meta_intent': validated_intents['meta'],
'intent_combinations': generate_intent_combinations(validated_intents),
'confidence_scores': calculate_intent_confidence(query, validated_intents),
'execution_plan': create_execution_plan(validated_intents)
}
def extract_primary_intent(query):
"""Extract the primary intent from the query"""
intent_patterns = {
'analyze': [
r'(?i)(analyze|analysis|examine|study|evaluate|review)\s+',
r'(?i)(what\s+is|how\s+does)\s+.*\s+(perform|work|behave)',
r'(?i)(tell\s+me\s+about|explain)\s+'
],
'create': [
r'(?i)(create|build|make|generate|develop)\s+',
r'(?i)(I\s+need|I\s+want)\s+(a|an)\s+',
r'(?i)(help\s+me\s+)(create|build|make)\s+'
],
'compare': [
r'(?i)(compare|comparison|vs|versus)\s+',
r'(?i)(which\s+is\s+better|what\s+is\s+the\s+difference)\s+',
r'(?i)(rank|rating|scoring)\s+'
],
'monitor': [
r'(?i)(monitor|track|watch|observe)\s+',
r'(?i)(keep\s+an\s+eye\s+on|follow)\s+',
r'(?i)(alert\s+me\s+when|notify\s+me)\s+'
],
'transform': [
r'(?i)(convert|transform|change|turn)\s+.*\s+(into|to)\s+',
r'(?i)(format|structure|organize)\s+',
r'(?i)(extract|parse|process)\s+'
]
}
best_match = None
highest_score = 0
for intent, patterns in intent_patterns.items():
for pattern in patterns:
if re.search(pattern, query):
score = calculate_intent_match_score(query, intent, pattern)
if score > highest_score:
highest_score = score
best_match = intent
return best_match or 'unknown'
def extract_secondary_intents(query):
"""Extract secondary intents from conjunctions and phrases"""
secondary_patterns = {
'and_visualize': [
r'(?i)(and\s+)?(show|visualize|display|chart|graph)\s+',
r'(?i)(create\s+)?(visualization|chart|graph|dashboard)\s+'
],
'and_save': [
r'(?i)(and\s+)?(save|store|export|download)\s+',
r'(?i)(keep|record|archive)\s+(the\s+)?(results|data)\s+'
],
'and_explain': [
r'(?i)(and\s+)?(explain|clarify|describe|detail)\s+',
r'(?i)(what\s+does\s+this\s+mean|why\s+is\s+this)\s+'
],
'and_compare': [
r'(?i)(and\s+)?(compare|vs|versus|against)\s+',
r'(?i)(relative\s+to|compared\s+with)\s+'
],
'and_alert': [
r'(?i)(and\s+)?(alert|notify|warn)\s+(me\s+)?(when|if)\s+',
r'(?i)(set\s+up\s+)?(notification|alert)\s+'
]
}
detected_intents = []
for intent, patterns in secondary_patterns.items():
for pattern in patterns:
if re.search(pattern, query):
detected_intents.append(intent)
break
return detected_intents
def extract_contextual_intents(query):
"""Extract contextual modifiers and presentation preferences"""
contextual_patterns = {
'quick_summary': [
r'(?i)(quick|brief|short|summary|overview)\s+',
r'(?i)(just\s+the\s+highlights|key\s+points)\s+'
],
'detailed_analysis': [
r'(?i)(detailed|in-depth|comprehensive|thorough)\s+',
r'(?i)(deep\s+dive|full\s+analysis)\s+'
],
'step_by_step': [
r'(?i)(step\s+by\s+step|how\s+to|process|procedure)\s+',
r'(?i)(walk\s+me\s+through|guide\s+me)\s+'
],
'real_time': [
r'(?i)(real\s+time|live|current|now|today)\s+',
r'(?i)(right\s+now|as\s+of\s+today)\s+'
],
'historical': [
r'(?i)(historical|past|previous|last\s+year|ytd)\s+',
r'(?i)(over\s+the\s+last\s+|historically)\s+'
]
}
detected_intents = []
for intent, patterns in contextual_patterns.items():
for pattern in patterns:
if re.search(pattern, query):
detected_intents.append(intent)
break
return detected_intents
```
### **Intent Validation System**
```python
def validate_intents_against_capabilities(primary, secondary, contextual, capabilities):
"""Validate detected intents against skill capabilities"""
validated = {
'primary': None,
'secondary': [],
'contextual': [],
'meta': None,
'validation_issues': []
}
# Validate primary intent
if primary in capabilities.get('primary_intents', []):
validated['primary'] = primary
else:
validated['validation_issues'].append(
f"Primary intent '{primary}' not supported by skill"
)
# Validate secondary intents
for intent in secondary:
if intent in capabilities.get('secondary_intents', []):
validated['secondary'].append(intent)
else:
validated['validation_issues'].append(
f"Secondary intent '{intent}' not supported by skill"
)
# Validate contextual intents
for intent in contextual:
if intent in capabilities.get('contextual_intents', []):
validated['contextual'].append(intent)
else:
validated['validation_issues'].append(
f"Contextual intent '{intent}' not supported by skill"
)
# If no valid primary intent, try to find best alternative
if not validated['primary'] and secondary:
validated['primary'] = find_best_alternative_primary(primary, secondary, capabilities)
validated['validation_issues'].append(
f"Used alternative primary intent: {validated['primary']}"
)
return validated
def generate_intent_combinations(validated_intents):
"""Generate possible combinations of validated intents"""
combinations = []
primary = validated_intents['primary']
secondary = validated_intents['secondary']
contextual = validated_intents['contextual']
if primary:
# Base combination: primary only
combinations.append({
'combination_id': 'primary_only',
'intents': [primary],
'priority': 1,
'complexity': 'low'
})
# Primary + each secondary
for sec_intent in secondary:
combinations.append({
'combination_id': f'primary_{sec_intent}',
'intents': [primary, sec_intent],
'priority': 2,
'complexity': 'medium'
})
# Primary + all secondary
if len(secondary) > 1:
combinations.append({
'combination_id': 'primary_all_secondary',
'intents': [primary] + secondary,
'priority': 3,
'complexity': 'high'
})
# Add contextual modifiers
for combo in combinations:
for context in contextual:
new_combo = combo.copy()
new_combo['intents'] = combo['intents'] + [context]
new_combo['combination_id'] = f"{combo['combination_id']}_{context}"
new_combo['priority'] = combo['priority'] + 0.1
new_combo['complexity'] = increase_complexity(combo['complexity'])
combinations.append(new_combo)
# Sort by priority and complexity
combinations.sort(key=lambda x: (x['priority'], x['complexity']))
return combinations
def create_execution_plan(validated_intents):
"""Create an execution plan for handling multiple intents"""
plan = {
'steps': [],
'parallel_tasks': [],
'sequential_dependencies': [],
'estimated_complexity': 'medium',
'estimated_time': 'medium'
}
primary = validated_intents['primary']
secondary = validated_intents['secondary']
contextual = validated_intents['contextual']
if primary:
# Step 1: Execute primary intent
plan['steps'].append({
'step_id': 1,
'intent': primary,
'action': f'execute_{primary}',
'dependencies': [],
'estimated_time': 'medium'
})
# Step 2: Execute secondary intents (can be parallel if compatible)
for i, intent in enumerate(secondary):
if can_execute_parallel(primary, intent):
plan['parallel_tasks'].append({
'task_id': f'secondary_{i}',
'intent': intent,
'action': f'execute_{intent}',
'dependencies': ['step_1']
})
else:
plan['steps'].append({
'step_id': len(plan['steps']) + 1,
'intent': intent,
'action': f'execute_{intent}',
'dependencies': [f'step_{len(plan["steps"])}'],
'estimated_time': 'short'
})
# Step 3: Apply contextual modifiers
for i, intent in enumerate(contextual):
plan['steps'].append({
'step_id': len(plan['steps']) + 1,
'intent': intent,
'action': f'apply_{intent}',
'dependencies': ['step_1'] + [f'secondary_{j}' for j in range(len(secondary))],
'estimated_time': 'short'
})
# Calculate overall complexity
total_intents = 1 + len(secondary) + len(contextual)
if total_intents <= 2:
plan['estimated_complexity'] = 'low'
elif total_intents <= 4:
plan['estimated_complexity'] = 'medium'
else:
plan['estimated_complexity'] = 'high'
return plan
```
---
## 📋 **Enhanced Marketplace Configuration**
### **Multi-Intent Configuration Structure**
```json
{
"name": "skill-name",
"activation": {
"keywords": [...],
"patterns": [...],
"contextual_filters": {...},
"_comment": "NEW: Multi-intent detection (v1.0)",
"intent_hierarchy": {
"primary_intents": {
"analyze": {
"description": "Analyze data or information",
"keywords": ["analyze", "examine", "evaluate", "study"],
"required_capabilities": ["data_processing", "analysis"],
"base_confidence": 0.9
},
"compare": {
"description": "Compare multiple items",
"keywords": ["compare", "versus", "vs", "ranking"],
"required_capabilities": ["comparison", "evaluation"],
"base_confidence": 0.85
},
"monitor": {
"description": "Track or monitor data",
"keywords": ["monitor", "track", "watch", "alert"],
"required_capabilities": ["monitoring", "notification"],
"base_confidence": 0.8
}
},
"secondary_intents": {
"and_visualize": {
"description": "Also create visualization",
"keywords": ["show", "chart", "graph", "visualize"],
"required_capabilities": ["visualization"],
"compatibility": ["analyze", "compare", "monitor"],
"confidence_modifier": 0.1
},
"and_save": {
"description": "Also save results",
"keywords": ["save", "export", "download", "store"],
"required_capabilities": ["file_operations"],
"compatibility": ["analyze", "compare", "transform"],
"confidence_modifier": 0.05
},
"and_explain": {
"description": "Also provide explanation",
"keywords": ["explain", "clarify", "describe", "detail"],
"required_capabilities": ["explanation", "reporting"],
"compatibility": ["analyze", "compare", "transform"],
"confidence_modifier": 0.05
}
},
"contextual_intents": {
"quick_summary": {
"description": "Provide brief overview",
"keywords": ["quick", "summary", "brief", "overview"],
"impact": "reduce_detail",
"confidence_modifier": 0.02
},
"detailed_analysis": {
"description": "Provide in-depth analysis",
"keywords": ["detailed", "comprehensive", "thorough", "in-depth"],
"impact": "increase_detail",
"confidence_modifier": 0.03
},
"real_time": {
"description": "Use current/live data",
"keywords": ["real-time", "live", "current", "now"],
"impact": "require_live_data",
"confidence_modifier": 0.04
}
},
"intent_combinations": {
"analyze_and_visualize": {
"description": "Analyze data and create visualization",
"primary": "analyze",
"secondary": ["and_visualize"],
"confidence_threshold": 0.85,
"execution_order": ["analyze", "and_visualize"]
},
"compare_and_explain": {
"description": "Compare items and explain differences",
"primary": "compare",
"secondary": ["and_explain"],
"confidence_threshold": 0.8,
"execution_order": ["compare", "and_explain"]
},
"monitor_and_alert": {
"description": "Monitor data and send alerts",
"primary": "monitor",
"secondary": ["and_alert"],
"confidence_threshold": 0.8,
"execution_order": ["monitor", "and_alert"]
}
},
"intent_processing": {
"max_secondary_intents": 3,
"max_contextual_intents": 2,
"parallel_execution_threshold": 0.8,
"fallback_to_primary": true,
"intent_confidence_threshold": 0.7
}
}
},
"capabilities": {
"primary_intents": ["analyze", "compare", "monitor"],
"secondary_intents": ["and_visualize", "and_save", "and_explain"],
"contextual_intents": ["quick_summary", "detailed_analysis", "real_time"],
"supported_combinations": [
"analyze_and_visualize",
"compare_and_explain",
"monitor_and_alert"
]
}
}
```
---
## 🧪 **Multi-Intent Testing Framework**
### **Test Case Generation**
```python
def generate_multi_intent_test_cases(skill_config):
"""Generate test cases for multi-intent detection"""
test_cases = []
# Single intent tests (baseline)
single_intents = [
{
'query': 'Analyze AAPL stock',
'intents': {'primary': 'analyze', 'secondary': [], 'contextual': []},
'expected': True,
'complexity': 'low'
},
{
'query': 'Compare MSFT vs GOOGL',
'intents': {'primary': 'compare', 'secondary': [], 'contextual': []},
'expected': True,
'complexity': 'low'
}
]
# Double intent tests
double_intents = [
{
'query': 'Analyze AAPL stock and show me a chart',
'intents': {'primary': 'analyze', 'secondary': ['and_visualize'], 'contextual': []},
'expected': True,
'complexity': 'medium'
},
{
'query': 'Compare these stocks and explain the differences',
'intents': {'primary': 'compare', 'secondary': ['and_explain'], 'contextual': []},
'expected': True,
'complexity': 'medium'
},
{
'query': 'Monitor this stock and alert me on changes',
'intents': {'primary': 'monitor', 'secondary': ['and_alert'], 'contextual': []},
'expected': True,
'complexity': 'medium'
}
]
# Triple intent tests
triple_intents = [
{
'query': 'Analyze AAPL stock, show me a chart, and save the results',
'intents': {'primary': 'analyze', 'secondary': ['and_visualize', 'and_save'], 'contextual': []},
'expected': True,
'complexity': 'high'
},
{
'query': 'Compare these stocks, explain differences, and give me a quick summary',
'intents': {'primary': 'compare', 'secondary': ['and_explain'], 'contextual': ['quick_summary']},
'expected': True,
'complexity': 'high'
}
]
# Complex natural language tests
complex_queries = [
{
'query': 'I need to analyze the performance of these tech stocks, create some visualizations to compare them, and save everything to a file for my presentation',
'intents': {'primary': 'analyze', 'secondary': ['and_visualize', 'and_compare', 'and_save'], 'contextual': []},
'expected': True,
'complexity': 'very_high'
},
{
'query': 'Can you help me monitor my portfolio in real-time and send me alerts if anything significant happens, with detailed analysis of what\'s going on?',
'intents': {'primary': 'monitor', 'secondary': ['and_alert', 'and_explain'], 'contextual': ['real_time', 'detailed_analysis']},
'expected': True,
'complexity': 'very_high'
}
]
# Edge cases and invalid combinations
edge_cases = [
{
'query': 'Analyze this stock and teach me how to cook',
'intents': {'primary': 'analyze', 'secondary': [], 'contextual': []},
'expected': True,
'complexity': 'low',
'note': 'Unsupported secondary intent should be filtered out'
},
{
'query': 'Compare these charts while explaining that theory',
'intents': {'primary': 'compare', 'secondary': ['and_explain'], 'contextual': []},
'expected': True,
'complexity': 'medium',
'note': 'Mixed context - should prioritize domain-relevant parts'
}
]
test_cases.extend(single_intents)
test_cases.extend(double_intents)
test_cases.extend(triple_intents)
test_cases.extend(complex_queries)
test_cases.extend(edge_cases)
return test_cases
def run_multi_intent_tests(skill_config, test_cases):
"""Run multi-intent detection tests"""
results = []
for i, test_case in enumerate(test_cases):
query = test_case['query']
expected_intents = test_case['intents']
expected = test_case['expected']
# Parse intents from query
detected_intents = parse_multiple_intents(query, skill_config['capabilities'])
# Validate results
result = {
'test_id': i + 1,
'query': query,
'expected_intents': expected_intents,
'detected_intents': detected_intents,
'expected_activation': expected,
'actual_activation': detected_intents['primary_intent'] is not None,
'intent_accuracy': calculate_intent_accuracy(expected_intents, detected_intents),
'complexity_match': test_case['complexity'] == detected_intents.get('complexity', 'unknown'),
'notes': test_case.get('note', '')
}
# Determine if test passed
primary_correct = expected_intents['primary'] == detected_intents.get('primary_intent')
secondary_correct = set(expected_intents['secondary']) == set(detected_intents.get('secondary_intents', []))
activation_correct = expected == result['actual_activation']
result['test_passed'] = primary_correct and secondary_correct and activation_correct
results.append(result)
# Log result
status = "✅" if result['test_passed'] else "❌"
print(f"{status} Test {i+1}: {query[:60]}...")
if not result['test_passed']:
print(f" Expected primary: {expected_intents['primary']}, Got: {detected_intents.get('primary_intent')}")
print(f" Expected secondary: {expected_intents['secondary']}, Got: {detected_intents.get('secondary_intents', [])}")
# Calculate metrics
total_tests = len(results)
passed_tests = sum(1 for r in results if r['test_passed'])
accuracy = passed_tests / total_tests if total_tests > 0 else 0
avg_intent_accuracy = sum(r['intent_accuracy'] for r in results) / total_tests if total_tests > 0 else 0
return {
'total_tests': total_tests,
'passed_tests': passed_tests,
'accuracy': accuracy,
'avg_intent_accuracy': avg_intent_accuracy,
'results': results
}
```
---
## 📊 **Performance Monitoring**
### **Multi-Intent Metrics**
```python
class MultiIntentMonitor:
"""Monitor multi-intent detection performance"""
def __init__(self):
self.metrics = {
'total_queries': 0,
'single_intent_queries': 0,
'multi_intent_queries': 0,
'intent_detection_accuracy': [],
'intent_combination_success': [],
'complexity_distribution': {'low': 0, 'medium': 0, 'high': 0, 'very_high': 0},
'execution_plan_accuracy': []
}
def log_intent_detection(self, query, detected_intents, execution_success=None):
"""Log intent detection results"""
self.metrics['total_queries'] += 1
# Count intent types
total_intents = 1 + len(detected_intents.get('secondary_intents', [])) + len(detected_intents.get('contextual_intents', []))
if total_intents == 1:
self.metrics['single_intent_queries'] += 1
else:
self.metrics['multi_intent_queries'] += 1
# Track complexity distribution
complexity = detected_intents.get('complexity', 'medium')
if complexity in self.metrics['complexity_distribution']:
self.metrics['complexity_distribution'][complexity] += 1
# Track execution success if provided
if execution_success is not None:
self.metrics['execution_plan_accuracy'].append(execution_success)
def calculate_multi_intent_rate(self):
"""Calculate the rate of multi-intent queries"""
if self.metrics['total_queries'] == 0:
return 0.0
return self.metrics['multi_intent_queries'] / self.metrics['total_queries']
def generate_performance_report(self):
"""Generate multi-intent performance report"""
total = self.metrics['total_queries']
if total == 0:
return "No data available"
multi_intent_rate = self.calculate_multi_intent_rate()
avg_execution_accuracy = (sum(self.metrics['execution_plan_accuracy']) / len(self.metrics['execution_plan_accuracy'])
if self.metrics['execution_plan_accuracy'] else 0)
report = f"""
Multi-Intent Detection Performance Report
========================================
Total Queries Analyzed: {total}
Single-Intent Queries: {self.metrics['single_intent_queries']} ({(self.metrics['single_intent_queries']/total)*100:.1f}%)
Multi-Intent Queries: {self.metrics['multi_intent_queries']} ({multi_intent_rate*100:.1f}%)
Complexity Distribution:
- Low: {self.metrics['complexity_distribution']['low']} ({(self.metrics['complexity_distribution']['low']/total)*100:.1f}%)
- Medium: {self.metrics['complexity_distribution']['medium']} ({(self.metrics['complexity_distribution']['medium']/total)*100:.1f}%)
- High: {self.metrics['complexity_distribution']['high']} ({(self.metrics['complexity_distribution']['high']/total)*100:.1f}%)
- Very High: {self.metrics['complexity_distribution']['very_high']} ({(self.metrics['complexity_distribution']['very_high']/total)*100:.1f}%)
Execution Plan Accuracy: {avg_execution_accuracy*100:.1f}%
"""
return report
```
---
## ✅ **Implementation Checklist**
### **Configuration Requirements**
- [ ] Add `intent_hierarchy` section to marketplace.json
- [ ] Define supported `primary_intents` with capabilities
- [ ] Define supported `secondary_intents` with compatibility rules
- [ ] Define supported `contextual_intents` with impact modifiers
- [ ] Configure `intent_combinations` with execution plans
- [ ] Set appropriate `intent_processing` thresholds
### **Testing Requirements**
- [ ] Generate multi-intent test cases for each combination
- [ ] Test single-intent queries (baseline)
- [ ] Test double-intent queries
- [ ] Test triple-intent queries
- [ ] Test complex natural language queries
- [ ] Validate edge cases and invalid combinations
### **Performance Requirements**
- [ ] Intent detection accuracy > 95%
- [ ] Multi-intent processing time < 200ms
- [ ] Execution plan accuracy > 90%
- [ ] Support for up to 5 concurrent intents
- [ ] Graceful fallback to primary intent
---
## 📈 **Expected Outcomes**
### **Performance Improvements**
- **Multi-Intent Support**: 0% → **100%**
- **Complex Query Handling**: 20% → **95%**
- **User Intent Accuracy**: 70% → **95%**
- **Natural Language Understanding**: 60% → **90%**
### **User Experience Benefits**
- Natural handling of complex requests
- Better understanding of user goals
- More comprehensive responses
- Reduced need for follow-up queries
---
**Version:** 1.0
**Last Updated:** 2025-10-24
**Maintained By:** Agent-Skill-Creator Team