Initial commit
This commit is contained in:
Zhongwei Li
637
skills/appsec/dast-nuclei/references/template_development.md
Normal file
637
skills/appsec/dast-nuclei/references/template_development.md
Normal file
@@ -0,0 +1,637 @@
|
||||
# Nuclei Template Development Guide
|
||||
|
||||
## Table of Contents
|
||||
- [Template Structure](#template-structure)
|
||||
- [Template Types](#template-types)
|
||||
- [Matchers and Extractors](#matchers-and-extractors)
|
||||
- [Advanced Techniques](#advanced-techniques)
|
||||
- [Testing and Validation](#testing-and-validation)
|
||||
- [Best Practices](#best-practices)
|
||||
|
||||
## Template Structure
|
||||
|
||||
### Basic Template Anatomy
|
||||
|
||||
```yaml
|
||||
id: unique-template-id
|
||||
info:
|
||||
name: Human-readable template name
|
||||
author: your-name
|
||||
severity: critical|high|medium|low|info
|
||||
description: Detailed description of what this template detects
|
||||
reference:
|
||||
- https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2024-XXXXX
|
||||
- https://nvd.nist.gov/vuln/detail/CVE-2024-XXXXX
|
||||
tags: cve,owasp,misconfig,custom
|
||||
|
||||
# Template type: http, dns, network, file, etc.
|
||||
http:
|
||||
- method: GET
|
||||
path:
|
||||
- "{{BaseURL}}/vulnerable-endpoint"
|
||||
|
||||
matchers:
|
||||
- type: status
|
||||
status:
|
||||
- 200
|
||||
|
||||
- type: word
|
||||
words:
|
||||
- "vulnerable signature"
|
||||
```
|
||||
|
||||
### Required Fields
|
||||
|
||||
- **id**: Unique identifier (kebab-case, organization-scoped for custom templates)
|
||||
- **info.name**: Clear, descriptive name
|
||||
- **info.author**: Template author
|
||||
- **info.severity**: One of: critical, high, medium, low, info
|
||||
- **info.description**: What vulnerability this detects
|
||||
- **info.tags**: Searchable tags for filtering
|
||||
|
||||
### Optional but Recommended Fields
|
||||
|
||||
- **info.reference**: Links to CVE, advisories, documentation
|
||||
- **info.classification**: CWE, CVE, OWASP mappings
|
||||
- **info.metadata**: Additional metadata (max-request, verified, etc.)
|
||||
|
||||
## Template Types
|
||||
|
||||
### HTTP Templates
|
||||
|
||||
Most common template type for web application testing:
|
||||
|
||||
```yaml
|
||||
id: http-example
|
||||
info:
|
||||
name: HTTP Template Example
|
||||
author: security-team
|
||||
severity: high
|
||||
tags: web,http
|
||||
|
||||
http:
|
||||
- method: GET
|
||||
path:
|
||||
- "{{BaseURL}}/api/users"
|
||||
- "{{BaseURL}}/api/admin"
|
||||
|
||||
headers:
|
||||
Authorization: "Bearer {{token}}"
|
||||
|
||||
matchers-condition: and
|
||||
matchers:
|
||||
- type: status
|
||||
status:
|
||||
- 200
|
||||
|
||||
- type: word
|
||||
part: body
|
||||
words:
|
||||
- "\"role\":\"admin\""
|
||||
- "sensitive_data"
|
||||
|
||||
extractors:
|
||||
- type: regex
|
||||
name: user_ids
|
||||
regex:
|
||||
- '"id":([0-9]+)'
|
||||
```
|
||||
|
||||
### DNS Templates
|
||||
|
||||
Test for DNS misconfigurations and subdomain takeovers:
|
||||
|
||||
```yaml
|
||||
id: dns-takeover-check
|
||||
info:
|
||||
name: DNS Subdomain Takeover Detection
|
||||
author: security-team
|
||||
severity: high
|
||||
tags: dns,takeover
|
||||
|
||||
dns:
|
||||
- name: "{{FQDN}}"
|
||||
type: CNAME
|
||||
|
||||
matchers:
|
||||
- type: word
|
||||
words:
|
||||
- "amazonaws.com"
|
||||
- "azurewebsites.net"
|
||||
- "herokuapp.com"
|
||||
```
|
||||
|
||||
### Network Templates
|
||||
|
||||
TCP/UDP port scanning and service detection:
|
||||
|
||||
```yaml
|
||||
id: exposed-redis
|
||||
info:
|
||||
name: Exposed Redis Instance
|
||||
author: security-team
|
||||
severity: critical
|
||||
tags: network,redis,exposure
|
||||
|
||||
network:
|
||||
- inputs:
|
||||
- data: "*1\r\n$4\r\ninfo\r\n"
|
||||
|
||||
host:
|
||||
- "{{Hostname}}"
|
||||
- "{{Hostname}}:6379"
|
||||
|
||||
matchers:
|
||||
- type: word
|
||||
words:
|
||||
- "redis_version"
|
||||
```
|
||||
|
||||
## Matchers and Extractors
|
||||
|
||||
### Matcher Types
|
||||
|
||||
#### Status Matcher
|
||||
```yaml
|
||||
matchers:
|
||||
- type: status
|
||||
status:
|
||||
- 200
|
||||
- 201
|
||||
condition: or
|
||||
```
|
||||
|
||||
#### Word Matcher
|
||||
```yaml
|
||||
matchers:
|
||||
- type: word
|
||||
part: body # body, header, all
|
||||
words:
|
||||
- "error"
|
||||
- "exception"
|
||||
condition: and
|
||||
case-insensitive: true
|
||||
```
|
||||
|
||||
#### Regex Matcher
|
||||
```yaml
|
||||
matchers:
|
||||
- type: regex
|
||||
regex:
|
||||
- "(?i)password\\s*=\\s*['\"]([^'\"]+)['\"]"
|
||||
part: body
|
||||
```
|
||||
|
||||
#### Binary Matcher
|
||||
```yaml
|
||||
matchers:
|
||||
- type: binary
|
||||
binary:
|
||||
- "504B0304" # ZIP file signature (hex)
|
||||
part: body
|
||||
```
|
||||
|
||||
#### DSL Matcher (Dynamic Expressions)
|
||||
```yaml
|
||||
matchers:
|
||||
- type: dsl
|
||||
dsl:
|
||||
- "status_code == 200 && len(body) > 1000"
|
||||
- "contains(tolower(body), 'admin')"
|
||||
```
|
||||
|
||||
### Matcher Conditions
|
||||
|
||||
- **and**: All matchers must match
|
||||
- **or**: At least one matcher must match (default)
|
||||
|
||||
```yaml
|
||||
matchers-condition: and
|
||||
matchers:
|
||||
- type: status
|
||||
status:
|
||||
- 200
|
||||
- type: word
|
||||
words:
|
||||
- "admin"
|
||||
```
|
||||
|
||||
### Extractors
|
||||
|
||||
Extract data from responses for reporting or chaining:
|
||||
|
||||
#### Regex Extractor
|
||||
```yaml
|
||||
extractors:
|
||||
- type: regex
|
||||
name: api_keys
|
||||
part: body
|
||||
regex:
|
||||
- 'api[_-]?key["\s:=]+([a-zA-Z0-9_-]{32,})'
|
||||
group: 1
|
||||
```
|
||||
|
||||
#### JSON Extractor
|
||||
```yaml
|
||||
extractors:
|
||||
- type: json
|
||||
name: user_data
|
||||
json:
|
||||
- ".users[].email"
|
||||
- ".users[].id"
|
||||
```
|
||||
|
||||
#### XPath Extractor
|
||||
```yaml
|
||||
extractors:
|
||||
- type: xpath
|
||||
name: titles
|
||||
xpath:
|
||||
- "//title"
|
||||
```
|
||||
|
||||
## Advanced Techniques
|
||||
|
||||
### Request Chaining (Workflows)
|
||||
|
||||
Execute templates in sequence, passing data between them:
|
||||
|
||||
```yaml
|
||||
id: workflow-example
|
||||
info:
|
||||
name: Multi-Step Authentication Test
|
||||
author: security-team
|
||||
|
||||
workflows:
|
||||
templates:
|
||||
- template: login.yaml
|
||||
- template: fetch-user-data.yaml
|
||||
```
|
||||
|
||||
**login.yaml**:
|
||||
```yaml
|
||||
id: login-template
|
||||
info:
|
||||
name: Login and Extract Token
|
||||
author: security-team
|
||||
severity: info
|
||||
|
||||
http:
|
||||
- method: POST
|
||||
path:
|
||||
- "{{BaseURL}}/api/login"
|
||||
|
||||
body: '{"username":"test","password":"test"}'
|
||||
|
||||
extractors:
|
||||
- type: json
|
||||
name: auth_token
|
||||
json:
|
||||
- ".token"
|
||||
internal: true # Pass to next template
|
||||
```
|
||||
|
||||
### Variables and Helpers
|
||||
|
||||
Use dynamic variables and helper functions:
|
||||
|
||||
```yaml
|
||||
http:
|
||||
- method: GET
|
||||
path:
|
||||
- "{{BaseURL}}/api/users/{{username}}"
|
||||
|
||||
# Available variables:
|
||||
# {{BaseURL}}, {{Hostname}}, {{Host}}, {{Port}}, {{Path}}
|
||||
# {{RootURL}}, {{Scheme}}, {{username}} (from previous extractor)
|
||||
|
||||
matchers:
|
||||
- type: dsl
|
||||
dsl:
|
||||
# Helper functions: len(), contains(), regex_match(), etc.
|
||||
- 'len(body) > 500'
|
||||
- 'contains(tolower(header), "x-api-key")'
|
||||
- 'status_code >= 200 && status_code < 300'
|
||||
```
|
||||
|
||||
### Payloads and Fuzzing
|
||||
|
||||
Use payload files for fuzzing:
|
||||
|
||||
```yaml
|
||||
id: sqli-fuzzing
|
||||
info:
|
||||
name: SQL Injection Fuzzing
|
||||
author: security-team
|
||||
severity: critical
|
||||
|
||||
http:
|
||||
- method: GET
|
||||
path:
|
||||
- "{{BaseURL}}/api/users?id={{payload}}"
|
||||
|
||||
payloads:
|
||||
payload:
|
||||
- "1' OR '1'='1"
|
||||
- "1' UNION SELECT NULL--"
|
||||
- "'; DROP TABLE users--"
|
||||
|
||||
matchers:
|
||||
- type: word
|
||||
words:
|
||||
- "SQL syntax"
|
||||
- "mysql_fetch"
|
||||
- "ORA-01756"
|
||||
```
|
||||
|
||||
Or use external payload file:
|
||||
|
||||
```yaml
|
||||
payloads:
|
||||
payload: payloads/sql-injection.txt
|
||||
|
||||
attack: clusterbomb # pitchfork, clusterbomb, batteringram
|
||||
```
|
||||
|
||||
### Rate Limiting and Threads
|
||||
|
||||
Control request rate to avoid overwhelming targets:
|
||||
|
||||
```yaml
|
||||
id: rate-limited-scan
|
||||
info:
|
||||
name: Rate-Limited Vulnerability Scan
|
||||
author: security-team
|
||||
severity: medium
|
||||
metadata:
|
||||
max-request: 50 # Maximum requests per template execution
|
||||
|
||||
http:
|
||||
- method: GET
|
||||
path:
|
||||
- "{{BaseURL}}/api/endpoint"
|
||||
|
||||
threads: 5 # Concurrent requests (default: 25)
|
||||
```
|
||||
|
||||
## Testing and Validation
|
||||
|
||||
### Local Testing
|
||||
|
||||
Test templates against local test servers:
|
||||
|
||||
```bash
|
||||
# Test single template
|
||||
nuclei -t custom-templates/my-template.yaml -u http://localhost:8080 -debug
|
||||
|
||||
# Validate template syntax
|
||||
nuclei -t custom-templates/my-template.yaml -validate
|
||||
|
||||
# Test with verbose output
|
||||
nuclei -t custom-templates/my-template.yaml -u https://target.com -verbose
|
||||
```
|
||||
|
||||
### Template Validation
|
||||
|
||||
Use the bundled validation script:
|
||||
|
||||
```bash
|
||||
python3 scripts/template_validator.py custom-templates/my-template.yaml
|
||||
```
|
||||
|
||||
### Test Lab Setup
|
||||
|
||||
Create a vulnerable test application for template development:
|
||||
|
||||
```bash
|
||||
# Use DVWA (Damn Vulnerable Web Application)
|
||||
docker run -d -p 80:80 vulnerables/web-dvwa
|
||||
|
||||
# Or OWASP Juice Shop
|
||||
docker run -d -p 3000:3000 bkimminich/juice-shop
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
### 1. Accurate Severity Classification
|
||||
|
||||
- **Critical**: RCE, authentication bypass, full system compromise
|
||||
- **High**: SQL injection, XSS, significant data exposure
|
||||
- **Medium**: Missing security headers, information disclosure
|
||||
- **Low**: Minor misconfigurations, best practice violations
|
||||
- **Info**: Technology detection, non-security findings
|
||||
|
||||
### 2. Minimize False Positives
|
||||
|
||||
```yaml
|
||||
# Use multiple matchers with AND condition
|
||||
matchers-condition: and
|
||||
matchers:
|
||||
- type: status
|
||||
status:
|
||||
- 200
|
||||
|
||||
- type: word
|
||||
words:
|
||||
- "admin"
|
||||
- "dashboard"
|
||||
condition: and
|
||||
|
||||
- type: regex
|
||||
regex:
|
||||
- '<title>.*Admin.*Panel.*</title>'
|
||||
case-insensitive: true
|
||||
```
|
||||
|
||||
### 3. Clear Naming Conventions
|
||||
|
||||
- **id**: `organization-vulnerability-type-identifier`
|
||||
- Example: `acme-api-key-exposure-config`
|
||||
- **name**: Descriptive, clear purpose
|
||||
- Example: "ACME Corp API Key Exposure in Config Endpoint"
|
||||
|
||||
### 4. Comprehensive Documentation
|
||||
|
||||
```yaml
|
||||
info:
|
||||
name: Detailed Template Name
|
||||
description: |
|
||||
Comprehensive description of what this template detects,
|
||||
why it's important, and how it works.
|
||||
|
||||
References:
|
||||
- CVE-2024-XXXXX
|
||||
- Internal ticket: SEC-1234
|
||||
|
||||
reference:
|
||||
- https://nvd.nist.gov/vuln/detail/CVE-2024-XXXXX
|
||||
- https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2024-XXXXX
|
||||
|
||||
classification:
|
||||
cvss-metrics: CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H
|
||||
cvss-score: 9.8
|
||||
cve-id: CVE-2024-XXXXX
|
||||
cwe-id: CWE-89
|
||||
|
||||
metadata:
|
||||
verified: true
|
||||
max-request: 10
|
||||
shodan-query: 'http.title:"Admin Panel"'
|
||||
|
||||
tags: cve,owasp,sqli,high-severity,verified
|
||||
```
|
||||
|
||||
### 5. Responsible Testing Parameters
|
||||
|
||||
```yaml
|
||||
# Avoid aggressive fuzzing in default templates
|
||||
info:
|
||||
metadata:
|
||||
max-request: 10 # Limit total requests
|
||||
|
||||
http:
|
||||
- method: GET
|
||||
threads: 5 # Limit concurrent requests
|
||||
|
||||
# Use specific, targeted payloads
|
||||
payloads:
|
||||
test: ["safe-payload-1", "safe-payload-2"]
|
||||
```
|
||||
|
||||
### 6. Error Handling
|
||||
|
||||
```yaml
|
||||
http:
|
||||
- method: GET
|
||||
path:
|
||||
- "{{BaseURL}}/api/test"
|
||||
|
||||
# Handle various response scenarios
|
||||
matchers:
|
||||
- type: dsl
|
||||
dsl:
|
||||
- "status_code == 200 && contains(body, 'vulnerable')"
|
||||
- "status_code == 500 && contains(body, 'error')"
|
||||
condition: or
|
||||
|
||||
# Negative matchers (must NOT match)
|
||||
matchers:
|
||||
- type: word
|
||||
negative: true
|
||||
words:
|
||||
- "404 Not Found"
|
||||
- "403 Forbidden"
|
||||
```
|
||||
|
||||
### 7. Template Organization
|
||||
|
||||
```
|
||||
custom-templates/
|
||||
├── api/
|
||||
│ ├── api-key-exposure.yaml
|
||||
│ ├── graphql-introspection.yaml
|
||||
│ └── rest-api-misconfig.yaml
|
||||
├── cves/
|
||||
│ ├── 2024/
|
||||
│ │ ├── CVE-2024-12345.yaml
|
||||
│ │ └── CVE-2024-67890.yaml
|
||||
├── exposures/
|
||||
│ ├── sensitive-files.yaml
|
||||
│ └── backup-exposure.yaml
|
||||
└── misconfig/
|
||||
├── cors-misconfiguration.yaml
|
||||
└── debug-mode-enabled.yaml
|
||||
```
|
||||
|
||||
### 8. Version Control and Maintenance
|
||||
|
||||
- Use Git to track template changes
|
||||
- Tag templates with version numbers in metadata
|
||||
- Document changes in template comments
|
||||
- Regularly test templates against updated applications
|
||||
|
||||
```yaml
|
||||
info:
|
||||
metadata:
|
||||
version: 1.2.0
|
||||
last-updated: 2024-11-20
|
||||
changelog: |
|
||||
1.2.0 - Added additional matcher for new vulnerability variant
|
||||
1.1.0 - Improved regex pattern to reduce false positives
|
||||
1.0.0 - Initial release
|
||||
```
|
||||
|
||||
## Example: Complete Custom Template
|
||||
|
||||
```yaml
|
||||
id: acme-corp-api-debug-exposure
|
||||
info:
|
||||
name: ACME Corp API Debug Endpoint Exposure
|
||||
author: acme-security-team
|
||||
severity: high
|
||||
description: |
|
||||
Detects exposed debug endpoint in ACME Corp API that leaks
|
||||
sensitive configuration including database credentials,
|
||||
API keys, and internal service URLs.
|
||||
|
||||
reference:
|
||||
- https://internal-wiki.acme.com/security/SEC-1234
|
||||
|
||||
classification:
|
||||
cvss-metrics: CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:N/A:N
|
||||
cvss-score: 7.5
|
||||
cwe-id: CWE-200
|
||||
|
||||
metadata:
|
||||
verified: true
|
||||
max-request: 3
|
||||
version: 1.0.0
|
||||
|
||||
tags: acme,api,exposure,debug,high-severity
|
||||
|
||||
http:
|
||||
- method: GET
|
||||
path:
|
||||
- "{{BaseURL}}/api/v1/debug/config"
|
||||
- "{{BaseURL}}/api/v2/debug/config"
|
||||
- "{{BaseURL}}/debug/config"
|
||||
|
||||
matchers-condition: and
|
||||
matchers:
|
||||
- type: status
|
||||
status:
|
||||
- 200
|
||||
|
||||
- type: word
|
||||
part: body
|
||||
words:
|
||||
- "database_url"
|
||||
- "api_secret_key"
|
||||
condition: or
|
||||
|
||||
- type: regex
|
||||
part: body
|
||||
regex:
|
||||
- '"(password|secret|token)":\s*"[^"]+"'
|
||||
|
||||
extractors:
|
||||
- type: regex
|
||||
name: exposed_secrets
|
||||
part: body
|
||||
regex:
|
||||
- '"(database_url|api_secret_key|jwt_secret)":\s*"([^"]+)"'
|
||||
group: 2
|
||||
|
||||
- type: json
|
||||
name: config_data
|
||||
json:
|
||||
- ".database_url"
|
||||
- ".api_secret_key"
|
||||
```
|
||||
|
||||
## Resources
|
||||
|
||||
- [Official Nuclei Template Guide](https://docs.projectdiscovery.io/templates/introduction)
|
||||
- [Nuclei Templates Repository](https://github.com/projectdiscovery/nuclei-templates)
|
||||
- [Template Editor](https://templates.nuclei.sh/)
|
||||
- [DSL Functions Reference](https://docs.projectdiscovery.io/templates/reference/matchers#dsl-matcher)
|
||||
Reference in New Issue
Block a user