zhongwei/gh-agentsecops-secopsagentkit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial commit

Browse Source
This commit is contained in:
Zhongwei Li
2025-11-29 17:51:02 +08:00
commit ff1f4bd119
252 changed files with 72682 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

550
skills/incident-response/ir-velociraptor/references/EXAMPLE.md Normal file
View File

@@ -0,0 +1,550 @@
# Reference Document Template
This file demonstrates how to structure detailed reference material that Claude loads on-demand.
**When to use this reference**: Include a clear statement about when Claude should consult this document.
For example: "Consult this reference when analyzing Python code for security vulnerabilities and needing detailed remediation patterns."
**Document purpose**: Briefly explain what this reference provides that's not in SKILL.md.
---
## Table of Contents
**For documents >100 lines, always include a table of contents** to help Claude navigate quickly.
- [When to Use References](#when-to-use-references)
- [Document Organization](#document-organization)
- [Detailed Technical Content](#detailed-technical-content)
- [Security Framework Mappings](#security-framework-mappings)
- [OWASP Top 10](#owasp-top-10)
- [CWE Mappings](#cwe-mappings)
- [MITRE ATT&CK](#mitre-attck)
- [Remediation Patterns](#remediation-patterns)
- [Advanced Configuration](#advanced-configuration)
- [Examples and Code Samples](#examples-and-code-samples)
---
## When to Use References
**Move content from SKILL.md to references/** when:
1. **Content exceeds 100 lines** - Keep SKILL.md concise
2. **Framework-specific details** - Detailed OWASP/CWE/MITRE mappings
3. **Advanced user content** - Deep technical details for expert users
4. **Lookup-oriented content** - Rule libraries, configuration matrices, comprehensive lists
5. **Language-specific patterns** - Separate files per language/framework
6. **Historical context** - Old patterns and deprecated approaches
**Keep in SKILL.md**:
- Core workflows (top 3-5 use cases)
- Decision points and branching logic
- Quick start guidance
- Essential security considerations
---
## Document Organization
### Structure for Long Documents
For references >100 lines:
```markdown
# Title
**When to use**: Clear trigger statement
**Purpose**: What this provides
## Table of Contents
- Links to all major sections
## Quick Reference
- Key facts or commands for fast lookup
## Detailed Content
- Comprehensive information organized logically
## Framework Mappings
- OWASP, CWE, MITRE ATT&CK references
## Examples
- Code samples and patterns
```
### Section Naming Conventions
- Use **imperative** or **declarative** headings
- ✅ "Detecting SQL Injection" not "How to detect SQL Injection"
- ✅ "Common Patterns" not "These are common patterns"
- Make headings **searchable** and **specific**
---
## Detailed Technical Content
This section demonstrates the type of detailed content that belongs in references rather than SKILL.md.
### Example: Comprehensive Vulnerability Detection
#### SQL Injection Detection Patterns
**Pattern 1: String Concatenation in Queries**
```python
# Vulnerable pattern
query = "SELECT * FROM users WHERE id = " + user_id
cursor.execute(query)
# Detection criteria:
# - SQL keyword (SELECT, INSERT, UPDATE, DELETE)
# - String concatenation operator (+, f-string)
# - Variable user input (request params, form data)
# Severity: HIGH
# CWE: CWE-89
# OWASP: A03:2021 - Injection
```
**Remediation**:
```python
# Fixed: Parameterized query
query = "SELECT * FROM users WHERE id = ?"
cursor.execute(query, (user_id,))
# OR using ORM
user = User.objects.get(id=user_id)
```
**Pattern 2: Unsafe String Formatting**
```python
# Vulnerable patterns
query = f"SELECT * FROM users WHERE name = '{username}'"
query = "SELECT * FROM users WHERE name = '%s'" % username
query = "SELECT * FROM users WHERE name = '{}'".format(username)
# All three patterns are vulnerable to SQL injection
```
#### Cross-Site Scripting (XSS) Detection
**Pattern 1: Unescaped Output in Templates**
```javascript
// Vulnerable: Direct HTML injection
element.innerHTML = userInput;
document.write(userInput);
// Vulnerable: React dangerouslySetInnerHTML
<div dangerouslySetInnerHTML={{__html: userComment}} />
// Detection criteria:
# - Direct DOM manipulation (innerHTML, document.write)
# - React dangerouslySetInnerHTML with user data
# - Template engines with autoescaping disabled
// Severity: HIGH
// CWE: CWE-79
// OWASP: A03:2021 - Injection
```
**Remediation**:
```javascript
// Fixed: Escaped output
element.textContent = userInput; // Auto-escapes
// Fixed: Sanitization library
import DOMPurify from 'dompurify';
const clean = DOMPurify.sanitize(userComment);
<div dangerouslySetInnerHTML={{__html: clean}} />
```
---
## Security Framework Mappings
This section provides comprehensive security framework mappings for findings.
### OWASP Top 10
Map security findings to OWASP Top 10 (2021) categories:
| Category | Title | Common Vulnerabilities |
|----------|-------|----------------------|
| **A01:2021** | Broken Access Control | Authorization bypass, privilege escalation, IDOR |
| **A02:2021** | Cryptographic Failures | Weak crypto, plaintext storage, insecure TLS |
| **A03:2021** | Injection | SQL injection, XSS, command injection, LDAP injection |
| **A04:2021** | Insecure Design | Missing security controls, threat modeling gaps |
| **A05:2021** | Security Misconfiguration | Default configs, verbose errors, unnecessary features |
| **A06:2021** | Vulnerable Components | Outdated libraries, unpatched dependencies |
| **A07:2021** | Auth & Session Failures | Weak passwords, session fixation, missing MFA |
| **A08:2021** | Software & Data Integrity | Unsigned updates, insecure CI/CD, deserialization |
| **A09:2021** | Logging & Monitoring Failures | Insufficient logging, no alerting, log injection |
| **A10:2021** | SSRF | Server-side request forgery, unvalidated redirects |
**Usage**: When reporting findings, map to primary OWASP category and reference the identifier (e.g., "A03:2021 - Injection").
### CWE Mappings
Map to relevant Common Weakness Enumeration categories for precise vulnerability classification:
#### Injection Vulnerabilities
- **CWE-78**: OS Command Injection
- **CWE-79**: Cross-site Scripting (XSS)
- **CWE-89**: SQL Injection
- **CWE-90**: LDAP Injection
- **CWE-91**: XML Injection
- **CWE-94**: Code Injection
#### Authentication & Authorization
- **CWE-287**: Improper Authentication
- **CWE-288**: Authentication Bypass Using Alternate Path
- **CWE-290**: Authentication Bypass by Spoofing
- **CWE-294**: Authentication Bypass by Capture-replay
- **CWE-306**: Missing Authentication for Critical Function
- **CWE-307**: Improper Restriction of Excessive Authentication Attempts
- **CWE-352**: Cross-Site Request Forgery (CSRF)
#### Cryptographic Issues
- **CWE-256**: Plaintext Storage of Password
- **CWE-259**: Use of Hard-coded Password
- **CWE-261**: Weak Encoding for Password
- **CWE-321**: Use of Hard-coded Cryptographic Key
- **CWE-326**: Inadequate Encryption Strength
- **CWE-327**: Use of Broken or Risky Cryptographic Algorithm
- **CWE-329**: Not Using a Random IV with CBC Mode
- **CWE-798**: Use of Hard-coded Credentials
#### Input Validation
- **CWE-20**: Improper Input Validation
- **CWE-73**: External Control of File Name or Path
- **CWE-434**: Unrestricted Upload of File with Dangerous Type
- **CWE-601**: URL Redirection to Untrusted Site
#### Sensitive Data Exposure
- **CWE-200**: Information Exposure
- **CWE-209**: Information Exposure Through Error Message
- **CWE-312**: Cleartext Storage of Sensitive Information
- **CWE-319**: Cleartext Transmission of Sensitive Information
- **CWE-532**: Information Exposure Through Log Files
**Usage**: Include CWE identifier in all vulnerability reports for standardized classification.
### MITRE ATT&CK
Reference relevant tactics and techniques for threat context:
#### Initial Access (TA0001)
- **T1190**: Exploit Public-Facing Application
- **T1133**: External Remote Services
- **T1078**: Valid Accounts
#### Execution (TA0002)
- **T1059**: Command and Scripting Interpreter
- **T1203**: Exploitation for Client Execution
#### Persistence (TA0003)
- **T1098**: Account Manipulation
- **T1136**: Create Account
- **T1505**: Server Software Component
#### Privilege Escalation (TA0004)
- **T1068**: Exploitation for Privilege Escalation
- **T1548**: Abuse Elevation Control Mechanism
#### Defense Evasion (TA0005)
- **T1027**: Obfuscated Files or Information
- **T1140**: Deobfuscate/Decode Files or Information
- **T1562**: Impair Defenses
#### Credential Access (TA0006)
- **T1110**: Brute Force
- **T1555**: Credentials from Password Stores
- **T1552**: Unsecured Credentials
#### Discovery (TA0007)
- **T1083**: File and Directory Discovery
- **T1046**: Network Service Scanning
#### Collection (TA0009)
- **T1005**: Data from Local System
- **T1114**: Email Collection
#### Exfiltration (TA0010)
- **T1041**: Exfiltration Over C2 Channel
- **T1567**: Exfiltration Over Web Service
**Usage**: When identifying vulnerabilities, consider which ATT&CK techniques an attacker could use to exploit them.
---
## Remediation Patterns
This section provides specific remediation guidance for common vulnerability types.
### SQL Injection Remediation
**Step 1: Identify vulnerable queries**
- Search for string concatenation in SQL queries
- Check for f-strings or format() with SQL keywords
- Review all database interaction code
**Step 2: Apply parameterized queries**
```python
# Python with sqlite3
cursor.execute("SELECT * FROM users WHERE id = ?", (user_id,))
# Python with psycopg2 (PostgreSQL)
cursor.execute("SELECT * FROM users WHERE id = %s", (user_id,))
# Python with SQLAlchemy (ORM)
from sqlalchemy import text
result = session.execute(text("SELECT * FROM users WHERE id = :id"), {"id": user_id})
```
**Step 3: Validate and sanitize input** (defense in depth)
```python
import re
# Validate input format
if not re.match(r'^\d+$', user_id):
raise ValueError("Invalid user ID format")
# Use ORM query builders
user = User.query.filter_by(id=user_id).first()
```
**Step 4: Implement least privilege**
- Database user should have minimum required permissions
- Use read-only accounts for SELECT operations
- Never use admin/root accounts for application queries
### XSS Remediation
**Step 1: Enable auto-escaping**
- Most modern frameworks escape by default
- Ensure auto-escaping is not disabled
**Step 2: Use framework-specific safe methods**
```javascript
// React: Use JSX (auto-escapes)
<div>{userInput}</div>
// Vue: Use template syntax (auto-escapes)
<div>{{ userInput }}</div>
// Angular: Use property binding (auto-escapes)
<div [textContent]="userInput"></div>
```
**Step 3: Sanitize when HTML is required**
```javascript
import DOMPurify from 'dompurify';
// Sanitize HTML content
const clean = DOMPurify.sanitize(userHTML, {
ALLOWED_TAGS: ['b', 'i', 'em', 'strong', 'p'],
ALLOWED_ATTR: []
});
```
**Step 4: Content Security Policy (CSP)**
```html
<!-- Add CSP header -->
Content-Security-Policy: default-src 'self'; script-src 'self' 'nonce-{random}'
```
---
## Advanced Configuration
This section contains detailed configuration options and tuning parameters.
### Example: SAST Tool Configuration
```yaml
# Advanced security scanner configuration
scanner:
# Severity threshold
severity_threshold: MEDIUM
# Rule configuration
rules:
enabled:
- sql-injection
- xss
- hardcoded-secrets
disabled:
- informational-only
# False positive reduction
confidence_threshold: HIGH
exclude_patterns:
- "*/test/*"
- "*/tests/*"
- "*/node_modules/*"
- "*.test.js"
- "*.spec.ts"
# Performance tuning
max_file_size_kb: 2048
timeout_seconds: 300
parallel_jobs: 4
# Output configuration
output_format: json
include_code_snippets: true
max_snippet_lines: 10
```
---
## Examples and Code Samples
This section provides comprehensive code examples for various scenarios.
### Example 1: Secure API Authentication
```python
# Secure API key handling
import os
from functools import wraps
from flask import Flask, request, jsonify
app = Flask(__name__)
# Load API key from environment (never hardcode)
VALID_API_KEY = os.environ.get('API_KEY')
if not VALID_API_KEY:
raise ValueError("API_KEY environment variable not set")
def require_api_key(f):
@wraps(f)
def decorated_function(*args, **kwargs):
api_key = request.headers.get('X-API-Key')
if not api_key:
return jsonify({'error': 'API key required'}), 401
# Constant-time comparison to prevent timing attacks
import hmac
if not hmac.compare_digest(api_key, VALID_API_KEY):
return jsonify({'error': 'Invalid API key'}), 403
return f(*args, **kwargs)
return decorated_function
@app.route('/api/secure-endpoint')
@require_api_key
def secure_endpoint():
return jsonify({'message': 'Access granted'})
```
### Example 2: Secure Password Hashing
```python
# Secure password storage with bcrypt
import bcrypt
def hash_password(password: str) -> str:
"""Hash a password using bcrypt."""
# Generate salt and hash password
salt = bcrypt.gensalt(rounds=12) # Cost factor: 12 (industry standard)
hashed = bcrypt.hashpw(password.encode('utf-8'), salt)
return hashed.decode('utf-8')
def verify_password(password: str, hashed: str) -> bool:
"""Verify a password against a hash."""
return bcrypt.checkpw(
password.encode('utf-8'),
hashed.encode('utf-8')
)
# Usage
stored_hash = hash_password("user_password")
is_valid = verify_password("user_password", stored_hash) # True
```
### Example 3: Secure File Upload
```python
# Secure file upload with validation
import os
import magic
from werkzeug.utils import secure_filename
ALLOWED_EXTENSIONS = {'pdf', 'png', 'jpg', 'jpeg'}
ALLOWED_MIME_TYPES = {
'application/pdf',
'image/png',
'image/jpeg'
}
MAX_FILE_SIZE = 5 * 1024 * 1024 # 5 MB
def is_allowed_file(filename: str, file_content: bytes) -> bool:
"""Validate file extension and MIME type."""
# Check extension
if '.' not in filename:
return False
ext = filename.rsplit('.', 1)[1].lower()
if ext not in ALLOWED_EXTENSIONS:
return False
# Check MIME type (prevent extension spoofing)
mime = magic.from_buffer(file_content, mime=True)
if mime not in ALLOWED_MIME_TYPES:
return False
return True
def handle_upload(file):
"""Securely handle file upload."""
# Check file size
file.seek(0, os.SEEK_END)
size = file.tell()
file.seek(0)
if size > MAX_FILE_SIZE:
raise ValueError("File too large")
# Read content for validation
content = file.read()
file.seek(0)
# Validate file type
if not is_allowed_file(file.filename, content):
raise ValueError("Invalid file type")
# Sanitize filename
filename = secure_filename(file.filename)
# Generate unique filename to prevent overwrite attacks
import uuid
unique_filename = f"{uuid.uuid4()}_{filename}"
# Save to secure location (outside web root)
upload_path = os.path.join('/secure/uploads', unique_filename)
file.save(upload_path)
return unique_filename
```
---
## Best Practices for Reference Documents
1. **Start with "When to use"** - Help Claude know when to load this reference
2. **Include table of contents** - For documents >100 lines
3. **Use concrete examples** - Code samples with vulnerable and fixed versions
4. **Map to frameworks** - OWASP, CWE, MITRE ATT&CK for context
5. **Provide remediation** - Don't just identify issues, show how to fix them
6. **Organize logically** - Group related content, use clear headings
7. **Keep examples current** - Use modern patterns and current framework versions
8. **Be concise** - Even in references, challenge every sentence

253
skills/incident-response/ir-velociraptor/references/WORKFLOW_CHECKLIST.md Normal file
View File

@@ -0,0 +1,253 @@
# Workflow Checklist Template
This template demonstrates workflow patterns for security operations. Copy and adapt these checklists to your specific skill needs.
## Pattern 1: Sequential Workflow Checklist
Use this pattern for operations that must be completed in order, step-by-step.
### Security Assessment Workflow
Progress:
[ ] 1. Identify application entry points and attack surface
[ ] 2. Map authentication and authorization flows
[ ] 3. Identify data flows and sensitive data handling
[ ] 4. Review existing security controls
[ ] 5. Document findings with framework references (OWASP, CWE)
[ ] 6. Prioritize findings by severity (CVSS scores)
[ ] 7. Generate report with remediation recommendations
Work through each step systematically. Check off completed items.
---
## Pattern 2: Conditional Workflow
Use this pattern when the workflow branches based on findings or conditions.
### Vulnerability Remediation Workflow
1. Identify vulnerability type
- If SQL Injection → See [sql-injection-remediation.md](sql-injection-remediation.md)
- If XSS (Cross-Site Scripting) → See [xss-remediation.md](xss-remediation.md)
- If Authentication flaw → See [auth-remediation.md](auth-remediation.md)
- If Authorization flaw → See [authz-remediation.md](authz-remediation.md)
- If Cryptographic issue → See [crypto-remediation.md](crypto-remediation.md)
2. Assess severity using CVSS calculator
- If CVSS >= 9.0 → Priority: Critical (immediate action)
- If CVSS 7.0-8.9 → Priority: High (action within 24h)
- If CVSS 4.0-6.9 → Priority: Medium (action within 1 week)
- If CVSS < 4.0 → Priority: Low (action within 30 days)
3. Apply appropriate remediation pattern
4. Validate fix with security testing
5. Document changes and update security documentation
---
## Pattern 3: Iterative Workflow
Use this pattern for operations that repeat across multiple targets or items.
### Code Security Review Workflow
For each file in the review scope:
1. Identify security-sensitive operations (auth, data access, crypto, input handling)
2. Check against secure coding patterns for the language
3. Flag potential vulnerabilities with severity rating
4. Map findings to CWE and OWASP categories
5. Suggest specific remediation approaches
6. Document finding with code location and fix priority
Continue until all files in scope have been reviewed.
---
## Pattern 4: Feedback Loop Workflow
Use this pattern when validation and iteration are required.
### Secure Configuration Generation Workflow
1. Generate initial security configuration based on requirements
2. Run validation script: `./scripts/validate_config.py config.yaml`
3. Review validation output:
- Note all errors (must fix)
- Note all warnings (should fix)
- Note all info items (consider)
4. Fix identified issues in configuration
5. Repeat steps 2-4 until validation passes with zero errors
6. Review warnings and determine if they should be addressed
7. Apply configuration once validation is clean
**Validation Loop**: Run validator → Fix errors → Repeat until clean
---
## Pattern 5: Parallel Analysis Workflow
Use this pattern when multiple independent analyses can run concurrently.
### Comprehensive Security Scan Workflow
Run these scans in parallel:
**Static Analysis**:
[ ] 1a. Run SAST scan (Semgrep/Bandit)
[ ] 1b. Run dependency vulnerability scan (Safety/npm audit)
[ ] 1c. Run secrets detection (Gitleaks/TruffleHog)
[ ] 1d. Run license compliance check
**Dynamic Analysis**:
[ ] 2a. Run DAST scan (ZAP/Burp)
[ ] 2b. Run API security testing
[ ] 2c. Run authentication/authorization testing
**Infrastructure Analysis**:
[ ] 3a. Run infrastructure-as-code scan (Checkov/tfsec)
[ ] 3b. Run container image scan (Trivy/Grype)
[ ] 3c. Run configuration review
**Consolidation**:
[ ] 4. Aggregate all findings
[ ] 5. Deduplicate and correlate findings
[ ] 6. Prioritize by risk (CVSS + exploitability + business impact)
[ ] 7. Generate unified security report
---
## Pattern 6: Research and Documentation Workflow
Use this pattern for security research and documentation tasks.
### Threat Modeling Workflow
Research Progress:
[ ] 1. Identify system components and boundaries
[ ] 2. Map data flows between components
[ ] 3. Identify trust boundaries
[ ] 4. Enumerate assets (data, services, credentials)
[ ] 5. Apply STRIDE framework to each component:
- Spoofing threats
- Tampering threats
- Repudiation threats
- Information disclosure threats
- Denial of service threats
- Elevation of privilege threats
[ ] 6. Map threats to MITRE ATT&CK techniques
[ ] 7. Identify existing mitigations
[ ] 8. Document residual risks
[ ] 9. Recommend additional security controls
[ ] 10. Generate threat model document
Work through each step systematically. Check off completed items.
---
## Pattern 7: Compliance Validation Workflow
Use this pattern for compliance checks against security standards.
### Security Compliance Audit Workflow
**SOC 2 Controls Review**:
[ ] 1. Review access control policies (CC6.1, CC6.2, CC6.3)
[ ] 2. Verify logical access controls implementation (CC6.1)
[ ] 3. Review authentication mechanisms (CC6.1)
[ ] 4. Verify encryption implementation (CC6.1, CC6.7)
[ ] 5. Review audit logging configuration (CC7.2)
[ ] 6. Verify security monitoring (CC7.2, CC7.3)
[ ] 7. Review incident response procedures (CC7.3, CC7.4)
[ ] 8. Verify backup and recovery processes (A1.2, A1.3)
**Evidence Collection**:
[ ] 9. Collect policy documents
[ ] 10. Collect configuration screenshots
[ ] 11. Collect audit logs
[ ] 12. Document control gaps
[ ] 13. Generate compliance report
---
## Pattern 8: Incident Response Workflow
Use this pattern for security incident handling.
### Security Incident Response Workflow
**Detection and Analysis**:
[ ] 1. Confirm security incident (rule out false positive)
[ ] 2. Determine incident severity (SEV1/2/3/4)
[ ] 3. Identify affected systems and data
[ ] 4. Preserve evidence (logs, memory dumps, network captures)
**Containment**:
[ ] 5. Isolate affected systems (network segmentation)
[ ] 6. Disable compromised accounts
[ ] 7. Block malicious indicators (IPs, domains, hashes)
[ ] 8. Implement temporary compensating controls
**Eradication**:
[ ] 9. Identify root cause
[ ] 10. Remove malicious artifacts (malware, backdoors, webshells)
[ ] 11. Patch vulnerabilities exploited
[ ] 12. Reset compromised credentials
**Recovery**:
[ ] 13. Restore systems from clean backups (if needed)
[ ] 14. Re-enable systems with monitoring
[ ] 15. Verify system integrity
[ ] 16. Resume normal operations
**Post-Incident**:
[ ] 17. Document incident timeline
[ ] 18. Identify lessons learned
[ ] 19. Update security controls to prevent recurrence
[ ] 20. Update incident response procedures
[ ] 21. Communicate with stakeholders
---
## Usage Guidelines
### When to Use Workflow Checklists
✅ **Use checklists for**:
- Complex multi-step operations
- Operations requiring specific order
- Security assessments and audits
- Incident response procedures
- Compliance validation tasks
❌ **Don't use checklists for**:
- Simple single-step operations
- Highly dynamic exploratory work
- Operations that vary significantly each time
### Adapting This Template
1. **Copy relevant pattern** to your skill's SKILL.md or create new reference file
2. **Customize steps** to match your specific security tool or process
3. **Add framework references** (OWASP, CWE, NIST) where applicable
4. **Include tool-specific commands** for automation
5. **Add decision points** where manual judgment is required
### Checklist Best Practices
- **Be specific**: "Run semgrep --config=auto ." not "Scan the code"
- **Include success criteria**: "Validation passes with 0 errors"
- **Reference standards**: Link to OWASP, CWE, NIST where relevant
- **Show progress**: Checkbox format helps track completion
- **Provide escape hatches**: "If validation fails, see troubleshooting.md"
### Integration with Feedback Loops
Combine checklists with validation scripts for maximum effectiveness:
1. Create checklist for the workflow
2. Provide validation script that checks quality
3. Include "run validator" step in checklist
4. Loop: Complete step → Validate → Fix issues → Re-validate
This pattern dramatically improves output quality through systematic validation.

627
skills/incident-response/ir-velociraptor/references/artifact-development.md Normal file
View File

@@ -0,0 +1,627 @@
# Velociraptor Artifact Development Guide
Guide to creating custom VQL artifacts for specific investigation and threat hunting scenarios.
## Table of Contents
- [Artifact Structure](#artifact-structure)
- [Parameter Types](#parameter-types)
- [Source Types](#source-types)
- [Best Practices](#best-practices)
- [Common Patterns](#common-patterns)
- [Testing Artifacts](#testing-artifacts)
## Artifact Structure
Velociraptor artifacts are YAML files with a defined structure:
```yaml
name: Category.Subcategory.ArtifactName
description: |
Detailed description of what this artifact collects and why.
Include use cases and expected output.
author: Your Name <email@domain.com>
type: CLIENT # CLIENT, SERVER, or CLIENT_EVENT
parameters:
- name: ParameterName
default: "default_value"
type: string
description: Parameter description
precondition: |
SELECT OS FROM info() WHERE OS = 'windows'
sources:
- name: SourceName
query: |
SELECT * FROM plugin()
WHERE condition
reports:
- type: CLIENT
template: |
# Report Title
{{ .Description }}
{{ range .Rows }}
- {{ .Column }}
{{ end }}
```
### Required Fields
- **name**: Unique artifact identifier in dot notation
- **description**: What the artifact does and when to use it
- **sources**: At least one VQL query source
### Optional Fields
- **author**: Creator information
- **type**: Artifact type (CLIENT, SERVER, CLIENT_EVENT)
- **parameters**: User-configurable inputs
- **precondition**: Check before running (OS, software presence)
- **reports**: Output formatting templates
- **references**: External documentation links
## Parameter Types
### String Parameters
```yaml
parameters:
- name: SearchPath
default: "C:/Windows/System32/"
type: string
description: Directory path to search
```
### Integer Parameters
```yaml
parameters:
- name: DaysBack
default: 7
type: int
description: Number of days to look back
```
### Boolean Parameters
```yaml
parameters:
- name: IncludeSystem
default: Y
type: bool
description: Include system files
```
### Regex Parameters
```yaml
parameters:
- name: ProcessPattern
default: "(?i)(powershell|cmd)"
type: regex
description: Process name pattern to match
```
### Choice Parameters
```yaml
parameters:
- name: LogLevel
default: "INFO"
type: choices
choices:
- DEBUG
- INFO
- WARNING
- ERROR
description: Logging verbosity
```
### CSV Parameters
```yaml
parameters:
- name: IOCList
default: |
evil.com
malicious.net
type: csv
description: List of IOC domains
```
## Source Types
### Query Sources
Standard VQL query that collects data:
```yaml
sources:
- name: ProcessCollection
query: |
SELECT Pid, Name, CommandLine, Username
FROM pslist()
WHERE Name =~ ProcessPattern
```
### Event Sources
Continuous monitoring queries for CLIENT_EVENT artifacts:
```yaml
sources:
- name: ProcessCreation
query: |
SELECT * FROM watch_evtx(
filename="C:/Windows/System32/winevt/Logs/Security.evtx"
)
WHERE System.EventID.Value = 4688
```
### Multiple Sources
Artifacts can have multiple sources for different data collection:
```yaml
sources:
- name: Processes
query: |
SELECT * FROM pslist()
- name: NetworkConnections
query: |
SELECT * FROM netstat()
- name: LoadedDLLs
query: |
SELECT * FROM modules()
```
## Best Practices
### 1. Use Preconditions
Prevent artifact execution on incompatible systems:
```yaml
# Windows-only artifact
precondition: |
SELECT OS FROM info() WHERE OS = 'windows'
# Requires specific tool
precondition: |
SELECT * FROM stat(filename="C:/Tools/sysinternals/psexec.exe")
# Version check
precondition: |
SELECT * FROM info() WHERE OS = 'windows' AND OSVersion =~ '10'
```
### 2. Parameterize Paths and Patterns
Make artifacts flexible and reusable:
```yaml
parameters:
- name: TargetPath
default: "C:/Users/**/AppData/**"
type: string
- name: FilePattern
default: "*.exe"
type: string
sources:
- query: |
SELECT * FROM glob(globs=TargetPath + "/" + FilePattern)
```
### 3. Use LET for Query Composition
Break complex queries into manageable parts:
```yaml
sources:
- query: |
-- Define reusable subqueries
LET SuspiciousProcesses = SELECT Pid, Name, CommandLine
FROM pslist()
WHERE CommandLine =~ "(?i)(bypass|hidden)"
LET NetworkConnections = SELECT Pid, Raddr.IP AS RemoteIP
FROM netstat()
WHERE Status = "ESTABLISHED"
-- Join and correlate
SELECT sp.Name,
sp.CommandLine,
nc.RemoteIP
FROM SuspiciousProcesses sp
JOIN NetworkConnections nc ON sp.Pid = nc.Pid
```
### 4. Add Error Handling
Handle missing data gracefully:
```yaml
sources:
- query: |
SELECT * FROM foreach(
row={
SELECT FullPath FROM glob(globs=SearchPath)
},
query={
SELECT FullPath,
hash(path=FullPath, accessor="file").SHA256 AS SHA256
FROM scope()
WHERE log(message="Processing: " + FullPath)
},
workers=5
)
WHERE SHA256 -- Filter out hash failures
```
### 5. Include Documentation
Add inline comments and comprehensive descriptions:
```yaml
description: |
## Overview
This artifact hunts for suspicious scheduled tasks.
## Use Cases
- Persistence mechanism detection
- Lateral movement artifact collection
- Threat hunting campaigns
## Output
Returns task name, actions, triggers, and creation time.
## References
- MITRE ATT&CK T1053.005 (Scheduled Task/Job)
```
## Common Patterns
### Pattern: File Collection with Hashing
```yaml
name: Custom.Windows.FileCollection
description: Collect files matching patterns with hashes
parameters:
- name: GlobPatterns
default: |
C:/Users/**/AppData/**/*.exe
C:/Windows/Temp/**/*.dll
type: csv
sources:
- query: |
SELECT FullPath,
Size,
timestamp(epoch=Mtime) AS Modified,
timestamp(epoch=Btime) AS Created,
hash(path=FullPath, accessor="file") AS Hashes
FROM foreach(
row={
SELECT * FROM parse_csv(filename=GlobPatterns, accessor="data")
},
query={
SELECT * FROM glob(globs=_value)
}
)
WHERE NOT IsDir
```
### Pattern: Event Log Analysis
```yaml
name: Custom.Windows.EventLogHunt
description: Hunt for specific event IDs with context
parameters:
- name: LogFile
default: "C:/Windows/System32/winevt/Logs/Security.evtx"
type: string
- name: EventIDs
default: "4624,4625,4672"
type: csv
sources:
- query: |
LET EventIDList = SELECT parse_string_with_regex(
string=EventIDs,
regex="(\\d+)"
).g1 AS EventID FROM scope()
SELECT timestamp(epoch=System.TimeCreated.SystemTime) AS EventTime,
System.EventID.Value AS EventID,
System.Computer AS Computer,
EventData
FROM parse_evtx(filename=LogFile)
WHERE str(str=System.EventID.Value) IN EventIDList.EventID
ORDER BY EventTime DESC
```
### Pattern: Process Tree Analysis
```yaml
name: Custom.Windows.ProcessTree
description: Build process tree from a starting PID
parameters:
- name: RootPID
default: 0
type: int
description: Starting process PID (0 for all)
sources:
- query: |
LET ProcessList = SELECT Pid, Ppid, Name, CommandLine, Username, CreateTime
FROM pslist()
LET RECURSIVE GetChildren(ParentPID) = SELECT *
FROM ProcessList
WHERE Ppid = ParentPID
LET RECURSIVE BuildTree(Level, ParentPID) = SELECT
Level,
Pid,
Ppid,
Name,
CommandLine,
Username,
CreateTime
FROM GetChildren(ParentPID=ParentPID)
UNION ALL
SELECT * FROM BuildTree(Level=Level+1, ParentPID=Pid)
SELECT * FROM if(
condition=RootPID > 0,
then={
SELECT * FROM BuildTree(Level=0, ParentPID=RootPID)
},
else={
SELECT 0 AS Level, * FROM ProcessList
}
)
ORDER BY CreateTime
```
### Pattern: Network IOC Matching
```yaml
name: Custom.Windows.NetworkIOCMatch
description: Match network connections against IOC list
parameters:
- name: IOCList
default: |
IP,Description
192.0.2.1,C2 Server
198.51.100.50,Malicious Host
type: csv
sources:
- query: |
LET IOCs = SELECT IP, Description
FROM parse_csv(filename=IOCList, accessor="data")
LET Connections = SELECT
Raddr.IP AS RemoteIP,
Raddr.Port AS RemotePort,
Pid,
process_tracker_get(id=Pid).Name AS ProcessName,
process_tracker_get(id=Pid).CommandLine AS CommandLine
FROM netstat()
WHERE Status = "ESTABLISHED"
SELECT c.RemoteIP,
c.RemotePort,
c.ProcessName,
c.CommandLine,
i.Description AS IOCMatch
FROM Connections c
JOIN IOCs i ON c.RemoteIP = i.IP
```
### Pattern: Registry Timeline
```yaml
name: Custom.Windows.RegistryTimeline
description: Timeline registry modifications in specific keys
parameters:
- name: RegistryPaths
default: |
HKEY_LOCAL_MACHINE/SOFTWARE/Microsoft/Windows/CurrentVersion/Run/**
HKEY_CURRENT_USER/SOFTWARE/Microsoft/Windows/CurrentVersion/Run/**
type: csv
- name: DaysBack
default: 7
type: int
sources:
- query: |
LET StartTime = timestamp(epoch=now() - DaysBack * 86400)
SELECT timestamp(epoch=Key.Mtime) AS Modified,
Key.FullPath AS RegistryPath,
ValueName,
ValueData.value AS Value
FROM foreach(
row={
SELECT * FROM parse_csv(filename=RegistryPaths, accessor="data")
},
query={
SELECT * FROM read_reg_key(globs=_value)
}
)
WHERE Key.Mtime > StartTime
ORDER BY Modified DESC
```
## Testing Artifacts
### 1. Local Testing with GUI
```bash
# Start Velociraptor in GUI mode
velociraptor gui
# Navigate to: View Artifacts → Add Artifact
# Paste your artifact YAML and click Save
# Run artifact via Collected Artifacts → New Collection
```
### 2. Command Line Testing
```bash
# Test artifact syntax
velociraptor artifacts show Custom.Artifact.Name
# Run artifact locally
velociraptor artifacts collect Custom.Artifact.Name \
--args ParameterName=value \
--format json
# Run with output file
velociraptor artifacts collect Custom.Artifact.Name \
--output results.json
```
### 3. Notebook Testing
Use VQL notebooks for interactive development:
```sql
-- Test query components in isolation
SELECT * FROM pslist() WHERE Name =~ "powershell" LIMIT 10
-- Test parameter substitution
LET ProcessPattern = "(?i)(powershell|cmd)"
SELECT * FROM pslist() WHERE Name =~ ProcessPattern
-- Test full artifact query
/* Paste your artifact query here */
```
### 4. Validation Checklist
Before deploying artifacts:
- [ ] Artifact name follows convention: Category.Subcategory.Name
- [ ] Description includes use cases and expected output
- [ ] Parameters have sensible defaults
- [ ] Precondition prevents incompatible execution
- [ ] Query tested in notebook mode
- [ ] Error handling for missing data
- [ ] Performance acceptable on test system
- [ ] Output format is useful and parseable
- [ ] Documentation includes MITRE ATT&CK mapping if applicable
## Performance Considerations
### Limit Scope
```yaml
# BAD: Scans entire filesystem
SELECT * FROM glob(globs="C:/**/*.exe")
# GOOD: Targeted scope
SELECT * FROM glob(globs=[
"C:/Users/**/AppData/**/*.exe",
"C:/Windows/Temp/**/*.exe"
])
```
### Use Workers for Parallel Processing
```yaml
sources:
- query: |
SELECT * FROM foreach(
row={SELECT * FROM glob(globs=SearchPath)},
query={
SELECT FullPath,
hash(path=FullPath, accessor="file").SHA256 AS SHA256
FROM scope()
},
workers=10 -- Process 10 files concurrently
)
```
### Rate Limiting
```yaml
sources:
- query: |
SELECT * FROM foreach(
row={SELECT * FROM glob(globs="C:/**")},
query={
SELECT * FROM scope()
WHERE rate(query_name="my_query", ops_per_sec=100)
}
)
```
## MITRE ATT&CK Mapping
Map artifacts to MITRE ATT&CK techniques:
```yaml
name: Custom.Windows.PersistenceHunt
description: |
Hunt for persistence mechanisms.
MITRE ATT&CK Techniques:
- T1547.001: Registry Run Keys / Startup Folder
- T1053.005: Scheduled Task/Job
- T1543.003: Windows Service
- T1546.003: Windows Management Instrumentation Event Subscription
references:
- https://attack.mitre.org/techniques/T1547/001/
- https://attack.mitre.org/techniques/T1053/005/
```
## Artifact Distribution
### Export Artifacts
```bash
# Export single artifact
velociraptor artifacts show Custom.Artifact.Name > artifact.yaml
# Export all custom artifacts
velociraptor artifacts list --filter Custom > all_artifacts.yaml
```
### Import Artifacts
```bash
# Via command line
velociraptor --config server.config.yaml artifacts import artifact.yaml
# Via GUI
# Navigate to: View Artifacts → Upload Artifact Pack
```
### Share via Artifact Exchange
Contribute artifacts to the community:
1. Test thoroughly across different systems
2. Document clearly with examples
3. Add MITRE ATT&CK mappings
4. Submit to: https://docs.velociraptor.app/exchange/

657
skills/incident-response/ir-velociraptor/references/deployment-guide.md Normal file
View File

@@ -0,0 +1,657 @@
# Velociraptor Enterprise Deployment Guide
Comprehensive guide for deploying Velociraptor in enterprise environments.
## Table of Contents
- [Architecture Overview](#architecture-overview)
- [Server Deployment](#server-deployment)
- [Client Deployment](#client-deployment)
- [High Availability](#high-availability)
- [Security Hardening](#security-hardening)
- [Monitoring and Maintenance](#monitoring-and-maintenance)
- [Scaling Considerations](#scaling-considerations)
## Architecture Overview
### Components
**Frontend Server**:
- Handles client communication (gRPC)
- Serves web GUI
- Manages TLS connections
- Default port: TCP 8000 (clients), TCP 8889 (GUI)
**Datastore**:
- Filesystem-based by default
- Stores artifacts, collections, and configurations
- Can use external storage (S3, GCS)
**Clients (Agents)**:
- Lightweight endpoint agents
- Execute VQL queries
- Report results to server
- Self-updating capability
### Deployment Models
**Single Server** (< 1000 endpoints):
```
[Clients] ──→ [Frontend + GUI + Datastore]
```
**Multi-Frontend** (1000-10000 endpoints):
```
┌─→ [Frontend 1]
[Clients] ──→ [LB]├─→ [Frontend 2] ──→ [Shared Datastore]
└─→ [Frontend 3]
```
**Distributed** (> 10000 endpoints):
```
┌─→ [Frontend Pool 1] ──→ [Datastore Region 1]
[Clients by region]├─→ [Frontend Pool 2] ──→ [Datastore Region 2]
└─→ [Frontend Pool 3] ──→ [Datastore Region 3]
```
## Server Deployment
### Prerequisites
**System Requirements**:
- OS: Linux (Ubuntu 20.04+, RHEL 8+), Windows Server 2019+
- RAM: 8GB minimum, 16GB+ recommended for large deployments
- CPU: 4 cores minimum, 8+ for production
- Storage: 100GB+ for datastore (grows with collections)
- Network: Public IP or internal with client access
**Software Requirements**:
- No external dependencies (single binary)
- Optional: MySQL/PostgreSQL for metadata (future enhancement)
### Installation Steps
#### 1. Download Velociraptor
```bash
# Linux
wget https://github.com/Velocidex/velociraptor/releases/download/v0.72/velociraptor-v0.72.3-linux-amd64
# Make executable
chmod +x velociraptor-v0.72.3-linux-amd64
sudo mv velociraptor-v0.72.3-linux-amd64 /usr/local/bin/velociraptor
```
#### 2. Generate Server Configuration
```bash
# Interactive configuration generation
velociraptor config generate -i
# Or automated with defaults
velociraptor config generate \
--deployment linux \
--frontend_hostname velociraptor.company.com \
--frontend_port 8000 \
--gui_port 8889 \
--datastore /var/lib/velociraptor \
> /etc/velociraptor/server.config.yaml
```
**Key Configuration Options**:
```yaml
# server.config.yaml
version:
name: velociraptor
version: "0.72"
Client:
server_urls:
- https://velociraptor.company.com:8000/
ca_certificate: |
-----BEGIN CERTIFICATE-----
[CA cert]
-----END CERTIFICATE-----
API:
bind_address: 0.0.0.0
bind_port: 8001
bind_scheme: tcp
GUI:
bind_address: 0.0.0.0
bind_port: 8889
use_plain_http: false
internal_cidr:
- 10.0.0.0/8
- 172.16.0.0/12
- 192.168.0.0/16
Frontend:
hostname: velociraptor.company.com
bind_address: 0.0.0.0
bind_port: 8000
Datastore:
implementation: FileBaseDataStore
location: /var/lib/velociraptor
filestore_directory: /var/lib/velociraptor
```
#### 3. Setup Systemd Service (Linux)
```bash
# Create service file
sudo cat > /etc/systemd/system/velociraptor.service <<'EOF'
[Unit]
Description=Velociraptor DFIR Agent
After=network.target
[Service]
Type=simple
ExecStart=/usr/local/bin/velociraptor --config /etc/velociraptor/server.config.yaml frontend -v
Restart=on-failure
RestartSec=10
User=velociraptor
Group=velociraptor
StandardOutput=journal
StandardError=journal
SyslogIdentifier=velociraptor
# Security hardening
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=full
ProtectHome=true
ReadWritePaths=/var/lib/velociraptor
[Install]
WantedBy=multi-user.target
EOF
# Create user
sudo useradd -r -s /bin/false velociraptor
# Setup directories
sudo mkdir -p /etc/velociraptor /var/lib/velociraptor
sudo chown -R velociraptor:velociraptor /etc/velociraptor /var/lib/velociraptor
# Start service
sudo systemctl daemon-reload
sudo systemctl enable velociraptor
sudo systemctl start velociraptor
```
#### 4. Create Initial Admin User
```bash
# Create admin user
velociraptor --config /etc/velociraptor/server.config.yaml \
user add admin --role administrator
# Verify
velociraptor --config /etc/velociraptor/server.config.yaml \
user show admin
```
#### 5. Access Web Interface
```bash
# Access GUI at: https://velociraptor.company.com:8889/
# Login with admin credentials created above
```
### TLS Certificate Configuration
**Option 1: Self-Signed (Testing)**:
```bash
# Already generated during config creation
# Certificates in server.config.yaml
```
**Option 2: Let's Encrypt**:
```bash
# Install certbot
sudo apt install certbot
# Generate certificate
sudo certbot certonly --standalone \
-d velociraptor.company.com \
--non-interactive --agree-tos \
-m admin@company.com
# Update server.config.yaml with Let's Encrypt certs
```
**Option 3: Corporate CA**:
```yaml
# Update server.config.yaml
Frontend:
certificate: /path/to/server-cert.pem
private_key: /path/to/server-key.pem
GUI:
use_plain_http: false
certificate: /path/to/gui-cert.pem
private_key: /path/to/gui-key.pem
```
## Client Deployment
### Generate Client Configuration
```bash
# Generate client config from server config
velociraptor --config /etc/velociraptor/server.config.yaml \
config client > /tmp/client.config.yaml
```
### Deployment Methods
#### Method 1: MSI Installer (Windows)
```bash
# Generate MSI installer
velociraptor --config /etc/velociraptor/server.config.yaml \
config msi --binary velociraptor.exe \
--output VelociraptorClient.msi
# Deploy via GPO, SCCM, or Intune
# Silent install: msiexec /i VelociraptorClient.msi /quiet
```
#### Method 2: DEB/RPM Package (Linux)
```bash
# Generate DEB package
velociraptor --config /etc/velociraptor/server.config.yaml \
debian client --binary velociraptor-linux-amd64 \
--output velociraptor-client.deb
# Deploy via Ansible, Puppet, or package manager
# Install: sudo dpkg -i velociraptor-client.deb
```
#### Method 3: Manual Installation
**Windows**:
```powershell
# Copy binary and config
Copy-Item velociraptor.exe C:\Program Files\Velociraptor\
Copy-Item client.config.yaml C:\Program Files\Velociraptor\
# Install as service
& "C:\Program Files\Velociraptor\velociraptor.exe" `
--config "C:\Program Files\Velociraptor\client.config.yaml" `
service install
# Start service
Start-Service Velociraptor
```
**Linux**:
```bash
# Copy binary and config
sudo cp velociraptor /usr/local/bin/
sudo cp client.config.yaml /etc/velociraptor/
# Create systemd service
sudo cat > /etc/systemd/system/velociraptor-client.service <<'EOF'
[Unit]
Description=Velociraptor DFIR Client
After=network.target
[Service]
Type=simple
ExecStart=/usr/local/bin/velociraptor --config /etc/velociraptor/client.config.yaml client -v
Restart=on-failure
User=root
[Install]
WantedBy=multi-user.target
EOF
# Start service
sudo systemctl enable velociraptor-client
sudo systemctl start velociraptor-client
```
### Client Configuration Options
```yaml
# client.config.yaml
Client:
server_urls:
- https://velociraptor.company.com:8000/
# Connection tuning
max_poll: 60 # Max seconds between polls
max_poll_std: 10 # Jitter to prevent thundering herd
# Performance
max_upload_size: 104857600 # 100MB
cpu_limit: 80 # CPU usage percentage limit
progress_timeout: 3600 # Query timeout
# Writeback file (client state)
writeback_linux: /etc/velociraptor/writeback.yaml
writeback_windows: C:\Program Files\Velociraptor\writeback.yaml
```
## High Availability
### Load Balancer Configuration
**HAProxy Example**:
```conf
# /etc/haproxy/haproxy.cfg
frontend velociraptor_frontend
bind *:8000 ssl crt /etc/ssl/certs/velociraptor.pem
mode tcp
default_backend velociraptor_servers
backend velociraptor_servers
mode tcp
balance leastconn
option tcp-check
server velo1 10.0.1.10:8000 check
server velo2 10.0.1.11:8000 check
server velo3 10.0.1.12:8000 check
frontend velociraptor_gui
bind *:8889 ssl crt /etc/ssl/certs/velociraptor.pem
mode http
default_backend velociraptor_gui_servers
backend velociraptor_gui_servers
mode http
balance roundrobin
option httpchk GET /
server velo1 10.0.1.10:8889 check
server velo2 10.0.1.11:8889 check
server velo3 10.0.1.12:8889 check
```
### Shared Datastore
**NFS Configuration**:
```bash
# On NFS server
sudo apt install nfs-kernel-server
sudo mkdir -p /export/velociraptor
sudo chown nobody:nogroup /export/velociraptor
# /etc/exports
/export/velociraptor 10.0.1.0/24(rw,sync,no_subtree_check,no_root_squash)
# On Velociraptor servers
sudo mount -t nfs nfs-server:/export/velociraptor /var/lib/velociraptor
```
**S3 Datastore (Future)**:
```yaml
# server.config.yaml
Datastore:
implementation: S3DataStore
s3_bucket: velociraptor-datastore
s3_region: us-east-1
credentials_file: /etc/velociraptor/aws-credentials
```
## Security Hardening
### Network Security
**Firewall Rules** (iptables):
```bash
# Allow client connections
sudo iptables -A INPUT -p tcp --dport 8000 -j ACCEPT
# Allow GUI access from management network only
sudo iptables -A INPUT -p tcp --dport 8889 -s 10.0.0.0/8 -j ACCEPT
sudo iptables -A INPUT -p tcp --dport 8889 -j DROP
# Save rules
sudo iptables-save > /etc/iptables/rules.v4
```
**TLS Configuration**:
```yaml
# Enforce TLS 1.2+
Frontend:
min_tls_version: "1.2"
GUI:
min_tls_version: "1.2"
```
### Access Control
**Role-Based Access**:
```bash
# Create read-only analyst role
velociraptor --config server.config.yaml \
acl grant analyst --role reader
# Create hunt operator role
velociraptor --config server.config.yaml \
acl grant hunter --role analyst
# Create admin role
velociraptor --config server.config.yaml \
acl grant admin --role administrator
```
**Permissions Matrix**:
| Role | View Artifacts | Run Collections | Create Hunts | Manage Users | View All Clients |
|------|---------------|-----------------|--------------|--------------|------------------|
| Reader | ✓ | ✗ | ✗ | ✗ | ✗ |
| Analyst | ✓ | ✓ | ✗ | ✗ | ✓ |
| Investigator | ✓ | ✓ | ✓ | ✗ | ✓ |
| Administrator | ✓ | ✓ | ✓ | ✓ | ✓ |
### Audit Logging
**Enable Comprehensive Logging**:
```yaml
# server.config.yaml
Logging:
output_directory: /var/log/velociraptor
separate_logs_per_component: true
max_age: 365
# Log queries
log_queries: true
# Log all API calls
log_api_calls: true
```
**Audit Log Monitoring**:
```bash
# Monitor authentication events
tail -f /var/log/velociraptor/frontend.log | grep -i "auth"
# Monitor collection starts
tail -f /var/log/velociraptor/frontend.log | grep -i "collection"
# Monitor hunt creation
tail -f /var/log/velociraptor/frontend.log | grep -i "hunt"
```
## Monitoring and Maintenance
### Health Checks
**Server Health**:
```bash
# Check server status
systemctl status velociraptor
# Check connected clients
velociraptor --config server.config.yaml \
query "SELECT client_id, os_info.hostname, last_seen_at FROM clients()"
# Check resource usage
velociraptor --config server.config.yaml \
query "SELECT * FROM monitoring()"
```
**Client Health Monitoring**:
```sql
-- Find offline clients (>24 hours)
SELECT client_id,
os_info.hostname AS Hostname,
timestamp(epoch=last_seen_at) AS LastSeen
FROM clients()
WHERE last_seen_at < now() - 86400
ORDER BY last_seen_at
```
### Backup and Recovery
**Backup Strategy**:
```bash
#!/bin/bash
# velociraptor-backup.sh
BACKUP_DIR="/backup/velociraptor"
DATASTORE="/var/lib/velociraptor"
DATE=$(date +%Y%m%d-%H%M%S)
# Stop server (optional for consistency)
# systemctl stop velociraptor
# Backup datastore
tar -czf "$BACKUP_DIR/datastore-$DATE.tar.gz" "$DATASTORE"
# Backup configuration
cp /etc/velociraptor/server.config.yaml "$BACKUP_DIR/server.config-$DATE.yaml"
# Restart server
# systemctl start velociraptor
# Rotate old backups (keep 30 days)
find "$BACKUP_DIR" -name "*.tar.gz" -mtime +30 -delete
```
**Recovery**:
```bash
# Stop server
systemctl stop velociraptor
# Restore datastore
tar -xzf /backup/velociraptor/datastore-20240115.tar.gz -C /var/lib/
# Restore config
cp /backup/velociraptor/server.config-20240115.yaml /etc/velociraptor/server.config.yaml
# Start server
systemctl start velociraptor
```
### Maintenance Tasks
**Database Cleanup**:
```bash
# Delete old collections
velociraptor --config server.config.yaml \
query "DELETE FROM collections WHERE timestamp < now() - 7776000" # 90 days
# Vacuum datastore (reclaim space)
velociraptor --config server.config.yaml \
datastore vacuum
```
**Client Updates**:
```bash
# Update clients via server
# 1. Upload new binary to server
velociraptor --config server.config.yaml \
tools upload --file velociraptor-v0.72.4.exe --name velociraptor
# 2. Create update hunt
velociraptor --config server.config.yaml \
query "SELECT * FROM hunt(artifact='Generic.Client.Update')"
```
## Scaling Considerations
### Performance Tuning
**Server Configuration**:
```yaml
# server.config.yaml
Frontend:
# Increase concurrent connections
max_connections: 10000
# Connection timeouts
keep_alive_timeout: 300
Datastore:
# Filesystem tuning
max_dir_size: 10000 # Files per directory
Resources:
# Increase worker pools
expected_clients: 10000
max_poll_threads: 100
```
**System Tuning**:
```bash
# Increase file descriptors
echo "velociraptor soft nofile 65536" >> /etc/security/limits.conf
echo "velociraptor hard nofile 65536" >> /etc/security/limits.conf
# Kernel tuning
cat >> /etc/sysctl.conf <<EOF
net.core.somaxconn = 4096
net.ipv4.tcp_max_syn_backlog = 4096
net.ipv4.ip_local_port_range = 10000 65000
EOF
sysctl -p
```
### Capacity Planning
**Client-to-Server Ratio**:
- Single server: Up to 10,000 clients
- Multi-frontend: Up to 100,000 clients
- Distributed: 100,000+ clients
**Storage Requirements**:
- Base install: ~200MB
- Per-client metadata: ~100KB
- Per-collection: Varies (typically 1-50MB)
- Retention: Plan for 90-180 days of data
**Network Bandwidth**:
- Baseline: ~1KB/client/minute (polling)
- Collection: Depends on artifacts (10MB-1GB)
- Hunt: Multiply collection size by client count
**Formula**:
```
Storage = (Clients × 100KB) + (Collections/day × AvgSize × RetentionDays)
Bandwidth = (Clients × 1KB × 60 × 24) + (Hunts/day × Clients × AvgCollection)
```
### Monitoring Metrics
**Key Performance Indicators**:
- Client check-in rate (target: >99%)
- Average query execution time
- Collection success rate
- Datastore growth rate
- Server CPU/memory usage
- Network throughput
**Prometheus Metrics** (if enabled):
```yaml
# server.config.yaml
Monitoring:
bind_address: localhost
bind_port: 9090
```

597
skills/incident-response/ir-velociraptor/references/mitre-attack-mapping.md Normal file
View File

@@ -0,0 +1,597 @@
# MITRE ATT&CK Technique Detection with Velociraptor
Mapping of MITRE ATT&CK techniques to Velociraptor artifacts and VQL queries.
## Table of Contents
- [Initial Access](#initial-access)
- [Execution](#execution)
- [Persistence](#persistence)
- [Privilege Escalation](#privilege-escalation)
- [Defense Evasion](#defense-evasion)
- [Credential Access](#credential-access)
- [Discovery](#discovery)
- [Lateral Movement](#lateral-movement)
- [Collection](#collection)
- [Exfiltration](#exfiltration)
- [Command and Control](#command-and-control)
## Initial Access
### T1078: Valid Accounts
**Artifacts**:
- `Windows.EventLogs.EvtxHunter` (EventID 4624, 4625)
- `Windows.EventLogs.RDP`
**VQL Query**:
```sql
-- Detect unusual logon patterns
SELECT timestamp(epoch=System.TimeCreated.SystemTime) AS LogonTime,
EventData.TargetUserName AS Username,
EventData.IpAddress AS SourceIP,
EventData.LogonType AS LogonType,
EventData.WorkstationName AS Workstation
FROM parse_evtx(filename="C:/Windows/System32/winevt/Logs/Security.evtx")
WHERE System.EventID.Value = 4624
AND (
EventData.LogonType IN (3, 10) -- Network or RemoteInteractive
OR timestamp(epoch=System.TimeCreated.SystemTime).Hour NOT IN (8,9,10,11,12,13,14,15,16,17) -- Off-hours
)
ORDER BY LogonTime DESC
```
### T1566: Phishing
**Artifacts**:
- `Windows.Forensics.Lnk`
- `Windows.Applications.Office.Keywords`
**VQL Query**:
```sql
-- Suspicious Office document execution
SELECT FullPath,
Mtime,
read_file(filename=FullPath, length=100000) AS Content
FROM glob(globs=[
"C:/Users/*/Downloads/**/*.doc*",
"C:/Users/*/Downloads/**/*.xls*"
])
WHERE Content =~ "(?i)(macro|vba|shell|exec|powershell)"
AND Mtime > timestamp(epoch=now() - 604800)
```
## Execution
### T1059.001: PowerShell
**Artifacts**:
- `Windows.EventLogs.PowershellScriptblock`
- `Windows.System.Powershell.PSReadline`
**VQL Query**:
```sql
-- Malicious PowerShell execution
SELECT timestamp(epoch=System.TimeCreated.SystemTime) AS ExecutionTime,
EventData.ScriptBlockText AS Command,
EventData.Path AS ScriptPath
FROM parse_evtx(filename="C:/Windows/System32/winevt/Logs/Microsoft-Windows-PowerShell%4Operational.evtx")
WHERE System.EventID.Value = 4104 -- Script Block Logging
AND EventData.ScriptBlockText =~ "(?i)(invoke-expression|iex|downloadstring|webclient|bypass|hidden|encodedcommand)"
ORDER BY ExecutionTime DESC
```
### T1059.003: Windows Command Shell
**Artifacts**:
- `Windows.System.Pslist`
- `Windows.EventLogs.ProcessCreation`
**VQL Query**:
```sql
-- Suspicious cmd.exe usage
SELECT Pid, Ppid, Name, CommandLine, Username, CreateTime
FROM pslist()
WHERE Name =~ "(?i)cmd.exe"
AND CommandLine =~ "(?i)(/c|/k|/r)"
AND Ppid IN (
SELECT Pid FROM pslist()
WHERE Name =~ "(?i)(winword|excel|powerpnt|acrobat|outlook)"
)
```
### T1053.005: Scheduled Task
**Artifacts**:
- `Windows.System.TaskScheduler`
- `Windows.EventLogs.ScheduledTasks`
**VQL Query**:
```sql
-- Recently created scheduled tasks
SELECT FullPath AS TaskPath,
parse_xml(file=FullPath).Task.Actions.Exec.Command AS Command,
parse_xml(file=FullPath).Task.Principals.Principal.UserId AS RunAsUser,
timestamp(epoch=Mtime) AS Created
FROM glob(globs="C:/Windows/System32/Tasks/**")
WHERE NOT IsDir
AND Mtime > timestamp(epoch=now() - 86400)
AND Command != ""
ORDER BY Created DESC
```
## Persistence
### T1547.001: Registry Run Keys
**Artifacts**:
- `Windows.Persistence.PermanentRuns`
- `Windows.System.StartupItems`
**VQL Query**:
```sql
-- Autorun registry entries
SELECT Key.FullPath AS RegistryKey,
ValueName,
ValueData.value AS ExecutablePath,
timestamp(epoch=Key.Mtime) AS LastModified
FROM read_reg_key(globs=[
"HKEY_LOCAL_MACHINE/SOFTWARE/Microsoft/Windows/CurrentVersion/Run/*",
"HKEY_LOCAL_MACHINE/SOFTWARE/Microsoft/Windows/CurrentVersion/RunOnce/*",
"HKEY_CURRENT_USER/SOFTWARE/Microsoft/Windows/CurrentVersion/Run/*",
"HKEY_LOCAL_MACHINE/SOFTWARE/WOW6432Node/Microsoft/Windows/CurrentVersion/Run/*"
])
WHERE ValueData.value != ""
ORDER BY LastModified DESC
```
### T1543.003: Windows Service
**Artifacts**:
- `Windows.System.Services`
- `Windows.EventLogs.ServiceCreation`
**VQL Query**:
```sql
-- Suspicious services
SELECT Key.Name AS ServiceName,
ImagePath.value AS ExecutablePath,
DisplayName.value AS DisplayName,
Start.value AS StartType,
timestamp(epoch=Key.Mtime) AS LastModified
FROM read_reg_key(globs="HKEY_LOCAL_MACHINE/SYSTEM/CurrentControlSet/Services/*")
WHERE ImagePath.value != ""
AND (
ImagePath.value =~ "(?i)(temp|appdata|users)"
OR ImagePath.value =~ "(?i)(powershell|cmd|wscript)"
OR Key.Mtime > timestamp(epoch=now() - 604800)
)
```
### T1546.003: WMI Event Subscription
**Artifacts**:
- `Windows.Persistence.PermanentWMIEvents`
**VQL Query**:
```sql
-- Malicious WMI event subscriptions
SELECT Namespace,
FilterName,
Query,
ConsumerName,
ConsumerType,
ConsumerData
FROM wmi(
query="SELECT * FROM __FilterToConsumerBinding",
namespace="ROOT/Subscription"
)
WHERE ConsumerData =~ "(?i)(powershell|cmd|wscript|executable)"
```
## Privilege Escalation
### T1548.002: Bypass User Account Control
**Artifacts**:
- `Windows.EventLogs.EvtxHunter` (EventID 4688 with elevated token)
**VQL Query**:
```sql
-- UAC bypass indicators
SELECT timestamp(epoch=System.TimeCreated.SystemTime) AS EventTime,
EventData.NewProcessName AS ProcessName,
EventData.CommandLine AS CommandLine,
EventData.ParentProcessName AS ParentProcess
FROM parse_evtx(filename="C:/Windows/System32/winevt/Logs/Security.evtx")
WHERE System.EventID.Value = 4688
AND EventData.TokenElevationType = "%%1937" -- Full token elevated
AND (
EventData.NewProcessName =~ "(?i)(fodhelper|computerdefaults|sdclt)"
OR EventData.CommandLine =~ "(?i)(eventvwr|ms-settings)"
)
```
### T1134: Access Token Manipulation
**Artifacts**:
- `Windows.EventLogs.EvtxHunter` (EventID 4672, 4673)
**VQL Query**:
```sql
-- Sensitive privilege use
SELECT timestamp(epoch=System.TimeCreated.SystemTime) AS EventTime,
EventData.SubjectUserName AS Username,
EventData.PrivilegeList AS Privileges
FROM parse_evtx(filename="C:/Windows/System32/winevt/Logs/Security.evtx")
WHERE System.EventID.Value = 4672
AND EventData.PrivilegeList =~ "(SeDebugPrivilege|SeTcbPrivilege|SeLoadDriverPrivilege)"
```
## Defense Evasion
### T1070.001: Clear Windows Event Logs
**Artifacts**:
- `Windows.EventLogs.Cleared`
**VQL Query**:
```sql
-- Event log clearing
SELECT timestamp(epoch=System.TimeCreated.SystemTime) AS ClearedTime,
System.Channel AS LogName,
EventData.SubjectUserName AS Username
FROM parse_evtx(filename="C:/Windows/System32/winevt/Logs/Security.evtx")
WHERE System.EventID.Value IN (1102, 104) -- Audit log cleared
ORDER BY ClearedTime DESC
```
### T1562.001: Disable or Modify Tools
**Artifacts**:
- `Windows.Forensics.Timeline`
- `Windows.Registry.RecentDocs`
**VQL Query**:
```sql
-- Security tool tampering
SELECT Key.FullPath AS RegistryKey,
ValueName,
ValueData.value AS Value,
timestamp(epoch=Key.Mtime) AS Modified
FROM read_reg_key(globs=[
"HKEY_LOCAL_MACHINE/SOFTWARE/Microsoft/Windows Defender/**",
"HKEY_LOCAL_MACHINE/SOFTWARE/Policies/Microsoft/Windows Defender/**",
"HKEY_LOCAL_MACHINE/SYSTEM/CurrentControlSet/Services/WinDefend/**"
])
WHERE (
ValueName =~ "(?i)(DisableAntiSpyware|DisableRealtimeMonitoring|Start)"
AND (ValueData.value = 1 OR ValueData.value = 4)
)
```
### T1055: Process Injection
**Artifacts**:
- `Windows.Detection.ProcessInjection`
- `Windows.Memory.Acquisition`
**VQL Query**:
```sql
-- Detect process injection via memory protections
SELECT Pid,
process_tracker_get(id=Pid).Name AS ProcessName,
Address,
Size,
Protection,
Type
FROM vad()
WHERE Protection =~ "EXECUTE.*WRITE" -- RWX memory
AND Type = "Private"
AND process_tracker_get(id=Pid).Name NOT IN ("chrome.exe", "firefox.exe") -- Exclude known JIT
```
## Credential Access
### T1003.001: LSASS Memory
**Artifacts**:
- `Windows.EventLogs.ProcessAccess`
- `Windows.Detection.Mimikatz`
**VQL Query**:
```sql
-- LSASS access attempts
SELECT timestamp(epoch=System.TimeCreated.SystemTime) AS AccessTime,
EventData.SourceProcessId AS SourcePID,
EventData.SourceImage AS SourceImage,
EventData.TargetImage AS TargetImage,
EventData.GrantedAccess AS AccessRights
FROM parse_evtx(filename="C:/Windows/System32/winevt/Logs/Microsoft-Windows-Sysmon%4Operational.evtx")
WHERE System.EventID.Value = 10 -- ProcessAccess
AND EventData.TargetImage =~ "(?i)lsass.exe"
AND EventData.GrantedAccess =~ "(0x1010|0x1410|0x143A)" -- Suspicious access rights
```
### T1003.002: Security Account Manager
**Artifacts**:
- `Windows.Forensics.SAM`
- `Windows.EventLogs.EvtxHunter`
**VQL Query**:
```sql
-- SAM registry hive access
SELECT FullPath,
timestamp(epoch=Atime) AS AccessTime,
timestamp(epoch=Mtime) AS ModifiedTime
FROM glob(globs=[
"C:/Windows/System32/config/SAM",
"C:/Windows/System32/config/SYSTEM",
"C:/Windows/System32/config/SECURITY"
])
WHERE Atime > timestamp(epoch=now() - 86400)
```
### T1555: Credentials from Password Stores
**Artifacts**:
- `Windows.Forensics.DPAPI`
- `Windows.Browsers.ChromeHistory`
**VQL Query**:
```sql
-- Browser credential access
SELECT FullPath,
timestamp(epoch=Atime) AS AccessTime
FROM glob(globs=[
"C:/Users/*/AppData/Local/Google/Chrome/User Data/*/Login Data",
"C:/Users/*/AppData/Roaming/Mozilla/Firefox/Profiles/*/logins.json"
])
WHERE Atime > timestamp(epoch=now() - 86400)
ORDER BY AccessTime DESC
```
## Discovery
### T1082: System Information Discovery
**Artifacts**:
- `Generic.Client.Info`
- `Windows.System.SystemInfo`
**VQL Query**:
```sql
-- System enumeration commands
SELECT Pid, Name, CommandLine, Username, CreateTime
FROM pslist()
WHERE CommandLine =~ "(?i)(systeminfo|whoami|ipconfig|hostname|ver)"
AND CreateTime > timestamp(epoch=now() - 3600)
ORDER BY CreateTime DESC
```
### T1083: File and Directory Discovery
**Artifacts**:
- `Windows.EventLogs.ProcessCreation`
**VQL Query**:
```sql
-- File system enumeration
SELECT Pid, Name, CommandLine, CreateTime
FROM pslist()
WHERE CommandLine =~ "(?i)(dir|tree|findstr|where)"
AND CommandLine =~ "(?i)(\\*|recursive|/s|/b)"
ORDER BY CreateTime DESC
```
### T1049: System Network Connections Discovery
**Artifacts**:
- `Windows.Network.Netstat`
**VQL Query**:
```sql
-- Network enumeration commands
SELECT Pid, Name, CommandLine, CreateTime
FROM pslist()
WHERE CommandLine =~ "(?i)(netstat|net use|net view|arp|route print|nslookup)"
ORDER BY CreateTime DESC
```
## Lateral Movement
### T1021.001: Remote Desktop Protocol
**Artifacts**:
- `Windows.EventLogs.RDP`
- `Windows.EventLogs.EvtxHunter`
**VQL Query**:
```sql
-- RDP lateral movement
SELECT timestamp(epoch=System.TimeCreated.SystemTime) AS LogonTime,
EventData.TargetUserName AS Username,
EventData.IpAddress AS SourceIP,
System.Computer AS DestinationHost
FROM parse_evtx(filename="C:/Windows/System32/winevt/Logs/Security.evtx")
WHERE System.EventID.Value = 4624
AND EventData.LogonType = 10 -- RemoteInteractive
AND EventData.IpAddress != "127.0.0.1"
ORDER BY LogonTime DESC
```
### T1021.002: SMB/Windows Admin Shares
**Artifacts**:
- `Windows.EventLogs.EvtxHunter` (EventID 5140, 5145)
**VQL Query**:
```sql
-- Admin share access
SELECT timestamp(epoch=System.TimeCreated.SystemTime) AS AccessTime,
EventData.SubjectUserName AS Username,
EventData.IpAddress AS SourceIP,
EventData.ShareName AS Share,
EventData.RelativeTargetName AS FilePath
FROM parse_evtx(filename="C:/Windows/System32/winevt/Logs/Security.evtx")
WHERE System.EventID.Value = 5140
AND EventData.ShareName =~ "(?i)(ADMIN\\$|C\\$|IPC\\$)"
```
### T1047: Windows Management Instrumentation
**Artifacts**:
- `Windows.EventLogs.WMIActivity`
- `Windows.System.Pslist`
**VQL Query**:
```sql
-- WMI process creation
SELECT Pid, Name, CommandLine, Username, CreateTime
FROM pslist()
WHERE (
-- WMI spawned processes
Ppid IN (SELECT Pid FROM pslist() WHERE Name =~ "(?i)wmiprvse.exe")
-- Or WMIC usage
OR (Name =~ "(?i)wmic.exe" AND CommandLine =~ "(?i)(process call create|/node:)")
)
ORDER BY CreateTime DESC
```
## Collection
### T1005: Data from Local System
**Artifacts**:
- `Windows.Forensics.Timeline`
- `Windows.Detection.Yara`
**VQL Query**:
```sql
-- Data staging detection
SELECT FullPath, Size,
timestamp(epoch=Ctime) AS Created,
timestamp(epoch=Mtime) AS Modified
FROM glob(globs=[
"C:/Users/*/AppData/**/*.zip",
"C:/Users/*/AppData/**/*.rar",
"C:/Users/*/AppData/**/*.7z",
"C:/Windows/Temp/**/*.zip"
])
WHERE Size > 10485760 -- > 10MB
AND Ctime > timestamp(epoch=now() - 86400)
ORDER BY Size DESC
```
### T1119: Automated Collection
**Artifacts**:
- `Windows.System.Pslist`
- `Windows.EventLogs.ProcessCreation`
**VQL Query**:
```sql
-- Automated collection tools
SELECT Pid, Name, CommandLine, Username, CreateTime
FROM pslist()
WHERE CommandLine =~ "(?i)(robocopy|xcopy|tar|7z|winrar)"
AND CommandLine =~ "(?i)(/s|recursive|mirror)"
```
## Exfiltration
### T1041: Exfiltration Over C2 Channel
**Artifacts**:
- `Windows.Network.NetstatEnriched`
- `Windows.Detection.NetworkAlerts`
**VQL Query**:
```sql
-- Large outbound transfers
SELECT Laddr.Port AS LocalPort,
Raddr.IP AS RemoteIP,
Raddr.Port AS RemotePort,
Pid,
process_tracker_get(id=Pid).Name AS ProcessName,
process_tracker_get(id=Pid).CommandLine AS CommandLine
FROM netstat()
WHERE Status = "ESTABLISHED"
AND Raddr.IP !~ "^(10\\.|172\\.(1[6-9]|2[0-9]|3[01])\\.|192\\.168\\.)"
AND Raddr.Port NOT IN (80, 443, 22)
```
### T1052: Exfiltration Over Physical Medium
**Artifacts**:
- `Windows.Forensics.USBDevices`
- `Windows.EventLogs.USBActivity`
**VQL Query**:
```sql
-- USB file transfers
SELECT FullPath, Size,
timestamp(epoch=Mtime) AS Modified
FROM glob(globs=["D:/**", "E:/**", "F:/**"]) -- Removable drives
WHERE Mtime > timestamp(epoch=now() - 86400)
AND Size > 1048576 -- > 1MB
ORDER BY Mtime DESC, Size DESC
```
## Command and Control
### T1071: Application Layer Protocol
**Artifacts**:
- `Windows.Network.NetstatEnriched`
- `Windows.Detection.Sigma`
**VQL Query**:
```sql
-- Unusual outbound connections
SELECT Raddr.IP AS RemoteIP,
Raddr.Port AS RemotePort,
COUNT(*) AS ConnectionCount,
GROUP_CONCAT(DISTINCT process_tracker_get(id=Pid).Name) AS Processes
FROM netstat()
WHERE Status = "ESTABLISHED"
AND Raddr.IP !~ "^(10\\.|172\\.(1[6-9]|2[0-9]|3[01])\\.|192\\.168\\.)"
AND Raddr.Port NOT IN (80, 443, 53, 22, 3389)
GROUP BY Raddr.IP, Raddr.Port
HAVING ConnectionCount > 10
```
### T1095: Non-Application Layer Protocol
**Artifacts**:
- `Windows.Network.RawConnections`
**VQL Query**:
```sql
-- Raw socket usage (ICMP tunneling, etc.)
SELECT Pid,
process_tracker_get(id=Pid).Name AS ProcessName,
process_tracker_get(id=Pid).CommandLine AS CommandLine,
Protocol,
Laddr.IP AS LocalIP,
Raddr.IP AS RemoteIP
FROM netstat()
WHERE Protocol NOT IN ("TCP", "UDP")
AND Raddr.IP != ""
```
### T1219: Remote Access Software
**Artifacts**:
- `Windows.System.Pslist`
- `Windows.Persistence.PermanentRuns`
**VQL Query**:
```sql
-- Remote access tools
SELECT Pid, Name, Exe, CommandLine, Username
FROM pslist()
WHERE Name =~ "(?i)(teamviewer|anydesk|logmein|ammyy|vnc|radmin|screenconnect)"
OR Exe =~ "(?i)(remote|rdp|desktop|viewer)"
```

535
skills/incident-response/ir-velociraptor/references/vql-patterns.md Normal file
View File

@@ -0,0 +1,535 @@
# VQL Query Patterns for Incident Response
Comprehensive VQL query patterns for common incident response and threat hunting scenarios.
## Table of Contents
- [Process Analysis](#process-analysis)
- [Network Forensics](#network-forensics)
- [File System Analysis](#file-system-analysis)
- [Registry Forensics](#registry-forensics)
- [Memory Analysis](#memory-analysis)
- [Event Log Analysis](#event-log-analysis)
- [Persistence Mechanisms](#persistence-mechanisms)
- [Lateral Movement Detection](#lateral-movement-detection)
- [Data Exfiltration](#data-exfiltration)
- [Malware Analysis](#malware-analysis)
## Process Analysis
### Suspicious Process Detection
```sql
-- Processes with suspicious characteristics
SELECT Pid, Ppid, Name, CommandLine, Username, Exe, CreateTime
FROM pslist()
WHERE (
-- Suspicious parent-child relationships
(Ppid IN (SELECT Pid FROM pslist() WHERE Name =~ "(?i)(winword|excel|powerpnt|acrobat)")
AND Name =~ "(?i)(powershell|cmd|wscript|cscript)")
-- Processes running from temp directories
OR Exe =~ "(?i)(temp|tmp|appdata)"
-- Processes with obfuscated command lines
OR CommandLine =~ "(?i)(iex|invoke-expression|downloadstring|webclient|hidden|bypass)"
)
```
### Living-off-the-Land Binaries (LOLBins)
```sql
-- Detect abuse of legitimate Windows binaries
SELECT Pid, Name, CommandLine, Username, Exe
FROM pslist()
WHERE (
-- certutil for downloading
(Name =~ "(?i)certutil" AND CommandLine =~ "(?i)(urlcache|url)")
-- bitsadmin for downloading
OR (Name =~ "(?i)bitsadmin" AND CommandLine =~ "(?i)(transfer|download)")
-- mshta for code execution
OR (Name =~ "(?i)mshta" AND CommandLine =~ "(?i)(http|javascript|vbscript)")
-- rundll32 suspicious usage
OR (Name =~ "(?i)rundll32" AND CommandLine =~ "(?i)(javascript|url)")
)
```
### Process Injection Detection
```sql
-- Identify potential process injection
SELECT Pid, Name,
AllocatedMemory,
ProtectionFlags,
Handles
FROM handles()
WHERE Type = "Section"
AND ProtectionFlags =~ "EXECUTE"
AND Name != ""
```
## Network Forensics
### External Connections
```sql
-- All external network connections with process context
SELECT Laddr.IP AS LocalIP,
Laddr.Port AS LocalPort,
Raddr.IP AS RemoteIP,
Raddr.Port AS RemotePort,
Status, Pid,
process_tracker_get(id=Pid).Name AS ProcessName,
process_tracker_get(id=Pid).Exe AS ProcessPath,
process_tracker_get(id=Pid).CommandLine AS CommandLine
FROM netstat()
WHERE Status = "ESTABLISHED"
AND Raddr.IP != ""
AND Raddr.IP !~ "^(10\\.|172\\.(1[6-9]|2[0-9]|3[01])\\.|192\\.168\\.)" -- Exclude RFC1918
AND Raddr.IP !~ "^(127\\.|169\\.254\\.)" -- Exclude localhost and link-local
```
### Unusual Port Activity
```sql
-- Connections on unusual ports
SELECT Raddr.IP AS RemoteIP,
Raddr.Port AS RemotePort,
COUNT(*) AS ConnectionCount,
GROUP_CONCAT(DISTINCT process_tracker_get(id=Pid).Name) AS Processes
FROM netstat()
WHERE Status = "ESTABLISHED"
AND Raddr.Port NOT IN (80, 443, 22, 3389, 445, 139, 53)
GROUP BY Raddr.IP, Raddr.Port
HAVING ConnectionCount > 5
```
### DNS Query Analysis
```sql
-- Suspicious DNS queries
SELECT query AS Domain,
response AS IPAddress,
timestamp(epoch=Time) AS QueryTime
FROM parse_evtx(filename="C:/Windows/System32/winevt/Logs/Microsoft-Windows-DNS-Client%4Operational.evtx")
WHERE System.EventID.Value = 3008
AND (
-- Long domain names (possible DGA)
length(query) > 50
-- High entropy domains
OR query =~ "[a-z0-9]{20,}"
-- Suspicious TLDs
OR query =~ "\\.(tk|ml|ga|cf|gq)$"
)
```
## File System Analysis
### Recently Modified Executables
```sql
-- Executables modified in last 7 days
SELECT FullPath, Size,
timestamp(epoch=Mtime) AS ModifiedTime,
timestamp(epoch=Ctime) AS CreatedTime,
hash(path=FullPath, accessor="file") AS SHA256
FROM glob(globs=[
"C:/Windows/System32/**/*.exe",
"C:/Windows/SysWOW64/**/*.exe",
"C:/Users/*/AppData/**/*.exe",
"C:/ProgramData/**/*.exe"
])
WHERE Mtime > timestamp(epoch=now() - 604800) -- 7 days
ORDER BY Mtime DESC
```
### Webshell Detection
```sql
-- Potential webshells in web directories
SELECT FullPath, Size,
timestamp(epoch=Mtime) AS ModifiedTime,
read_file(filename=FullPath, length=1000) AS Content
FROM glob(globs=[
"C:/inetpub/wwwroot/**/*.asp",
"C:/inetpub/wwwroot/**/*.aspx",
"C:/inetpub/wwwroot/**/*.php",
"C:/xampp/htdocs/**/*.php"
])
WHERE Content =~ "(?i)(eval|base64_decode|exec|shell_exec|system|passthru|WScript\\.Shell)"
OR FullPath =~ "(?i)(cmd|shell|upload|backdoor|c99)"
```
### Suspicious File Timestamps
```sql
-- Files with timestamp anomalies (timestomping detection)
SELECT FullPath,
timestamp(epoch=Mtime) AS ModifiedTime,
timestamp(epoch=Ctime) AS ChangeTime,
timestamp(epoch=Btime) AS BornTime
FROM glob(globs="C:/Users/**/*.exe")
WHERE Mtime < Btime -- Modified time before birth time (anomaly)
OR Ctime < Btime -- Change time before birth time
```
## Registry Forensics
### Autorun Locations
```sql
-- Comprehensive autorun registry key enumeration
SELECT Key.FullPath AS RegistryPath,
ValueName,
ValueData.value AS Value,
timestamp(epoch=Key.Mtime) AS LastModified
FROM read_reg_key(globs=[
"HKEY_LOCAL_MACHINE/SOFTWARE/Microsoft/Windows/CurrentVersion/Run/*",
"HKEY_LOCAL_MACHINE/SOFTWARE/Microsoft/Windows/CurrentVersion/RunOnce/*",
"HKEY_CURRENT_USER/SOFTWARE/Microsoft/Windows/CurrentVersion/Run/*",
"HKEY_LOCAL_MACHINE/SOFTWARE/WOW6432Node/Microsoft/Windows/CurrentVersion/Run/*",
"HKEY_LOCAL_MACHINE/SYSTEM/CurrentControlSet/Services/*"
])
WHERE ValueData.value != ""
```
### Recent Registry Modifications
```sql
-- Recently modified registry keys in security-sensitive locations
SELECT FullPath,
timestamp(epoch=Mtime) AS ModifiedTime
FROM glob(globs=[
"HKEY_LOCAL_MACHINE/SOFTWARE/Microsoft/Windows/CurrentVersion/**",
"HKEY_LOCAL_MACHINE/SYSTEM/CurrentControlSet/**",
"HKEY_CURRENT_USER/SOFTWARE/Microsoft/Windows/CurrentVersion/**"
], accessor="registry")
WHERE Mtime > timestamp(epoch=now() - 86400) -- Last 24 hours
ORDER BY Mtime DESC
```
### AppInit DLL Injection
```sql
-- Detect AppInit DLL injection mechanism
SELECT ValueName,
ValueData.value AS DLLPath,
timestamp(epoch=Key.Mtime) AS LastModified
FROM read_reg_key(globs=[
"HKEY_LOCAL_MACHINE/SOFTWARE/Microsoft/Windows NT/CurrentVersion/Windows/AppInit_DLLs",
"HKEY_LOCAL_MACHINE/SOFTWARE/WOW6432Node/Microsoft/Windows NT/CurrentVersion/Windows/AppInit_DLLs"
])
WHERE ValueData.value != ""
```
## Memory Analysis
### Suspicious Memory Regions
```sql
-- Memory regions with unusual protections
SELECT Pid,
process_tracker_get(id=Pid).Name AS ProcessName,
Address,
Size,
Protection
FROM vad()
WHERE Protection =~ "EXECUTE.*WRITE" -- RWX memory (suspicious)
AND Type = "Private"
```
### Injected Code Detection
```sql
-- Detect potentially injected code
SELECT Pid,
Name AS ProcessName,
Vad.Address AS MemoryAddress,
Vad.Protection AS Protection,
Vad.Type AS MemoryType
FROM pslist()
LET Vad <= SELECT * FROM vad(pid=Pid)
WHERE Vad.Protection =~ "EXECUTE"
AND Vad.Type = "Private"
AND Vad.Name = ""
```
## Event Log Analysis
### Failed Logon Attempts
```sql
-- Failed authentication attempts
SELECT timestamp(epoch=System.TimeCreated.SystemTime) AS EventTime,
EventData.TargetUserName AS Username,
EventData.IpAddress AS SourceIP,
EventData.WorkstationName AS Workstation,
EventData.FailureReason AS Reason
FROM parse_evtx(filename="C:/Windows/System32/winevt/Logs/Security.evtx")
WHERE System.EventID.Value = 4625 -- Failed logon
ORDER BY EventTime DESC
LIMIT 1000
```
### Privilege Escalation Events
```sql
-- Privilege elevation and sensitive privilege use
SELECT timestamp(epoch=System.TimeCreated.SystemTime) AS EventTime,
System.EventID.Value AS EventID,
EventData.SubjectUserName AS User,
EventData.PrivilegeList AS Privileges
FROM parse_evtx(filename="C:/Windows/System32/winevt/Logs/Security.evtx")
WHERE System.EventID.Value IN (4672, 4673, 4674) -- Special privilege events
AND EventData.PrivilegeList =~ "(SeDebugPrivilege|SeTcbPrivilege|SeLoadDriverPrivilege)"
```
### Scheduled Task Creation
```sql
-- Detect scheduled task creation for persistence
SELECT timestamp(epoch=System.TimeCreated.SystemTime) AS EventTime,
EventData.TaskName AS TaskName,
EventData.UserContext AS RunAsUser,
EventData.TaskContent AS TaskXML
FROM parse_evtx(filename="C:/Windows/System32/winevt/Logs/Microsoft-Windows-TaskScheduler%4Operational.evtx")
WHERE System.EventID.Value = 106 -- Task registered
ORDER BY EventTime DESC
```
## Persistence Mechanisms
### Comprehensive Persistence Hunt
```sql
-- Multi-vector persistence detection
LET RegistryAutoRuns = SELECT "Registry" AS Method, Key.FullPath AS Location, ValueData.value AS Value
FROM read_reg_key(globs="HKEY_LOCAL_MACHINE/SOFTWARE/Microsoft/Windows/CurrentVersion/Run/*")
LET ScheduledTasks = SELECT "Scheduled Task" AS Method, FullPath AS Location, "" AS Value
FROM glob(globs="C:/Windows/System32/Tasks/**")
WHERE NOT IsDir
LET Services = SELECT "Service" AS Method, Key.Name AS Location, ImagePath.value AS Value
FROM read_reg_key(globs="HKEY_LOCAL_MACHINE/SYSTEM/CurrentControlSet/Services/**/ImagePath")
LET StartupFolders = SELECT "Startup Folder" AS Method, FullPath AS Location, "" AS Value
FROM glob(globs=[
"C:/Users/*/AppData/Roaming/Microsoft/Windows/Start Menu/Programs/Startup/*",
"C:/ProgramData/Microsoft/Windows/Start Menu/Programs/Startup/*"
])
SELECT * FROM chain(
a=RegistryAutoRuns,
b=ScheduledTasks,
c=Services,
d=StartupFolders
)
```
### WMI Event Subscription Persistence
```sql
-- Detect malicious WMI event subscriptions
SELECT Name,
EventFilter,
Consumer,
timestamp(epoch=CreationDate) AS Created
FROM wmi_persist()
WHERE EventFilter != "" OR Consumer != ""
```
## Lateral Movement Detection
### PsExec Activity
```sql
-- PsExec service creation and execution
SELECT timestamp(epoch=System.TimeCreated.SystemTime) AS EventTime,
EventData.ServiceName AS ServiceName,
EventData.ImagePath AS ExecutablePath,
EventData.AccountName AS Account
FROM parse_evtx(filename="C:/Windows/System32/winevt/Logs/System.evtx")
WHERE System.EventID.Value = 7045 -- Service installed
AND (
EventData.ServiceName =~ "(?i)PSEXESVC"
OR EventData.ImagePath =~ "(?i)(\\\\\\\\.*\\\\.*\\\\|admin\\$|c\\$)"
)
```
### Remote Desktop Activity
```sql
-- RDP logon activity
SELECT timestamp(epoch=System.TimeCreated.SystemTime) AS LogonTime,
EventData.TargetUserName AS Username,
EventData.IpAddress AS SourceIP,
EventData.LogonType AS LogonType
FROM parse_evtx(filename="C:/Windows/System32/winevt/Logs/Security.evtx")
WHERE System.EventID.Value = 4624 -- Successful logon
AND EventData.LogonType = 10 -- RemoteInteractive (RDP)
ORDER BY LogonTime DESC
```
### SMB/Admin Share Access
```sql
-- Network share access from remote systems
SELECT timestamp(epoch=System.TimeCreated.SystemTime) AS AccessTime,
EventData.SubjectUserName AS Username,
EventData.IpAddress AS SourceIP,
EventData.ShareName AS ShareAccessed,
EventData.ObjectName AS FileAccessed
FROM parse_evtx(filename="C:/Windows/System32/winevt/Logs/Security.evtx")
WHERE System.EventID.Value = 5140 -- Network share accessed
AND EventData.ShareName =~ "(?i)(ADMIN\\$|C\\$|IPC\\$)"
```
## Data Exfiltration
### Large File Transfers
```sql
-- Files copied to removable media or network shares
SELECT FullPath,
Size,
timestamp(epoch=Mtime) AS LastModified,
hash(path=FullPath, accessor="file").SHA256 AS SHA256
FROM glob(globs=[
"D:/**", -- Removable drive
"E:/**",
"\\\\*/**" -- Network paths
])
WHERE Size > 10485760 -- Files larger than 10MB
AND Mtime > timestamp(epoch=now() - 86400)
ORDER BY Size DESC
```
### USB Device History
```sql
-- USB device connection history
SELECT Key.Name AS DeviceID,
FriendlyName.value AS DeviceName,
timestamp(epoch=Key.Mtime) AS LastConnected
FROM read_reg_key(globs="HKEY_LOCAL_MACHINE/SYSTEM/CurrentControlSet/Enum/USBSTOR/**/FriendlyName")
ORDER BY LastConnected DESC
```
### Cloud Storage Activity
```sql
-- Files in cloud sync directories
SELECT FullPath, Size,
timestamp(epoch=Mtime) AS LastModified
FROM glob(globs=[
"C:/Users/*/OneDrive/**",
"C:/Users/*/Dropbox/**",
"C:/Users/*/Google Drive/**"
])
WHERE Mtime > timestamp(epoch=now() - 86400)
ORDER BY Mtime DESC
```
## Malware Analysis
### Suspicious File Indicators
```sql
-- Files with malware-associated characteristics
SELECT FullPath,
Size,
timestamp(epoch=Mtime) AS ModifiedTime,
hash(path=FullPath, accessor="file") AS Hashes
FROM glob(globs=[
"C:/Windows/Temp/**/*.exe",
"C:/Users/*/AppData/Local/Temp/**/*.exe",
"C:/ProgramData/**/*.exe"
])
WHERE (
-- Small executables (potential droppers)
Size < 102400
-- Or recently created
OR Mtime > timestamp(epoch=now() - 3600)
)
```
### Packed Executable Detection
```sql
-- Detect potentially packed executables (high entropy)
SELECT FullPath,
parse_pe(file=FullPath).Entropy AS Entropy,
parse_pe(file=FullPath).Sections AS Sections
FROM glob(globs="C:/Users/**/*.exe")
WHERE parse_pe(file=FullPath).Entropy > 7.0 -- High entropy suggests packing
```
### Malicious Scripts
```sql
-- Suspicious PowerShell/VBS scripts
SELECT FullPath,
Size,
timestamp(epoch=Mtime) AS ModifiedTime,
read_file(filename=FullPath, length=5000) AS Content
FROM glob(globs=[
"C:/Users/**/*.ps1",
"C:/Users/**/*.vbs",
"C:/Users/**/*.js",
"C:/Windows/Temp/**/*.ps1"
])
WHERE Content =~ "(?i)(invoke-expression|iex|downloadstring|webclient|bypass|hidden|encodedcommand)"
```
## Advanced Hunting Patterns
### Threat Hunting with Multiple Indicators
```sql
-- Correlate multiple suspicious indicators
LET SuspiciousProcesses = SELECT Pid, Name, CommandLine
FROM pslist()
WHERE CommandLine =~ "(?i)(bypass|hidden|encodedcommand)"
LET SuspiciousConnections = SELECT Pid, Raddr.IP AS RemoteIP
FROM netstat()
WHERE Status = "ESTABLISHED"
AND Raddr.IP !~ "^(10\\.|172\\.(1[6-9]|2[0-9]|3[01])\\.|192\\.168\\.)"
SELECT sp.Pid,
sp.Name,
sp.CommandLine,
GROUP_CONCAT(sc.RemoteIP) AS ConnectedIPs
FROM SuspiciousProcesses sp
JOIN SuspiciousConnections sc ON sp.Pid = sc.Pid
GROUP BY sp.Pid
```
### Timeline Analysis
```sql
-- Comprehensive timeline of system activity
SELECT timestamp(epoch=Timestamp) AS EventTime,
Source,
EventType,
Details
FROM chain(
a={SELECT Mtime AS Timestamp, "FileSystem" AS Source, "FileCreated" AS EventType, FullPath AS Details
FROM glob(globs="C:/Users/**") WHERE Mtime > timestamp(epoch=now() - 86400)},
b={SELECT System.TimeCreated.SystemTime AS Timestamp, "EventLog" AS Source,
format(format="EventID:%v", args=System.EventID.Value) AS EventType,
EventData AS Details
FROM parse_evtx(filename="C:/Windows/System32/winevt/Logs/Security.evtx")
WHERE System.TimeCreated.SystemTime > timestamp(epoch=now() - 86400)},
c={SELECT Key.Mtime AS Timestamp, "Registry" AS Source, "KeyModified" AS EventType, Key.FullPath AS Details
FROM glob(globs="HKEY_LOCAL_MACHINE/SOFTWARE/**", accessor="registry")
WHERE Key.Mtime > timestamp(epoch=now() - 86400)}
)
ORDER BY EventTime DESC
```