Initial commit
This commit is contained in:
460
skills/dependency-evaluator/ECOSYSTEM_GUIDES.md
Normal file
460
skills/dependency-evaluator/ECOSYSTEM_GUIDES.md
Normal file
@@ -0,0 +1,460 @@
|
||||
# Ecosystem-Specific Evaluation Guides
|
||||
|
||||
Different language ecosystems have different norms, risks, and best practices. Use this guide to adjust your evaluation criteria based on the package ecosystem.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
- [Ecosystem Baselines](#ecosystem-baselines)
|
||||
- [Node.js / npm](#nodejs--npm)
|
||||
- [Python / PyPI](#python--pypi)
|
||||
- [Rust / Cargo](#rust--cargo)
|
||||
- [Go](#go)
|
||||
- [Ruby / RubyGems](#ruby--rubygems)
|
||||
- [Java / Maven Central](#java--maven-central)
|
||||
- [Cross-Ecosystem Patterns](#cross-ecosystem-patterns)
|
||||
- [Adjusting Your Evaluation](#adjusting-your-evaluation)
|
||||
|
||||
---
|
||||
|
||||
## Ecosystem Baselines
|
||||
|
||||
Use these baselines for ecosystem-relative comparisons. These represent typical patterns as of 2025; use as context not rigid rules.
|
||||
|
||||
### Release Cadence Norms
|
||||
|
||||
| Ecosystem | Actively Developed | Mature/Stable | Concerning |
|
||||
|-----------|-------------------|---------------|------------|
|
||||
| npm | Monthly+ releases | Quarterly releases | >6 months no release |
|
||||
| PyPI | Monthly-quarterly | Bi-annual releases | >9 months no release |
|
||||
| Cargo | Bi-monthly to quarterly | Annual releases OK | >12 months no release |
|
||||
| Go | Quarterly typical | Annual releases OK | >12 months no release |
|
||||
| RubyGems | Monthly for Rails-related | Quarterly for utilities | >6 months no release |
|
||||
| Maven | Quarterly typical | Bi-annual for mature | >9 months no release |
|
||||
|
||||
**Key:** "Concerning" means outlier for actively developed packages; mature packages may legitimately have longer gaps.
|
||||
|
||||
### Dependency Count Norms
|
||||
|
||||
| Ecosystem | Light | Typical | Heavy | Extreme |
|
||||
|-----------|-------|---------|-------|---------|
|
||||
| npm | <10 | 20-50 | 100-150 | 200+ |
|
||||
| PyPI | <5 | 10-30 | 50-80 | 100+ |
|
||||
| Cargo | <10 | 20-40 | 60-80 | 100+ |
|
||||
| Go | <5 | 5-20 | 30-40 | 50+ |
|
||||
| RubyGems | <5 | 10-25 | 40-60 | 80+ |
|
||||
| Maven | <10 | 20-50 | 80-120 | 150+ |
|
||||
|
||||
**Counts are total transitive dependencies.** Adjust expectations based on package type (frameworks have more).
|
||||
|
||||
### Download Thresholds (Weekly)
|
||||
|
||||
| Ecosystem | Niche | Moderate | Popular | Very Popular |
|
||||
|-----------|-------|----------|---------|--------------|
|
||||
| npm | <500 | 1k-10k | 50k-100k | 500k+ |
|
||||
| PyPI | <100 | 500-5k | 20k-50k | 200k+ |
|
||||
| Cargo | <50 | 200-2k | 10k-30k | 100k+ |
|
||||
| RubyGems | <100 | 500-5k | 20k-50k | 200k+ |
|
||||
|
||||
**Note:** Downloads alone don't indicate quality. Niche packages can be excellent; popular packages can be deprecated.
|
||||
|
||||
### Issue Response Time Norms
|
||||
|
||||
| Ecosystem | Excellent | Good | Acceptable | Concerning |
|
||||
|-----------|-----------|------|------------|------------|
|
||||
| npm (popular) | Hours-1 day | 2-7 days | 2-4 weeks | >1 month |
|
||||
| npm (smaller) | 1-3 days | 1-2 weeks | 1 month | >2 months |
|
||||
| PyPI | 1-3 days | 1-2 weeks | 3-4 weeks | >1 month |
|
||||
| Cargo | 1-2 days | 3-7 days | 2-3 weeks | >1 month |
|
||||
| Go | 1-3 days | 1-2 weeks | 3-4 weeks | >1 month |
|
||||
|
||||
**For security issues:** Expect 24-48hr acknowledgment regardless of ecosystem.
|
||||
|
||||
### Documentation Expectations
|
||||
|
||||
| Ecosystem | Minimum Expected | Excellent |
|
||||
|-----------|------------------|-----------|
|
||||
| npm | README with examples, TypeScript types | Dedicated docs site, migration guides, playground |
|
||||
| PyPI | README with examples, type hints | ReadTheDocs site, Sphinx docs, examples repo |
|
||||
| Cargo | README with examples, rustdoc | docs.rs complete, examples in repo, book/guide |
|
||||
| Go | README with examples, godoc | pkg.go.dev complete, examples, design docs |
|
||||
| RubyGems | README with examples | RDoc/YARD docs, Rails integration guide |
|
||||
|
||||
### Comparative Assessment Guidelines
|
||||
|
||||
**Use these baselines to ask:**
|
||||
- Is this package's release cadence below the norm for its ecosystem and maturity level?
|
||||
- Is the dependency count in the top quartile for similar packages in this ecosystem?
|
||||
- Is the issue response time significantly slower than ecosystem expectations?
|
||||
- Are downloads declining while ecosystem overall is growing?
|
||||
|
||||
**Example application:**
|
||||
- npm package with 150 transitive deps → "Heavy" but not extreme; acceptable for framework, concerning for utility
|
||||
- Cargo crate with no release in 10 months → Not yet concerning for mature stable crate
|
||||
- PyPI package with 200 deps → Extreme; investigate why so many
|
||||
- Go module with 40 deps → Unusual for Go (stdlib-first culture); investigate
|
||||
|
||||
---
|
||||
|
||||
## Node.js / npm
|
||||
|
||||
### Ecosystem Characteristics
|
||||
- **Philosophy**: Micropackages are common; many tiny single-purpose modules
|
||||
- **Package count**: Over 2 million packages (largest ecosystem)
|
||||
- **Dependency culture**: Deep dependency trees are normalized
|
||||
- **Versioning**: Semver is standard but not always followed strictly
|
||||
|
||||
### Unique Risks
|
||||
|
||||
**Left-pad Risk**
|
||||
The infamous "left-pad incident" (2016) highlighted npm's vulnerability to tiny, critical packages being removed. Characteristics:
|
||||
- Single-function packages with disproportionate usage
|
||||
- High download counts but minimal functionality
|
||||
- Supply chain risk when widely used packages are yanked
|
||||
|
||||
**npm-specific Supply Chain Attacks**
|
||||
- Typosquatting is common (react vs. reakt)
|
||||
- Package name confusion attacks
|
||||
- Malicious install scripts in postinstall hooks
|
||||
- Maintainer account compromises
|
||||
|
||||
### What to Watch For
|
||||
- Packages with hundreds of transitive dependencies for simple tasks
|
||||
- Postinstall scripts that download external code
|
||||
- Packages that wrap simple native functionality unnecessarily
|
||||
- Extremely high download counts but minimal GitHub activity (bot inflation)
|
||||
|
||||
### Preferred Patterns
|
||||
- Packages with minimal dependencies
|
||||
- Well-established micro-utilities from trusted authors
|
||||
- Scoped packages (@organization/package) from known orgs
|
||||
- Packages with verified publishers
|
||||
|
||||
### Recommended Tools
|
||||
```bash
|
||||
npm ls --all # Visualize full dependency tree
|
||||
npm audit # Security vulnerability scanning
|
||||
npm pack --dry-run # Check bundle size
|
||||
```
|
||||
|
||||
### Ecosystem-Specific Red Flags
|
||||
- Packages requiring sudo or elevated permissions
|
||||
- Packages with network calls in postinstall
|
||||
- Packages with native dependencies when pure JS would suffice
|
||||
- Suspicious similarity to popular package names
|
||||
|
||||
### Ecosystem-Specific Green Flags
|
||||
- TypeScript type definitions included
|
||||
- ES modules support
|
||||
- Tree-shakeable exports
|
||||
- Zero dependencies for utility packages
|
||||
|
||||
## Python / PyPI
|
||||
|
||||
### Ecosystem Characteristics
|
||||
- **Philosophy**: "Batteries included" - stdlib-first approach
|
||||
- **Package count**: Over 400,000 packages
|
||||
- **Dependency culture**: Lighter dependency trees than npm
|
||||
- **Versioning**: Mix of semver and date-based versioning
|
||||
|
||||
### Unique Risks
|
||||
|
||||
**PyPI Supply Chain Attacks**
|
||||
- Notable typosquatting incidents (e.g., python3-dateutil vs. dateutil)
|
||||
- Malicious packages targeting data scientists (fake ML libraries)
|
||||
- Native code in wheels may contain malware
|
||||
- setup.py can execute arbitrary code during install
|
||||
|
||||
**Dependency Confusion**
|
||||
- Public PyPI packages with same names as private packages
|
||||
- pip installs public version instead of intended private one
|
||||
|
||||
### What to Watch For
|
||||
- Packages with names very similar to popular packages
|
||||
- Unusual wheel distributions without source code
|
||||
- Packages targeting specific communities (ML, data science) with suspicious features
|
||||
- setup.py files with network calls or obfuscated code
|
||||
|
||||
### Preferred Patterns
|
||||
- Packages from known maintainers and organizations
|
||||
- Packages with signed releases (GPG signatures)
|
||||
- Pure Python packages (no compiled extensions) when possible
|
||||
- Packages maintained by Python Software Foundation or sub-projects
|
||||
|
||||
### Recommended Tools
|
||||
```bash
|
||||
pip show <package> # View package metadata
|
||||
pip index versions <package> # Check version history
|
||||
# Use pip-audit for security scanning (install separately)
|
||||
```
|
||||
|
||||
### Ecosystem-Specific Red Flags
|
||||
- Packages requesting unnecessary permissions in setup
|
||||
- Typosquatting of popular packages (reqeusts vs. requests)
|
||||
- Obfuscated code in setup.py
|
||||
- Wheels only (no source distribution)
|
||||
|
||||
### Ecosystem-Specific Green Flags
|
||||
- Listed in Python Packaging Authority (PyPA)
|
||||
- Type hints (PEP 484) included
|
||||
- Both source distributions and wheels available
|
||||
- Active maintenance by known Python community members
|
||||
|
||||
## Rust / Cargo
|
||||
|
||||
### Ecosystem Characteristics
|
||||
- **Philosophy**: Safety and correctness-first; explicit is better than implicit
|
||||
- **Package count**: Over 100,000 crates
|
||||
- **Dependency culture**: Moderate dependencies; emphasis on correctness
|
||||
- **Versioning**: Strict semver adherence is cultural norm
|
||||
|
||||
### Unique Strengths
|
||||
- Strong compile-time guarantees reduce certain vulnerability classes
|
||||
- Cargo's built-in tooling is excellent (cargo tree, cargo metadata)
|
||||
- Culture of good documentation (docs.rs)
|
||||
- `#![forbid(unsafe_code)]` for packages avoiding unsafe blocks
|
||||
|
||||
### What to Watch For
|
||||
- Crates pulling in many proc-macro dependencies (slow compile times)
|
||||
- Heavy use of `unsafe` blocks without justification
|
||||
- Transitive dependencies with unsafe code when unnecessary
|
||||
- Version conflicts in dependency tree (Cargo is strict about this)
|
||||
|
||||
### Preferred Patterns
|
||||
- Crates with `#![forbid(unsafe_code)]` for non-performance-critical code
|
||||
- Well-documented use of unsafe with safety invariants explained
|
||||
- Minimal proc-macro dependencies
|
||||
- Idiomatic Rust patterns
|
||||
|
||||
### Recommended Tools
|
||||
```bash
|
||||
cargo tree -p <crate> # Dependency tree visualization
|
||||
cargo metadata --format-version 1 # Machine-readable metadata
|
||||
cargo audit # Security vulnerability scanning (install separately)
|
||||
```
|
||||
|
||||
### Ecosystem-Specific Red Flags
|
||||
- Excessive unsafe code without documentation
|
||||
- Non-idiomatic Rust (indicates unfamiliarity)
|
||||
- Proc-macro heavy for simple functionality
|
||||
- Breaking semver (very rare in Rust ecosystem)
|
||||
|
||||
### Ecosystem-Specific Green Flags
|
||||
- Published on docs.rs with comprehensive documentation
|
||||
- `#![forbid(unsafe_code)]` or well-justified unsafe usage
|
||||
- Fast compile times relative to functionality
|
||||
- Active maintenance by Rust community members
|
||||
- Inclusion in "awesome-rust" lists
|
||||
|
||||
## Go
|
||||
|
||||
### Ecosystem Characteristics
|
||||
- **Philosophy**: Simplicity, minimalism, stdlib-first
|
||||
- **Package count**: Smaller than npm/PyPI (by design)
|
||||
- **Dependency culture**: Fewer dependencies is idiomatic
|
||||
- **Versioning**: Go modules with semantic versioning
|
||||
|
||||
### Unique Strengths
|
||||
- Strong standard library reduces dependency needs
|
||||
- Built-in dependency management (go mod)
|
||||
- Static linking produces standalone binaries
|
||||
- Import paths explicitly reference source repositories
|
||||
|
||||
### What to Watch For
|
||||
- Packages that wrap stdlib with minimal added value
|
||||
- Deep dependency trees (unusual in Go)
|
||||
- Packages that violate Go idioms and conventions
|
||||
- Module paths not matching repository structure
|
||||
|
||||
### Preferred Patterns
|
||||
- Prefer stdlib solutions when available
|
||||
- Minimal external dependencies
|
||||
- Clear, simple APIs following Go conventions
|
||||
- Well-structured module paths (github.com/org/project)
|
||||
|
||||
### Recommended Tools
|
||||
```bash
|
||||
go list -m -versions <module> # List module versions
|
||||
go mod graph # Dependency graph
|
||||
go mod why <module> # Why is this dependency included
|
||||
```
|
||||
|
||||
### Ecosystem-Specific Red Flags
|
||||
- Wrapping stdlib unnecessarily
|
||||
- Complex APIs when simple would suffice
|
||||
- Not following Go Project Layout
|
||||
- Vendoring dependencies (uncommon with go mod)
|
||||
|
||||
### Ecosystem-Specific Green Flags
|
||||
- Minimal dependencies (< 5 direct deps)
|
||||
- Follows effective Go guidelines
|
||||
- Clear documentation and examples
|
||||
- Used in prominent Go projects
|
||||
|
||||
## Ruby / RubyGems
|
||||
|
||||
### Ecosystem Characteristics
|
||||
- **Philosophy**: Convention over configuration, developer happiness
|
||||
- **Package count**: Over 175,000 gems
|
||||
- **Dependency culture**: Moderate; gems often do a lot
|
||||
- **Versioning**: Generally follows semver
|
||||
|
||||
### Unique Characteristics
|
||||
- Gems often monkey-patch core classes (can cause conflicts)
|
||||
- Rails ecosystem dominates Ruby gem ecosystem
|
||||
- Strong community conventions
|
||||
|
||||
### What to Watch For
|
||||
- Gems that extensively monkey-patch core classes
|
||||
- Dependencies that conflict with Rails (if using Rails)
|
||||
- Gems that override standard library behavior
|
||||
- Unmaintained gems for Rails version compatibility
|
||||
|
||||
### Preferred Patterns
|
||||
- Well-documented gems with clear upgrade paths
|
||||
- Gems that minimize monkey-patching
|
||||
- Rails-compatible versioning (if applicable)
|
||||
- Active maintenance matching Rails release cycles
|
||||
|
||||
### Recommended Tools
|
||||
```bash
|
||||
gem list <gem> # List installed versions
|
||||
gem dependency <gem> # Show dependencies
|
||||
bundle outdated # Check for updates (in bundler projects)
|
||||
```
|
||||
|
||||
### Ecosystem-Specific Red Flags
|
||||
- Extensive monkey-patching without documentation
|
||||
- Incompatibility with major Rails versions
|
||||
- Gems requiring old Ruby versions
|
||||
- No Bundler compatibility
|
||||
|
||||
### Ecosystem-Specific Green Flags
|
||||
- Rails-compatible (if relevant)
|
||||
- Minimal monkey-patching or well-documented overrides
|
||||
- Active maintenance matching Ruby version releases
|
||||
- Listed in awesome-ruby or Ruby Toolbox
|
||||
|
||||
## Java / Maven Central
|
||||
|
||||
### Ecosystem Characteristics
|
||||
- **Philosophy**: Enterprise-ready, battle-tested
|
||||
- **Package count**: Over 500,000 artifacts
|
||||
- **Dependency culture**: Can be heavy; mature dependency resolution
|
||||
- **Versioning**: Mix of semver and date-based
|
||||
|
||||
### Unique Strengths
|
||||
- Mature ecosystem with established governance
|
||||
- Strong backward compatibility culture
|
||||
- Extensive enterprise adoption and vetting
|
||||
- Maven Central has quality standards
|
||||
|
||||
### What to Watch For
|
||||
- Dependency version conflicts (dependency hell)
|
||||
- Transitive dependencies pulling in multiple versions
|
||||
- Large artifact sizes
|
||||
- Complex dependency trees
|
||||
|
||||
### Preferred Patterns
|
||||
- Well-maintained artifacts from reputable organizations
|
||||
- Clear compatibility matrices (Java version, framework version)
|
||||
- Semantic versioning adherence
|
||||
- Artifacts hosted on Maven Central (not random repos)
|
||||
|
||||
### Recommended Tools
|
||||
```bash
|
||||
mvn dependency:tree # Dependency tree visualization
|
||||
mvn dependency:analyze # Unused dependency analysis
|
||||
mvn versions:display-dependency-updates # Check for updates
|
||||
```
|
||||
|
||||
### Ecosystem-Specific Red Flags
|
||||
- Artifacts only in obscure Maven repos
|
||||
- Complex dependency resolution issues
|
||||
- No Java version compatibility documented
|
||||
- Transitive dependencies with licensing issues
|
||||
|
||||
### Ecosystem-Specific Green Flags
|
||||
- Published to Maven Central
|
||||
- Apache or Eclipse Foundation backing
|
||||
- Clear Java version support policy
|
||||
- Spring ecosystem compatibility (if relevant)
|
||||
- OSGi bundle metadata (for OSGi projects)
|
||||
|
||||
## Cross-Ecosystem Patterns
|
||||
|
||||
### Supply Chain Security Varies by Ecosystem
|
||||
|
||||
**Highest Risk:**
|
||||
- npm (largest attack surface, numerous incidents)
|
||||
- PyPI (targeted attacks on data scientists)
|
||||
|
||||
**Medium Risk:**
|
||||
- Maven (occasional but usually caught quickly)
|
||||
- RubyGems (smaller ecosystem, fewer incidents)
|
||||
|
||||
**Lower Risk:**
|
||||
- Cargo (newer, security-conscious culture)
|
||||
- Go (stdlib-first reduces attack surface)
|
||||
|
||||
### Dependency Tree Norms
|
||||
|
||||
**Expect Heavier Trees:**
|
||||
- npm (100+ transitive deps can be normal)
|
||||
- Maven (enterprise frameworks bring many deps)
|
||||
|
||||
**Expect Lighter Trees:**
|
||||
- Go (< 20 transitive deps typical)
|
||||
- Rust (20-50 deps common)
|
||||
- Python (30-60 deps typical)
|
||||
|
||||
### Versioning Discipline
|
||||
|
||||
**Strict Semver:**
|
||||
- Rust/Cargo (breaking semver is rare)
|
||||
- npm (expected but not always followed)
|
||||
|
||||
**Flexible Versioning:**
|
||||
- Maven (mix of approaches)
|
||||
- Python (mix of semver and datever)
|
||||
|
||||
### Documentation Culture
|
||||
|
||||
**Excellent Documentation Expected:**
|
||||
- Rust (docs.rs standard)
|
||||
- Python (ReadTheDocs common)
|
||||
|
||||
**Variable Documentation:**
|
||||
- npm (ranges from excellent to none)
|
||||
- Maven (often enterprise-focused docs)
|
||||
|
||||
## Adjusting Your Evaluation
|
||||
|
||||
### For npm Packages
|
||||
- **Increase weight on**: Dependency Footprint, Security Posture
|
||||
- **Be more lenient on**: Single maintainer (common for utilities)
|
||||
- **Extra scrutiny for**: Packages with < 50 lines of code but high usage
|
||||
|
||||
### For Python Packages
|
||||
- **Increase weight on**: Security Posture (typosquatting risk)
|
||||
- **Be more lenient on**: Lower download counts (smaller ecosystem)
|
||||
- **Extra scrutiny for**: Packages targeting data scientists/ML engineers
|
||||
|
||||
### For Rust Crates
|
||||
- **Increase weight on**: API Stability, Documentation Quality
|
||||
- **Be more lenient on**: Compile-time dependencies (proc-macros)
|
||||
- **Extra scrutiny for**: Excessive unsafe code usage
|
||||
|
||||
### For Go Modules
|
||||
- **Increase weight on**: Simplicity, Minimal Dependencies
|
||||
- **Be more lenient on**: Lower GitHub stars (smaller community)
|
||||
- **Extra scrutiny for**: Packages wrapping stdlib unnecessarily
|
||||
|
||||
### For Ruby Gems
|
||||
- **Increase weight on**: Rails compatibility (if applicable)
|
||||
- **Be more lenient on**: Monkey-patching (if well-documented)
|
||||
- **Extra scrutiny for**: Core class modifications
|
||||
|
||||
### For Java Artifacts
|
||||
- **Increase weight on**: Enterprise Adoption, Backward Compatibility
|
||||
- **Be more lenient on**: Larger dependency trees (framework norm)
|
||||
- **Extra scrutiny for**: Artifacts not on Maven Central
|
||||
Reference in New Issue
Block a user