461 lines
16 KiB
Markdown
461 lines
16 KiB
Markdown
# Ecosystem-Specific Evaluation Guides
|
|
|
|
Different language ecosystems have different norms, risks, and best practices. Use this guide to adjust your evaluation criteria based on the package ecosystem.
|
|
|
|
## Table of Contents
|
|
|
|
- [Ecosystem Baselines](#ecosystem-baselines)
|
|
- [Node.js / npm](#nodejs--npm)
|
|
- [Python / PyPI](#python--pypi)
|
|
- [Rust / Cargo](#rust--cargo)
|
|
- [Go](#go)
|
|
- [Ruby / RubyGems](#ruby--rubygems)
|
|
- [Java / Maven Central](#java--maven-central)
|
|
- [Cross-Ecosystem Patterns](#cross-ecosystem-patterns)
|
|
- [Adjusting Your Evaluation](#adjusting-your-evaluation)
|
|
|
|
---
|
|
|
|
## Ecosystem Baselines
|
|
|
|
Use these baselines for ecosystem-relative comparisons. These represent typical patterns as of 2025; use as context not rigid rules.
|
|
|
|
### Release Cadence Norms
|
|
|
|
| Ecosystem | Actively Developed | Mature/Stable | Concerning |
|
|
|-----------|-------------------|---------------|------------|
|
|
| npm | Monthly+ releases | Quarterly releases | >6 months no release |
|
|
| PyPI | Monthly-quarterly | Bi-annual releases | >9 months no release |
|
|
| Cargo | Bi-monthly to quarterly | Annual releases OK | >12 months no release |
|
|
| Go | Quarterly typical | Annual releases OK | >12 months no release |
|
|
| RubyGems | Monthly for Rails-related | Quarterly for utilities | >6 months no release |
|
|
| Maven | Quarterly typical | Bi-annual for mature | >9 months no release |
|
|
|
|
**Key:** "Concerning" means outlier for actively developed packages; mature packages may legitimately have longer gaps.
|
|
|
|
### Dependency Count Norms
|
|
|
|
| Ecosystem | Light | Typical | Heavy | Extreme |
|
|
|-----------|-------|---------|-------|---------|
|
|
| npm | <10 | 20-50 | 100-150 | 200+ |
|
|
| PyPI | <5 | 10-30 | 50-80 | 100+ |
|
|
| Cargo | <10 | 20-40 | 60-80 | 100+ |
|
|
| Go | <5 | 5-20 | 30-40 | 50+ |
|
|
| RubyGems | <5 | 10-25 | 40-60 | 80+ |
|
|
| Maven | <10 | 20-50 | 80-120 | 150+ |
|
|
|
|
**Counts are total transitive dependencies.** Adjust expectations based on package type (frameworks have more).
|
|
|
|
### Download Thresholds (Weekly)
|
|
|
|
| Ecosystem | Niche | Moderate | Popular | Very Popular |
|
|
|-----------|-------|----------|---------|--------------|
|
|
| npm | <500 | 1k-10k | 50k-100k | 500k+ |
|
|
| PyPI | <100 | 500-5k | 20k-50k | 200k+ |
|
|
| Cargo | <50 | 200-2k | 10k-30k | 100k+ |
|
|
| RubyGems | <100 | 500-5k | 20k-50k | 200k+ |
|
|
|
|
**Note:** Downloads alone don't indicate quality. Niche packages can be excellent; popular packages can be deprecated.
|
|
|
|
### Issue Response Time Norms
|
|
|
|
| Ecosystem | Excellent | Good | Acceptable | Concerning |
|
|
|-----------|-----------|------|------------|------------|
|
|
| npm (popular) | Hours-1 day | 2-7 days | 2-4 weeks | >1 month |
|
|
| npm (smaller) | 1-3 days | 1-2 weeks | 1 month | >2 months |
|
|
| PyPI | 1-3 days | 1-2 weeks | 3-4 weeks | >1 month |
|
|
| Cargo | 1-2 days | 3-7 days | 2-3 weeks | >1 month |
|
|
| Go | 1-3 days | 1-2 weeks | 3-4 weeks | >1 month |
|
|
|
|
**For security issues:** Expect 24-48hr acknowledgment regardless of ecosystem.
|
|
|
|
### Documentation Expectations
|
|
|
|
| Ecosystem | Minimum Expected | Excellent |
|
|
|-----------|------------------|-----------|
|
|
| npm | README with examples, TypeScript types | Dedicated docs site, migration guides, playground |
|
|
| PyPI | README with examples, type hints | ReadTheDocs site, Sphinx docs, examples repo |
|
|
| Cargo | README with examples, rustdoc | docs.rs complete, examples in repo, book/guide |
|
|
| Go | README with examples, godoc | pkg.go.dev complete, examples, design docs |
|
|
| RubyGems | README with examples | RDoc/YARD docs, Rails integration guide |
|
|
|
|
### Comparative Assessment Guidelines
|
|
|
|
**Use these baselines to ask:**
|
|
- Is this package's release cadence below the norm for its ecosystem and maturity level?
|
|
- Is the dependency count in the top quartile for similar packages in this ecosystem?
|
|
- Is the issue response time significantly slower than ecosystem expectations?
|
|
- Are downloads declining while ecosystem overall is growing?
|
|
|
|
**Example application:**
|
|
- npm package with 150 transitive deps → "Heavy" but not extreme; acceptable for framework, concerning for utility
|
|
- Cargo crate with no release in 10 months → Not yet concerning for mature stable crate
|
|
- PyPI package with 200 deps → Extreme; investigate why so many
|
|
- Go module with 40 deps → Unusual for Go (stdlib-first culture); investigate
|
|
|
|
---
|
|
|
|
## Node.js / npm
|
|
|
|
### Ecosystem Characteristics
|
|
- **Philosophy**: Micropackages are common; many tiny single-purpose modules
|
|
- **Package count**: Over 2 million packages (largest ecosystem)
|
|
- **Dependency culture**: Deep dependency trees are normalized
|
|
- **Versioning**: Semver is standard but not always followed strictly
|
|
|
|
### Unique Risks
|
|
|
|
**Left-pad Risk**
|
|
The infamous "left-pad incident" (2016) highlighted npm's vulnerability to tiny, critical packages being removed. Characteristics:
|
|
- Single-function packages with disproportionate usage
|
|
- High download counts but minimal functionality
|
|
- Supply chain risk when widely used packages are yanked
|
|
|
|
**npm-specific Supply Chain Attacks**
|
|
- Typosquatting is common (react vs. reakt)
|
|
- Package name confusion attacks
|
|
- Malicious install scripts in postinstall hooks
|
|
- Maintainer account compromises
|
|
|
|
### What to Watch For
|
|
- Packages with hundreds of transitive dependencies for simple tasks
|
|
- Postinstall scripts that download external code
|
|
- Packages that wrap simple native functionality unnecessarily
|
|
- Extremely high download counts but minimal GitHub activity (bot inflation)
|
|
|
|
### Preferred Patterns
|
|
- Packages with minimal dependencies
|
|
- Well-established micro-utilities from trusted authors
|
|
- Scoped packages (@organization/package) from known orgs
|
|
- Packages with verified publishers
|
|
|
|
### Recommended Tools
|
|
```bash
|
|
npm ls --all # Visualize full dependency tree
|
|
npm audit # Security vulnerability scanning
|
|
npm pack --dry-run # Check bundle size
|
|
```
|
|
|
|
### Ecosystem-Specific Red Flags
|
|
- Packages requiring sudo or elevated permissions
|
|
- Packages with network calls in postinstall
|
|
- Packages with native dependencies when pure JS would suffice
|
|
- Suspicious similarity to popular package names
|
|
|
|
### Ecosystem-Specific Green Flags
|
|
- TypeScript type definitions included
|
|
- ES modules support
|
|
- Tree-shakeable exports
|
|
- Zero dependencies for utility packages
|
|
|
|
## Python / PyPI
|
|
|
|
### Ecosystem Characteristics
|
|
- **Philosophy**: "Batteries included" - stdlib-first approach
|
|
- **Package count**: Over 400,000 packages
|
|
- **Dependency culture**: Lighter dependency trees than npm
|
|
- **Versioning**: Mix of semver and date-based versioning
|
|
|
|
### Unique Risks
|
|
|
|
**PyPI Supply Chain Attacks**
|
|
- Notable typosquatting incidents (e.g., python3-dateutil vs. dateutil)
|
|
- Malicious packages targeting data scientists (fake ML libraries)
|
|
- Native code in wheels may contain malware
|
|
- setup.py can execute arbitrary code during install
|
|
|
|
**Dependency Confusion**
|
|
- Public PyPI packages with same names as private packages
|
|
- pip installs public version instead of intended private one
|
|
|
|
### What to Watch For
|
|
- Packages with names very similar to popular packages
|
|
- Unusual wheel distributions without source code
|
|
- Packages targeting specific communities (ML, data science) with suspicious features
|
|
- setup.py files with network calls or obfuscated code
|
|
|
|
### Preferred Patterns
|
|
- Packages from known maintainers and organizations
|
|
- Packages with signed releases (GPG signatures)
|
|
- Pure Python packages (no compiled extensions) when possible
|
|
- Packages maintained by Python Software Foundation or sub-projects
|
|
|
|
### Recommended Tools
|
|
```bash
|
|
pip show <package> # View package metadata
|
|
pip index versions <package> # Check version history
|
|
# Use pip-audit for security scanning (install separately)
|
|
```
|
|
|
|
### Ecosystem-Specific Red Flags
|
|
- Packages requesting unnecessary permissions in setup
|
|
- Typosquatting of popular packages (reqeusts vs. requests)
|
|
- Obfuscated code in setup.py
|
|
- Wheels only (no source distribution)
|
|
|
|
### Ecosystem-Specific Green Flags
|
|
- Listed in Python Packaging Authority (PyPA)
|
|
- Type hints (PEP 484) included
|
|
- Both source distributions and wheels available
|
|
- Active maintenance by known Python community members
|
|
|
|
## Rust / Cargo
|
|
|
|
### Ecosystem Characteristics
|
|
- **Philosophy**: Safety and correctness-first; explicit is better than implicit
|
|
- **Package count**: Over 100,000 crates
|
|
- **Dependency culture**: Moderate dependencies; emphasis on correctness
|
|
- **Versioning**: Strict semver adherence is cultural norm
|
|
|
|
### Unique Strengths
|
|
- Strong compile-time guarantees reduce certain vulnerability classes
|
|
- Cargo's built-in tooling is excellent (cargo tree, cargo metadata)
|
|
- Culture of good documentation (docs.rs)
|
|
- `#![forbid(unsafe_code)]` for packages avoiding unsafe blocks
|
|
|
|
### What to Watch For
|
|
- Crates pulling in many proc-macro dependencies (slow compile times)
|
|
- Heavy use of `unsafe` blocks without justification
|
|
- Transitive dependencies with unsafe code when unnecessary
|
|
- Version conflicts in dependency tree (Cargo is strict about this)
|
|
|
|
### Preferred Patterns
|
|
- Crates with `#![forbid(unsafe_code)]` for non-performance-critical code
|
|
- Well-documented use of unsafe with safety invariants explained
|
|
- Minimal proc-macro dependencies
|
|
- Idiomatic Rust patterns
|
|
|
|
### Recommended Tools
|
|
```bash
|
|
cargo tree -p <crate> # Dependency tree visualization
|
|
cargo metadata --format-version 1 # Machine-readable metadata
|
|
cargo audit # Security vulnerability scanning (install separately)
|
|
```
|
|
|
|
### Ecosystem-Specific Red Flags
|
|
- Excessive unsafe code without documentation
|
|
- Non-idiomatic Rust (indicates unfamiliarity)
|
|
- Proc-macro heavy for simple functionality
|
|
- Breaking semver (very rare in Rust ecosystem)
|
|
|
|
### Ecosystem-Specific Green Flags
|
|
- Published on docs.rs with comprehensive documentation
|
|
- `#![forbid(unsafe_code)]` or well-justified unsafe usage
|
|
- Fast compile times relative to functionality
|
|
- Active maintenance by Rust community members
|
|
- Inclusion in "awesome-rust" lists
|
|
|
|
## Go
|
|
|
|
### Ecosystem Characteristics
|
|
- **Philosophy**: Simplicity, minimalism, stdlib-first
|
|
- **Package count**: Smaller than npm/PyPI (by design)
|
|
- **Dependency culture**: Fewer dependencies is idiomatic
|
|
- **Versioning**: Go modules with semantic versioning
|
|
|
|
### Unique Strengths
|
|
- Strong standard library reduces dependency needs
|
|
- Built-in dependency management (go mod)
|
|
- Static linking produces standalone binaries
|
|
- Import paths explicitly reference source repositories
|
|
|
|
### What to Watch For
|
|
- Packages that wrap stdlib with minimal added value
|
|
- Deep dependency trees (unusual in Go)
|
|
- Packages that violate Go idioms and conventions
|
|
- Module paths not matching repository structure
|
|
|
|
### Preferred Patterns
|
|
- Prefer stdlib solutions when available
|
|
- Minimal external dependencies
|
|
- Clear, simple APIs following Go conventions
|
|
- Well-structured module paths (github.com/org/project)
|
|
|
|
### Recommended Tools
|
|
```bash
|
|
go list -m -versions <module> # List module versions
|
|
go mod graph # Dependency graph
|
|
go mod why <module> # Why is this dependency included
|
|
```
|
|
|
|
### Ecosystem-Specific Red Flags
|
|
- Wrapping stdlib unnecessarily
|
|
- Complex APIs when simple would suffice
|
|
- Not following Go Project Layout
|
|
- Vendoring dependencies (uncommon with go mod)
|
|
|
|
### Ecosystem-Specific Green Flags
|
|
- Minimal dependencies (< 5 direct deps)
|
|
- Follows effective Go guidelines
|
|
- Clear documentation and examples
|
|
- Used in prominent Go projects
|
|
|
|
## Ruby / RubyGems
|
|
|
|
### Ecosystem Characteristics
|
|
- **Philosophy**: Convention over configuration, developer happiness
|
|
- **Package count**: Over 175,000 gems
|
|
- **Dependency culture**: Moderate; gems often do a lot
|
|
- **Versioning**: Generally follows semver
|
|
|
|
### Unique Characteristics
|
|
- Gems often monkey-patch core classes (can cause conflicts)
|
|
- Rails ecosystem dominates Ruby gem ecosystem
|
|
- Strong community conventions
|
|
|
|
### What to Watch For
|
|
- Gems that extensively monkey-patch core classes
|
|
- Dependencies that conflict with Rails (if using Rails)
|
|
- Gems that override standard library behavior
|
|
- Unmaintained gems for Rails version compatibility
|
|
|
|
### Preferred Patterns
|
|
- Well-documented gems with clear upgrade paths
|
|
- Gems that minimize monkey-patching
|
|
- Rails-compatible versioning (if applicable)
|
|
- Active maintenance matching Rails release cycles
|
|
|
|
### Recommended Tools
|
|
```bash
|
|
gem list <gem> # List installed versions
|
|
gem dependency <gem> # Show dependencies
|
|
bundle outdated # Check for updates (in bundler projects)
|
|
```
|
|
|
|
### Ecosystem-Specific Red Flags
|
|
- Extensive monkey-patching without documentation
|
|
- Incompatibility with major Rails versions
|
|
- Gems requiring old Ruby versions
|
|
- No Bundler compatibility
|
|
|
|
### Ecosystem-Specific Green Flags
|
|
- Rails-compatible (if relevant)
|
|
- Minimal monkey-patching or well-documented overrides
|
|
- Active maintenance matching Ruby version releases
|
|
- Listed in awesome-ruby or Ruby Toolbox
|
|
|
|
## Java / Maven Central
|
|
|
|
### Ecosystem Characteristics
|
|
- **Philosophy**: Enterprise-ready, battle-tested
|
|
- **Package count**: Over 500,000 artifacts
|
|
- **Dependency culture**: Can be heavy; mature dependency resolution
|
|
- **Versioning**: Mix of semver and date-based
|
|
|
|
### Unique Strengths
|
|
- Mature ecosystem with established governance
|
|
- Strong backward compatibility culture
|
|
- Extensive enterprise adoption and vetting
|
|
- Maven Central has quality standards
|
|
|
|
### What to Watch For
|
|
- Dependency version conflicts (dependency hell)
|
|
- Transitive dependencies pulling in multiple versions
|
|
- Large artifact sizes
|
|
- Complex dependency trees
|
|
|
|
### Preferred Patterns
|
|
- Well-maintained artifacts from reputable organizations
|
|
- Clear compatibility matrices (Java version, framework version)
|
|
- Semantic versioning adherence
|
|
- Artifacts hosted on Maven Central (not random repos)
|
|
|
|
### Recommended Tools
|
|
```bash
|
|
mvn dependency:tree # Dependency tree visualization
|
|
mvn dependency:analyze # Unused dependency analysis
|
|
mvn versions:display-dependency-updates # Check for updates
|
|
```
|
|
|
|
### Ecosystem-Specific Red Flags
|
|
- Artifacts only in obscure Maven repos
|
|
- Complex dependency resolution issues
|
|
- No Java version compatibility documented
|
|
- Transitive dependencies with licensing issues
|
|
|
|
### Ecosystem-Specific Green Flags
|
|
- Published to Maven Central
|
|
- Apache or Eclipse Foundation backing
|
|
- Clear Java version support policy
|
|
- Spring ecosystem compatibility (if relevant)
|
|
- OSGi bundle metadata (for OSGi projects)
|
|
|
|
## Cross-Ecosystem Patterns
|
|
|
|
### Supply Chain Security Varies by Ecosystem
|
|
|
|
**Highest Risk:**
|
|
- npm (largest attack surface, numerous incidents)
|
|
- PyPI (targeted attacks on data scientists)
|
|
|
|
**Medium Risk:**
|
|
- Maven (occasional but usually caught quickly)
|
|
- RubyGems (smaller ecosystem, fewer incidents)
|
|
|
|
**Lower Risk:**
|
|
- Cargo (newer, security-conscious culture)
|
|
- Go (stdlib-first reduces attack surface)
|
|
|
|
### Dependency Tree Norms
|
|
|
|
**Expect Heavier Trees:**
|
|
- npm (100+ transitive deps can be normal)
|
|
- Maven (enterprise frameworks bring many deps)
|
|
|
|
**Expect Lighter Trees:**
|
|
- Go (< 20 transitive deps typical)
|
|
- Rust (20-50 deps common)
|
|
- Python (30-60 deps typical)
|
|
|
|
### Versioning Discipline
|
|
|
|
**Strict Semver:**
|
|
- Rust/Cargo (breaking semver is rare)
|
|
- npm (expected but not always followed)
|
|
|
|
**Flexible Versioning:**
|
|
- Maven (mix of approaches)
|
|
- Python (mix of semver and datever)
|
|
|
|
### Documentation Culture
|
|
|
|
**Excellent Documentation Expected:**
|
|
- Rust (docs.rs standard)
|
|
- Python (ReadTheDocs common)
|
|
|
|
**Variable Documentation:**
|
|
- npm (ranges from excellent to none)
|
|
- Maven (often enterprise-focused docs)
|
|
|
|
## Adjusting Your Evaluation
|
|
|
|
### For npm Packages
|
|
- **Increase weight on**: Dependency Footprint, Security Posture
|
|
- **Be more lenient on**: Single maintainer (common for utilities)
|
|
- **Extra scrutiny for**: Packages with < 50 lines of code but high usage
|
|
|
|
### For Python Packages
|
|
- **Increase weight on**: Security Posture (typosquatting risk)
|
|
- **Be more lenient on**: Lower download counts (smaller ecosystem)
|
|
- **Extra scrutiny for**: Packages targeting data scientists/ML engineers
|
|
|
|
### For Rust Crates
|
|
- **Increase weight on**: API Stability, Documentation Quality
|
|
- **Be more lenient on**: Compile-time dependencies (proc-macros)
|
|
- **Extra scrutiny for**: Excessive unsafe code usage
|
|
|
|
### For Go Modules
|
|
- **Increase weight on**: Simplicity, Minimal Dependencies
|
|
- **Be more lenient on**: Lower GitHub stars (smaller community)
|
|
- **Extra scrutiny for**: Packages wrapping stdlib unnecessarily
|
|
|
|
### For Ruby Gems
|
|
- **Increase weight on**: Rails compatibility (if applicable)
|
|
- **Be more lenient on**: Monkey-patching (if well-documented)
|
|
- **Extra scrutiny for**: Core class modifications
|
|
|
|
### For Java Artifacts
|
|
- **Increase weight on**: Enterprise Adoption, Backward Compatibility
|
|
- **Be more lenient on**: Larger dependency trees (framework norm)
|
|
- **Extra scrutiny for**: Artifacts not on Maven Central
|