Files
gh-ypares-agent-skills-ypar…/skills/package-npm-nix/SKILL.md
2025-11-30 09:08:16 +08:00

591 lines
14 KiB
Markdown

---
name: package-npm-nix
description: Package npm/TypeScript/Bun CLI tools for Nix. Use when creating Nix derivations for JavaScript/TypeScript tools from npm registry or GitHub sources, handling pre-built packages or source builds with dependency management.
---
<objective>
Create Nix packages for npm-based CLI tools, covering both pre-built packages from npm registry and source builds with proper dependency management. This skill provides patterns for fetching, building, and packaging JavaScript/TypeScript/Bun tools in Nix environments.
</objective>
<quick_start>
<pre_built_from_npm>
For tools already built and published to npm (fastest approach):
```nix
{
lib,
stdenv,
fetchzip,
nodejs,
}:
stdenv.mkDerivation rec {
pname = "tool-name";
version = "1.0.0";
src = fetchzip {
url = "https://registry.npmjs.org/${pname}/-/${pname}-${version}.tgz";
hash = "sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=";
};
nativeBuildInputs = [ nodejs ];
installPhase = ''
runHook preInstall
mkdir -p $out/bin
cp $src/dist/cli.js $out/bin/tool-name
chmod +x $out/bin/tool-name
# Fix shebang
substituteInPlace $out/bin/tool-name \
--replace-quiet "#!/usr/bin/env node" "#!${nodejs}/bin/node"
runHook postInstall
'';
meta = with lib; {
description = "Tool description";
homepage = "https://github.com/org/repo";
license = licenses.mit;
sourceProvenance = with lib.sourceTypes; [ binaryBytecode ];
maintainers = with maintainers; [ ];
mainProgram = "tool-name";
platforms = platforms.all;
};
}
```
Get the hash:
```bash
nix-prefetch-url --unpack https://registry.npmjs.org/tool-name/-/tool-name-1.0.0.tgz
# Convert to SRI format:
nix hash convert --to sri --hash-algo sha256 <hash-output>
```
</pre_built_from_npm>
<source_build_with_bun>
For tools that need to be built from source using Bun:
```nix
{
lib,
stdenv,
stdenvNoCC,
fetchFromGitHub,
bun,
makeBinaryWrapper,
nodejs,
autoPatchelfHook,
}:
let
fetchBunDeps =
{ src, hash, ... }@args:
stdenvNoCC.mkDerivation {
pname = args.pname or "${src.name or "source"}-bun-deps";
version = args.version or src.version or "unknown";
inherit src;
nativeBuildInputs = [ bun ];
buildPhase = ''
export HOME=$TMPDIR
export npm_config_ignore_scripts=true
bun install --no-progress --frozen-lockfile --ignore-scripts
'';
installPhase = ''
mkdir -p $out
cp -R ./node_modules $out
cp ./bun.lock $out/
'';
dontFixup = true;
outputHash = hash;
outputHashAlgo = "sha256";
outputHashMode = "recursive";
};
version = "1.0.0";
src = fetchFromGitHub {
owner = "org";
repo = "repo";
rev = "v${version}";
hash = "sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=";
};
node_modules = fetchBunDeps {
pname = "tool-name-bun-deps";
inherit version src;
hash = "sha256-BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB=";
};
in
stdenv.mkDerivation rec {
pname = "tool-name";
inherit version src;
nativeBuildInputs = [
bun
nodejs
makeBinaryWrapper
autoPatchelfHook
];
buildInputs = [
stdenv.cc.cc.lib
];
buildPhase = ''
# Verify lockfile match
diff -q ./bun.lock ${node_modules}/bun.lock || exit 1
# Copy and patch node_modules
cp -R ${node_modules}/node_modules .
chmod -R u+w node_modules
patchShebangs node_modules
autoPatchelf node_modules
export HOME=$TMPDIR
export npm_config_ignore_scripts=true
bun run build
'';
installPhase = ''
mkdir -p $out/bin
cp dist/tool-name $out/bin/tool-name
chmod +x $out/bin/tool-name
'';
dontStrip = true;
meta = with lib; {
description = "Tool description";
homepage = "https://github.com/org/repo";
license = licenses.mit;
sourceProvenance = with lib.sourceTypes; [ fromSource ];
maintainers = with maintainers; [ ];
mainProgram = "tool-name";
platforms = [ "x86_64-linux" ];
};
}
```
</source_build_with_bun>
</quick_start>
<workflow>
<step_1_identify_package_type>
**Determine build approach**:
Check the npm package:
```bash
# Download and inspect
nix-prefetch-url --unpack https://registry.npmjs.org/package/-/package-1.0.0.tgz
ls -la /nix/store/<hash>-package-1.0.0.tgz/
```
If `dist/` directory exists with built files:
→ Use pre-built approach (simpler, faster)
If only `src/` exists or package.json has build scripts:
→ Use source build approach
Check package.json for:
- `"bin"` field: Shows what executables are provided
- `"type": "module"`: ES modules (common in modern packages)
- `"scripts"`: Build commands (indicates source build needed)
- Runtime: Look for bun, node, or specific version requirements
</step_1_identify_package_type>
<step_2_fetch_hashes>
**Get source and dependency hashes**:
For pre-built packages:
```bash
# Fetch npm tarball
nix-prefetch-url --unpack https://registry.npmjs.org/pkg/-/pkg-1.0.0.tgz
# Output: 1abc... (base32 format)
# Convert to SRI format
nix hash convert --to sri --hash-algo sha256 1abc...
# Output: sha256-xyz...
```
For source builds:
```bash
# Get GitHub source hash
nix-prefetch-url --unpack https://github.com/org/repo/archive/v1.0.0.tar.gz
# Get dependencies hash (requires iteration):
# 1. Use lib.fakeHash in fetchBunDeps
# 2. Try to build
# 3. Nix will show expected hash in error
# 4. Update hash and rebuild
```
</step_2_fetch_hashes>
<step_3_create_package_files>
**Create package structure**:
```bash
mkdir -p packages/tool-name
```
Create `packages/tool-name/package.nix` with full derivation (see quick_start).
Create `packages/tool-name/default.nix`:
```nix
{ pkgs }: pkgs.callPackage ./package.nix { }
```
This two-file pattern allows the package to be used standalone or integrated into a flake.
</step_3_create_package_files>
<step_4_handle_special_cases>
**Common additional requirements**:
**WASM files or other assets**:
```nix
installPhase = ''
mkdir -p $out/bin
cp $src/dist/cli.js $out/bin/tool
cp $src/dist/*.wasm $out/bin/ # Copy WASM alongside
chmod +x $out/bin/tool
substituteInPlace $out/bin/tool \
--replace-quiet "#!/usr/bin/env node" "#!${nodejs}/bin/node"
'';
```
**Multiple executables**:
```nix
# package.json might have:
# "bin": {
# "tool": "dist/cli.js",
# "tool-admin": "dist/admin.js"
# }
installPhase = ''
mkdir -p $out/bin
for exe in tool tool-admin; do
cp $src/dist/$exe.js $out/bin/$exe
chmod +x $out/bin/$exe
substituteInPlace $out/bin/$exe \
--replace-quiet "#!/usr/bin/env node" "#!${nodejs}/bin/node"
done
'';
meta.mainProgram = "tool"; # Primary command
```
**Platform-specific binaries**:
```nix
meta = {
platforms = [ "x86_64-linux" ]; # Bun-compiled binaries often Linux-only
# or
platforms = platforms.all; # Pure JS works everywhere
};
```
</step_4_handle_special_cases>
<step_5_test_build>
**Build and test**:
```bash
# Build
nix build .#tool-name
# Test the binary
./result/bin/tool-name --version
./result/bin/tool-name --help
# Check dependencies (Linux)
ldd ./result/bin/tool-name # Should show all deps resolved
# Format
nix fmt
# Run flake checks
nix flake check
```
</step_5_test_build>
</workflow>
<metadata_requirements>
<essential_fields>
Every package must have complete metadata:
```nix
meta = with lib; {
description = "Clear, concise description";
homepage = "https://project-homepage.com";
changelog = "https://github.com/org/repo/releases"; # Optional but nice
license = licenses.mit; # or licenses.unfree for proprietary
sourceProvenance = with lib.sourceTypes; [
fromSource # Built from source
# or
binaryBytecode # Pre-built JS/TS (npm dist/)
# or
binaryNativeCode # Compiled binaries
];
maintainers = with maintainers; [ ]; # Empty OK for community packages
mainProgram = "binary-name";
platforms = platforms.all; # or specific: [ "x86_64-linux" ]
};
```
</essential_fields>
<source_provenance_guide>
**Choose based on what you're packaging**:
- `fromSource`: Built from TypeScript/source during derivation
- `binaryBytecode`: Pre-compiled JS from npm registry
- `binaryNativeCode`: Native binaries (Rust, Go, Bun-compiled)
This affects security auditing and reproducibility expectations.
</source_provenance_guide>
</metadata_requirements>
<common_patterns>
<shebang_replacement>
**Always replace shebangs** for reproducibility:
```nix
# Single file
substituteInPlace $out/bin/tool \
--replace-quiet "#!/usr/bin/env node" "#!${nodejs}/bin/node"
# Multiple files
find $out/bin -type f -exec substituteInPlace {} \
--replace-quiet "#!/usr/bin/env node" "#!${nodejs}/bin/node" \;
```
The `--replace-quiet` flag suppresses warnings if pattern not found.
</shebang_replacement>
<native_dependencies>
**Handle native modules** (like sqlite, sharp):
```nix
nativeBuildInputs = [
bun
nodejs
makeBinaryWrapper
autoPatchelfHook # Linux: patches ELF binaries
];
buildInputs = [
stdenv.cc.cc.lib # Provides libgcc_s.so.1, libstdc++.so.6
];
autoPatchelfIgnoreMissingDeps = [
"libc.musl-x86_64.so.1" # Ignore musl if not available
];
```
`autoPatchelf` runs automatically on Linux, fixing RPATH for .so files.
</native_dependencies>
<bun_compiled_binaries>
**Don't strip Bun-compiled executables**:
```nix
# Bun embeds JavaScript in the binary
dontStrip = true;
```
Stripping would remove the embedded JS, breaking the program.
</bun_compiled_binaries>
<checking_tarball_contents>
**Inspect npm package structure**:
```bash
# After nix-prefetch-url
ls -la /nix/store/*-pkg-1.0.0.tgz/
# Common layouts:
# dist/cli.js → Pre-built, use directly
# dist/index.js → Main entry, check package.json "bin"
# src/index.ts → Source only, need to build
# lib/ → Built CommonJS
# esm/ → Built ES modules
```
Check package.json to find the correct entry point.
</checking_tarball_contents>
</common_patterns>
<anti_patterns>
<avoid_these>
**Don't do this**:
❌ Hardcode node paths:
```nix
# Bad
"#!/usr/bin/node" # Won't work on NixOS
```
✅ Use substituteInPlace:
```nix
# Good
substituteInPlace $out/bin/tool \
--replace-quiet "#!/usr/bin/env node" "#!${nodejs}/bin/node"
```
❌ Skip hash verification:
```nix
# Bad - insecure
hash = lib.fakeHash;
```
✅ Get real hash:
```nix
# Good - reproducible and secure
hash = "sha256-actual-hash-here";
```
❌ Forget to make executable:
```nix
# Bad - won't run
cp $src/dist/cli.js $out/bin/tool
```
✅ Set executable bit:
```nix
# Good
cp $src/dist/cli.js $out/bin/tool
chmod +x $out/bin/tool
```
❌ Strip Bun binaries:
```nix
# Bad - breaks Bun-compiled executables
# (default behavior strips binaries)
```
✅ Disable stripping:
```nix
# Good
dontStrip = true;
```
</avoid_these>
</anti_patterns>
<troubleshooting>
<hash_mismatch>
**Error: "hash mismatch in fixed-output derivation"**
The hash you provided doesn't match what Nix fetched.
Solution:
1. Nix error shows "got: sha256-XYZ..."
2. Copy that hash into your derivation
3. Rebuild
For `fetchBunDeps`, this is expected the first time—use the error output to get the correct hash.
</hash_mismatch>
<missing_executable>
**Error: Binary not found after build**
Check:
```bash
# List what was actually built
ls -R result/
# Check package.json "bin" field
cat /nix/store/*-source/package.json | jq .bin
# Check build output location
cat /nix/store/*-source/package.json | jq .scripts.build
```
The build might output to a different directory than expected.
</missing_executable>
<elf_interpreter_error>
**Error: "No such file or directory" when running binary (Linux)**
The binary needs ELF patching for native dependencies.
Solution:
```nix
nativeBuildInputs = [
autoPatchelfHook
];
buildInputs = [
stdenv.cc.cc.lib
];
```
For node_modules with native addons:
```nix
buildPhase = ''
cp -R ${node_modules}/node_modules .
chmod -R u+w node_modules
autoPatchelf node_modules # Patch .node files
'';
```
</elf_interpreter_error>
<bun_lock_mismatch>
**Error: "bun.lock mismatch"**
The lockfile in your source doesn't match the cached dependencies.
This happens when:
- Source version updated but dependency hash not updated
- Source repo has uncommitted lockfile changes
Solution:
1. Update source hash to match new version
2. Set dependency hash to `lib.fakeHash`
3. Build to get correct dependency hash
4. Update dependency hash
5. Rebuild
</bun_lock_mismatch>
</troubleshooting>
<validation>
<build_checklist>
Before considering the package done:
- [ ] `nix build .#package-name` succeeds
- [ ] `./result/bin/tool --version` works
- [ ] `./result/bin/tool --help` works
- [ ] `nix flake check` passes
- [ ] `meta.description` is clear and concise
- [ ] `meta.homepage` points to project site
- [ ] `meta.license` is correct
- [ ] `meta.sourceProvenance` matches what you packaged
- [ ] `meta.mainProgram` is set
- [ ] `meta.platforms` is appropriate for the tool
- [ ] All hashes are real (no `lib.fakeHash`)
- [ ] Shebangs use Nix store paths, not /usr/bin
- [ ] File is formatted with `nix fmt`
</build_checklist>
<testing_on_other_platforms>
If you only have Linux but package claims `platforms.all`:
Consider asking maintainers with macOS/ARM to test, or:
- Mark platforms conservatively based on what you can test
- Note in package that other platforms are untested
- Let CI or other contributors expand platform support
</testing_on_other_platforms>
</validation>
<success_criteria>
A well-packaged npm tool has:
- Clean build with no warnings or errors
- Working executable in `result/bin/`
- Complete and accurate metadata
- Proper source provenance classification
- All dependencies resolved (no missing libraries)
- Reproducible builds (real hashes, no network access during build)
- Follows Nix packaging conventions (shebang patching, proper phases)
</success_criteria>