zhongwei/gh-hugoduncan-library-skills-plugins-clojure-libraries
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial commit

Browse Source
This commit is contained in:
Zhongwei Li
2025-11-29 18:47:15 +08:00
commit d024d22cd1
30 changed files with 9249 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

777
skills/babashka.fs/SKILL.md Normal file
View File

@@ -0,0 +1,777 @@
---
name: babashka.fs
description: A guide to using babashka.fs.
---
# Babashka.fs File System Utilities Skill
## Overview
The `babashka.fs` library is a comprehensive file system utility library for Clojure, designed for cross-platform file operations. It provides a clean, functional API for working with files, directories, and paths, built on top of Java's NIO.2 API while offering a more idiomatic Clojure interface.
**When to use this skill:**
- When working with files and directories in Clojure/Babashka scripts
- When you need cross-platform file system operations
- When writing build tasks, file processing scripts, or automation tools
- When you need to search, filter, or manipulate file systems programmatically
## Setup and Requirements
### Adding to your project
```clojure
;; In deps.edn
{:deps {babashka/fs {:mvn/version "0.5.27"}}}
;; In your namespace
(ns my-script
(:require [babashka.fs :as fs]))
```
### Built-in to Babashka
The library is built into Babashka, so no additional dependencies are needed for bb scripts:
```clojure
#!/usr/bin/env bb
(require '[babashka.fs :as fs])
(fs/directory? ".") ; => true
```
## Core Concepts
### Path Objects
Most functions accept and return `java.nio.file.Path` objects, but also work with strings and other path-like objects. The library automatically coerces between types.
```clojure
;; All of these work
(fs/exists? ".")
(fs/exists? (fs/path "."))
(fs/exists? (java.io.File. "."))
```
### Cross-Platform Support
The library handles platform differences automatically, but provides utilities when you need platform-specific behavior:
```clojure
;; Works on all platforms
(fs/path "dir" "subdir" "file.txt")
;; Convert to Unix-style paths (useful for Windows)
(fs/unixify "C:\\Users\\name\\file.txt") ; => "C:/Users/name/file.txt"
```
## Path Operations
### Creating and Manipulating Paths
```clojure
;; Create paths
(fs/path "dir" "subdir" "file.txt") ; Join path components
(fs/file "dir" "subdir" "file.txt") ; Alias for fs/path
;; Path properties
(fs/absolute? "/tmp/file.txt") ; true
(fs/relative? "dir/file.txt") ; true
(fs/hidden? ".hidden-file") ; Check if hidden
;; Path transformations
(fs/absolutize "relative/path") ; Convert to absolute
(fs/canonicalize "/tmp/../file.txt") ; Resolve to canonical form
(fs/normalize "/tmp/./dir/../file.txt") ; Normalize path
;; Path components
(fs/file-name "/path/to/file.txt") ; "file.txt"
(fs/parent "/path/to/file.txt") ; "/path/to"
(fs/extension "file.txt") ; "txt"
(fs/split-ext "file.txt") ; ["file" "txt"]
(fs/strip-ext "file.txt") ; "file"
;; Path relationships
(fs/starts-with? "/foo/bar" "/foo") ; true
(fs/ends-with? "/foo/bar.txt" "bar.txt") ; true
(fs/relativize "/foo/bar" "/foo/bar/baz") ; "baz"
;; Get all components
(fs/components "/path/to/file.txt") ; Seq of path components
```
### Working with Extensions
```clojure
;; Get extension
(fs/extension "document.pdf") ; "pdf"
(fs/extension "archive.tar.gz") ; "gz"
;; Split filename and extension
(fs/split-ext "document.pdf") ; ["document" "pdf"]
;; Remove extension
(fs/strip-ext "document.pdf") ; "document"
(fs/strip-ext "archive.tar.gz") ; "archive.tar"
```
## File and Directory Checks
```clojure
;; Existence and type checks
(fs/exists? "file.txt") ; Does it exist?
(fs/directory? "path/to/dir") ; Is it a directory?
(fs/regular-file? "file.txt") ; Is it a regular file?
(fs/sym-link? "link") ; Is it a symbolic link?
(fs/hidden? ".hidden") ; Is it hidden?
;; Permission checks
(fs/readable? "file.txt") ; Can we read it?
(fs/writable? "file.txt") ; Can we write to it?
(fs/executable? "script.sh") ; Can we execute it?
;; Comparison
(fs/same-file? "file1.txt" "file2.txt") ; Are they the same file?
```
## Creating Files and Directories
```clojure
;; Create directories
(fs/create-dir "new-dir") ; Create single directory
(fs/create-dirs "path/to/new/dir") ; Create with parents
;; Create files
(fs/create-file "new-file.txt") ; Create empty file
;; Create temporary files/directories
(fs/create-temp-file) ; Creates temp file
(fs/create-temp-file {:prefix "data-" ; Custom prefix/suffix
:suffix ".json"})
(fs/create-temp-dir) ; Creates temp directory
(fs/create-temp-dir {:prefix "workdir-"})
;; Create links
(fs/create-link "link-name" "target") ; Hard link
(fs/create-sym-link "symlink" "target") ; Symbolic link
;; Temporary directory context
(fs/with-temp-dir [tmp-dir {:prefix "work-"}]
(println "Working in" (str tmp-dir))
;; Do work with tmp-dir
;; Directory automatically deleted after
)
```
## Reading and Writing Files
### Reading Files
```clojure
;; Read entire file
(slurp (fs/file "data.txt")) ; As string
;; Read lines
(with-open [rdr (io/reader (fs/file "data.txt"))]
(doall (line-seq rdr)))
;; Or use fs helpers
(fs/read-all-lines "data.txt") ; Returns seq of lines
(fs/read-all-bytes "binary-file") ; Returns byte array
```
### Writing Files
```clojure
;; Write text
(spit (fs/file "output.txt") "Hello, world!")
;; Write lines
(fs/write-lines "output.txt"
["Line 1" "Line 2" "Line 3"])
(fs/write-lines "output.txt"
["More lines"]
{:append true}) ; Append mode
;; Write bytes
(fs/write-bytes "output.bin" byte-array)
(fs/write-bytes "output.bin" byte-array
{:append true})
```
## Copying, Moving, and Deleting
```clojure
;; Copy files
(fs/copy "source.txt" "dest.txt") ; Copy file
(fs/copy "source.txt" "dest.txt"
{:replace-existing true}) ; Overwrite if exists
;; Copy entire directory trees
(fs/copy-tree "source-dir" "dest-dir") ; Recursive copy
(fs/copy-tree "source-dir" "dest-dir"
{:replace-existing true})
;; Move/rename
(fs/move "old-name.txt" "new-name.txt") ; Move or rename
(fs/move "file.txt" "other-dir/") ; Move to directory
;; Delete
(fs/delete "file.txt") ; Delete single file
(fs/delete-if-exists "maybe-file.txt") ; No error if missing
(fs/delete-tree "directory") ; Delete directory recursively
;; Delete on exit
(fs/delete-on-exit "temp-file.txt") ; Delete when JVM exits
```
## Listing and Traversing Directories
### Simple Listing
```clojure
;; List directory contents
(fs/list-dir ".") ; Seq of paths in directory
(fs/list-dir "." "*.txt") ; With glob pattern
;; List multiple directories
(fs/list-dirs ["dir1" "dir2"] "*.clj") ; Combine results
;; Get directory stream (more efficient for large dirs)
(with-open [ds (fs/directory-stream "." "*.txt")]
(doseq [path ds]
(println path)))
```
### Walking Directory Trees
```clojure
;; Walk directory tree
(fs/walk-file-tree "."
{:visit-file (fn [path attrs]
(println "File:" path)
:continue)
:pre-visit-dir (fn [path attrs]
(println "Entering:" path)
:continue)
:post-visit-dir (fn [path ex]
(println "Leaving:" path)
:continue)})
;; Common options
;; :max-depth - limit depth
;; :follow-links - follow symbolic links
;; :visit-file - called for each file
;; :pre-visit-dir - called before visiting directory
;; :post-visit-dir - called after visiting directory
;; :visit-file-failed - called when file access fails
```
## Searching and Filtering: Glob and Match
### Glob Patterns
The `glob` function is one of the most powerful features for finding files:
```clojure
;; Find all Clojure files recursively
(fs/glob "." "**/*.clj") ; ** means recursive
;; Find files in current directory only
(fs/glob "." "*.txt") ; * means any characters
;; Multiple extensions
(fs/glob "." "**{.clj,.cljc,.cljs}") ; Match multiple patterns
;; Complex patterns
(fs/glob "src" "**/test_*.clj") ; Test files anywhere
(fs/glob "." "data/*.{json,edn}") ; JSON or EDN in data dir
;; Exclude patterns (use filter)
(->> (fs/glob "." "**/*.clj")
(remove #(re-find #"/test/" (str %)))) ; Exclude test directories
;; Common glob patterns:
;; * - matches any characters (not including /)
;; ** - matches any characters including /
;; ? - matches single character
;; [abc] - matches any character in brackets
;; {a,b} - matches either a or b
```
### Match with Regular Expressions
For more complex matching, use `match`:
```clojure
;; Use regex for pattern matching
(fs/match "." "regex:.*\\.clj$" {:recursive true})
;; Or glob (explicit)
(fs/match "." "glob:**/*.clj" {:recursive true})
;; Options
(fs/match "src" "regex:test.*\\.clj"
{:recursive true
:hidden false ; Skip hidden files
:follow-links false ; Don't follow symlinks
:max-depth 5}) ; Limit depth
```
### Practical File Filtering Examples
```clojure
;; Find large files
(->> (fs/glob "." "**/*")
(filter fs/regular-file?)
(filter #(> (fs/size %) (* 10 1024 1024))) ; > 10MB
(map str))
;; Find recently modified files
(->> (fs/glob "." "**/*.clj")
(filter #(> (fs/file-time->millis (fs/last-modified-time %))
(- (System/currentTimeMillis)
(* 24 60 60 1000)))) ; Last 24 hours
(map str))
;; Find files by owner (Unix)
(->> (fs/glob "/var/log" "*")
(filter #(= "root" (str (fs/owner %))))
(map str))
;; Find executable scripts
(->> (fs/glob "." "**/*.sh")
(filter fs/executable?)
(map str))
```
## File Metadata and Attributes
```clojure
;; File size
(fs/size "file.txt") ; Size in bytes
;; Timestamps
(fs/creation-time "file.txt") ; FileTime object
(fs/last-modified-time "file.txt") ; FileTime object
(fs/set-last-modified-time "file.txt"
(fs/file-time 1234567890000))
;; Convert FileTime to millis
(fs/file-time->millis (fs/last-modified-time "file.txt"))
(fs/file-time->instant (fs/last-modified-time "file.txt"))
;; Create FileTime from millis
(fs/file-time 1234567890000)
;; Owner (Unix/Linux)
(fs/owner "file.txt") ; Returns owner object
(str (fs/owner "file.txt")) ; Owner name as string
;; POSIX permissions (Unix/Linux)
(fs/posix->str (fs/posix-file-permissions "file.txt")) ; "rwxr-xr-x"
(fs/set-posix-file-permissions "file.txt"
(fs/str->posix "rwxr-xr-x"))
;; Check for modified files since anchor
(fs/modified-since "target" "src") ; Files in src newer than target
```
## Archive Operations (Zip)
```clojure
;; Create zip archive
(fs/zip "archive.zip" "file1.txt") ; Single file
(fs/zip "archive.zip" ["file1.txt"
"file2.txt"
"dir"]) ; Multiple files/dirs
;; Zip with options
(fs/zip "archive.zip" "directory"
{:root "directory"}) ; Strip parent path
;; Extract zip archive
(fs/unzip "archive.zip" "output-dir") ; Extract all
;; Extract with filter
(fs/unzip "archive.zip" "output-dir"
{:extract-fn (fn [{:keys [name]}]
(re-find #"\\.txt$" name))}) ; Only .txt files
;; Manually work with zip entries
(fs/zip-path "archive.zip" "path/in/zip") ; Access file in zip as path
```
## System Paths and Utilities
```clojure
;; User directories
(fs/home) ; User home directory
(fs/temp-dir) ; System temp directory
(fs/cwd) ; Current working directory
;; XDG Base Directory Specification (Linux)
(fs/xdg-config-home) ; ~/.config
(fs/xdg-config-home "myapp") ; ~/.config/myapp
(fs/xdg-data-home) ; ~/.local/share
(fs/xdg-cache-home) ; ~/.cache
(fs/xdg-state-home) ; ~/.local/state
;; Executable paths
(fs/exec-paths) ; All dirs in PATH
(fs/which "java") ; Find executable in PATH
(fs/which "git") ; Returns path or nil
;; Find executable manually
(->> (fs/exec-paths)
(mapcat #(fs/list-dir % "java*"))
(filter fs/executable?)
first)
```
## Advanced Patterns and Best Practices
### Safe File Operations with Error Handling
```clojure
;; Check before operating
(when (fs/exists? "config.edn")
(fs/copy "config.edn" "config.backup.edn"))
;; Use delete-if-exists for optional deletion
(fs/delete-if-exists "temp-file.txt")
;; Handle walk-file-tree errors
(fs/walk-file-tree "."
{:visit-file-failed (fn [path ex]
(println "Failed to access:" path)
:skip-subtree)})
```
### Working with Temporary Files
```clojure
;; Pattern 1: with-temp-dir (automatic cleanup)
(fs/with-temp-dir [tmp-dir {:prefix "work-"}]
(let [work-file (fs/path tmp-dir "data.txt")]
(spit work-file "temporary data")
(process-file work-file)))
;; tmp-dir automatically deleted here
;; Pattern 2: Manual temp file management
(let [tmp-file (fs/create-temp-file {:prefix "data-"
:suffix ".json"})]
(try
(spit tmp-file (json/encode data))
(process-file tmp-file)
(finally
(fs/delete tmp-file))))
;; Pattern 3: Delete on exit
(let [tmp-file (fs/create-temp-file)]
(fs/delete-on-exit tmp-file)
(spit tmp-file data)
tmp-file) ; File deleted when JVM exits
```
### Efficient Directory Processing
```clojure
;; Process large directories efficiently
(with-open [stream (fs/directory-stream "." "*.txt")]
(doseq [path stream]
(process-file path))) ; Lazy processing, one at a time
;; Instead of realizing entire seq
(doseq [path (fs/list-dir "." "*.txt")]
(process-file path)) ; Realizes all paths first
```
### Cross-Platform Path Construction
```clojure
;; Always use fs/path for joining - it handles separators
(fs/path "dir" "subdir" "file.txt") ; Works everywhere
;; Don't manually concatenate with separators
;; BAD: (str "dir" "/" "subdir" "/" "file.txt") ; Breaks on Windows
;; Convert Windows paths to Unix style when needed
(fs/unixify (fs/path "C:" "Users" "name")) ; "C:/Users/name"
```
### File Filtering Pipeline Pattern
```clojure
;; Build reusable filters
(defn clojure-source? [path]
(and (fs/regular-file? path)
(re-find #"\.(clj|cljs|cljc)$" (str path))))
(defn recent? [days path]
(let [cutoff (- (System/currentTimeMillis)
(* days 24 60 60 1000))]
(> (fs/file-time->millis (fs/last-modified-time path)) cutoff)))
;; Compose filters
(->> (fs/glob "src" "**/*")
(filter clojure-source?)
(filter (partial recent? 7))
(map str))
```
### Atomic File Operations
```clojure
;; Write to temp file, then move (atomic on most filesystems)
(let [target (fs/path "important-data.edn")
tmp-file (fs/create-temp-file {:prefix ".tmp-"
:suffix ".edn"
:dir (fs/parent target)})]
(try
(spit tmp-file (pr-str data))
(fs/move tmp-file target {:replace-existing true})
(catch Exception e
(fs/delete-if-exists tmp-file)
(throw e))))
```
## Common Use Cases and Recipes
### Build Tool Tasks
```clojure
;; Clean target directory
(defn clean []
(when (fs/exists? "target")
(fs/delete-tree "target")))
;; Copy resources
(defn copy-resources []
(fs/create-dirs "target/resources")
(fs/copy-tree "resources" "target/resources"))
;; Find all source files
(defn source-files []
(fs/glob "src" "**/*.clj"))
```
### File Backup
```clojure
(defn backup-file [path]
(let [backup-name (str path ".backup."
(System/currentTimeMillis))]
(fs/copy path backup-name)))
(defn backup-directory [dir dest]
(let [timestamp (System/currentTimeMillis)
backup-dir (fs/path dest (str (fs/file-name dir)
"-" timestamp))]
(fs/copy-tree dir backup-dir)))
```
### Log Rotation
```clojure
(defn rotate-logs [log-dir max-age-days]
(let [cutoff (- (System/currentTimeMillis)
(* max-age-days 24 60 60 1000))]
(->> (fs/glob log-dir "*.log")
(filter #(< (fs/file-time->millis
(fs/last-modified-time %))
cutoff))
(run! fs/delete))))
```
### File Synchronization
```clojure
(defn sync-newer-files [src dest]
(doseq [src-file (fs/glob src "**/*")
:when (fs/regular-file? src-file)]
(let [rel-path (fs/relativize src src-file)
dest-file (fs/path dest rel-path)]
(when (or (not (fs/exists? dest-file))
(> (fs/file-time->millis (fs/last-modified-time src-file))
(fs/file-time->millis (fs/last-modified-time dest-file))))
(fs/create-dirs (fs/parent dest-file))
(fs/copy src-file dest-file {:replace-existing true})
(println "Synced:" src-file)))))
```
### Finding Duplicate Files
```clojure
(require '[clojure.java.io :as io])
(import '[java.security MessageDigest])
(defn file-hash [path]
(with-open [is (io/input-stream (fs/file path))]
(let [digest (MessageDigest/getInstance "MD5")
buffer (byte-array 8192)]
(loop []
(let [n (.read is buffer)]
(when (pos? n)
(.update digest buffer 0 n)
(recur))))
(format "%032x" (BigInteger. 1 (.digest digest))))))
(defn find-duplicates [dir]
(->> (fs/glob dir "**/*")
(filter fs/regular-file?)
(group-by file-hash)
(filter #(> (count (val %)) 1))
(map (fn [[hash paths]]
{:hash hash
:size (fs/size (first paths))
:files (map str paths)}))))
```
## Error Handling and Edge Cases
```clojure
;; Handle missing files gracefully
(when (fs/exists? "config.edn")
(process-config (slurp "config.edn")))
;; Or with try-catch
(try
(process-file "data.txt")
(catch java.nio.file.NoSuchFileException e
(println "File not found:" (.getMessage e)))
(catch java.nio.file.AccessDeniedException e
(println "Access denied:" (.getMessage e))))
;; Check permissions before operations
(when (and (fs/exists? "file.txt")
(fs/readable? "file.txt"))
(slurp "file.txt"))
;; Handle walk errors
(fs/walk-file-tree "."
{:visit-file-failed (fn [path ex]
(println "Cannot access:" path)
:continue)}) ; Continue despite errors
```
## Performance Tips
1. **Use directory-stream for large directories**: It's lazy and doesn't load all entries into memory
2. **Filter early**: Apply filters in glob patterns when possible rather than filtering in Clojure
3. **Avoid repeated file system calls**: Cache results like file-exists? checks
4. **Use walk-file-tree for deep recursion**: More efficient than recursive list-dir
5. **Batch operations**: Group multiple files when possible instead of individual operations
## Testing and Mocking
```clojure
;; Use with-temp-dir for tests
(deftest test-file-processing
(fs/with-temp-dir [tmp-dir {}]
(let [test-file (fs/path tmp-dir "test.txt")]
(spit test-file "test data")
(is (fs/exists? test-file))
(is (= "test data" (slurp test-file)))
;; No cleanup needed - automatic
)))
```
## Platform-Specific Considerations
### Windows
- Use `fs/unixify` to normalize paths for cross-platform code
- Hidden files require the hidden attribute, not just a leading dot
- POSIX permission functions won't work
### Unix/Linux/macOS
- Full POSIX permissions support
- XDG base directory functions available
- Hidden files start with dot
- Owner functions work
### General
- Always use `fs/path` to join paths - it handles separators correctly
- Test on target platforms when possible
- Use relative paths when portability matters
## Integration with Babashka Tasks
```clojure
;; In bb.edn
{:tasks
{:requires ([babashka.fs :as fs])
clean {:doc "Remove build artifacts"
:task (fs/delete-tree "target")}
test {:doc "Run tests"
:task (do
(doseq [test-file (fs/glob "test" "**/*_test.clj")]
(load-file (str test-file))))}
build {:doc "Build project"
:depends [clean]
:task (do
(fs/create-dirs "target")
(println "Building..."))}}}
```
## Quick Reference: Most Common Functions
```clojure
;; Checking
(fs/exists? path)
(fs/directory? path)
(fs/regular-file? path)
;; Creating
(fs/create-dirs path)
(fs/create-file path)
(fs/create-temp-dir)
;; Reading/Writing
(slurp (fs/file path))
(spit (fs/file path) content)
(fs/read-all-lines path)
(fs/write-lines path lines)
;; Copying/Moving/Deleting
(fs/copy src dest)
(fs/copy-tree src dest)
(fs/move src dest)
(fs/delete path)
(fs/delete-tree path)
;; Finding
(fs/glob root "**/*.clj")
(fs/match root pattern {:recursive true})
(fs/list-dir dir)
(fs/which "executable")
;; Paths
(fs/path "dir" "file")
(fs/parent path)
(fs/file-name path)
(fs/extension path)
(fs/absolutize path)
(fs/relativize base target)
```
## Additional Resources
- [Official GitHub Repository](https://github.com/babashka/fs)
- [API Documentation](https://github.com/babashka/fs/blob/master/API.md)
- [Babashka Book](https://book.babashka.org/)
- [Java NIO.2 Path Documentation](https://docs.oracle.com/javase/tutorial/essential/io/fileio.html)
## Summary
The babashka.fs library provides a comprehensive, idiomatic Clojure interface for file system operations. Key strengths:
- **Cross-platform**: Handles OS differences automatically
- **Composable**: Functions work well together in pipelines
- **Efficient**: Built on NIO.2 for good performance
- **Practical**: Includes high-level functions for common tasks
- **Safe**: Provides options for atomic operations and error handling
When writing file system code in Clojure or Babashka, reach for babashka.fs first - it's likely to have exactly what you need with a clean, functional API.