Ferralk / Guide
Using Ferralk
Ferralk has one matcher crate and one walker crate. The walker re-exports the
matcher crate, but depending on ferralk-glob alone keeps applications that do
not traverse filesystems dependency-light.
Where to start
Pick the entry point by what you hold and how far a wildcard may reach:
Scroll sideways for more columns.
| You want to | Use | Wildcard scope |
|---|---|---|
| Find files on disk | Walker with include and exclude patterns | Ordinary wildcards stay inside one component; a whole-component ** is recursive. Braces and extglobs are enabled for you. |
| Match a path you already hold, as a shell glob would | Pattern::is_match_glob_path | Every ordinary wildcard stays inside one component; a whole-component ** crosses when recursive_double_star is enabled. |
Match a whole byte sequence in which / is not special | Pattern::is_match | Every wildcard may cross a separator. |
Filter a list the way zlob's matchPaths does | Pattern::is_match_path and the filter_paths family | zlob's list-filter rule, described below. |
The walker's defaults are conservative: it reads nothing it was not asked to read and reports everything it found. The table lists each default and the switch that changes it; the sections below explain the semantics.
Scroll sideways for more columns.
| Concern | Default | Switch |
|---|---|---|
| Which entries qualify | Every non-excluded entry when no include is set; otherwise any include, OR-ed | Walker::include, Walker::exclude, and the borrowed try_ forms |
| Wildcards and a leading period | Not matched by an include, so **/*.ts skips .cache/x.ts; always matched by an exclude, as in .gitignore | Walker::match_hidden(true) for includes |
| Hidden entries at all | Traversed and reported | WalkOptions::skip_hidden(true) drops them before any pattern runs |
.gitignore, .ignore, .git/info/exclude | Not read | Walker::respect_git_ignore(true) |
The .git directory under Git ignores | Skipped | WalkOptions::keep_git_dir(true) |
| Symlinks below a root | Reported, not followed | WalkOptions::follow_symlinks(true) |
Symlink kind for files_only and directories_only | The listing's kind: a symlink is a symlink | WalkOptions::resolve_symlink_kind(true) classifies by target |
| Entry kinds | Files, directories, and symlinks | WalkOptions::files_only(true) or directories_only(true) |
| Depth | Unlimited | WalkOptions::max_depth(n) |
| Ordering | Unsorted | WalkOptions::sort(true), for collect() and visit(); stream() and stream_parallel() ignore it |
| Metadata | Not fetched | WalkOptions::metadata(true) |
| Recoverable errors | Collected next to the entries | Walker::error_policy(ErrorPolicy::Skip) or ErrorPolicy::Abort |
| Threads | Available parallelism, held to one performance cluster on Apple Silicon macOS and clamped to 1..=256 | Walker::threads(n), for collect(), visit() and stream_parallel(); stream() is always single-threaded |
| Wildcard scope | Component-local, as in a shell | Walker::wildcard_mode(WildcardMode::SeparatorCrossing) for globset-style patterns |
| Stopping early | Runs to completion | Walker::cancellation(token), or Verdict::Stop from a visit predicate |
Match paths deliberately
Pattern accepts AsRef<[u8]>, so callers can match filenames without lossy
Unicode conversion. A &str is already such a value; for a Path or an
OsStr, ferralk_glob::path_bytes(path) borrows its native encoded bytes,
raw on Unix and lossless WTF-8 on Windows (ADR-0005). Pattern::is_match
compares an entire byte sequence. For paths, prefer is_match_glob_path:
ordinary *, ?, classes, and Extglob operators stay within one component,
while an explicitly enabled ** crosses components.
** is recursive only as a whole path component, as in gitignore, Bash
globstar, globset, and fast-glob: **, **/x, x/**, and x/**/y.
Whole means an unescaped / or an end of the pattern on both sides of the
star run; an escaped \/ is a literal byte and bounds nothing. A **/ hands
over only at a component start, so **/x matches x and a/x but not sx,
a/**/b matches a/b and a/x/b but not a/xb, and **/node_modules/**
does not reach my_node_modules. Anywhere else — a**, **b, a**/b,
**.ts — the run is ordinary, read exactly as it is with
recursive_double_star disabled: component-local under is_match_glob_path,
the position rule below under is_match_path, and separator-crossing like any
* under is_match. Braces expand first, so each alternative is judged as the
pattern it expands to: {**,x}/y holds a recursive **/y, a{**,x}/y an
ordinary a**/y. An extglob alternative stands where its group stands:
@(**)/y is recursive, x@(**)/y is not. A group then reads like its
alternatives written in its place, so the two ** rules above hold around
groups too: @(**)/y matches y as **/y does, and @(x)/**, !(y)/**,
and x/@(**) all accept x as x/** does. A whole-component run of three or
more stars stays recursive.
is_match_path preserves the zlob list-filter convention instead: a wildcard
in the root component may cross separators, and a wildcard standing directly
behind an explicit separator is component-local. That is zlob's exact rule and
it stops there: a later wildcard in the same component, the star of a/?* or
a/b*, may cross separators again, so a/?*/z accepts a/b/c/z while a/*/z
does not, exactly as zlob's matchPaths answers. Both path entry points ignore
one conventional leading ./ on the pattern and candidate; is_match compares
those bytes literally. With recursive_double_star disabled a consecutive **
is two ordinary stars: through is_match and is_match_glob_path that equals
*, while through is_match_path the second star stands behind the first
rather than behind the separator and follows the rule above.
Extglob groups do not introduce a fourth policy: their outer wildcards and
their alternatives use the same root and component rule selected by the entry
point. Through is_match_path a group directly behind a separator stands where
a wildcard would, so src/@(*.ts) keeps src/a/b.ts out exactly like
src/*.ts, src/!(x) matches within one component, and a later wildcard
inside an alternative crosses again as in a/b*. Only the first iteration of
+(…) or *(…) stands behind the separator, so src/+(*.ts) reads like
src/*.ts*.ts, and an outer star run of two or more reads as in plain syntax.
Brace alternatives are
judged one by one, so {src/@(*.ts),lib/*} and {src/@(*.ts),zz} answer
alike for src/a/b.ts. A recursive **/ prefix may consume zero directories
before a group, so **/@(x) matches both x and a/x, and not sx, when
both options are enabled.
use ferralk_glob::{Pattern, PatternOptions};
// Braces, extglobs and a recursive `**`; wildcards skip a leading period.
let pattern = Pattern::compile("{src,tests}/**/*.rs", PatternOptions::walker())?;
assert!(pattern.is_match_glob_path("src/lib.rs"));
assert!(pattern.is_match_glob_path("tests/unit/parser.rs"));
assert!(!pattern.is_match_glob_path(".cache/src/lib.rs"));
PatternOptions has two named starting points. PatternOptions::walker() is
the dialect Walker compiles its include and exclude patterns in, and the
walker builds its own options from it: recursive **, braces, and extglobs on,
everything else at its default, as the table below shows. Chain
.match_hidden(true) to mirror Walker::match_hidden(true) for an include,
and always for an exclude, which covers a leading period under either
setting, and .case_insensitive(true) to mirror
Walker::case_insensitive(true) for both. Match with
is_match_glob_path, or with is_match under
WildcardMode::SeparatorCrossing, to answer as the walker does for a
root-relative path. PatternOptions::default() is intentionally conservative:
under it src/**/*.rs does not match src/lib.rs, nor *.{rs,toml}
lib.rs, until the matching switch is enabled.
Scroll sideways for more columns.
| Option | default() | walker() | Effect when enabled |
|---|---|---|---|
braces | off | on | Enables nested {a,b} alternatives. |
recursive_double_star | off | on | Makes a whole-component ** (**/x, x/**, x/**/y) recursive. Any other star run, and every run while off, is ordinary stars. |
extglob | off | on | Enables Bash-style @(), ?(), *(), +(), and !(). |
match_hidden | off | off | Allows wildcard tokens to match a leading period. |
case_insensitive | off | off | Uses ASCII-only case folding; Walker::case_insensitive(true) on a walk. |
escape | on | on | Interprets backslash as an escape. |
Use Pattern::validate for syntax-only checks and Pattern::has_wildcards to
choose between a literal and glob path without compiling an application-level
policy twice.
Match against a list of globs
PatternSet is the counterpart of globset::GlobSet. It answers exactly what
asking each member Pattern in turn would, through the same three entry
points: is_match, is_match_path, and is_match_glob_path are the OR of the
members' verdicts, and matches_into, matches_path_into, and
matches_glob_path_into refill a caller's Vec<usize> with the index of every
member that matches, in ascending order.
use ferralk_glob::{PatternOptions, PatternSet};
let excludes = ["**/node_modules/**", "**/*.d.ts", "dist/**"];
let set = PatternSet::new(excludes, PatternOptions::walker().match_hidden(true))?;
assert!(set.is_match_glob_path("web/node_modules/pkg/index.js"));
assert!(!set.is_match_glob_path("src/main.ts"));
let mut matched = Vec::new();
set.matches_glob_path_into("dist/types/api.d.ts", &mut matched);
assert_eq!(matched, [1, 2]);
PatternSet::new compiles every glob with one PatternOptions; a glob that
does not compile is reported as a PatternSetError whose index() is its
position in the list and whose pattern_error() is the usual PatternError. For
per-glob options, compile the Patterns yourself and collect() them into a
set.
The set is faster than a loop, never different from one. At construction each
member is filed under a literal that every match must contain in a fixed
place: its final extension (**/*.rs), a whole path component
(**/node_modules/**, dist/**), or a fixed suffix or prefix. A path then
asks only the members whose literal it has. A member without such a literal
— *, **/*x*, or an extglob — is asked for every path, as a loop would ask
it; with many of those, globset's regex set is ahead, because this
crate does not translate globs into a regex
(ADR-0013).
Walk filesystems with explicit policy
Walker filters paths relative to its root. Include patterns are OR-ed; no
include accepts every non-excluded entry. Excluded directories are pruned only
when the walker can prove that no include can re-admit a descendant.
Patterns are anchored at the root. They are not matched at any depth the way a
slash-free .gitignore line is: exclude("target/**"), like a bare
exclude("target"), prunes only the target directly below the root, and a
target further down is walked as usual. Write exclude("**/target/**"), as
the example below does, to prune one at any depth. Includes read the same way:
src/**/*.rs selects below the top-level src only, **/src/**/*.rs below
every src. Rules read from ignore files under respect_git_ignore(true) keep
Git's own anchoring.
Matching is case-sensitive on every platform, whatever the filesystem does
with names: **/*.RS selects nothing in a tree of .rs files, and src/**
does not enter SRC/. Walker::case_insensitive(true) folds ASCII case in
includes and excludes, and in the pruning derived from them, so src/** then
walks into SRC/ and nowhere else. Two things keep their own case rule: the
walk root at the start of an absolute pattern, which is a path compared as
spelled, and ignore files, which follow git_ignore_case and the
repository's core.ignoreCase. The switch returns a Result, because
patterns added before it are recompiled; the only error it can report is a
pattern that exceeds the compiled-size limit under the new setting.
use ferralk::{CancellationToken, ErrorPolicy, WalkOptions, Walker};
let cancellation = CancellationToken::default();
let result = Walker::new("workspace")
.include("**/*.{rs,toml}")?
.exclude("**/target/**")?
.respect_git_ignore(true)
.cancellation(cancellation.clone())
.threads(4)
.error_policy(ErrorPolicy::Collect)
.options(
WalkOptions::default()
.files_only(true)
.sort(true)
.max_depth(8),
)
.collect()?;
if result.was_cancelled() {
eprintln!("walk stopped early");
}
# let _ = result;
# Ok::<(), Box<dyn std::error::Error>>(())
options() replaces every WalkOptions switch at once. It does not merge with
an earlier call, because a WalkOptions value cannot tell a switch left at its
default from one set to false. Build one value, as above, and pass it once:
.options(WalkOptions::default().files_only(true)).options(WalkOptions::default().sort(true))
is sorted but no longer files-only.
An entry's path() is the root exactly as it was given, a separator unless
the root is empty or already ends in one, and the path below the root:
Walker::new(".") yields ./src/lib.rs. WalkEntry::relative_path() is that
last part, src/lib.rs, and it is exactly the spelling the include and
exclude patterns were matched against, whether the root was written .,
./, x//y/ or a/... It borrows from the entry, and
WalkEntry::into_path() takes the whole path without copying it.
Keep a configured walker when input patterns are invalid
The chaining Walker::include and Walker::exclude methods intentionally
consume and return the builder, which keeps ordinary configuration concise. For
a caller-supplied pattern list, use their borrowed try_ counterparts instead.
Each rejected pattern leaves the complete Walker unchanged, so valid entries
before and after it still compose across all configured roots and matcher modes.
use ferralk::{WalkOptions, Walker};
let mut walker = Walker::new("workspace")
.add_root("generated-workspace")?
.options(WalkOptions::default().files_only(true));
for pattern in ["src/**/*.rs", "[a", "tests/**/*.rs"] {
if let Err(error) = walker.try_include(pattern) {
eprintln!("skipping invalid include {pattern:?}: {error}");
}
}
for pattern in ["**/generated/**", "[a", "**/*.tmp"] {
if let Err(error) = walker.try_exclude(pattern) {
eprintln!("skipping invalid exclude {pattern:?}: {error}");
}
}
let result = walker.collect()?;
# let _ = result;
# Ok::<(), Box<dyn std::error::Error>>(())
Important defaults:
Ordinary wildcards (
*,?, classes) stay inside one path component, so*.tsselects a file in the walk root and notsrc/main.ts. A**that is a whole path component is recursive under either setting —**/xselectsxanda/x, neversx— andWalker::wildcard_mode(WildcardMode::SeparatorCrossing)switches to the readingglobsetuses. fast-glob and globby already keep*in one component; see the migration table.Patterns use
/on every platform, and\is the escape character - on Windows too. A pattern built by joiningPathBufs therefore does not mean what it looks like:root.join("src").join("*.ts")producesC:\root\src\*.ts, where\*asks for a literal*in a filename. Writeinclude("src/**/*.ts")and letWalker::new(root)hold the path. Windows rejects the shapes that could never match, with a message saying so; see the compatibility guide.A leading
!is not negation. fast-glob, globby, and.gitignoreread!src/**as "notsrc/**"; a walker pattern list has no negation, soincludeandexcludereject a pattern, or a brace alternative, that starts with!instead of silently asking for a directory literally named!src. Pass what should be left out toexclude("src/**").!(…)is still the negated extglob, and\!still spells a literal!; see the compatibility guide.A walk may have several roots.
Walker::new(first).add_root(second)?walks both trees with one thread pool, andWalkEntry::rootsays which root an entry came from. Patterns apply under every root,depthis counted from the entry's own root, and roots that contain one another deliver their overlap once per root. A root must be a directory: a file root reportsread_dirwith a not-a-directory error and emits no entry for the file. A directory symlink supplied as a root is always traversed;follow_symlinksonly affects symlinks found below a root. See the compatibility guide.Include and exclude patterns may be absolute.
Walker::new("/repo") .include("/repo/src/**/*.ts")selects whatsrc/**/*.tsselects, so a caller holding absolute patterns does not have to strip the root itself. A pattern about a different tree selects nothing rather than erroring; a wildcard at or above the root, a.., and a pattern naming the root itself are rejected. A root whose name contains pattern syntax is spelled with that syntax escaped:/work/a\[1\]/**selects everything under/work/a[1]. A walk root with a..component rejects every absolute pattern, including one about a different tree, because the root cannot be related to any absolute path without resolving it; relative patterns work under such a root as usual. The conventional leading./is ignored once on every brace alternative, as the path matchers ignore it, so{./src,lib}/*.tsselects from both directories. Relative patterns receive the same guardrails:.and./name the root, and a.component after that prefix (such assrc/./main.rs), a real..component, or an empty component left by a repeated separator (src//*.ts) is rejected with guidance instead of silently selecting nothing. See the compatibility guide..gitignoreand.ignorefiles are considered only afterrespect_git_ignore(true). A subtree walk inherits the.gitignoreand.ignorechain from its repository root through the directory above the walk root, plus the repository's.git/info/exclude; the file closest to an entry decides, as in Git. The explicitly supplied walk root is always entered even when an inherited rule ignores that directory, matching ripgrep's explicit-root behavior; inherited rules still filter entries below it. Repository discovery walks parent directories to the filesystem root and does not stop at mount boundaries. Linked-worktree and submodule.gitpointer files are resolved (including onecommondir). In-tree.gitignoreand.ignoresymlinks are ignored rather than followed;.git/info/excludekeeps Git's link-following behavior. Each ignore or repository-metadata file is capped at 8 MiB, and each rule file at 100,000 lines. A rule file that exceeds either limit, or otherwise cannot be read, contributes no rules and is reported as aread_ignorefailure through the configuredErrorPolicy.Ferralk reads
core.ignoreCaseandcore.precomposeUnicodefrom the repository-local.git/config; a linked worktree also reads its privateconfig.worktreeafter the common config whenextensions.worktreeConfig=true.core.ignoreCase=trueuses Git-compatible ASCII folding for rules and candidates. A differently cased.Gitignoreis loaded only when opening the canonical.gitignorename resolves on the filesystem, so a case-sensitive filesystem never treats it as a magic file. On macOS only,core.precomposeUnicode=trueNFC-normalizes valid UTF-8 candidate components before matching and leaves invalid bytes untouched. These local booleans accept Git's named forms, bare keys, empty assignments, and signed base-zero integer forms (includingK/M/Gscaling); malformed values do not override an earlier valid repository-local value. Only the exact top-level[core]and[extensions]sections apply: quoted subsections are distinct. Git backslash-newline continuations are decoded before comments, quotes, escapes, and boolean parsing, preserving the next line's indentation.System/global config, includes, and environment overrides are intentionally not consulted: they are process-wide state outside a library walk. Use
Walker::git_ignore_case(value)andWalker::git_precompose_unicode(value)to supply Git's effective values; each explicit Walker value takes precedence over repository-local config.clear_git_ignore_caseandclear_git_precompose_unicoderesume repository-local detection on a reused builder.Ferralk deliberately continues into nested repositories, as ripgrep does: outer ignore rules remain active inside them and their in-tree
.gitignoreand.ignorefiles are loaded too. Their own.git/info/excludeand local Git config are not loaded. This differs from Git's repository-boundary behavior.Symlinks discovered below a root are not followed unless
WalkOptions::follow_symlinks(true)is selected; an ancestor-chain guard prevents directory cycles without suppressing acyclic symlink aliases. Separately supplied roots — including duplicates, overlaps and symlink aliases — start independent chains. A root that is itself a directory symlink is opened regardless, because the caller supplied it as the tree to walk.Repository discovery and ignore-rule anchoring resolve the root once, the way Git resolves its working directory:
.and..are folded and every symlink is followed, so a root reached through a link belongs to the repository that physically contains it whether it is spelledlink,./linkorwork/../link. Directories below the root derive their discovery spelling from that resolved root lexically, so an ignore file inside a followed symlinked directory anchors where its entries are spelled. Entry paths keep the caller's spelling throughout.files_only(true)anddirectories_only(true)filter on the kind a directory listing reports, and a listing reports a symlink as a symlink and nothing about its target. So by defaultfiles_onlykeeps every symlink - including a broken one and one pointing at a directory - anddirectories_onlykeeps none of them. That matches zlob'sZLOB_WALK_NO_REPORT_DIRS, which is why it is the default.WalkOptions::resolve_symlink_kind(true)classifies symlink entries by their target instead, which is what callers who meanPath::is_filewant:Scroll sideways for more columns.
the link points at files_onlydirectories_onlya file kept dropped a directory dropped kept nothing (broken) dropped dropped It costs one
statper symlink entry, paid only when one of the two filters is on and the walk is not already following symlinks. A broken link is dropped without an error - there is no target, so it is neither kind - while astatthat fails for any other reason leaves the kind unknown and is reported through theErrorPolicy. The entry's ownkind()still reportsSymlinkeither way: the switch answers what the link points at, not what the entry is.ErrorPolicy::Collectis the default. It returns accepted entries and recoverable filesystem errors in oneWalkResult. What each policy does with a recoverable error:Scroll sideways for more columns.
Policy Error below a root Root that cannot be opened collect()returnsCollect(default)collected in errors()collected in errors()Ok, with entries and errorsSkipdiscarded, reported nowhere collected in errors()Ok, with entries and root errorsAbortreturned as Err, ending the walkreturned as Err, ending the walkErrwith the first error; entries are discardedNeither
CollectnorSkipturns an error intoErr, socollect()?succeeds for a root that does not exist, with the failure only inWalkResult::errors(). Check it;WalkResultis#[must_use]for this reason. A missing, unreadable, or non-directory caller-supplied root is kept even underSkip, because it must not look like an empty tree, and other roots continue, so a multi-root walk can return their entries alongside the failed root's error.Skipis not "skip but tell me": useCollectfor that.visit()follows the same table.stream()yields each kept error as an item in place;stream_parallel()yields it as an item as soon as a worker meets it, which can be before entries of the directory it belongs to or of that directory's siblings. UnderAbortthe first error is the last item of either stream, unless a cancellation request is observed first, in which case the stream ends without it andwas_cancelled()is true. Ignore-rule read and safety-limit failures use theread_ignoreoperation; underSkipthey are discarded like other descendant-level recoverable failures.Results are unsorted unless
WalkOptions::sort(true)is set.stream()andstream_parallel()ignore that switch without a diagnostic; usecollect()when the order matters.Metadata is not fetched unless
WalkOptions::metadata(true)is set.stream()yields entries and recoverable errors incrementally. It is single-threaded and cannot provide global sorting. Because errors are items, iterator adapters count them:stream().take(10)can yield fewer than ten entries. Decide what an error means before counting, with.filter_map(Result::ok).take(10)to drop errors or.take(10).collect::<Result<Vec<_>, _>>()to stop at the first one.stream_parallel()yields the same items from the walkcollect()runs, onthreads(n)workers of its own, through a bounded channel to the thread that iterates it. Entries arrive in no particular order: workers interleave, so a directory is not necessarily yielded before its subtree. Workers pause while the consumer is behind, and dropping the stream stops the walk and joins its threads beforedropreturns. A worker panic resumes on the iterating thread. Nothing starts before the first item; withthreads(1)it isstream()itself, and otherwise it starts one thread even for a tiny tree, wherestream()is cheaper.A
WalkResultiterates with the same item type, owned or borrowed:for item in walker.collect()?yields every entry asOk, in the order ofentries(), and then every error asErr, so a loop cannot pass over the errors unseen.entries()andinto_parts()take the entries alone when the errors are handled separately.
Recoverable failures expose a typed WalkOperation; match it with a fallback
arm because the enum is non-exhaustive. as_str() provides the stable
machine-readable operation name. Human-readable WalkError and underlying
I/O error messages are diagnostic text, not a programmatic interface.
WalkError's Display names only the operation and the path, such as
read_dir src/locked; the io::Error that says why is its source(), and
WalkError::io_kind() is that error's io::ErrorKind, for branching without a
downcast. Error reporters that print the source chain, such as anyhow, therefore show the
cause once, and code that prints a WalkError by itself should print its
source as well.
CancellationToken::cancel requests a cooperative stop. It is safe to clone the
token and keep it outside the walker; walkers only observe it, so their own
abort errors, worker-start failures, visitor stops, and panics do not cancel a
token the caller may share or reuse.
The request is observed at a bounded granularity rather than between every two
filesystem operations. collect(), serial and parallel alike, reads the token
when a worker takes a directory and then once every 64 entries while it works
through the listing, so a cancelled walk classifies at most that many more
entries per worker before stopping and never opens another directory.
stream() reads it before every entry it yields. stream_parallel() does
both: its workers read it as collect()'s do, and the iterating thread reads
it before every item, so nothing found before the request is yielded after it.
Verdict::Stop reaches a
parallel walk's workers, the one that returned it included, at that same
granularity; a serial walk stops at the very next entry, because there the
verdict is a local flag rather than shared state.
Reading shared cancellation state per entry bought a promptness nothing can observe — a walk is free to finish the listing it has already read either way — and cost a load of a shared atomic on the walk's hottest loop.
Filtering with a matcher of your own
collect() hands back every entry it found, so a caller that applies its own
predicate afterwards runs it on one thread over the whole list. On a large tree
that pass is big enough to cancel out the threads the walk just used.
visit() asks the predicate on the worker that produced the entry. A
WalkEntry keeps its native Path, while Pattern is byte-first; pass
WalkEntry::path_bytes() to bridge them without allocating or converting
through UTF-8. The bytes are the native OsStr::as_encoded_bytes() form: raw
filesystem bytes on Unix and lossless WTF-8 on Windows (ADR-0005). They include
the root; for a pattern written relative to the root, as the walker's own are,
pass ferralk_glob::path_bytes(entry.relative_path()) instead.
use ferralk::{Verdict, Walker, ferralk_glob::{Pattern, PatternOptions}};
let my_matcher = Pattern::compile(
"**/*.rs",
PatternOptions::default().recursive_double_star(true),
)?;
let result = Walker::new("src")
.threads(4)
.visit(|entry| {
if my_matcher.is_match(entry.path_bytes()) {
Verdict::Keep
} else {
Verdict::Skip
}
})?;
# let _ = result;
# Ok::<(), Box<dyn std::error::Error>>(())
Everything else behaves as it does for collect(): cancellation, the error
policy, panic propagation and sorting are unchanged, and only which entries
survive differs.
Verdict::Skipdrops the entry from the result. It does not prune: a directory is still descended into.Verdict::Prunedrops the entry and does not walk below it: a pruned directory is never opened, so nothing inside it is visited, returned, or reported as an error. It isexclude()decided at run time, the wayfilter_entryprunes inwalkdirandignore, and for a file it is the same asSkip. The visitor is asked about a directory before anything inside it, but only about entries the walk would return: a directory that no include selects, or any directory underfiles_only(true), is walked without being offered for pruning.Verdict::Stopends the walk the way a cancellation request does, andWalkResult::was_cancelledreports it. A caller-ownedCancellationTokenis left alone.- The visitor is shared across workers rather than cloned, so it takes
&selfand must beSync. Per-worker state belongs in a thread-local.
Below a small tree size the walk stays on one thread whatever threads() says:
starting workers costs more than a handful of directories does. threads() is
a ceiling in the other direction too: a helper the operating system refuses to
start, because the process is at its thread or memory limit, leaves the walk
to the workers already running and is reported as one recoverable
spawn_worker error under ErrorPolicy::Collect.
Wide directories do not become equally wide task queues. stream() suspends
the current listing in one frame while it enters a child, and parallel walks
classify at most 1,024 entries before handing the remaining listing back
through one continuation. The listing names and retained results still scale
with the directory, but pending traversal work is bounded by depth for
stream() and by one batch per active worker for parallel collection. The
batch is wide enough that an ordinary directory is classified in one piece, so
its name buffers stay with the worker for the next directory instead of
traveling into a continuation.
A long-running host should know one allocator effect of parallel walks. Each
helper thread lands in its own glibc malloc arena, and the result vectors a
large walk frees raise glibc's dynamic trim threshold, so after a parallel
walk of a few hundred thousand entries the process can keep tens of megabytes
of freed heap resident with almost nothing live; serial and stream() walks
show no such retention. This is glibc's behavior, not a leak. A daemon or
language server that measures it can cap the arenas with MALLOC_ARENA_MAX
or install a different global allocator.
Hidden paths: two separate switches
An ordinary wildcard does not cover a leading period, so **/*.ts skips
.react-router/routes.ts — the period belongs to a directory component, and
the whole subtree stays out of the result. The rule holds inside one component
too: *.rs matches .rs only with match_hidden(true). A negated extglob
is an ordinary wildcard for this rule: !(x) covers neither .env nor the
hidden component of a/.env, and naming a hidden exception, as in !(.env),
opts no other hidden name in.
Walker::match_hidden(true) opts in for include patterns:
use ferralk::{WalkOptions, Walker};
let result = Walker::new("workspace")
.match_hidden(true)
.include("site/**/*.ts")?
.exclude("**/node_modules/**")?
.options(WalkOptions::default().sort(true))
.collect()?;
# let _ = result;
# Ok::<(), Box<dyn std::error::Error>>(())
Builder order does not matter; includes added before the call are recompiled.
A literal period is not a wildcard, so .claude/** selects a hidden directory
with either setting.
Excludes are not subject to this rule. They apply in every directory the walk
enters, hidden ones included, the way a .gitignore line does: a wildcard in
an exclude, ** included, covers a leading period whatever match_hidden
says. Without any include, exclude("**/node_modules/**") removes
.cache/node_modules/a.ts too, and exclude("*.log") removes .debug.log as
well as debug.log. An exclude also wins over an include that names a hidden
entry literally: include(".github/**") with exclude("**/*.yml") drops
.github/workflows/ci.yml. match_hidden therefore decides what a walk can
select, never what an exclude removes.
WalkOptions::skip_hidden(true) is a different mechanism, not the inverse of
this one. It is a traversal filter: it drops every entry with a leading-period
component and never descends into a hidden directory, before any pattern is
consulted. match_hidden only decides what a wildcard in an include is
allowed to cover.
With skip_hidden enabled no hidden path survives long enough for
match_hidden to matter.
Platform support
The portable backend uses std::fs on Linux, macOS, and Windows. Experimental
native backends are feature-gated:
ferralk = { version = "1.1.0", features = ["native-linux"] } # x-release-please-version
native-linux applies only on Linux and native-macos only on macOS; other
platforms retain the portable backend. The feature names and everything they
select are experimental and explicitly outside the 1.x semver contract: they
may be renamed or removed in a minor release. See the
stability contract and ADR-0010 for the rollout policy. The
default portable backend remains the stable target. native-macos links
Darwin's private __getdirentries64
stub: it is what libc's readdir uses today, but it is not an Apple public API
and can be rejected by App Store private-symbol checks. App-Store-distributed
applications should keep the default portable backend. If a filesystem or a
future OS refuses the native call at runtime, Ferralk maps that refusal to the
portable fallback; this preserves correctness but not native performance.
On Unix, a native no-follow walk opens scheduled non-root directories with
O_NOFOLLOW, closing the directory-to-symlink replacement window at the final
component while retaining the portable reader's support for a user-supplied
directory-symlink root. A changed scheduled path is rejected (ELOOP on
Linux; Darwin may report ENOTDIR with O_DIRECTORY). If a native syscall is
unsupported, Ferralk reports that scheduled directory rather than reopening it
through a symlink-following std::fs::read_dir(path) call. This is the native
reader's deliberate safety advantage; a portable-only walk cannot offer the
same atomic final-component guarantee.
Development and validation
Run the single canonical pull-request preflight in
CONTRIBUTING before changing
matcher or walker behavior. It covers both the root workspace and the
separate platform-neutral fuzz targets, while excluding the development-only
oracle package so no Zig installation is needed. CI additionally exercises
platform-specific, sanitizer, and coverage lanes; coverage includes oracle
and installs Zig itself. Install Zig 0.16 and libclang locally only when you
need to validate oracle too.
The minimum supported Rust version is written down once, as rust-version
under [workspace.package] in the root Cargo.toml; the CI MSRV job reads
that field instead of pinning a version of its own. Per
ADR-0004 the MSRV tracks current stable
minus two releases. The scheduled/manual policy check compares that field with
Rust's official stable-channel metadata, so a new stable release turns drift
into a targeted maintenance update rather than a surprise in ordinary CI.
The checked-in JSONL corpus is the behavioral source of truth. Read
corpus-format.md before adding a case. Fuzz targets live in
fuzz/; benchmark evidence and other deferred follow-up
are tracked in GitHub.