Ferralk / Contributing
Corpus format
The checked-in corpus is the executable compatibility contract. Each topic is
a UTF-8 JSON Lines (.jsonl) file under ../corpus. One
non-empty line is one independently replayable case. CI validates every file
with cargo run -p harness -- corpus; it never invokes Zig or the live zlob
oracle.
Record shape
Every record conforms to corpus.schema.json:
Scroll sideways for more columns.
| Field | Required | Meaning |
|---|---|---|
id | yes | Stable, topic-local identifier, for example wildcard-001. |
kind | yes | matcher, match_glob_path for the component-local reading, has_wildcards for syntax preflight, match_paths / _at for borrowed list filtering, match_path_indices / _at for positions, compile_error for a pattern the compiler must reject, ignore for a Git-ignore verdict, or absolute_pattern for an absolute walker pattern rewritten against a walk root. |
paths / matches | no | Input and Ferralk-selected path lists for match_paths, preserving input order. |
oracle_matches | no | zlob's selected list for a deliberate list-result divergence. |
base_path | no | Base directory stripped before matching each input path in a match_paths_at case, and the walk root an absolute_pattern case rewrites against. |
rewritten | no | The root-relative pattern an absolute_pattern case must produce. Absent means the pattern names paths outside the root and can select nothing. |
windows_paths | no | true if an absolute_pattern case is written in Windows path syntax. Unlike platform, this does not restrict the case to one host: what makes a path absolute is a property of the pattern, so both spellings are replayed everywhere. |
indices / oracle_indices | no | Ferralk and divergent-oracle input positions for a match_path_indices operation. |
pattern | yes | Glob or ignore expression using the byte codec below. |
path | yes | Candidate path using the byte codec below; empty for syntax-only records. |
flags | no | Ordered behavior switches from the compatibility matrix. |
ignore_rules | no | Lines written to a synthetic .gitignore for an ignore case. |
nested_ignore_rules | no | Further .gitignore files below the root, each with its directory and rules. |
exclude_rules | no | Repository-wide rules written to .git/info/exclude; every ignore file overrides them. |
candidate_is_dir | no | For an ignore case, create path as a directory instead of a regular file. |
candidate_is_symlink | no | For a POSIX ignore case, create path as a symlink to a directory instead of a regular file. Mutually exclusive with candidate_is_dir. |
git_ignorecase | no | For an ignore case, set the isolated repository's core.ignorecase to true. |
expected | yes | Whether the expression accepts the candidate; for list operations, whether the real selection is non-empty; for absolute_pattern, whether rewriting produces a match-capable relative pattern. Exact list and rewrite results remain in their dedicated fields. |
oracle_expected | no | The external-oracle result when it deliberately differs from expected. |
error_offset | no | Byte offset the rejected construct must be reported at, for a compile_error or absolute_pattern case. |
error_message | no | Stable error text a compile_error or absolute_pattern case must produce. Its presence is what makes an absolute_pattern case a rejection. |
platform | no | posix or windows; restricts a separator-dependent verdict to one host. |
source | yes | zlob_1_6_3, fast_glob, git_check_ignore, or handwritten. |
adr | no | Four-digit ADR number that adopts Ferralk's deliberate divergence from the recorded oracle. |
oracle_defect | no | true when the disagreement is an oracle defect or documented limit of a secondary oracle. |
note | no | Short explanation or cross-oracle disagreement. |
The topic files are basic.jsonl, braces.jsonl, bytes.jsonl,
case-folding.jsonl, classes.jsonl, dotfiles.jsonl, edge-cases.jsonl, errors.jsonl,
extglob-suite.jsonl, fast-glob.jsonl, fnmatch.jsonl,
glibc-recursive.jsonl, globstar.jsonl, ignore.jsonl, match-paths.jsonl,
path-matcher.jsonl, platform.jsonl, preflight.jsonl, and
wildcards.jsonl. Files are added when a topic gains a case. Case IDs do
not change once published; a changed expected value is a new case plus an
explanation in note. The one-time explicit-kind migration also normalized
the previously self-compared list and absolute-pattern expected fields from
their unchanged matches, indices, and rewritten results; those metadata
normalizations are not matcher-verdict changes.
An ignore case may live in any topic file; routing depends on kind, never
on the filename. In ignore.jsonl, pattern is the primary rule for quick
review and ignore_rules is the complete ordered rule chain. The Git oracle
creates an isolated repository, writes those lines into .gitignore, creates
path, and uses git check-ignore --no-index --quiet -- path.
candidate_is_dir keeps
directory-only rules tied to an actual directory entry;
candidate_is_symlink verifies that a link to a directory remains a symlink
candidate; git_ignorecase opts into Git's repository-local ASCII case
folding. This keeps nested/negated behavior tied to Git rather than to Ferralk
implementation details.
nested_ignore_rules adds .gitignore files below the root. Git consults the
file closest to the candidate last, so a deeper file overrides a shallower one,
and only these records can express that precedence.
The oracle runs Git with GIT_CONFIG_GLOBAL, GIT_CONFIG_SYSTEM and HOME
neutralized. Without that the developer's own core.excludesFile decides
corpus verdicts: a global *.log rule makes a case pass locally and fail in
CI, or worse, pass in both for the wrong reason.
Ignore verdicts use Git 2.52.0 as the pinned CI reference and require Git
2.52.0 or newer for local oracle replay. Git 2.52 fixed the X**/Y pathname
reading so that the separator cannot disappear and fuse the literal
components; earlier releases therefore disagree with the version-sensitive
ignore-074 through ignore-076 records. The harness reports and skips the
Git-backed replay on those older installations instead of presenting their
version-dependent result as a corpus regression.
Two readings of one pattern
A matcher case is the fnmatch reading zlob defines: an ordinary wildcard may
cross a separator. A match_glob_path case is the filesystem-glob reading of
Pattern::is_match_glob_path, where an ordinary wildcard stays inside one path
component and only ** crosses. The same pattern and candidate can appear
under both kinds with different verdicts, and corpus/fast-glob.jsonl does
exactly that, because fast-glob implements the component-local reading while
zlob implements the fnmatch one.
An absolute_pattern case describes Walker::include and Walker::exclude
rewriting an absolute pattern for a walk root, and has three outcomes: it
produces rewritten, it names paths outside the root and so can select nothing
(neither rewritten nor error_message), or it is rejected with
error_message. A relative pattern passes through the same step unchanged,
so the walker-only rules for relative patterns, such as refusing a leading !
that is not an extglob, are recorded as this kind too. The zlob oracle skips
these, having no walker to compare against.
The zlob oracle skips match_glob_path cases: zlob has no component-local
mode to compare against.
Rejected patterns
A compile_error case records a pattern the compiler must refuse. path is
empty and expected is false, so a rejected pattern stays comparable with
every other kind: it accepts nothing. error_offset and error_message are
optional; when present the harness asserts the exact byte offset and the
stable message, which pins the diagnostic and not only the failure. Both
fields are rejected on any other kind. Constructs zlob treats as ordinary text
rather than as errors — an unmatched brace, an unterminated extglob — stay
matcher cases so the difference is written down instead of assumed.
{"id":"error-unclosed-class","kind":"compile_error","pattern":"[abc","path":"","flags":[],"expected":false,"error_offset":0,"error_message":"unclosed character class","source":"handwritten"}
Platform-specific verdicts
Most verdicts hold everywhere, but the separator set does not: per
ADR-0005 a backslash separates
components on Windows and is an ordinary byte elsewhere. An optional
platform of posix or windows restricts such a case to the host it
describes; every other host skips it and reports the skip count. A verdict
that differs per platform is therefore two records with the same pattern and
path, one per platform, rather than one record with a hidden assumption.
Byte codec
pattern and path are JSON strings that carry either normal UTF-8 text or
canonical \\xNN byte escapes. NN is exactly two uppercase hexadecimal
digits. The escape represents one byte; it is applied after JSON decoding.
Printable ASCII other than a backslash is written literally. A backslash,
control byte, or any byte that is not part of valid UTF-8 is emitted as
\\xNN. Valid non-ASCII Unicode is also escaped byte-by-byte, so corpus diffs
remain ASCII-only and unambiguous. For example:
{"id":"bytes-001","pattern":"*.txt","path":"caf\\xC3\\xA9.txt","flags":[],"expected":true,"source":"handwritten"}
The corpus crate owns the strict Rust encoder and decoder. Malformed
backslashes such as \\n, incomplete escapes, and lower-level JSON decoding
errors are validation failures, never silently interpreted as paths.
Evidence and disputes
zlob_1_6_3 cases originate from the pinned oracle. fast_glob is used only
where its syntax overlaps. git_check_ignore is normative for ignore topics.
Handwritten records are allowed for a documented policy decision or a reduced
regression case. A disagreement must carry exactly one provenance marker:
adr for a deliberate Ferralk policy or oracle_defect: true for a defect or
documented limit in the reference implementation. expected is always the
Ferralk contract and still runs in the harness; oracle_expected holds the
diverging reference result and runs only in the appropriate oracle adapter.
The harness rejects a divergence without provenance, so neither it nor a
future matcher can silently discard a known difference.
The zlob 1.6.3 public API disagrees with its own frozen source assertions for
fnmatch-l0092 and fnmatch-l0097, and with its filesystem tests for the
empty branch of a leading repeating extglob in
path-matcher-repeating-extglob-hidden-on and
absolute-path-repeating-extglob-hidden-off. Those four records are oracle
defects rather than Ferralk policy decisions.
fast-glob 1.1 is a secondary differential oracle, not Ferralk's semantic
authority. Its eight recorded disagreements cover the trailing-** elision,
backslash and POSIX-class syntax, separator handling in classes, leading ./
normalization, and its brace-group limit. They carry oracle_defect: true to
state that no Ferralk contract decision is pending on their result.
What the zlob oracle skips
The pinned oracle is a &str-based Rust API over zlob 1.6.3, so parts of the
contract have no representation there. The adapter in
../tools/oracle/tests/zlob_oracle.rs
skips six categories and prints a count for each: a case_insensitive case
(zlob 1.6.3 has no case-folding flag), a case whose pattern, path, or candidate
list is not UTF-8, a compile_error case, a match_glob_path case, an
absolute_pattern case (zlob has no walker), and a case written for another
platform. It also skips two whole topics, ignore.jsonl
and fast-glob.jsonl, whose oracle_expected records a different oracle's
verdict. The empty-list NOCHECK case is also reported as a dedicated skip:
zlob's Rust FFI cannot safely expose its synthetic result, so the adapter must
not count it as replayed. A skipped case is not a weaker case: it
still replays in normal CI through the harness, which is the Ferralk
contract. The adapter asserts the exact replayed and skipped totals and prints
a per-file replay tally, so any corpus inventory change requires an explicit
review instead of disappearing behind a broad skip condition.