Skip to content
Ferralk

Ferralk / Contributing

Corpus format

The checked-in corpus is the executable compatibility contract. Each topic is a UTF-8 JSON Lines (.jsonl) file under ../corpus. One non-empty line is one independently replayable case. CI validates every file with cargo run -p harness -- corpus; it never invokes Zig or the live zlob oracle.

Record shape

Every record conforms to corpus.schema.json:

FieldRequiredMeaning
idyesStable, topic-local identifier, for example wildcard-001.
kindyesmatcher, match_glob_path for the component-local reading, has_wildcards for syntax preflight, match_paths / _at for borrowed list filtering, match_path_indices / _at for positions, compile_error for a pattern the compiler must reject, ignore for a Git-ignore verdict, or absolute_pattern for an absolute walker pattern rewritten against a walk root.
paths / matchesnoInput and Ferralk-selected path lists for match_paths, preserving input order.
oracle_matchesnozlob's selected list for a deliberate list-result divergence.
base_pathnoBase directory stripped before matching each input path in a match_paths_at case, and the walk root an absolute_pattern case rewrites against.
rewrittennoThe root-relative pattern an absolute_pattern case must produce. Absent means the pattern names paths outside the root and can select nothing.
windows_pathsnotrue if an absolute_pattern case is written in Windows path syntax. Unlike platform, this does not restrict the case to one host: what makes a path absolute is a property of the pattern, so both spellings are replayed everywhere.
indices / oracle_indicesnoFerralk and divergent-oracle input positions for a match_path_indices operation.
patternyesGlob or ignore expression using the byte codec below.
pathyesCandidate path using the byte codec below; empty for syntax-only records.
flagsnoOrdered behavior switches from the compatibility matrix.
ignore_rulesnoLines written to a synthetic .gitignore for an ignore case.
nested_ignore_rulesnoFurther .gitignore files below the root, each with its directory and rules.
exclude_rulesnoRepository-wide rules written to .git/info/exclude; every ignore file overrides them.
candidate_is_dirnoFor an ignore case, create path as a directory instead of a regular file.
candidate_is_symlinknoFor a POSIX ignore case, create path as a symlink to a directory instead of a regular file. Mutually exclusive with candidate_is_dir.
git_ignorecasenoFor an ignore case, set the isolated repository's core.ignorecase to true.
expectedyesWhether the expression accepts the candidate; for list operations, whether the real selection is non-empty; for absolute_pattern, whether rewriting produces a match-capable relative pattern. Exact list and rewrite results remain in their dedicated fields.
oracle_expectednoThe external-oracle result when it deliberately differs from expected.
error_offsetnoByte offset the rejected construct must be reported at, for a compile_error or absolute_pattern case.
error_messagenoStable error text a compile_error or absolute_pattern case must produce. Its presence is what makes an absolute_pattern case a rejection.
platformnoposix or windows; restricts a separator-dependent verdict to one host.
sourceyeszlob_1_6_3, fast_glob, git_check_ignore, or handwritten.
adrnoFour-digit ADR number that adopts Ferralk's deliberate divergence from the recorded oracle.
oracle_defectnotrue when the disagreement is an oracle defect or documented limit of a secondary oracle.
notenoShort explanation or cross-oracle disagreement.

The topic files are basic.jsonl, braces.jsonl, bytes.jsonl, case-folding.jsonl, classes.jsonl, dotfiles.jsonl, edge-cases.jsonl, errors.jsonl, extglob-suite.jsonl, fast-glob.jsonl, fnmatch.jsonl, glibc-recursive.jsonl, globstar.jsonl, ignore.jsonl, match-paths.jsonl, path-matcher.jsonl, platform.jsonl, preflight.jsonl, and wildcards.jsonl. Files are added when a topic gains a case. Case IDs do not change once published; a changed expected value is a new case plus an explanation in note. The one-time explicit-kind migration also normalized the previously self-compared list and absolute-pattern expected fields from their unchanged matches, indices, and rewritten results; those metadata normalizations are not matcher-verdict changes.

An ignore case may live in any topic file; routing depends on kind, never on the filename. In ignore.jsonl, pattern is the primary rule for quick review and ignore_rules is the complete ordered rule chain. The Git oracle creates an isolated repository, writes those lines into .gitignore, creates path, and uses git check-ignore --no-index --quiet -- path. candidate_is_dir keeps directory-only rules tied to an actual directory entry; candidate_is_symlink verifies that a link to a directory remains a symlink candidate; git_ignorecase opts into Git's repository-local ASCII case folding. This keeps nested/negated behavior tied to Git rather than to Ferralk implementation details.

nested_ignore_rules adds .gitignore files below the root. Git consults the file closest to the candidate last, so a deeper file overrides a shallower one, and only these records can express that precedence.

The oracle runs Git with GIT_CONFIG_GLOBAL, GIT_CONFIG_SYSTEM and HOME neutralized. Without that the developer's own core.excludesFile decides corpus verdicts: a global *.log rule makes a case pass locally and fail in CI, or worse, pass in both for the wrong reason.

Ignore verdicts use Git 2.52.0 as the pinned CI reference and require Git 2.52.0 or newer for local oracle replay. Git 2.52 fixed the X**/Y pathname reading so that the separator cannot disappear and fuse the literal components; earlier releases therefore disagree with the version-sensitive ignore-074 through ignore-076 records. The harness reports and skips the Git-backed replay on those older installations instead of presenting their version-dependent result as a corpus regression.

Two readings of one pattern

A matcher case is the fnmatch reading zlob defines: an ordinary wildcard may cross a separator. A match_glob_path case is the filesystem-glob reading of Pattern::is_match_glob_path, where an ordinary wildcard stays inside one path component and only ** crosses. The same pattern and candidate can appear under both kinds with different verdicts, and corpus/fast-glob.jsonl does exactly that, because fast-glob implements the component-local reading while zlob implements the fnmatch one.

An absolute_pattern case describes Walker::include and Walker::exclude rewriting an absolute pattern for a walk root, and has three outcomes: it produces rewritten, it names paths outside the root and so can select nothing (neither rewritten nor error_message), or it is rejected with error_message. A relative pattern passes through the same step unchanged, so the walker-only rules for relative patterns, such as refusing a leading ! that is not an extglob, are recorded as this kind too. The zlob oracle skips these, having no walker to compare against.

The zlob oracle skips match_glob_path cases: zlob has no component-local mode to compare against.

Rejected patterns

A compile_error case records a pattern the compiler must refuse. path is empty and expected is false, so a rejected pattern stays comparable with every other kind: it accepts nothing. error_offset and error_message are optional; when present the harness asserts the exact byte offset and the stable message, which pins the diagnostic and not only the failure. Both fields are rejected on any other kind. Constructs zlob treats as ordinary text rather than as errors — an unmatched brace, an unterminated extglob — stay matcher cases so the difference is written down instead of assumed.

{"id":"error-unclosed-class","kind":"compile_error","pattern":"[abc","path":"","flags":[],"expected":false,"error_offset":0,"error_message":"unclosed character class","source":"handwritten"}

Platform-specific verdicts

Most verdicts hold everywhere, but the separator set does not: per ADR-0005 a backslash separates components on Windows and is an ordinary byte elsewhere. An optional platform of posix or windows restricts such a case to the host it describes; every other host skips it and reports the skip count. A verdict that differs per platform is therefore two records with the same pattern and path, one per platform, rather than one record with a hidden assumption.

Byte codec

pattern and path are JSON strings that carry either normal UTF-8 text or canonical \\xNN byte escapes. NN is exactly two uppercase hexadecimal digits. The escape represents one byte; it is applied after JSON decoding.

Printable ASCII other than a backslash is written literally. A backslash, control byte, or any byte that is not part of valid UTF-8 is emitted as \\xNN. Valid non-ASCII Unicode is also escaped byte-by-byte, so corpus diffs remain ASCII-only and unambiguous. For example:

{"id":"bytes-001","pattern":"*.txt","path":"caf\\xC3\\xA9.txt","flags":[],"expected":true,"source":"handwritten"}

The corpus crate owns the strict Rust encoder and decoder. Malformed backslashes such as \\n, incomplete escapes, and lower-level JSON decoding errors are validation failures, never silently interpreted as paths.

Evidence and disputes

zlob_1_6_3 cases originate from the pinned oracle. fast_glob is used only where its syntax overlaps. git_check_ignore is normative for ignore topics. Handwritten records are allowed for a documented policy decision or a reduced regression case. A disagreement must carry exactly one provenance marker: adr for a deliberate Ferralk policy or oracle_defect: true for a defect or documented limit in the reference implementation. expected is always the Ferralk contract and still runs in the harness; oracle_expected holds the diverging reference result and runs only in the appropriate oracle adapter. The harness rejects a divergence without provenance, so neither it nor a future matcher can silently discard a known difference.

The zlob 1.6.3 public API disagrees with its own frozen source assertions for fnmatch-l0092 and fnmatch-l0097, and with its filesystem tests for the empty branch of a leading repeating extglob in path-matcher-repeating-extglob-hidden-on and absolute-path-repeating-extglob-hidden-off. Those four records are oracle defects rather than Ferralk policy decisions.

fast-glob 1.1 is a secondary differential oracle, not Ferralk's semantic authority. Its eight recorded disagreements cover the trailing-** elision, backslash and POSIX-class syntax, separator handling in classes, leading ./ normalization, and its brace-group limit. They carry oracle_defect: true to state that no Ferralk contract decision is pending on their result.

What the zlob oracle skips

The pinned oracle is a &str-based Rust API over zlob 1.6.3, so parts of the contract have no representation there. The adapter in ../tools/oracle/tests/zlob_oracle.rs skips six categories and prints a count for each: a case_insensitive case (zlob 1.6.3 has no case-folding flag), a case whose pattern, path, or candidate list is not UTF-8, a compile_error case, a match_glob_path case, an absolute_pattern case (zlob has no walker), and a case written for another platform. It also skips two whole topics, ignore.jsonl and fast-glob.jsonl, whose oracle_expected records a different oracle's verdict. The empty-list NOCHECK case is also reported as a dedicated skip: zlob's Rust FFI cannot safely expose its synthetic result, so the adapter must not count it as replayed. A skipped case is not a weaker case: it still replays in normal CI through the harness, which is the Ferralk contract. The adapter asserts the exact replayed and skipped totals and prints a per-file replay tally, so any corpus inventory change requires an explicit review instead of disappearing behind a broad skip condition.