Build Instructions: A jq Interpreter

Objective

Build an interpreter for the jq language as described in sources/jq-manual.txt. Correctness is measured by the upstream jq conformance corpus, sources/jq.test, taken verbatim from jq 1.8.2. The goal is to pass every case the corpus supplies, with none failed and none errored. The suite's size is a property of the pinned corpus; never assert a case count.

The implementation language is Python, fixed by this Target's TECHNOLOGY_STACK.md and governed by stack/python.md.

jq is a small language with a large semantic core. Almost every filter is a generator: it takes one input and produces a stream of zero, one, or many outputs, and downstream filters run once per upstream output. Backtracking through that stream is not an optimisation, it is the evaluation model, and reduce, foreach, limit, first, label/break, and the ?// destructuring alternative are all defined in terms of it. An implementation that treats a filter as a function returning one value will pass the early cases and then stall permanently. Decide the evaluation model before writing builtins.

Run Harness

sources/full_test.sh is the single scoring entry point. It is supplied, not authored: it is staged verbatim into the build directory alongside the other imported assets, and drydock uat runs sh sources/full_test.sh from the completed application root and takes its exit code and output as the score. It reads:

#!/bin/sh
# full_test.sh — scoring entry point. Do not filter, skip, or reinterpret.
set -eu
if [ ! -x ./jq ]; then
    echo "error: no executable ./jq at the application root." >&2
    echo "The deliverable is an executable named jq that reads JSON on stdin." >&2
    exit 1
fi
JQ="$PWD/jq" exec python3 sources/run_conformance.py

Before relying on any path above, run ls sources/ in the application directory and correct the paths in the harness against what is actually on disk. Correcting a path is the only edit permitted to this script. Do not add flags, filters, skips, or a redirection of the exit code.

The interface check is deliberately separate from the conformance run so that a missing program and a genuine conformance failure are distinguishable in the evidence. JQ is the harness's only knowledge of the implementation language; the harness itself is language-neutral.

The scoring assets are read-only. sources/full_test.sh, sources/run_conformance.py, sources/exclusions.txt, and sources/jq.test are hash-verified against the import and restored before grading, so a modification is reported as tampering rather than honoured. Do not write to them. Build ./jq so that the supplied entry point succeeds; changing the entry point is not a repair.

Interface Contract

The program is a filter: an executable file named jq at the application root, invoked as

./jq -c '<program>'

with JSON on stdin. It writes each value the program produces to stdout as one compact JSON value per line, and exits 0.

-c is the only option exercised. The program need not implement any other jq command-line option, and the manual's "Invoking jq" section is omitted from sources/jq-manual.txt for that reason.

Exit codes follow jq's own, and the distinction is load-bearing because the harness grades on it:

A case may legitimately emit several values and then raise; the harness compares the values produced before the raise, so exit 5 is not by itself a failure. Exit 3 on a valid program is always a failure. Diagnostics go to stderr and are never compared.

Any implementation shape that satisfies this contract is acceptable. A #!/usr/bin/env python3 script named jq that imports the real work from a package alongside it is the obvious one; main should parse arguments and delegate.

Test / Verification Process

The imported source files are placed in a sources/ subdirectory of the application directory. No installation step and no network access are required at any point.

JQ="$PWD/jq" python3 sources/run_conformance.py                     # the scored run
JQ="$PWD/jq" python3 sources/run_conformance.py -v                  # list passing cases too
JQ="$PWD/jq" python3 sources/run_conformance.py --json              # machine-readable
JQ="$PWD/jq" python3 sources/run_conformance.py --list              # print cases, run nothing
JQ="$PWD/jq" python3 sources/run_conformance.py --select 'reduce'   # one construct at a time

During development, sh sources/full_test.sh does the interface check and the conformance run together, and is the same command the score is taken from.

--select takes a regular expression matched against the case's program text and exists for development only. The acceptance gate always runs the whole corpus; there is no scoped gate and none may be created.

The summary line is:

jq conformance: NNN passed, N failed, N errored, N skipped (corpus jq.test @ jq-1.8.2)

The harness reserves exit 2 for its own faults — a missing corpus, an unset JQ, a stale exclusion. Exit 2 never means the interpreter is wrong.

The corpus format

sources/jq.test documents its own format in its header. Cases are separated by blank lines; blank lines and # lines are ignored. A case is a program line, an input line, and then the expected output values, one per line. A case preceded by %%FAIL is a program that must be rejected at compile time: the following lines are upstream jq's diagnostic, which this harness records but never compares. Reproducing jq's exact error text is reverse-engineering a C implementation, not conforming to a specification, so a %%FAIL case passes on exit 3 alone.

Values are compared structurally, not textually. 1 and 1.0 are the same jq value; so are two objects whose keys are printed in a different order. Formatting of output is therefore not under test, but the number and order of values is.

Declared exclusions

sources/exclusions.txt names the corpus cases this kit cannot run, with the reason. They are the module-loader cases: import and include resolved against a search path of fixture files that this kit's flat source import cannot carry. They are reported as skipped and are not part of the score.

The module grammar cases are not excluded and must pass. module (.+1); 0, module []; 0, include "a" (.+1); 0, include "a" []; 0, include "\ "; 0, include "\(a)"; 0, and %::wat are all %%FAIL cases: the front end must parse the module syntax far enough to reject them, without ever touching the filesystem.

Files the LLM Needs

• sources/jq-manual.txt — the jq language manual at 1.8.2, rendered to plain text.

The primary specification. Required, and the normative description of every builtin.

• sources/jq.test — the conformance corpus. Also the most precise available statement of

the semantics, especially for generators and backtracking.

• sources/parser.y — upstream's yacc grammar. The authority on operator precedence,

associativity, and the shape of every syntactic form.

• sources/lexer.l — upstream's lexer. The authority on tokens, string interpolation, and

escape handling.

• sources/builtin.jq — the subset of jq's builtins that upstream defines in jq itself.

Read it as a specification of those builtins' semantics.

• sources/run_conformance.py, sources/full_test.sh, sources/exclusions.txt —

the scoring instruments. Read-only.

Source Roles

Record this table in the Analysis so every asset is staged onto disk in the build directory. sources/jq-manual.txt, sources/jq.test, sources/parser.y, and sources/lexer.l are large and must be readable from disk during implementation rather than carried in prompt text.

Suggested Implementation Order

The difficulty is concentrated in one place — the evaluation model — and not spread evenly across the corpus. Build the core correctly before reaching for coverage.

1. Lexer and parser. Follow sources/lexer.l and sources/parser.y directly. Produce

an AST. Precedence, ? suffixes, string interpolation, and the def forms are all settled there. Reject invalid programs with exit 3.

1. The generator core. Evaluate a filter as something that yields a stream of values:

., literals, |, ,, field access, iteration, arithmetic, comparison, and empty. Every later feature is expressed in terms of this. Get [.[] | f], cartesian products over multi-output arguments, and short-circuiting right.

1. Paths and assignment. path(f), getpath, setpath, delpaths, del, and then

=, |=, +=, and friends. Assignment is defined over path expressions, so this cannot precede step 2.

1. Control flow. if/then/elif/else/end, try/catch and ?, //,

reduce, foreach, label/break, limit, first, last, until, while, recurse. This is where backtracking is tested hardest.

1. Functions, variables, and destructuring. def with arity and closures, as

bindings, object and array patterns, and the ?// alternative operator.

1. Builtins. Work outward from sources/builtin.jq and the manual: strings, arrays,

objects, sort_by/group_by/unique_by, @base64/@uri/@csv/@tsv/@sh formats, the date functions, tostream, input/inputs, $__loc__, debug.

1. Numbers and edge cases. nan, infinite, integer/float equality, large literals,

and the have_decnum builtin — return false from it and the corpus takes its non-decNumber branch, which native floats satisfy.

The manual is normative and the corpus is precise. Follow both directly rather than inferring behaviour from jq's printed output.

Definition of Done

• sh sources/full_test.sh runs cleanly with zero errors and exits zero.
• The program satisfies the ./jq -c → stdin → one-JSON-value-per-line → exit-code

contract.

• Every corpus case that runs passes: the failed and errored counts are both zero, and the

skipped count matches the declared exclusions. The harness exit status is the verdict and the whole verdict — assert returncode == 0 and stop there. Do not assert on the text of the summary line at all: the case totals belong to the pinned corpus, and a check that reads a runner's printed output is measuring the runner rather than the interpreter.

• Do not create acceptance checks asserting that imported or staged files merely exist.
• The interpreter is written from the specification. **Every third-party jq

implementation or binding is forbidden** — jq.py, pyjq, jqlang, gojq, jaq, and any other — as is shelling out to a system jq binary. A wrapper around real jq scores perfectly and makes the exercise meaningless.

• The project declares no third-party runtime dependency. The standard library is

sufficient: json, decimal, math, re, datetime, time, base64, unicodedata, itertools, functools, dataclasses, argparse, sys.

• No network access at any point, including at test time.
• Deliver a concise project README.md documenting the stdin/stdout interface, the exit

codes, and the sh sources/full_test.sh command.