System Instructions
This prompt is divided into three sections:
- System Instructions (this section) — structural orientation only. Do not treat this
section as task input.
- Input Context — begins with the heading
# Input Context. All blocks are wrapped in
<pblock> tags. Two block types:
- File blocks:
<pblock filename="<name>" role="<role>" guidance="...">— optional
guidance attribute carries context-specific instructions; content is in a fenced block.
- Metadata/section blocks:
<pblock label="<label>" kind="<kind>">— job parameters,
rules, instructions, or group headers.
- Agent Task — begins with the heading
# Agent Task. Defines your persona, constraints,
and required outputs. Read all input context before acting on this section.
Input Context
<pblock label="Build block job" kind="job">
Build block job
- TARGET: jq
- BUILD_DIRECTORY: /mnt/c/Users/barlo/projects/drydock/uat/jq/runs/20260822.044627/build/jq
- WORKING_DIRECTORY: /mnt/c/Users/barlo/projects/drydock/uat/jq/runs/20260822.044627/build/jq
- BUILD_BLOCK: Block 8 · Service (block-8)
- STORIES: Implement jq filter expression grammar. (PARSE-003), Implement declarations and control syntax parsing. (PARSE-004)
- DATE: 2026-08-22
- BUILD_SCOPE: exactly one MANIFEST.md build block
</pblock>
<pblock label="Stories in this block" kind="section">
Stories in this block
- Implement jq filter expression grammar. (PARSE-003) [story]
- Implement declarations and control syntax parsing. (PARSE-004) [story]
</pblock>
<pblock label="Files on disk" kind="section">
Files on disk in the build directory
- sources/builtin.jq
- sources/exclusions.txt
- sources/full_test.sh
- sources/jq-manual.txt
- sources/jq.test
- sources/lexer.l
- sources/parser.y
- sources/run_conformance.py
These are the only imported files present on disk. Every other file named in this prompt is supplied as prompt context and is not on disk; read it here and do not report it as a missing input.
</pblock>
COMPASS - Target Orientation
<pblock filename="COMPASS.md" role="compass" path="/mnt/c/Users/barlo/projects/drydock/uat/jq/runs/20260822.044627/workspace/targets/jq/COMPASS.md" guidance="Important: This is the core project intent, constraints, and guardrails. It should have presecedence in conflicts.">
# COMPASS: jq
## Compass
Build a standalone interpreter for the jq language as described in `sources/jq-manual.txt`. The
product is an executable file named `jq` at the application root. It reads JSON from standard
input, evaluates a jq filter as an ordered generator, and writes each value the filter produces to
standard output as one compact JSON value per line.
Correctness is measured by the upstream jq conformance corpus `sources/jq.test`, taken verbatim
from jq 1.8.2, minus the cases named in `sources/exclusions.txt`. The goal is every case passing,
none failed and none errored.
## Constraints
- Implement in Python using only the standard library.
- Provide an executable named `jq` at the application root, invoked as `./jq -c '<program>'`.
- `-c` is the only option exercised. No other command-line option is required.
- Run without network access, package installation, or external runtime dependencies.
- Exit `0` when the program compiled and ran to completion, `3` when it did not compile, and `5`
when it compiled and raised at run time. The harness grades on this distinction.
- Diagnostics go to standard error and are never compared.
## Guardrails
- Do not shell out to a system `jq` executable.
- Do not use a third-party jq implementation or binding.
- Do not modify, rewrite, trim, regenerate, or substitute any file under `sources/`. Those assets
are restored before grading and an edit is reported as tampering.
- Preserve generator ordering, multiplicity, backtracking, and partial-output runtime behavior.
- Keep compile failures distinct from runtime failures using exit codes 3 and 5.
## Verification Protocol
This section is normative. It governs which story may invoke the supplied harness, and how.
### Invoking the harness
`sources/run_conformance.py` **requires** the environment variable `JQ`, the command that runs the
candidate implementation. Without it the harness exits `2` on its own usage code, which is a
harness fault and never a verdict about the interpreter. Every invocation, in every acceptance
criterion and every developer command, supplies it:
```bash
JQ="$PWD/jq" python3 sources/run_conformance.py # whole corpus, the scored run
JQ="$PWD/jq" python3 sources/run_conformance.py --select 'reduce' # run one construct for real
```
Those two commands are the only ways this build runs the harness. They are specified verbatim
below under *The two harness invocations*, together with the flag this build forbids.
`sources/` is read only. No story edits, patches, or regenerates `sources/run_conformance.py`,
`sources/jq.test`, or `sources/exclusions.txt`; a harness defect is reported, not repaired in
place. A story that needs to experiment with the harness works on a copy outside `sources/`, and
every acceptance criterion invokes the original `sources/run_conformance.py`.
An acceptance criterion written in Python supplies it by **extending** the inherited environment,
never by replacing it:
```python
env={**os.environ, "JQ": str(build_dir / "jq")}
```
`env={"JQ": ...}` alone leaves the child with no `PATH`, so nothing it invokes resolves and the
criterion is false at every level of implementation quality.
`sources/full_test.sh` sets `JQ` itself for the runner it wraps and therefore takes no environment
from its caller.
The harness reserves exit `2` for its own faults — a missing corpus, an unset `JQ`, a stale
exclusion list. Exit `2` never means the interpreter is wrong.
The summary line is:
```
jq conformance: NNN passed, N failed, N errored, N skipped (corpus jq.test @ jq-1.8.2)
```
### The two harness invocations
An acceptance criterion that runs `sources/run_conformance.py` uses one of these two commands. No
criterion in this build passes any other flag to the harness.
| Story kind | Command | Executes cases? | Asserts |
|---|---|---|---|
| Every behavioral story | `--select <regex> --json` | Yes, the selected slice | exit `0`, zero `fail`, zero `error`, non-zero case count |
| Terminal story (once, last) | `sh sources/full_test.sh` | Yes, all of them | exit `0` |
The staging story does not appear in this table. It does not run the harness at all; see *The
staging story* below.
#### `--list` is never run
`sources/run_conformance.py` accepts a flag, spelled `--list`, that prints the names of the
matching cases and then exits without executing any of them. `sources/INSTRUCTIONS.md`, the file
header, and `--help` all document it.
**This build never runs it. Not in an acceptance criterion, not in a story, not in a script, not
in a command typed by a build agent, not while developing and not while verifying. The string
`--list` does not appear anywhere in this project's output. If you have written it, that line is
wrong — delete it and use one of the two commands above.**
A Drydock build is headless. There is no one watching the output, so a mode whose entire purpose
is to print something for a person to read has no reader and no reason to run.
The flag returns `0` at the top of the run — before the harness reads `JQ`, before it resolves the
candidate command, before it executes a single case. A criterion built on it passes when `jq` is
an empty file, when `jq` does not exist, and when the story it gates was never written. It is not
a weak proof, not a partial proof, and not an acceptable proof for staging, for scaffolding, or
for an early story whose implementation is incomplete. It is not a proof. Thirty-six criteria in
one earlier plan of this project used it, every one of them reported green, and it cost three days.
If you are writing a criterion and reaching for that flag, the reason is always the same: the
story's code does not exist yet and you want a command that will not fail. That is the definition
of a criterion that proves nothing. Write the `--select ... --json` form instead and let it be red
until the story makes it green. **A criterion is supposed to fail before its story is built.**
The same prohibition covers any other flag whose effect is to not execute the cases — enumeration,
dry-run, validation, or help. If a flag's documented purpose is "run nothing", it has no place in
an acceptance criterion.
#### Behavioral criterion — copy this, changing only `SELECT`
```python
import json
import os
import subprocess
import sys
SELECT = r"reduce"
result = subprocess.run(
[sys.executable, "sources/run_conformance.py", "--select", SELECT, "--json"],
capture_output=True,
text=True,
env={**os.environ, "JQ": f"{os.getcwd()}/jq"},
)
print(result.stdout)
print(result.stderr, file=sys.stderr)
report = json.loads(result.stdout)
tally = report["summary"]
assert sum(tally.values()) > 0, f"selector matched no case: {SELECT}"
assert tally["fail"] == 0 and tally["error"] == 0, tally
assert result.returncode == 0, result.returncode
```
Three assertions, and all three are required.
1. **The selector matched something.** `--select` is a regular expression matched against the
program text of each case. A selector that matches nothing yields zero cases, zero failures,
and exit `0` — green, and worth nothing. Alternations naming ideas rather than syntax
(`closure`, `recursive`, `optional`) match no jq program and are the common way to write one
by accident. Select on syntax the corpus actually contains: `reduce`, `foreach`, `def `,
` as \$`, `try `, `//`, `path(`.
2. **No case failed or errored.** Read off the parsed JSON tally, not off any printed line.
3. **The exit status is `0`.** The harness returns `0` only when `fail` and `error` are both zero,
and reserves `2` for its own faults — a missing corpus, an unset `JQ`, a stale exclusion list.
Exit `2` is never a verdict about the interpreter.
`--json` writes the report and nothing else to stdout, so `json.loads(result.stdout)` is total. Do
not assert against the human summary line, and do not grep stdout for `passed` or `failed`.
### The terminal story
The **terminal story** is the last story in the build order: the one on which every other story is
a transitive dependency, and after which no further story runs. It is a verification story. Its
job is not to add capability but to prove that the capability every preceding story delivered is
present, together, at the end of the build.
The terminal story of this project runs `sh sources/full_test.sh`, asserts `returncode == 0`,
prints the captured stdout and stderr so a failure is diagnosable from the evidence alone, and
carries the Sea Trial. It is the only story permitted to run the whole corpus.
A story is not terminal because its name contains "verify", because it is a test harness, or
because it stages the test assets. Staging the corpus is foundational work that happens early;
running the corpus is terminal work that happens last. Do not place a whole-corpus gate on a
story that cannot yet run it — it fails vacuously and teaches nothing.
### Scope of every other story
Every non-terminal story is gated on its own declared behavior only, through `--select` against
the constructs that story implements, and the criterion asserts the selected slice passes. A
non-terminal story never invokes `sources/full_test.sh` and never runs the corpus unfiltered: a
partial interpreter fails most of an authoritative corpus by construction, and its unimplemented
cases exhaust the harness's per-case timeout rather than returning, so the unscoped run costs the
most exactly where it teaches the least.
Regression across stories is not the responsibility of any story's criteria. Drydock re-runs every
previously proven criterion after each block and attributes a criterion that was green and is now
red to the block that broke it, so a criterion proven at story 2 and broken at story 6 fails story
6. Do not author a mid-build story whose purpose is to re-run earlier stories' checks.
### The staging story
The story that stages the conformance assets is gated on the assets being present, complete, and
mutually consistent — not on a bare file-existence assertion, and not on the corpus running. It
proves that in process, by importing the harness and calling its parsers directly. It never
launches the harness, so the question of which flags to pass does not arise:
```python
import sys
sys.path.insert(0, "sources")
import run_conformance as harness
EXPECTED_CASES = 550
EXPECTED_EXCLUSIONS = 13
cases = harness.parse_corpus(harness.CORPUS.read_text(encoding="utf-8"))
excluded = harness.apply_exclusions(cases, harness.parse_exclusions(harness.EXCLUSIONS))
assert len(cases) == EXPECTED_CASES, len(cases)
assert len(excluded) == EXPECTED_EXCLUSIONS, len(excluded)
```
This reads state rather than output: the harness module imports, the corpus parses into the
expected number of cases, and every exclusion still matches a case — `apply_exclusions` raises on
a stale entry, so a corpus and an exclusion list that have drifted apart fail here rather than
silently skipping cases later.
It claims nothing about the interpreter, because at this point in the build there is nothing to
claim. Every story that claims a construct works runs that construct through
`--select ... --json`.
## The corpus
`sources/jq.test` documents its own format in its header. Cases are separated by blank lines;
blank lines and `#` lines are ignored. A case is a program line, an input line, and then the
expected output values, one per line. A case preceded by `%%FAIL` is a program that must be
rejected at compile time; the following lines are upstream jq's diagnostic, which the harness
records but never compares. Reproducing jq's exact error text is reverse-engineering a C
implementation rather than conforming to a specification, so a `%%FAIL` case passes on exit `3`
alone.
Values are compared structurally, not textually. `1` and `1.0` are the same jq value, and so are
two objects whose keys print in a different order. Output formatting is therefore not under test,
but the number and order of values is.
`sources/exclusions.txt` names the corpus cases this kit cannot run, with the reason. They are the
module-loader cases, whose `import` and `include` resolve against a search path of fixture files a
flat source import cannot carry. They are reported as `skipped` and are not part of the score.
The module *grammar* cases are not excluded and must pass. `module (.+1); 0`, `module []; 0`,
`include "a" (.+1); 0`, `include "a" []; 0`, `include "\ "; 0`, `include "\(a)"; 0`, and `%::wat`
are all `%%FAIL` cases: the front end parses the module syntax far enough to reject them, without
ever touching the filesystem.
<!-- drydock:build-write-guardrail:start -->
## Build Write Guardrail
- Authorized build directory: `/mnt/c/Users/barlo/projects/drydock/uat/jq/runs/20260822.044627/build/jq`
- Authorized Target directory: `/mnt/c/Users/barlo/projects/drydock/uat/jq/runs/20260822.044627/workspace/targets/jq`
- Build agents have permission to create, modify, and remove files required by the active build block inside these authorized directories.
- No path outside these authorized directories may be modified.
- Protected Drydock artifacts:
- `/mnt/c/Users/barlo/projects/drydock/uat/jq/runs/20260822.044627/workspace/targets/jq/blueprint/`
- `/mnt/c/Users/barlo/projects/drydock/uat/jq/runs/20260822.044627/workspace/targets/jq/MANIFEST.md`
- `/mnt/c/Users/barlo/projects/drydock/uat/jq/runs/20260822.044627/workspace/targets/jq/COMPASS.md`
- `/mnt/c/Users/barlo/projects/drydock/uat/jq/runs/20260822.044627/workspace/targets/jq/QuarterDeck/`
- `/mnt/c/Users/barlo/projects/drydock/uat/jq/runs/20260822.044627/workspace/targets/jq/evidence/`
<!-- drydock:build-write-guardrail:end --></pblock>
STACK - Technology HOW
<pblock filename="python_compact.md" role="stack" path="/mnt/c/Users/barlo/projects/drydock/Rigging/stack/python_compact.md">
<!-- Compacted from rigging/stack/python.md on 2026-07-16 (manual update to match python.md V3) -->
# Python — Compact
## Configuration Management
One typed frozen-dataclass `Config` is the only env reader — never read `os.environ` elsewhere. No `Dev`/`Prod`/`Test` subclasses; the environment (`.env`) selects configuration. Never hardcode secrets, ports, or paths. Secret hygiene and `.env.example`: see `stack/env_variables_and_secrets.md`.
```python
# config.py
import os
from dataclasses import dataclass
from dotenv import load_dotenv
load_dotenv()
@dataclass(frozen=True)
class Config:
secret_key: str
database_path: str
port: int
debug: bool = False
@classmethod
def load(cls) -> "Config":
try:
return cls(
secret_key=os.environ["SECRET_KEY"],
database_path=os.environ.get("DATABASE_PATH", "data/app.db"),
port=int(os.environ.get("APP_PORT", "5001")),
debug=os.environ.get("APP_DEBUG") == "1",
)
except KeyError as e:
raise RuntimeError(f"Missing required env var: {e}") from e
```
## Code Style and Understandability
Code must be understandable through naming, structure, small focused units, explicit types, clear interfaces, appropriate abstractions, and tests. Names state intent; one responsibility per module; functions do one thing at one level of abstraction; no speculative abstraction layers. Comments state constraints the code cannot express — never restate mechanics.
## Type Hints and Static Typing
Modern hints on all public interfaces; typed structures across boundaries; run a type checker when practical.
- Built-in generics (`list[str]`, `dict[str, int]`) and `X | None` — never `typing.List`, `Optional`, `Union`
- Type every public function, method, and class attribute
- Schemas/serializers/services/data structures are typed classes: frozen dataclasses internally, Pydantic/`TypedDict` at serialization boundaries
- No bare `dict`, positional tuples, or `Any` crossing a module boundary
- `uv add --dev mypy` then `uv run mypy .` (or pyright) alongside ruff and pytest in CI
## Logging
Use `logging` with named loggers, never `print()`. Configure formatter + console and file handlers at startup (`data/logs/app.log`); level from `APP_DEBUG`.
```python
import logging
logger = logging.getLogger(__name__)
logger.info('Server starting on port %s', port)
```
## Environment Separation
Distinct `.env` per environment; same typed `Config` reads whichever is present. Never run debug in production.
| Setting | Dev | Test | Prod |
|---------|-----|------|------|
| APP_DEBUG | 1 | 0 | 0 |
| DATABASE_PATH | data/app.db | :memory: | data/app.db |
| SECRET_KEY | .env value | .env value | .env value (required) |
| LOGGING | DEBUG | WARNING | INFO |
## Testing
`pytest` with fixtures; fresh in-memory DB per test; test at the boundary. Every Python build must include a complete pytest suite regardless of the specification — no tests, no ACTIVE conformity.
```python
# tests/conftest.py
import pytest
@pytest.fixture
def app(monkeypatch):
monkeypatch.setenv("SECRET_KEY", "test")
monkeypatch.setenv("DATABASE_PATH", ":memory:")
from app import create_app
from config import Config
yield create_app(Config.load())
@pytest.fixture
def client(app):
return app.test_client()
```
**test_smoke.py** — app factory works; `GET /health` returns 200 `{"status": "ok"}`; `GET /` returns 200.
**test_routes.py** — one test per route: `GET` pages assert 200; `POST` APIs assert status in `{200, 201, 204}`; HTMX routes send `HX-Request: true`; `{id}` routes use fixture-created records.
**test_db.py** (only if DATABASE.md exists) — expected tables exist; round-trip per major table; invalid FK raises `IntegrityError` (`PRAGMA foreign_keys=ON`).
Do not test third-party internals, config loading, or private helpers.
```ini
# pytest.ini
[pytest]
testpaths = tests
addopts = -v
```
## Security
Validate all user input; parameterized queries exclusively; never trust client data.
- `?` placeholders, never f-strings, for all DB operations
- `secure_filename()` for user-supplied paths
- Length and type validation on inputs
- Secret key from environment, never hardcoded
- Never expose stack traces to end users
## Dependency Management (uv)
`uv` only; `pyproject.toml` is the manifest; `uv.lock` committed; `.venv/` gitignored. Full toolchain conventions: `stack/uv_ruff.md`.
```bash
uv venv # creates .venv/
uv add flask python-dotenv # runtime deps → pyproject.toml + uv.lock
uv add --dev pytest ruff mypy # dev deps
uv sync --frozen # CI — fail if lock is stale
```
- Never bare `pip install` or `python -m venv`
- Dev deps in `[project.optional-dependencies].dev`; runtime deps minimal
## Startup Validation
`Config.load()` already validates required env vars; startup validation confirms DB connectivity. Crash early on misconfiguration.
```python
def validate_startup(config: Config, db: Database):
try:
db.healthcheck() # SELECT 1 inside the Database class
except Exception as e:
raise RuntimeError(f'Database not accessible: {e}')
logger.info('Startup validation passed')
```
## Directory Layout
```
project-name/
├── app.py # Entry point / app factory
├── routes.py # Route handlers
├── models.py # Data models and type registries
├── db.py # Database class: typed tables, connection, schema, migrations
├── ops.py # Business logic and operations
├── config.py # typed Config class — the only env reader
├── templates/ # base.html + types/ partials
├── static/ # css/, js/
├── tests/ # conftest.py, test_*.py
├── bin/ # (from common.md)
├── data/ # (from common.md)
├── pyproject.toml
├── uv.lock
├── .env # gitignored; .env.example committed
└── .gitignore
```</pblock>
CONTEXT - Read-Only Support
<pblock filename="parser.y" role="context" path="/mnt/c/Users/barlo/projects/drydock/uat/jq/runs/20260822.044627/workspace/targets/jq/blueprint/parser.y">
%{
#include <assert.h>
#include <math.h>
#include <stdio.h>
#include <string.h>
#include "compile.h"
#include "jv_alloc.h"
#include "builtin.h"
#define YYMALLOC jv_mem_alloc
#define YYFREE jv_mem_free
%}
%code requires {
#include "locfile.h"
struct lexer_param;
#define YYLTYPE location
#define YYLLOC_DEFAULT(Loc, Rhs, N) \
do { \
if (N) { \
(Loc).start = YYRHSLOC(Rhs, 1).start; \
(Loc).end = YYRHSLOC(Rhs, N).end; \
} else { \
(Loc).start = YYRHSLOC(Rhs, 0).end; \
(Loc).end = YYRHSLOC(Rhs, 0).end; \
} \
} while (0)
}
%locations
%define parse.error verbose
%define api.pure
%union {
jv literal;
block blk;
}
%destructor { jv_free($$); } <literal>
%destructor { block_free($$); } <blk>
%parse-param {block* answer}
%parse-param {int* errors}
%parse-param {struct locfile* locations}
%parse-param {struct lexer_param* lexer_param_ptr}
%lex-param {block* answer}
%lex-param {int* errors}
%lex-param {struct locfile* locations}
%lex-param {struct lexer_param* lexer_param_ptr}
%token INVALID_CHARACTER
%token <literal> IDENT
%token <literal> FIELD
%token <literal> BINDING
%token <literal> LITERAL
%token <literal> FORMAT
%token REC ".."
%token SETMOD "%="
%token EQ "=="
%token NEQ "!="
%token DEFINEDOR "//"
%token AS "as"
%token DEF "def"
%token MODULE "module"
%token IMPORT "import"
%token INCLUDE "include"
%token IF "if"
%token THEN "then"
%token ELSE "else"
%token ELSE_IF "elif"
%token REDUCE "reduce"
%token FOREACH "foreach"
%token END "end"
%token AND "and"
%token OR "or"
%token TRY "try"
%token CATCH "catch"
%token LABEL "label"
%token BREAK "break"
%token LOC "$__loc__"
%token SETPIPE "|="
%token SETPLUS "+="
%token SETMINUS "-="
%token SETMULT "*="
%token SETDIV "/="
%token SETDEFINEDOR "//="
%token LESSEQ "<="
%token GREATEREQ ">="
%token ALTERNATION "?//"
%token QQSTRING_START
%token <literal> QQSTRING_TEXT
%token QQSTRING_INTERP_START
%token QQSTRING_INTERP_END
%token QQSTRING_END
/* Instead of raising this, find a way to use precedence to resolve
* shift-reduce conflicts. */
%expect 0
%precedence FUNCDEF
%right '|'
%left ','
%right "//"
%nonassoc '=' SETPIPE SETPLUS SETMINUS SETMULT SETDIV SETMOD SETDEFINEDOR
%left OR
%left AND
%nonassoc NEQ EQ '<' '>' LESSEQ GREATEREQ
%left '+' '-'
%left '*' '/' '%'
%precedence NONOPT /* non-optional; rules for which a specialized
'?' rule should be preferred over Expr '?' */
%precedence '?' '.' '[' FIELD
%precedence "try"
%precedence "catch"
%type <blk> Query Expr Term
%type <blk> DictPairs DictPair DictExpr
%type <blk> ElseBody
%type <blk> String QQString
%type <blk> FuncDef FuncDefs
%type <blk> Module Import Imports ImportWhat ImportFrom
%type <blk> Param Params Arg Args
%type <blk> Patterns RepPatterns Pattern ArrayPats ObjPats ObjPat
%type <literal> Keyword
%type <literal> StringStart
%{
#include "lexer.h"
struct lexer_param {
yyscan_t lexer;
};
#define FAIL(loc, msg) \
do { \
location l = loc; \
yyerror(&l, answer, errors, locations, lexer_param_ptr, msg); \
/*YYERROR*/; \
} while (0)
void yyerror(YYLTYPE* loc, block* answer, int* errors,
struct locfile* locations, struct lexer_param* lexer_param_ptr, const char *s){
(*errors)++;
locfile_locate(locations, *loc, "jq: error: %s", s);
}
int yylex(YYSTYPE* yylval, YYLTYPE* yylloc, block* answer, int* errors,
struct locfile* locations, struct lexer_param* lexer_param_ptr) {
yyscan_t lexer = lexer_param_ptr->lexer;
int tok = jq_yylex(yylval, yylloc, lexer);
if ((tok == LITERAL || tok == QQSTRING_TEXT) && !jv_is_valid(yylval->literal)) {
jv msg = jv_invalid_get_msg(jv_copy(yylval->literal));
if (jv_get_kind(msg) == JV_KIND_STRING) {
FAIL(*yylloc, jv_string_value(msg));
} else {
FAIL(*yylloc, "Invalid literal");
}
jv_free(msg);
jv_free(yylval->literal);
yylval->literal = jv_null();
}
return tok;
}
/* Returns string message if the block is a constant that is not valid as an
* object key. */
static jv check_object_key(block k) {
if (block_is_const(k) && block_const_kind(k) != JV_KIND_STRING) {
char errbuf[30];
return jv_string_fmt("Cannot use %s (%s) as object key",
jv_kind_name(block_const_kind(k)),
jv_dump_string_trunc(block_const(k), errbuf, sizeof(errbuf)));
}
return jv_invalid();
}
static block gen_index(block obj, block key) {
return BLOCK(gen_subexp(key), obj, gen_op_simple(INDEX));
}
static block gen_index_opt(block obj, block key) {
return BLOCK(gen_subexp(key), obj, gen_op_simple(INDEX_OPT));
}
static block gen_slice_index(block obj, block start, block end, opcode idx_op) {
block key = BLOCK(gen_subexp(gen_const(jv_object())),
gen_subexp(gen_const(jv_string("start"))),
gen_subexp(start),
gen_op_simple(INSERT),
gen_subexp(gen_const(jv_string("end"))),
gen_subexp(end),
gen_op_simple(INSERT));
return BLOCK(key, obj, gen_op_simple(idx_op));
}
static block constant_fold(block a, block b, int op) {
if (!block_is_single(a) || !block_is_const(a) ||
!block_is_single(b) || !block_is_const(b))
return gen_noop();
jv jv_a = block_const(a);
block_free(a);
jv jv_b = block_const(b);
block_free(b);
jv res = jv_invalid();
switch (op) {
case '+': res = binop_plus(jv_a, jv_b); break;
case '-': res = binop_minus(jv_a, jv_b); break;
case '*': res = binop_multiply(jv_a, jv_b); break;
case '/': res = binop_divide(jv_a, jv_b); break;
case '%': res = binop_mod(jv_a, jv_b); break;
case EQ: res = binop_equal(jv_a, jv_b); break;
case NEQ: res = binop_notequal(jv_a, jv_b); break;
case '<': res = binop_less(jv_a, jv_b); break;
case '>': res = binop_greater(jv_a, jv_b); break;
case LESSEQ: res = binop_lesseq(jv_a, jv_b); break;
case GREATEREQ: res = binop_greatereq(jv_a, jv_b); break;
}
if (jv_is_valid(res))
return gen_const(res);
return gen_error(jv_invalid_get_msg(res));
}
static block gen_binop(block a, block b, int op) {
block folded = constant_fold(a, b, op);
if (!block_is_noop(folded))
return folded;
const char* funcname = 0;
switch (op) {
case '+': funcname = "_plus"; break;
case '-': funcname = "_minus"; break;
case '*': funcname = "_multiply"; break;
case '/': funcname = "_divide"; break;
case '%': funcname = "_mod"; break;
case EQ: funcname = "_equal"; break;
case NEQ: funcname = "_notequal"; break;
case '<': funcname = "_less"; break;
case '>': funcname = "_greater"; break;
case LESSEQ: funcname = "_lesseq"; break;
case GREATEREQ: funcname = "_greatereq"; break;
}
assert(funcname);
return gen_call(funcname, BLOCK(gen_lambda(a), gen_lambda(b)));
}
static block gen_format(block a, jv fmt) {
return BLOCK(a, gen_call("format", gen_lambda(gen_const(fmt))));
}
static block gen_definedor_assign(block object, block val) {
block tmp = gen_op_var_fresh(STOREV, "tmp");
return BLOCK(gen_op_simple(DUP),
val, tmp,
gen_call("_modify", BLOCK(gen_lambda(object),
gen_lambda(gen_definedor(gen_noop(),
gen_op_bound(LOADV, tmp))))));
}
static block gen_update(block object, block val, int optype) {
block tmp = gen_op_var_fresh(STOREV, "tmp");
return BLOCK(gen_op_simple(DUP),
val,
tmp,
gen_call("_modify", BLOCK(gen_lambda(object),
gen_lambda(gen_binop(gen_noop(),
gen_op_bound(LOADV, tmp),
optype)))));
}
static block gen_loc_object(location *loc, struct locfile *locations) {
return gen_const(JV_OBJECT(jv_string("file"), jv_copy(locations->fname),
jv_string("line"), jv_number(locfile_get_line(locations, loc->start) + 1)));
}
%}
%%
TopLevel:
Module Imports Query {
*answer = BLOCK($1, $2, gen_op_simple(TOP), $3);
} |
Module Imports FuncDefs {
*answer = BLOCK($1, $2, $3);
}
Module:
%empty {
$$ = gen_noop();
} |
"module" Query ';' {
if (!block_is_const($2)) {
FAIL(@2, "Module metadata must be constant");
$$ = gen_noop();
block_free($2);
} else if (block_const_kind($2) != JV_KIND_OBJECT) {
FAIL(@2, "Module metadata must be an object");
$$ = gen_noop();
block_free($2);
} else {
$$ = gen_module($2);
}
}
Imports:
%empty {
$$ = gen_noop();
} |
Import Imports {
$$ = BLOCK($1, $2);
}
FuncDefs:
%empty {
$$ = gen_noop();
} |
FuncDef FuncDefs {
$$ = block_join($1, $2);
}
Query:
FuncDef Query %prec FUNCDEF {
$$ = block_bind_referenced($1, $2, OP_IS_CALL_PSEUDO);
} |
Expr "as" Patterns '|' Query {
$$ = gen_destructure($1, $3, $5);
} |
"label" BINDING '|' Query {
jv v = jv_string_fmt("*label-%s", jv_string_value($2));
$$ = gen_location(@$, locations, gen_label(jv_string_value(v), $4));
jv_free($2);
jv_free(v);
} |
Query '|' Query {
$$ = block_join($1, $3);
} |
Query ',' Query {
$$ = gen_both($1, $3);
} |
Expr {
$$ = $1;
}
Expr:
Expr "//" Expr {
$$ = gen_definedor($1, $3);
} |
Expr '=' Expr {
$$ = gen_call("_assign", BLOCK(gen_lambda($1), gen_lambda($3)));
} |
Expr "or" Expr {
$$ = gen_or($1, $3);
} |
Expr "and" Expr {
$$ = gen_and($1, $3);
} |
Expr "//=" Expr {
$$ = gen_definedor_assign($1, $3);
} |
Expr "|=" Expr {
$$ = gen_call("_modify", BLOCK(gen_lambda($1), gen_lambda($3)));
} |
Expr '+' Expr {
$$ = gen_binop($1, $3, '+');
} |
Expr "+=" Expr {
$$ = gen_update($1, $3, '+');
} |
Expr '-' Expr {
$$ = gen_binop($1, $3, '-');
} |
Expr "-=" Expr {
$$ = gen_update($1, $3, '-');
} |
Expr '*' Expr {
$$ = gen_binop($1, $3, '*');
} |
Expr "*=" Expr {
$$ = gen_update($1, $3, '*');
} |
Expr '/' Expr {
$$ = gen_binop($1, $3, '/');
} |
Expr '%' Expr {
$$ = gen_binop($1, $3, '%');
} |
Expr "/=" Expr {
$$ = gen_update($1, $3, '/');
} |
Expr SETMOD Expr {
$$ = gen_update($1, $3, '%');
} |
Expr "==" Expr {
$$ = gen_binop($1, $3, EQ);
} |
Expr "!=" Expr {
$$ = gen_binop($1, $3, NEQ);
} |
Expr '<' Expr {
$$ = gen_binop($1, $3, '<');
} |
Expr '>' Expr {
$$ = gen_binop($1, $3, '>');
} |
Expr "<=" Expr {
$$ = gen_binop($1, $3, LESSEQ);
} |
Expr ">=" Expr {
$$ = gen_binop($1, $3, GREATEREQ);
} |
Term %prec NONOPT {
$$ = $1;
}
Import:
ImportWhat ';' {
$$ = $1;
} |
ImportWhat Query ';' {
if (!block_is_const($2)) {
FAIL(@2, "Module metadata must be constant");
$$ = gen_noop();
block_free($1);
block_free($2);
} else if (block_const_kind($2) != JV_KIND_OBJECT) {
FAIL(@2, "Module metadata must be an object");
$$ = gen_noop();
block_free($1);
block_free($2);
} else {
$$ = gen_import_meta($1, $2);
}
}
ImportWhat:
"import" ImportFrom "as" BINDING {
$$ = gen_import(block_const($2), $4, 1);
block_free($2);
} |
"import" ImportFrom "as" IDENT {
$$ = gen_import(block_const($2), $4, 0);
block_free($2);
} |
"include" ImportFrom {
$$ = gen_import(block_const($2), jv_invalid(), 0);
block_free($2);
}
ImportFrom:
String {
if (!block_is_const($1)) {
FAIL(@1, "Import path must be constant");
$$ = gen_const(jv_string(""));
block_free($1);
} else {
$$ = $1;
}
}
FuncDef:
"def" IDENT ':' Query ';' {
$$ = gen_function(jv_string_value($2), gen_noop(), $4);
jv_free($2);
} |
"def" IDENT '(' Params ')' ':' Query ';' {
$$ = gen_function(jv_string_value($2), $4, $7);
jv_free($2);
}
Params:
Param {
$$ = $1;
} |
Params ';' Param {
$$ = BLOCK($1, $3);
}
Param:
BINDING {
$$ = gen_param_regular(jv_string_value($1));
jv_free($1);
} |
IDENT {
$$ = gen_param(jv_string_value($1));
jv_free($1);
}
StringStart:
FORMAT QQSTRING_START {
$$ = $1;
} |
QQSTRING_START {
$$ = jv_string("text");
}
String:
StringStart QQString QQSTRING_END {
$$ = $2;
jv_free($1);
};
QQString:
%empty {
$$ = gen_const(jv_string(""));
} |
QQString QQSTRING_TEXT {
$$ = gen_binop($1, gen_const($2), '+');
} |
QQString QQSTRING_INTERP_START Query QQSTRING_INTERP_END {
$$ = gen_binop($1, gen_format($3, jv_copy($<literal>0)), '+');
}
ElseBody:
"elif" Query "then" Query ElseBody {
$$ = gen_cond($2, $4, $5);
} |
"else" Query "end" {
$$ = $2;
} |
"end" {
$$ = gen_noop();
}
Term:
'.' {
$$ = gen_noop();
} |
REC {
$$ = gen_call("recurse", gen_noop());
} |
BREAK BINDING {
jv v = jv_string_fmt("*label-%s", jv_string_value($2)); // impossible symbol
$$ = gen_location(@$, locations,
BLOCK(gen_op_unbound(LOADV, jv_string_value(v)),
gen_call("error", gen_noop())));
jv_free(v);
jv_free($2);
} |
BREAK error {
FAIL(@$, "break requires a label to break to");
$$ = gen_noop();
} |
Term FIELD '?' {
$$ = gen_index_opt($1, gen_const($2));
} |
FIELD '?' {
$$ = gen_index_opt(gen_noop(), gen_const($1));
} |
Term '.' String '?' {
$$ = gen_index_opt($1, $3);
} |
'.' String '?' {
$$ = gen_index_opt(gen_noop(), $2);
} |
Term FIELD %prec NONOPT {
$$ = gen_index($1, gen_const($2));
} |
FIELD %prec NONOPT {
$$ = gen_index(gen_noop(), gen_const($1));
} |
Term '.' String %prec NONOPT {
$$ = gen_index($1, $3);
} |
'.' String %prec NONOPT {
$$ = gen_index(gen_noop(), $2);
} |
'.' error {
FAIL(@$, "try .[\"field\"] instead of .field for unusually named fields");
$$ = gen_noop();
} |
'.' IDENT error {
jv_free($2);
FAIL(@$, "try .[\"field\"] instead of .field for unusually named fields");
$$ = gen_noop();
} |
/* FIXME: string literals */
Term '[' Query ']' '?' {
$$ = gen_index_opt($1, $3);
} |
Term '[' Query ']' %prec NONOPT {
$$ = gen_index($1, $3);
} |
Term '.' '[' Query ']' '?' {
$$ = gen_index_opt($1, $4);
} |
Term '.' '[' Query ']' %prec NONOPT {
$$ = gen_index($1, $4);
} |
Term '[' ']' '?' {
$$ = block_join($1, gen_op_simple(EACH_OPT));
} |
Term '[' ']' %prec NONOPT {
$$ = block_join($1, gen_op_simple(EACH));
} |
Term '.' '[' ']' '?' {
$$ = block_join($1, gen_op_simple(EACH_OPT));
} |
Term '.' '[' ']' %prec NONOPT {
$$ = block_join($1, gen_op_simple(EACH));
} |
Term '[' Query ':' Query ']' '?' {
$$ = gen_slice_index($1, $3, $5, INDEX_OPT);
} |
Term '[' Query ':' ']' '?' {
$$ = gen_slice_index($1, $3, gen_const(jv_null()), INDEX_OPT);
} |
Term '[' ':' Query ']' '?' {
$$ = gen_slice_index($1, gen_const(jv_null()), $4, INDEX_OPT);
} |
Term '[' Query ':' Query ']' %prec NONOPT {
$$ = gen_slice_index($1, $3, $5, INDEX);
} |
Term '[' Query ':' ']' %prec NONOPT {
$$ = gen_slice_index($1, $3, gen_const(jv_null()), INDEX);
} |
Term '[' ':' Query ']' %prec NONOPT {
$$ = gen_slice_index($1, gen_const(jv_null()), $4, INDEX);
} |
Term '?' {
$$ = gen_try($1, gen_op_simple(BACKTRACK));
} |
LITERAL {
$$ = gen_const($1);
} |
String {
$$ = $1;
} |
FORMAT {
$$ = gen_format(gen_noop(), $1);
} |
'-' Term {
$$ = BLOCK($2, gen_call("_negate", gen_noop()));
} |
'(' Query ')' {
$$ = $2;
} |
'[' Query ']' {
$$ = gen_collect($2);
} |
'[' ']' {
$$ = gen_const(jv_array());
} |
'{' DictPairs '}' {
block o = gen_const_object($2);
if (o.first != NULL)
$$ = o;
else
$$ = BLOCK(gen_subexp(gen_const(jv_object())), $2, gen_op_simple(POP));
} |
"reduce" Expr "as" Patterns '(' Query ';' Query ')' {
$$ = gen_reduce($2, $4, $6, $8);
} |
"foreach" Expr "as" Patterns '(' Query ';' Query ';' Query ')' {
$$ = gen_foreach($2, $4, $6, $8, $10);
} |
"foreach" Expr "as" Patterns '(' Query ';' Query ')' {
$$ = gen_foreach($2, $4, $6, $8, gen_noop());
} |
"if" Query "then" Query ElseBody {
$$ = gen_cond($2, $4, $5);
} |
"if" Query "then" error {
FAIL(@$, "Possibly unterminated 'if' statement");
$$ = $2;
} |
"try" Expr "catch" Expr {
$$ = gen_try($2, $4);
} |
"try" Expr "catch" error {
FAIL(@$, "Possibly unterminated 'try' statement");
$$ = $2;
} |
"try" Expr {
$$ = gen_try($2, gen_op_simple(BACKTRACK));
} |
/*
* This `$$$$varname` hack is strictly private to jq builtins. DO NOT USE!!
*
* This is used in `_modify`, in src/builtin.jq, to avoid holding on to a
* reference to `.`.
*
* We could just have the compiler emit bytecode for `_modify` so it can use
* LOADVN w/o needing jq syntax for LOADVN.
*
* This syntax, `$$$$varname`, violates referential transparency: it has
* side-effects that are surprising.
*
* DO NOT USE!! I will break your jq code if you do use this outside
* src/builtin.jq.
*/
'$' '$' '$' BINDING {
$$ = gen_location(@$, locations, gen_op_unbound(LOADVN, jv_string_value($4)));
jv_free($4);
} |
BINDING {
$$ = gen_location(@$, locations, gen_op_unbound(LOADV, jv_string_value($1)));
jv_free($1);
} |
"$__loc__" {
$$ = gen_loc_object(&@$, locations);
} |
IDENT {
const char *s = jv_string_value($1);
if (strcmp(s, "false") == 0)
$$ = gen_const(jv_false());
else if (strcmp(s, "true") == 0)
$$ = gen_const(jv_true());
else if (strcmp(s, "null") == 0)
$$ = gen_const(jv_null());
else
$$ = gen_location(@$, locations, gen_call(s, gen_noop()));
jv_free($1);
} |
IDENT '(' Args ')' {
$$ = gen_call(jv_string_value($1), $3);
$$ = gen_location(@1, locations, $$);
jv_free($1);
} |
'(' error ')' { $$ = gen_noop(); } |
'[' error ']' { $$ = gen_noop(); } |
Term '[' error ']' { $$ = $1; } |
'{' error '}' { $$ = gen_noop(); }
Args:
Arg {
$$ = $1;
} |
Args ';' Arg {
$$ = BLOCK($1, $3);
}
Arg:
Query {
$$ = gen_lambda($1);
}
RepPatterns:
RepPatterns "?//" Pattern {
$$ = BLOCK($1, gen_destructure_alt($3));
} |
Pattern {
$$ = gen_destructure_alt($1);
}
Patterns:
RepPatterns "?//" Pattern {
$$ = BLOCK($1, $3);
} |
Pattern {
$$ = $1;
}
Pattern:
BINDING {
$$ = gen_op_unbound(STOREV, jv_string_value($1));
jv_free($1);
} |
'[' ArrayPats ']' {
$$ = BLOCK($2, gen_op_simple(POP));
} |
'{' ObjPats '}' {
$$ = BLOCK($2, gen_op_simple(POP));
}
ArrayPats:
Pattern {
$$ = gen_array_matcher(gen_noop(), $1);
} |
ArrayPats ',' Pattern {
$$ = gen_array_matcher($1, $3);
}
ObjPats:
ObjPat {
$$ = $1;
} |
ObjPats ',' ObjPat {
$$ = BLOCK($1, $3);
}
ObjPat:
BINDING {
$$ = gen_object_matcher(gen_const($1), gen_op_unbound(STOREV, jv_string_value($1)));
} |
BINDING ':' Pattern {
$$ = gen_object_matcher(gen_const($1), BLOCK(gen_op_simple(DUP), gen_op_unbound(STOREV, jv_string_value($1)), $3));
} |
IDENT ':' Pattern {
$$ = gen_object_matcher(gen_const($1), $3);
} |
Keyword ':' Pattern {
$$ = gen_object_matcher(gen_const($1), $3);
} |
String ':' Pattern {
$$ = gen_object_matcher($1, $3);
} |
'(' Query ')' ':' Pattern {
jv msg = check_object_key($2);
if (jv_is_valid(msg)) {
FAIL(@2, jv_string_value(msg));
}
jv_free(msg);
$$ = gen_object_matcher($2, $5);
} |
error ':' Pattern {
FAIL(@$, "May need parentheses around object key expression");
$$ = $3;
}
Keyword:
"as" {
$$ = jv_string("as");
} |
"def" {
$$ = jv_string("def");
} |
"module" {
$$ = jv_string("module");
} |
"import" {
$$ = jv_string("import");
} |
"include" {
$$ = jv_string("include");
} |
"if" {
$$ = jv_string("if");
} |
"then" {
$$ = jv_string("then");
} |
"else" {
$$ = jv_string("else");
} |
"elif" {
$$ = jv_string("elif");
} |
"reduce" {
$$ = jv_string("reduce");
} |
"foreach" {
$$ = jv_string("foreach");
} |
"end" {
$$ = jv_string("end");
} |
"and" {
$$ = jv_string("and");
} |
"or" {
$$ = jv_string("or");
} |
"try" {
$$ = jv_string("try");
} |
"catch" {
$$ = jv_string("catch");
} |
"label" {
$$ = jv_string("label");
} |
"break" {
$$ = jv_string("break");
}
DictPairs:
%empty {
$$ = gen_noop();
} |
DictPair {
$$ = $1;
} |
DictPair ',' DictPairs {
$$ = block_join($1, $3);
}
DictPair:
IDENT ':' DictExpr {
$$ = gen_dictpair(gen_const($1), $3);
} |
Keyword ':' DictExpr {
$$ = gen_dictpair(gen_const($1), $3);
} |
String ':' DictExpr {
$$ = gen_dictpair($1, $3);
} |
String {
$$ = gen_dictpair($1, BLOCK(gen_op_simple(POP), gen_op_simple(DUP2),
gen_op_simple(DUP2), gen_op_simple(INDEX)));
} |
BINDING ':' DictExpr {
$$ = gen_dictpair(gen_location(@$, locations, gen_op_unbound(LOADV, jv_string_value($1))),
$3);
jv_free($1);
} |
BINDING {
$$ = gen_dictpair(gen_const($1),
gen_location(@$, locations, gen_op_unbound(LOADV, jv_string_value($1))));
} |
IDENT {
$$ = gen_dictpair(gen_const(jv_copy($1)),
gen_index(gen_noop(), gen_const($1)));
} |
"$__loc__" {
$$ = gen_dictpair(gen_const(jv_string("__loc__")),
gen_loc_object(&@$, locations));
} |
Keyword {
$$ = gen_dictpair(gen_const(jv_copy($1)),
gen_index(gen_noop(), gen_const($1)));
} |
'(' Query ')' ':' DictExpr {
jv msg = check_object_key($2);
if (jv_is_valid(msg)) {
FAIL(@2, jv_string_value(msg));
}
jv_free(msg);
$$ = gen_dictpair($2, $5);
} |
error ':' DictExpr {
FAIL(@1, "May need parentheses around object key expression");
$$ = $3;
}
DictExpr:
DictExpr '|' DictExpr {
$$ = block_join($1, $3);
} |
Expr {
$$ = $1;
}
%%
int jq_parse(struct locfile* locations, block* answer) {
struct lexer_param scanner;
YY_BUFFER_STATE buf;
jq_yylex_init_extra(0, &scanner.lexer);
buf = jq_yy_scan_bytes(locations->data, locations->length, scanner.lexer);
int errors = 0;
*answer = gen_noop();
yyparse(answer, &errors, locations, &scanner);
jq_yy_delete_buffer(buf, scanner.lexer);
jq_yylex_destroy(scanner.lexer);
if (errors > 0) {
block_free(*answer);
*answer = gen_noop();
}
return errors;
}
int jq_parse_library(struct locfile* locations, block* answer) {
int errs = jq_parse(locations, answer);
if (errs) return errs;
if (block_has_main(*answer)) {
locfile_locate(locations, UNKNOWN_LOCATION, "jq: error: library should only have function definitions, not a main expression");
return 1;
}
assert(block_has_only_binders_and_imports(*answer, OP_IS_CALL_PSEUDO));
return 0;
}</pblock>
<pblock filename="lexer.l" role="context" path="/mnt/c/Users/barlo/projects/drydock/uat/jq/runs/20260822.044627/workspace/targets/jq/blueprint/lexer.l">
%{
#include <assert.h>
#include "jv_alloc.h"
#include "compile.h"
struct lexer_param;
#include "parser.h" /* Generated by bison. */
#define YY_USER_ACTION \
do { \
yylloc->start = yyget_extra(yyscanner); \
yylloc->end = yylloc->start + yyleng; \
yyset_extra(yylloc->end, yyscanner); \
} while (0);
%}
%s IN_PAREN
%s IN_BRACKET
%s IN_BRACE
%s IN_QQINTERP
%x IN_QQSTRING
%x IN_COMMENT
%{
static int enter(int opening, int state, yyscan_t yyscanner);
static int try_exit(int closing, int state, yyscan_t yyscanner);
%}
%option noyywrap nounput noinput nodefault
%option noyyalloc noyyrealloc noyyfree
%option reentrant
%option extra-type="int"
%option bison-bridge bison-locations
%option prefix="jq_yy"
%option stack
%%
"#" { yy_push_state(IN_COMMENT, yyscanner); }
<IN_COMMENT>{
\\(\\|\r?\n)|. { }
\r?\n { yy_pop_state(yyscanner); }
}
<IN_COMMENT><<EOF>> { yy_pop_state(yyscanner); }
"!=" { return NEQ; }
"==" { return EQ; }
"as" { return AS; }
"import" { return IMPORT; }
"include" { return INCLUDE; }
"module" { return MODULE; }
"def" { return DEF; }
"if" { return IF; }
"then" { return THEN; }
"else" { return ELSE; }
"elif" { return ELSE_IF; }
"and" { return AND; }
"or" { return OR; }
"end" { return END; }
"reduce" { return REDUCE; }
"foreach" { return FOREACH; }
"//" { return DEFINEDOR; }
"try" { return TRY; }
"catch" { return CATCH; }
"label" { return LABEL; }
"break" { return BREAK; }
"$__loc__" { return LOC; }
"|=" { return SETPIPE; }
"+=" { return SETPLUS; }
"-=" { return SETMINUS; }
"*=" { return SETMULT; }
"/=" { return SETDIV; }
"%=" { return SETMOD; }
"//=" { return SETDEFINEDOR; }
"<=" { return LESSEQ; }
">=" { return GREATEREQ; }
".." { return REC; }
"?//" { return ALTERNATION; }
"."|"?"|"="|";"|","|":"|"|"|"+"|"-"|"*"|"/"|"%"|"\$"|"<"|">" { return yytext[0];}
"["|"{"|"(" {
return enter(yytext[0], YY_START, yyscanner);
}
"]"|"}"|")" {
return try_exit(yytext[0], YY_START, yyscanner);
}
"@"[a-zA-Z0-9_]+ {
yylval->literal = jv_string_sized(yytext + 1, yyleng - 1); return FORMAT;
}
([0-9]+(\.[0-9]*)?|\.[0-9]+)([eE][+-]?[0-9]+)? {
yylval->literal = jv_parse_sized(yytext, yyleng); return LITERAL;
}
"\"" {
yy_push_state(IN_QQSTRING, yyscanner);
return QQSTRING_START;
}
<IN_QQSTRING>{
"\\(" {
return enter(QQSTRING_INTERP_START, YY_START, yyscanner);
}
"\"" {
yy_pop_state(yyscanner);
return QQSTRING_END;
}
(\\[^u(]|\\u[a-zA-Z0-9]{0,4})+ {
/* pass escapes to the json parser */
jv escapes = jv_string_fmt("\"%.*s\"", (int)yyleng, yytext);
yylval->literal = jv_parse_sized(jv_string_value(escapes), jv_string_length_bytes(jv_copy(escapes)));
jv_free(escapes);
return QQSTRING_TEXT;
}
[^\\\"]+ {
yylval->literal = jv_string_sized(yytext, yyleng);
return QQSTRING_TEXT;
}
. {
return INVALID_CHARACTER;
}
}
([a-zA-Z_][a-zA-Z_0-9]*::)*[a-zA-Z_][a-zA-Z_0-9]* { yylval->literal = jv_string(yytext); return IDENT;}
\.[a-zA-Z_][a-zA-Z_0-9]* { yylval->literal = jv_string(yytext+1); return FIELD;}
\$([a-zA-Z_][a-zA-Z_0-9]*::)*[a-zA-Z_][a-zA-Z_0-9]* { yylval->literal = jv_string(yytext+1); return BINDING;}
[ \r\n\t]+ {}
. { return INVALID_CHARACTER; }
%%
/* perhaps these should be calls... */
/*
"true" { return TRUE; }
"false" { return FALSE; }
"null" { return NULL; }
*/
static int try_exit(int c, int state, yyscan_t yyscanner) {
char match = 0;
int ret;
switch (state) {
case IN_PAREN: match = ret = ')'; break;
case IN_BRACKET: match = ret = ']'; break;
case IN_BRACE: match = ret = '}'; break;
case IN_QQINTERP:
match = ')';
ret = QQSTRING_INTERP_END;
break;
default:
// may not be the best error to give
return INVALID_CHARACTER;
}
assert(match);
if (match == c) {
yy_pop_state(yyscanner);
return ret;
} else {
// FIXME: should we pop? Give a better error at least
return INVALID_CHARACTER;
}
}
static int enter(int c, int currstate, yyscan_t yyscanner) {
int state = 0;
switch (c) {
case '(': state = IN_PAREN; break;
case '[': state = IN_BRACKET; break;
case '{': state = IN_BRACE; break;
case QQSTRING_INTERP_START: state = IN_QQINTERP; break;
}
assert(state);
yy_push_state(state, yyscanner);
return c;
}
void* yyalloc(size_t sz, void* extra) {
return jv_mem_alloc(sz);
}
void* yyrealloc(void* p, size_t sz, void* extra) {
return jv_mem_realloc(p, sz);
}
void yyfree(void* p, void* extra) {
jv_mem_free(p);
}</pblock>
<pblock filename="jq.test" role="context" path="/mnt/c/Users/barlo/projects/drydock/uat/jq/runs/20260822.044627/workspace/targets/jq/blueprint/jq.test">
# Tests are groups of three lines: program, input, expected output
# Blank lines and lines starting with # are ignored
#
# Simple value tests to check parser. Input is irrelevant
#
true
null
true
false
null
false
null
42
null
1
null
1
-1
null
-1
# FIXME: much more number testing needed
{}
null
{}
[]
null
[]
{x:-1},{x:-.},{x:-.|abs}
1
{"x":-1}
{"x":-1}
{"x":1}
# The input line starts with a 0xFEFF (byte order mark) codepoint
# No, there is no reason to have a byte order mark in UTF8 text.
# But apparently people do, so jq shouldn't break on it.
.
"byte order mark"
"byte order mark"
# We test escapes by matching them against Unicode codepoints
# FIXME: more tests needed for weird unicode stuff (e.g. utf16 pairs)
"Aa\r\n\t\b\f\u03bc"
null
"Aa\u000d\u000a\u0009\u0008\u000c\u03bc"
.
"Aa\r\n\t\b\f\u03bc"
"Aa\u000d\u000a\u0009\u0008\u000c\u03bc"
%%FAIL
"u\vw"
jq: error: Invalid escape at line 1, column 4 (while parsing '"\v"') at <top-level>, line 1, column 3:
"u\vw"
^^
"inter\("pol" + "ation")"
null
"interpolation"
@text,@json,([1,.]|@csv,@tsv),@html,(@uri|.,@urid),@sh,(@base64|.,@base64d)
"!()<>&'\"\t"
"!()<>&'\"\t"
"\"!()<>&'\\\"\\t\""
"1,\"!()<>&'\"\"\t\""
"1\t!()<>&'\"\\t"
"!()<>&'"\t"
"%21%28%29%3C%3E%26%27%22%09"
"!()<>&'\"\t"
"'!()<>&'\\''\"\t'"
"ISgpPD4mJyIJ"
"!()<>&'\"\t"
# regression test for #436
@base64
"foóbar\n"
"Zm/Ds2Jhcgo="
@base64d
"Zm/Ds2Jhcgo="
"foóbar\n"
@uri
"\u03bc"
"%CE%BC"
@urid
"%CE%BC"
"\u03bc"
@html "<b>\(.)</b>"
"<script>hax</script>"
"<b><script>hax</script></b>"
[.[]|tojson|fromjson]
["foo", 1, ["a", 1, "b", 2, {"foo":"bar"}]]
["foo",1,["a",1,"b",2,{"foo":"bar"}]]
#
# Dictionary construction syntax
#
{a: 1}
null
{"a":1}
{a,b,(.d):.a,e:.b}
{"a":1, "b":2, "c":3, "d":"c"}
{"a":1, "b":2, "c":1, "e":2}
{"a",b,"a$\(1+1)"}
{"a":1, "b":2, "c":3, "a$2":4}
{"a":1, "b":2, "a$2":4}
%%FAIL
{(0):1}
jq: error: Cannot use number (0) as object key at <top-level>, line 1, column 3:
{(0):1}
^
%%FAIL
{1+2:3}
jq: error: May need parentheses around object key expression at <top-level>, line 1, column 2:
{1+2:3}
^^^
%%FAIL
{non_const:., (0):1}
jq: error: Cannot use number (0) as object key at <top-level>, line 1, column 16:
{non_const:., (0):1}
^
#
# Field access, piping
#
.foo
{"foo": 42, "bar": 43}
42
.foo | .bar
{"foo": {"bar": 42}, "bar": "badvalue"}
42
.foo.bar
{"foo": {"bar": 42}, "bar": "badvalue"}
42
.foo_bar
{"foo_bar": 2}
2
.["foo"].bar
{"foo": {"bar": 42}, "bar": "badvalue"}
42
."foo"."bar"
{"foo": {"bar": 20}}
20
.e0, .E1, .E-1, .E+1
{"e0": 1, "E1": 2, "E": 3}
1
2
2
4
[.[]|.foo?]
[1,[2],{"foo":3,"bar":4},{},{"foo":5}]
[3,null,5]
[.[]|.foo?.bar?]
[1,[2],[],{"foo":3},{"foo":{"bar":4}},{}]
[4,null]
[..]
[1,[[2]],{ "a":[1]}]
[[1,[[2]],{"a":[1]}],1,[[2]],[2],2,{"a":[1]},[1],1]
[.[]|.[]?]
[1,null,[],[1,[2,[[3]]]],[{}],[{"a":[1,[2]]}]]
[1,[2,[[3]]],{},{"a":[1,[2]]}]
[.[]|.[1:3]?]
[1,null,true,false,"abcdef",{},{"a":1,"b":2},[],[1,2,3,4,5],[1,2]]
[null,"bc",[],[2,3],[2]]
# chaining/suffix-list, with and without dot
map(try .a[] catch ., try .a.[] catch ., .a[]?, .a.[]?)
[{"a": [1,2]}, {"a": 123}]
[1,2,1,2,1,2,1,2,"Cannot iterate over number (123)","Cannot iterate over number (123)"]
# oss-fuzz #66070: objects[] leaks if a non-last element throws an error
try ["OK", (.[] | error)] catch ["KO", .]
{"a":["b"],"c":["d"]}
["KO",["b"]]
#
# Negative array indices
#
try (.foo[-1] = 0) catch .
null
"Out of bounds negative array index"
try (.foo[-2] = 0) catch .
null
"Out of bounds negative array index"
.[-1] = 5
[0,1,2]
[0,1,5]
.[-2] = 5
[0,1,2]
[0,5,2]
try (.[999999999] = 0) catch .
null
"Array index too large"
#
# Multiple outputs, iteration
#
.[]
[1,2,3]
1
2
3
1,1
[]
1
1
1,.
[]
1
[]
[.]
[2]
[[2]]
[[2]]
[3]
[[2]]
[{}]
[2]
[{}]
[.[]]
["a"]
["a"]
[(.,1),((.,.[]),(2,3))]
["a","b"]
[["a","b"],1,["a","b"],"a","b",2,3]
[([5,5][]),.,.[]]
[1,2,3]
[5,5,[1,2,3],1,2,3]
{x: (1,2)},{x:3} | .x
null
1
2
3
[.[-4,-3,-2,-1,0,1,2,3]]
[1,2,3]
[null,1,2,3,1,2,3,null]
[range(0;10)]
null
[0,1,2,3,4,5,6,7,8,9]
[range(0,1;3,4)]
null
[0,1,2, 0,1,2,3, 1,2, 1,2,3]
[range(0;10;3)]
null
[0,3,6,9]
[range(0;10;-1)]
null
[]
[range(0;-5;-1)]
null
[0,-1,-2,-3,-4]
[range(0,1;4,5;1,2)]
null
[0,1,2,3,0,2, 0,1,2,3,4,0,2,4, 1,2,3,1,3, 1,2,3,4,1,3]
[while(.<100; .*2)]
1
[1,2,4,8,16,32,64]
[(label $here | .[] | if .>1 then break $here else . end), "hi!"]
[0,1,2]
[0,1,"hi!"]
[(label $here | .[] | if .>1 then break $here else . end), "hi!"]
[0,2,1]
[0,"hi!"]
%%FAIL
. as $foo | break $foo
jq: error: $*label-foo is not defined at <top-level>, line 1, column 13:
. as $foo | break $foo
^^^^^^^^^^
[.[]|[.,1]|until(.[0] < 1; [.[0] - 1, .[1] * .[0]])|.[1]]
[1,2,3,4,5]
[1,2,6,24,120]
[label $out | foreach .[] as $item ([3, null]; if .[0] < 1 then break $out else [.[0] -1, $item] end; .[1])]
[11,22,33,44,55,66,77,88,99]
[11,22,33]
[foreach range(5) as $item (0; $item)]
null
[0,1,2,3,4]
[foreach .[] as [$i, $j] (0; . + $i - $j)]
[[2,1], [5,3], [6,4]]
[1,3,5]
[foreach .[] as {a:$a} (0; . + $a; -.)]
[{"a":1}, {"b":2}, {"a":3, "b":4}]
[-1, -1, -4]
[-foreach -.[] as $x (0; . + $x)]
[1,2,3]
[1,3,6]
[foreach .[] / .[] as $i (0; . + $i)]
[1,2]
[1,3,3.5,4.5]
[foreach .[] as $x (0; . + $x) as $x | $x]
[1,2,3]
[1,3,6]
[limit(3; .[])]
[11,22,33,44,55,66,77,88,99]
[11,22,33]
[limit(0; error)]
"badness"
[]
[limit(1; 1, error)]
"badness"
[1]
try limit(-1; error) catch .
null
"limit doesn't support negative count"
[skip(3; .[])]
[1,2,3,4,5,6,7,8,9]
[4,5,6,7,8,9]
[skip(0,2,3,4; .[])]
[1,2,3]
[1,2,3,3]
[skip(3; .[])]
[]
[]
try skip(-1; error) catch .
null
"skip doesn't support negative count"
nth(1; 0,1,error("foo"))
null
1
[first(range(.)), last(range(.))]
10
[0,9]
[first(range(.)), last(range(.))]
0
[]
[nth(0,5,9,10,15; range(.)), try nth(-1; range(.)) catch .]
10
[0,5,9,"nth doesn't support negative indices"]
# Check that first(g) does not extract more than one value from g
first(1,error("foo"))
null
1
#
# Check that various builtins evaluate all arguments where appropriate,
# doing cartesian products where appropriate.
#
# Check that limit does work for each value produced by n!
[limit(5,7; range(9))]
null
[0,1,2,3,4,0,1,2,3,4,5,6]
# Same check for nth
[nth(5,7; range(9;0;-1))]
null
[4,2]
# Same check for range/3
[range(0,1,2;4,3,2;2,3)]
null
[0,2,0,3,0,2,0,0,0,1,3,1,1,1,1,1,2,2,2,2]
# Same check for range/1
[range(3,5)]
null
[0,1,2,0,1,2,3,4]
# Same check for index/1, rindex/1, indices/1
[(index(",","|"), rindex(",","|")), indices(",","|")]
"a,b|c,d,e||f,g,h,|,|,i,j"
[1,3,22,19,[1,5,7,12,14,16,18,20,22],[3,9,10,17,19]]
# Same check for join/1
join(",","/")
["a","b","c","d"]
"a,b,c,d"
"a/b/c/d"
[.[]|join("a")]
[[],[""],["",""],["","",""]]
["","","a","aa"]
# Same check for flatten/1
flatten(3,2,1)
[0, [1], [[2]], [[[3]]]]
[0,1,2,3]
[0,1,2,[3]]
[0,1,[2],[[3]]]
#
# Slices
#
[.[3:2], .[-5:4], .[:-2], .[-2:], .[3:3][1:], .[10:]]
[0,1,2,3,4,5,6]
[[], [2,3], [0,1,2,3,4], [5,6], [], []]
[.[3:2], .[-5:4], .[:-2], .[-2:], .[3:3][1:], .[10:]]
"abcdefghi"
["","","abcdefg","hi","",""]
del(.[2:4],.[0],.[-2:])
[0,1,2,3,4,5,6,7]
[1,4,5]
.[2:4] = ([], ["a","b"], ["a","b","c"])
[0,1,2,3,4,5,6,7]
[0,1,4,5,6,7]
[0,1,"a","b",4,5,6,7]
[0,1,"a","b","c",4,5,6,7]
# Slices at large offsets (issue #1108)
#
# This is written this way because [range(<large number>)] is
# significantly slower under valgrind than .[<large number>] = value.
#
# We range down rather than up so that we have just one realloc.
reduce range(65540;65536;-1) as $i ([]; .[$i] = $i)|.[65536:]
null
[null,65537,65538,65539,65540]
#
# Variables
#
1 as $x | 2 as $y | [$x,$y,$x]
null
[1,2,1]
[1,2,3][] as $x | [[4,5,6,7][$x]]
null
[5]
[6]
[7]
42 as $x | . | . | . + 432 | $x + 1
34324
43
1 + 2 as $x | -$x
null
-3
"x" as $x | "a"+"y" as $y | $x+","+$y
null
"x,ay"
1 as $x | [$x,$x,$x as $x | $x]
null
[1,1,1]
[1, {c:3, d:4}] as [$a, {c:$b, b:$c}] | $a, $b, $c
null
1
3
null
. as {as: $kw, "str": $str, ("e"+"x"+"p"): $exp} | [$kw, $str, $exp]
{"as": 1, "str": 2, "exp": 3}
[1, 2, 3]
.[] as [$a, $b] | [$b, $a]
[[1], [1, 2, 3]]
[null, 1]
[2, 1]
. as $i | . as [$i] | $i
[0]
0
. as [$i] | . as $i | $i
[0]
[0]
%%FAIL
. as [] | null
jq: error: syntax error, unexpected ']', expecting BINDING or '[' or '{' at <top-level>, line 1, column 7:
. as [] | null
^
%%FAIL
. as {} | null
jq: error: syntax error, unexpected '}' at <top-level>, line 1, column 7:
. as {} | null
^
%%FAIL
. as $foo | [$foo, $bar]
jq: error: $bar is not defined at <top-level>, line 1, column 20:
. as $foo | [$foo, $bar]
^^^^
%%FAIL
. as {(true):$foo} | $foo
jq: error: Cannot use boolean (true) as object key at <top-level>, line 1, column 8:
. as {(true):$foo} | $foo
^^^^
# [.,(.[] | {x:.},.),.,.[]]
#
# Builtin functions
#
1+1
null
2
1+1
"wtasdf"
2.0
2-1
null
1
2-(-1)
null
3
1e+0+0.001e3
"I wonder what this will be?"
20e-1
.+4
15
19.0
.+null
{"a":42}
{"a":42}
null+.
null
null
.a+.b
{"a":42}
42
[1,2,3] + [.]
null
[1,2,3,null]
{"a":1} + {"b":2} + {"c":3}
"asdfasdf"
{"a":1, "b":2, "c":3}
"asdf" + "jkl;" + . + . + .
"some string"
"asdfjkl;some stringsome stringsome string"
"\u0000\u0020\u0000" + .
"\u0000\u0020\u0000"
"\u0000 \u0000\u0000 \u0000"
42 - .
11
31
[1,2,3,4,1] - [.,3]
1
[2,4]
[-1 as $x | 1,$x]
null
[1,-1]
[10 * 20, 20 / .]
4
[200, 5]
1 + 2 * 2 + 10 / 2
null
10
[16 / 4 / 2, 16 / 4 * 2, 16 - 4 - 2, 16 - 4 + 2]
null
[2, 8, 10, 14]
1e-19 + 1e-20 - 5e-21
null
1.05e-19
1 / 1e-17
null
1e+17
9E999999999, 9999999999E999999990, 1E-999999999, 0.000000001E-999999990
null
9E+999999999
9.999999999E+999999999
1E-999999999
1E-999999999
5E500000000 > 5E-5000000000, 10000E500000000 > 10000E-5000000000
null
true
true
# #2825
(1e999999999, 10e999999999) > (1e-1147483646, 0.1e-1147483646)
null
true
true
true
true
25 % 7
null
4
49732 % 472
null
172
[(infinite, -infinite) % (1, -1, infinite)]
null
[0,0,0,0,0,-1]
[nan % 1, 1 % nan | isnan]
null
[true,true]
1 + tonumber + ("10" | tonumber)
4
15
"123\u0000456" | try tonumber catch .
null
"string (\"123\\u0000456\") cannot be parsed as a number"
map(toboolean)
["false","true",false,true]
[false,true,false,true]
.[] | try toboolean catch .
[null,0,"tru","truee","fals","falsee",[],{}]
"null (null) cannot be parsed as a boolean"
"number (0) cannot be parsed as a boolean"
"string (\"tru\") cannot be parsed as a boolean"
"string (\"truee\") cannot be parsed as a boolean"
"string (\"fals\") cannot be parsed as a boolean"
"string (\"falsee\") cannot be parsed as a boolean"
"array ([]) cannot be parsed as a boolean"
"object ({}) cannot be parsed as a boolean"
"true\u0000x", "false\u0000" | try toboolean catch .
null
"string (\"true\\u0000x\") cannot be parsed as a boolean"
"string (\"false\\u0000\") cannot be parsed as a boolean"
[{"a":42},.object,10,.num,false,true,null,"b",[1,4]] | .[] as $x | [$x == .[]]
{"object": {"a":42}, "num":10.0}
[true, true, false, false, false, false, false, false, false]
[true, true, false, false, false, false, false, false, false]
[false, false, true, true, false, false, false, false, false]
[false, false, true, true, false, false, false, false, false]
[false, false, false, false, true, false, false, false, false]
[false, false, false, false, false, true, false, false, false]
[false, false, false, false, false, false, true, false, false]
[false, false, false, false, false, false, false, true, false]
[false, false, false, false, false, false, false, false, true ]
[.[] | length]
[[], {}, [1,2], {"a":42}, "asdf", "\u03bc"]
[0, 0, 2, 1, 4, 1]
utf8bytelength
"asdf\u03bc"
6
[.[] | try utf8bytelength catch .]
[[], {}, [1,2], 55, true, false]
["array ([]) only strings have UTF-8 byte length","object ({}) only strings have UTF-8 byte length","array ([1,2]) only strings have UTF-8 byte length","number (55) only strings have UTF-8 byte length","boolean (true) only strings have UTF-8 byte length","boolean (false) only strings have UTF-8 byte length"]
map(keys)
[{}, {"abcd":1,"abc":2,"abcde":3}, {"x":1, "z": 3, "y":2}]
[[], ["abc","abcd","abcde"], ["x","y","z"]]
[1,2,empty,3,empty,4]
null
[1,2,3,4]
map(add)
[[], [1,2,3], ["a","b","c"], [[3],[4,5],[6]], [{"a":1}, {"b":2}, {"a":3}]]
[null, 6, "abc", [3,4,5,6], {"a":3, "b": 2}]
map_values(.+1)
[0,1,2]
[1,2,3]
[add(null), add(range(range(10))), add(empty), add(10,range(10))]
null
[null,120,null,55]
# Real-world use case for add(empty)
.sum = add(.arr[])
{"arr":[]}
{"arr":[],"sum":null}
add({(.[]):1}) | keys
["a","a","b","a","d","b","d","a","d"]
["a","b","d"]
#
# User-defined functions
# Oh god.
#
def f: . + 1; def g: def g: . + 100; f | g | f; (f | g), g
3.0
106.0
105.0
def f: (1000,2000); f
123412345
1000
2000
def f(a;b;c;d;e;f): [a+1,b,c,d,e,f]; f(.[0];.[1];.[0];.[0];.[0];.[0])
[1,2]
[2,2,1,1,1,1]
def f: 1; def g: f, def f: 2; def g: 3; f, def f: g; f, g; def f: 4; [f, def f: g; def g: 5; f, g]+[f,g]
null
[4,1,2,3,3,5,4,1,2,3,3]
# Test precedence of 'def' vs '|'
def a: 0; . | a
null
0
# Many arguments
def f(a;b;c;d;e;f;g;h;i;j): [j,i,h,g,f,e,d,c,b,a]; f(.[0];.[1];.[2];.[3];.[4];.[5];.[6];.[7];.[8];.[9])
[0,1,2,3,4,5,6,7,8,9]
[9,8,7,6,5,4,3,2,1,0]
([1,2] + [4,5])
[1,2,3]
[1,2,4,5]
true
[1]
true
null,1,null
"hello"
null
1
null
[1,2,3]
[5,6]
[1,2,3]
[.[]|floor]
[-1.1,1.1,1.9]
[-2, 1, 1]
[.[]|sqrt]
[4,9]
[2,3]
(add / length) as $m | map((. - $m) as $d | $d * $d) | add / length | sqrt
[2,4,4,4,5,5,7,9]
2
# Should write a test that calls the -lm function from C (or bc(1)) to
# check that they match the corresponding jq functions. However,
# there's so little template code standing between that it suffices to
# test a handful of these. The results were checked by eye against
# bc(1).
atan * 4 * 1000000|floor / 1000000
1
3.141592
[(3.141592 / 2) * (range(0;20) / 20)|cos * 1000000|floor / 1000000]
null
[1,0.996917,0.987688,0.972369,0.951056,0.923879,0.891006,0.85264,0.809017,0.760406,0.707106,0.649448,0.587785,0.522498,0.45399,0.382683,0.309017,0.233445,0.156434,0.078459]
[(3.141592 / 2) * (range(0;20) / 20)|sin * 1000000|floor / 1000000]
null
[0,0.078459,0.156434,0.233445,0.309016,0.382683,0.45399,0.522498,0.587785,0.649447,0.707106,0.760405,0.809016,0.85264,0.891006,0.923879,0.951056,0.972369,0.987688,0.996917]
def f(x): x | x; f([.], . + [42])
[1,2,3]
[[[1,2,3]]]
[[1,2,3],42]
[[1,2,3,42]]
[1,2,3,42,42]
# test multiple function arities and redefinition
def f: .+1; def g: f; def f: .+100; def f(a):a+.+11; [(g|f(20)), f]
1
[33,101]
# test closures and lexical scoping
def id(x):x; 2000 as $x | def f(x):1 as $x | id([$x, x, x]); def g(x): 100 as $x | f($x,$x+x); g($x)
"more testing"
[1,100,2100.0,100,2100.0]
# test def f($a) syntax
def x(a;b): a as $a | b as $b | $a + $b; def y($a;$b): $a + $b; def check(a;b): [x(a;b)] == [y(a;b)]; check(.[];.[]*2)
[1,2,3]
true
# test backtracking through function calls and returns
# this test is *evil*
[[20,10][1,0] as $x | def f: (100,200) as $y | def g: [$x + $y, .]; . + $x | g; f[0] | [f][0][1] | f]
999999999
[[110.0, 130.0], [210.0, 130.0], [110.0, 230.0], [210.0, 230.0], [120.0, 160.0], [220.0, 160.0], [120.0, 260.0], [220.0, 260.0]]
# test recursion
def fac: if . == 1 then 1 else . * (. - 1 | fac) end; [.[] | fac]
[1,2,3,4]
[1,2,6,24]
# test stack overflow and reallocation
# this test is disabled for now, it takes a realllllly long time.
# def f: if length > 1000 then . else .+[1]|f end; f | length
# []
# 1001
reduce .[] as $x (0; . + $x)
[1,2,4]
7
reduce .[] as [$i, {j:$j}] (0; . + $i - $j)
[[2,{"j":1}], [5,{"j":3}], [6,{"j":4}]]
5
reduce [[1,2,10], [3,4,10]][] as [$i,$j] (0; . + $i * $j)
null
14
[-reduce -.[] as $x (0; . + $x)]
[1,2,3]
[6]
[reduce .[] / .[] as $i (0; . + $i)]
[1,2]
[4.5]
reduce .[] as $x (0; . + $x) as $x | $x
[1,2,3]
6
# This, while useless, should still compile.
reduce . as $n (.; .)
null
null
# Destructuring
. as {$a, b: [$c, {$d}]} | [$a, $c, $d]
{"a":1, "b":[2,{"d":3}]}
[1,2,3]
. as {$a, $b:[$c, $d]}| [$a, $b, $c, $d]
{"a":1, "b":[2,{"d":3}]}
[1,[2,{"d":3}],2,{"d":3}]
# Destructuring with alternation
.[] | . as {$a, b: [$c, {$d}]} ?// [$a, {$b}, $e] ?// $f | [$a, $b, $c, $d, $e, $f]
[{"a":1, "b":[2,{"d":3}]}, [4, {"b":5, "c":6}, 7, 8, 9], "foo"]
[1, null, 2, 3, null, null]
[4, 5, null, null, 7, null]
[null, null, null, null, null, "foo"]
# Destructuring DUP/POP issues
.[] | . as {a:$a} ?// {a:$a} ?// {a:$a} | $a
[[3],[4],[5],6]
# Runtime error: "jq: Cannot index array with string (\"c\")"
.[] as {a:$a} ?// {a:$a} ?// {a:$a} | $a
[[3],[4],[5],6]
# Runtime error: "jq: Cannot index array with string (\"c\")"
[[3],[4],[5],6][] | . as {a:$a} ?// {a:$a} ?// {a:$a} | $a
null
# Runtime error: "jq: Cannot index array with string (\"c\")"
[[3],[4],[5],6] | .[] as {a:$a} ?// {a:$a} ?// {a:$a} | $a
null
# Runtime error: "jq: Cannot index array with string (\"c\")"
.[] | . as {a:$a} ?// {a:$a} ?// $a | $a
[[3],[4],[5],6]
[3]
[4]
[5]
6
.[] as {a:$a} ?// {a:$a} ?// $a | $a
[[3],[4],[5],6]
[3]
[4]
[5]
6
[[3],[4],[5],6][] | . as {a:$a} ?// {a:$a} ?// $a | $a
null
[3]
[4]
[5]
6
[[3],[4],[5],6] | .[] as {a:$a} ?// {a:$a} ?// $a | $a
null
[3]
[4]
[5]
6
.[] | . as {a:$a} ?// $a ?// {a:$a} | $a
[[3],[4],[5],6]
[3]
[4]
[5]
6
.[] as {a:$a} ?// $a ?// {a:$a} | $a
[[3],[4],[5],6]
[3]
[4]
[5]
6
[[3],[4],[5],6][] | . as {a:$a} ?// $a ?// {a:$a} | $a
null
[3]
[4]
[5]
6
[[3],[4],[5],6] | .[] as {a:$a} ?// $a ?// {a:$a} | $a
null
[3]
[4]
[5]
6
.[] | . as $a ?// {a:$a} ?// {a:$a} | $a
[[3],[4],[5],6]
[3]
[4]
[5]
6
.[] as $a ?// {a:$a} ?// {a:$a} | $a
[[3],[4],[5],6]
[3]
[4]
[5]
6
[[3],[4],[5],6][] | . as $a ?// {a:$a} ?// {a:$a} | $a
null
[3]
[4]
[5]
6
[[3],[4],[5],6] | .[] as $a ?// {a:$a} ?// {a:$a} | $a
null
[3]
[4]
[5]
6
. as $dot|any($dot[];not)
[1,2,3,4,true,false,1,2,3,4,5]
true
. as $dot|any($dot[];not)
[1,2,3,4,true]
false
. as $dot|all($dot[];.)
[1,2,3,4,true,false,1,2,3,4,5]
false
. as $dot|all($dot[];.)
[1,2,3,4,true]
true
# Check short-circuiting
any(true, error; .)
"badness"
true
all(false, error; .)
"badness"
false
any(not)
[]
false
all(not)
[]
true
any(not)
[false]
true
all(not)
[false]
true
[any,all]
[]
[false,true]
[any,all]
[true]
[true,true]
[any,all]
[false]
[false,false]
[any,all]
[true,false]
[true,false]
[any,all]
[null,null,true]
[true,false]
#
# Paths
#
path(.foo[0,1])
null
["foo", 0]
["foo", 1]
path(.[] | select(.>3))
[1,5,3]
[1]
path(.)
42
[]
try path(.a | map(select(.b == 0))) catch .
{"a":[{"b":0}]}
"Invalid path expression with result [{\"b\":0}]"
try path(.a | map(select(.b == 0)) | .[0]) catch .
{"a":[{"b":0}]}
"Invalid path expression near attempt to access element 0 of [{\"b\":0}]"
try path(.a | map(select(.b == 0)) | .c) catch .
{"a":[{"b":0}]}
"Invalid path expression near attempt to access element \"c\" of [{\"b\":0}]"
try path(.a | map(select(.b == 0)) | .[]) catch .
{"a":[{"b":0}]}
"Invalid path expression near attempt to iterate through [{\"b\":0}]"
path(.a[path(.b)[0]])
{"a":{"b":0}}
["a","b"]
[paths]
[1,[[],{"a":2}]]
[[0],[1],[1,0],[1,1],[1,1,"a"]]
["foo",1] as $p | getpath($p), setpath($p; 20), delpaths([$p])
{"bar": 42, "foo": ["a", "b", "c", "d"]}
"b"
{"bar": 42, "foo": ["a", 20, "c", "d"]}
{"bar": 42, "foo": ["a", "c", "d"]}
map(getpath([2])), map(setpath([2]; 42)), map(delpaths([[2]]))
[[0], [0,1], [0,1,2]]
[null, null, 2]
[[0,null,42], [0,1,42], [0,1,42]]
[[0], [0,1], [0,1]]
map(delpaths([[0,"foo"]]))
[[{"foo":2, "x":1}], [{"bar":2}]]
[[{"x":1}], [{"bar":2}]]
["foo",1] as $p | getpath($p), setpath($p; 20), delpaths([$p])
{"bar":false}
null
{"bar":false, "foo": [null, 20]}
{"bar":false}
delpaths([[-200]])
[1,2,3]
[1,2,3]
try delpaths(0) catch .
{}
"Paths must be specified as an array"
del(.), del(empty), del((.foo,.bar,.baz) | .[2,3,0]), del(.foo[0], .bar[0], .foo, .baz.bar[0].x)
{"foo": [0,1,2,3,4], "bar": [0,1]}
null
{"foo": [0,1,2,3,4], "bar": [0,1]}
{"foo": [1,4], "bar": [1]}
{"bar": [1]}
del(.[1], .[-6], .[2], .[-3:9])
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
[0, 3, 5, 6, 9]
del(.[nan])
[1,2,3]
[1,2,3]
del(.[nan,nan])
[1,2,3]
[1,2,3]
# negative index
setpath([-1]; 1)
[0]
[1]
pick(.a.b.c)
null
{"a":{"b":{"c":null}}}
pick(first)
[1,2]
[1]
pick(first|first)
[[10,20],30]
[[10]]
# negative indices in path expressions (since last/1 is .[-1])
try pick(last) catch .
[1,2]
"Out of bounds negative array index"
#
# Assignment
#
.message = "goodbye"
{"message": "hello"}
{"message": "goodbye"}
.foo = .bar
{"bar":42}
{"foo":42, "bar":42}
.foo |= .+1
{"foo": 42}
{"foo": 43}
.[] += 2, .[] *= 2, .[] -= 2, .[] /= 2, .[] %=2
[1,3,5]
[3,5,7]
[2,6,10]
[-1,1,3]
[0.5, 1.5, 2.5]
[1,1,1]
[.[] % 7]
[-7,-6,-5,-4,-3,-2,-1,0,1,2,3,4,5,6,7]
[0,-6,-5,-4,-3,-2,-1,0,1,2,3,4,5,6,0]
.foo += .foo
{"foo":2}
{"foo":4}
.[0].a |= {"old":., "new":(.+1)}
[{"a":1,"b":2}]
[{"a":{"old":1, "new":2},"b":2}]
def inc(x): x |= .+1; inc(.[].a)
[{"a":1,"b":2},{"a":2,"b":4},{"a":7,"b":8}]
[{"a":2,"b":2},{"a":3,"b":4},{"a":8,"b":8}]
# #1358, getpath/1 should work in path expressions
.[] | try (getpath(["a",0,"b"]) |= 5) catch .
[null,{"b":0},{"a":0},{"a":null},{"a":[0,1]},{"a":{"b":1}},{"a":[{}]},{"a":[{"c":3}]}]
{"a":[{"b":5}]}
{"b":0,"a":[{"b":5}]}
"Cannot index number with number (0)"
{"a":[{"b":5}]}
"Cannot index number with string (\"b\")"
"Cannot index object with number (0)"
{"a":[{"b":5}]}
{"a":[{"c":3,"b":5}]}
# #2051, deletion using assigning empty against arrays
(.[] | select(. >= 2)) |= empty
[1,5,3,0,7]
[1,0]
.[] |= select(. % 2 == 0)
[0,1,2,3,4,5]
[0,2,4]
.foo[1,4,2,3] |= empty
{"foo":[0,1,2,3,4,5]}
{"foo":[0,5]}
.[2][3] = 1
[4]
[4, null, [null, null, null, 1]]
.foo[2].bar = 1
{"foo":[11], "bar":42}
{"foo":[11,null,{"bar":1}], "bar":42}
try ((map(select(.a == 1))[].b) = 10) catch .
[{"a":0},{"a":1}]
"Invalid path expression near attempt to iterate through [{\"a\":1}]"
try ((map(select(.a == 1))[].a) |= .+1) catch .
[{"a":0},{"a":1}]
"Invalid path expression near attempt to iterate through [{\"a\":1}]"
def x: .[1,2]; x=10
[0,1,2]
[0,10,10]
try (def x: reverse; x=10) catch .
[0,1,2]
"Invalid path expression with result [2,1,0]"
.[] = 1
[1,null,Infinity,-Infinity,NaN,-NaN]
[1,1,1,1,1,1]
#
# Conditionals
#
[.[] | if .foo then "yep" else "nope" end]
[{"foo":0},{"foo":1},{"foo":[]},{"foo":true},{"foo":false},{"foo":null},{"foo":"foo"},{}]
["yep","yep","yep","yep","nope","nope","yep","nope"]
[.[] | if .baz then "strange" elif .foo then "yep" else "nope" end]
[{"foo":0},{"foo":1},{"foo":[]},{"foo":true},{"foo":false},{"foo":null},{"foo":"foo"},{}]
["yep","yep","yep","yep","nope","nope","yep","nope"]
[if 1,null,2 then 3 else 4 end]
null
[3,4,3]
[if empty then 3 else 4 end]
null
[]
[if 1 then 3,4 else 5 end]
null
[3,4]
[if null then 3 else 5,6 end]
null
[5,6]
[if true then 3 end]
7
[3]
[if false then 3 end]
7
[7]
[if false then 3 else . end]
7
[7]
[if false then 3 elif false then 4 end]
7
[7]
[if false then 3 elif false then 4 else . end]
7
[7]
[-if true then 1 else 2 end]
null
[-1]
{x: if true then 1 else 2 end}
null
{"x":1}
if true then [.] else . end []
null
null
[.[] | [.foo[] // .bar]]
[{"foo":[1,2], "bar": 42}, {"foo":[1], "bar": null}, {"foo":[null,false,3], "bar": 18}, {"foo":[], "bar":42}, {"foo": [null,false,null], "bar": 41}]
[[1,2], [1], [3], [42], [41]]
.[] //= .[0]
["hello",true,false,[false],null]
["hello",true,"hello",[false],"hello"]
.[] | [.[0] and .[1], .[0] or .[1]]
[[true,[]], [false,1], [42,null], [null,false]]
[true,true]
[false,true]
[false,true]
[false,false]
[.[] | not]
[1,0,false,null,true,"hello"]
[false,false,true,true,false,false]
# Check numeric comparison binops
[10 > 0, 10 > 10, 10 > 20, 10 < 0, 10 < 10, 10 < 20]
{}
[true,false,false,false,false,true]
[10 >= 0, 10 >= 10, 10 >= 20, 10 <= 0, 10 <= 10, 10 <= 20]
{}
[true,true,false,false,true,true]
# And some in/equality tests
[ 10 == 10, 10 != 10, 10 != 11, 10 == 11]
{}
[true,false,true,false]
["hello" == "hello", "hello" != "hello", "hello" == "world", "hello" != "world" ]
{}
[true,false,false,true]
[[1,2,3] == [1,2,3], [1,2,3] != [1,2,3], [1,2,3] == [4,5,6], [1,2,3] != [4,5,6]]
{}
[true,false,false,true]
[{"foo":42} == {"foo":42},{"foo":42} != {"foo":42}, {"foo":42} != {"bar":42}, {"foo":42} == {"bar":42}]
{}
[true,false,true,false]
# ugly complicated thing
[{"foo":[1,2,{"bar":18},"world"]} == {"foo":[1,2,{"bar":18},"world"]},{"foo":[1,2,{"bar":18},"world"]} == {"foo":[1,2,{"bar":19},"world"]}]
{}
[true,false]
# containment operator
[("foo" | contains("foo")), ("foobar" | contains("foo")), ("foo" | contains("foobar"))]
{}
[true, true, false]
# containment operator (embedded NULs!)
[contains(""), contains("\u0000")]
"\u0000"
[true, true]
[contains(""), contains("a"), contains("ab"), contains("c"), contains("d")]
"ab\u0000cd"
[true, true, true, true, true]
[contains("cd"), contains("b\u0000"), contains("ab\u0000")]
"ab\u0000cd"
[true, true, true]
[contains("b\u0000c"), contains("b\u0000cd"), contains("b\u0000cd")]
"ab\u0000cd"
[true, true, true]
[contains("@"), contains("\u0000@"), contains("\u0000what")]
"ab\u0000cd"
[false, false, false]
# Try/catch and general `?` operator
[.[]|try if . == 0 then error("foo") elif . == 1 then .a elif . == 2 then empty else . end catch .]
[0,1,2,3]
["foo","Cannot index number with string (\"a\")",3]
[.[]|(.a, .a)?]
[null,true,{"a":1}]
[null,null,1,1]
[[.[]|[.a,.a]]?]
[null,true,{"a":1}]
[]
[if error then 1 else 2 end?]
"foo"
[]
try error(0) // 1
null
1
1, try error(2), 3
null
1
3
1 + try 2 catch 3 + 4
null
7
[-try .]
1
[-1]
try -.? catch .
"foo"
"string (\"foo\") cannot be negated"
{x: try 1, y: try error catch 2, z: if true then 3 end}
null
{"x":1,"y":2,"z":3}
{x: 1 + 2, y: false or true, z: null // 3}
null
{"x":3,"y":true,"z":3}
.[] | try error catch .
[1,null,2]
1
null
2
try error("\($__loc__)") catch .
null
"{\"file\":\"<top-level>\",\"line\":1}"
# string operations
[.[]|startswith("foo")]
["fo", "foo", "barfoo", "foobar", "barfoob"]
[false, true, false, true, false]
[.[]|endswith("foo")]
["fo", "foo", "barfoo", "foobar", "barfoob"]
[false, true, true, false, false]
[.[] | split(", ")]
["a,b, c, d, e,f",", a,b, c, d, e,f, "]
[["a,b","c","d","e,f"],["","a,b","c","d","e,f",""]]
split("")
"abc"
["a","b","c"]
[.[]|ltrimstr("foo")]
["fo", "foo", "barfoo", "foobar", "afoo"]
["fo","","barfoo","bar","afoo"]
[.[]|rtrimstr("foo")]
["fo", "foo", "barfoo", "foobar", "foob"]
["fo","","bar","foobar","foob"]
[.[]|trimstr("foo")]
["fo", "foo", "barfoo", "foobarfoo", "foob"]
["fo","","bar","bar","b"]
[.[]|ltrimstr("")]
["a", "xx", ""]
["a", "xx", ""]
[.[]|rtrimstr("")]
["a", "xx", ""]
["a", "xx", ""]
[.[]|trimstr("")]
["a", "xx", ""]
["a", "xx", ""]
[(index(","), rindex(",")), indices(",")]
"a,bc,def,ghij,klmno"
[1,13,[1,4,8,13]]
[ index("aba"), rindex("aba"), indices("aba") ]
"xababababax"
[1,7,[1,3,5,7]]
# _strindices is used by indices/1 but is callable
try _strindices("abc") catch .
123
"number (123) cannot be searched, as it is not a string"
try _strindices(123) catch .
"abc"
"number (123) is not a string"
# trim
# \u000b is vertical tab (\v not supported by json)
map(trim), map(ltrim), map(rtrim)
[" \n\t\r\f\u000b", ""," ", "a", " a ", "abc", " abc ", " abc", "abc "]
["", "", "", "a", "a", "abc", "abc", "abc", "abc"]
["", "", "", "a", "a ", "abc", "abc ", "abc", "abc "]
["", "", "", "a", " a", "abc", " abc", " abc", "abc"]
trim, ltrim, rtrim
"\u0009\u000A\u000B\u000C\u000D\u0020\u0085\u00A0\u1680\u2000\u2001\u2002\u2003\u2004\u2005\u2006\u2007\u2008\u2009\u200A\u2028\u2029\u202F\u205F\u3000abc\u0009\u000A\u000B\u000C\u000D\u0020\u0085\u00A0\u1680\u2000\u2001\u2002\u2003\u2004\u2005\u2006\u2007\u2008\u2009\u200A\u2028\u2029\u202F\u205F\u3000"
"abc"
"abc\u0009\u000A\u000B\u000C\u000D\u0020\u0085\u00A0\u1680\u2000\u2001\u2002\u2003\u2004\u2005\u2006\u2007\u2008\u2009\u200A\u2028\u2029\u202F\u205F\u3000"
"\u0009\u000A\u000B\u000C\u000D\u0020\u0085\u00A0\u1680\u2000\u2001\u2002\u2003\u2004\u2005\u2006\u2007\u2008\u2009\u200A\u2028\u2029\u202F\u205F\u3000abc"
try trim catch ., try ltrim catch ., try rtrim catch .
123
"trim input must be a string"
"trim input must be a string"
"trim input must be a string"
indices(1)
[0,1,1,2,3,4,1,5]
[1,2,6]
indices([1,2])
[0,1,2,3,1,4,2,5,1,2,6,7]
[1,8]
indices([1,2])
[1]
[]
indices(", ")
"a,b, cd,e, fgh, ijkl"
[3,9,14]
index("!")
"здравствуй мир!"
14
.[:rindex("x")]
"正xyz"
"正"
indices("o")
"🇬🇧oo"
[2,3]
indices("o")
"ƒoo"
[1,2]
[.[]|split(",")]
["a, bc, def, ghij, jklmn, a,b, c,d, e,f", "a,b,c,d, e,f,g,h"]
[["a"," bc"," def"," ghij"," jklmn"," a","b"," c","d"," e","f"],["a","b","c","d"," e","f","g","h"]]
[.[]|split(", ")]
["a, bc, def, ghij, jklmn, a,b, c,d, e,f", "a,b,c,d, e,f,g,h"]
[["a","bc","def","ghij","jklmn","a,b","c,d","e,f"],["a,b,c,d","e,f,g,h"]]
[.[] * 3]
["a", "ab", "abc"]
["aaa", "ababab", "abcabcabc"]
[.[] * "abc"]
[-1.0, -0.5, 0.0, 0.5, 1.0, 1.5, 3.7, 10.0]
[null,null,"","","abc","abc","abcabcabc","abcabcabcabcabcabcabcabcabcabc"]
[. * (nan,-nan)]
"abc"
[null,null]
. * 100000 | [.[:10],.[-10:]]
"abc"
["abcabcabca","cabcabcabc"]
. * 1000000000
""
""
try (. * 1000000000) catch .
"abc"
"Repeat string result too long"
[.[] / ","]
["a, bc, def, ghij, jklmn, a,b, c,d, e,f", "a,b,c,d, e,f,g,h"]
[["a"," bc"," def"," ghij"," jklmn"," a","b"," c","d"," e","f"],["a","b","c","d"," e","f","g","h"]]
[.[] / ", "]
["a, bc, def, ghij, jklmn, a,b, c,d, e,f", "a,b,c,d, e,f,g,h"]
[["a","bc","def","ghij","jklmn","a,b","c,d","e,f"],["a,b,c,d","e,f,g,h"]]
map(.[1] as $needle | .[0] | contains($needle))
[[[],[]], [[1,2,3], [1,2]], [[1,2,3], [3,1]], [[1,2,3], [4]], [[1,2,3], [1,4]]]
[true, true, true, false, false]
map(.[1] as $needle | .[0] | contains($needle))
[[["foobar", "foobaz"], ["baz", "bar"]], [["foobar", "foobaz"], ["foo"]], [["foobar", "foobaz"], ["blap"]]]
[true, true, false]
[({foo: 12, bar:13} | contains({foo: 12})), ({foo: 12} | contains({})), ({foo: 12, bar:13} | contains({baz:14}))]
{}
[true, true, false]
{foo: {baz: 12, blap: {bar: 13}}, bar: 14} | contains({bar: 14, foo: {blap: {}}})
{}
true
{foo: {baz: 12, blap: {bar: 13}}, bar: 14} | contains({bar: 14, foo: {blap: {bar: 14}}})
{}
false
sort
[42,[2,5,3,11],10,{"a":42,"b":2},{"a":42},true,2,[2,6],"hello",null,[2,5,6],{"a":[],"b":1},"abc","ab",[3,10],{},false,"abcd",null]
[null,null,false,true,2,10,42,"ab","abc","abcd","hello",[2,5,3,11],[2,5,6],[2,6],[3,10],{},{"a":42},{"a":42,"b":2},{"a":[],"b":1}]
(sort_by(.b) | sort_by(.a)), sort_by(.a, .b), sort_by(.b, .c), group_by(.b), group_by(.a + .b - .c == 2)
[{"a": 1, "b": 4, "c": 14}, {"a": 4, "b": 1, "c": 3}, {"a": 1, "b": 4, "c": 3}, {"a": 0, "b": 2, "c": 43}]
[{"a": 0, "b": 2, "c": 43}, {"a": 1, "b": 4, "c": 14}, {"a": 1, "b": 4, "c": 3}, {"a": 4, "b": 1, "c": 3}]
[{"a": 0, "b": 2, "c": 43}, {"a": 1, "b": 4, "c": 14}, {"a": 1, "b": 4, "c": 3}, {"a": 4, "b": 1, "c": 3}]
[{"a": 4, "b": 1, "c": 3}, {"a": 0, "b": 2, "c": 43}, {"a": 1, "b": 4, "c": 3}, {"a": 1, "b": 4, "c": 14}]
[[{"a": 4, "b": 1, "c": 3}], [{"a": 0, "b": 2, "c": 43}], [{"a": 1, "b": 4, "c": 14}, {"a": 1, "b": 4, "c": 3}]]
[[{"a": 1, "b": 4, "c": 14}, {"a": 0, "b": 2, "c": 43}], [{"a": 4, "b": 1, "c": 3}, {"a": 1, "b": 4, "c": 3}]]
unique
[1,2,5,3,5,3,1,3]
[1,2,3,5]
unique
[]
[]
[min, max, min_by(.[1]), max_by(.[1]), min_by(.[2]), max_by(.[2])]
[[4,2,"a"],[3,1,"a"],[2,4,"a"],[1,3,"a"]]
[[1,3,"a"],[4,2,"a"],[3,1,"a"],[2,4,"a"],[4,2,"a"],[1,3,"a"]]
[min,max,min_by(.),max_by(.)]
[]
[null,null,null,null]
.foo[.baz]
{"foo":{"bar":4},"baz":"bar"}
4
.[] | .error = "no, it's OK"
[{"error":true}]
{"error": "no, it's OK"}
[{a:1}] | .[] | .a=999
null
{"a": 999}
to_entries
{"a": 1, "b": 2}
[{"key":"a", "value":1}, {"key":"b", "value":2}]
from_entries
[{"key":"a", "value":1}, {"Key":"b", "Value":2}, {"name":"c", "value":3}, {"Name":"d", "Value":4}]
{"a": 1, "b": 2, "c": 3, "d": 4}
with_entries(.key |= "KEY_" + .)
{"a": 1, "b": 2}
{"KEY_a": 1, "KEY_b": 2}
map(has("foo"))
[{"foo": 42}, {}]
[true, false]
map(has(2))
[[0,1], ["a","b","c"]]
[false, true]
has(nan)
[0,1,2]
false
keys
[42,3,35]
[0,1,2]
[][.]
1000000000000000000
null
map([1,2][0:.])
[-1, 1, 2, 3, 1000000000000000000]
[[1], [1], [1,2], [1,2], [1,2]]
# Test recursive object merge
{"k": {"a": 1, "b": 2}} * .
{"k": {"a": 0,"c": 3}}
{"k": {"a": 0, "b": 2, "c": 3}}
{"k": {"a": 1, "b": 2}, "hello": {"x": 1}} * .
{"k": {"a": 0,"c": 3}, "hello": 1}
{"k": {"a": 0, "b": 2, "c": 3}, "hello": 1}
{"k": {"a": 1, "b": 2}, "hello": 1} * .
{"k": {"a": 0,"c": 3}, "hello": {"x": 1}}
{"k": {"a": 0, "b": 2, "c": 3}, "hello": {"x": 1}}
{"a": {"b": 1}, "c": {"d": 2}, "e": 5} * .
{"a": {"b": 2}, "c": {"d": 3, "f": 9}}
{"a": {"b": 2}, "c": {"d": 3, "f": 9}, "e": 5}
[.[]|arrays]
[1,2,"foo",[],[3,[]],{},true,false,null]
[[],[3,[]]]
[.[]|objects]
[1,2,"foo",[],[3,[]],{},true,false,null]
[{}]
[.[]|iterables]
[1,2,"foo",[],[3,[]],{},true,false,null]
[[],[3,[]],{}]
[.[]|scalars]
[1,2,"foo",[],[3,[]],{},true,false,null]
[1,2,"foo",true,false,null]
[.[]|values]
[1,2,"foo",[],[3,[]],{},true,false,null]
[1,2,"foo",[],[3,[]],{},true,false]
[.[]|booleans]
[1,2,"foo",[],[3,[]],{},true,false,null]
[true,false]
[.[]|nulls]
[1,2,"foo",[],[3,[]],{},true,false,null]
[null]
flatten
[0, [1], [[2]], [[[3]]]]
[0, 1, 2, 3]
flatten(0)
[0, [1], [[2]], [[[3]]]]
[0, [1], [[2]], [[[3]]]]
flatten(2)
[0, [1], [[2]], [[[3]]]]
[0, 1, 2, [3]]
flatten(2)
[0, [1, [2]], [1, [[3], 2]]]
[0, 1, 2, 1, [3], 2]
try flatten(-1) catch .
[0, [1], [[2]], [[[3]]]]
"flatten depth must not be negative"
transpose
[[1], [2,3]]
[[1,2],[null,3]]
transpose
[]
[]
ascii_upcase
"useful but not for é"
"USEFUL BUT NOT FOR é"
bsearch(0,1,2,3,4)
[1,2,3]
-1
0
1
2
-4
bsearch({x:1})
[{ "x": 0 },{ "x": 1 },{ "x": 2 }]
1
try ["OK", bsearch(0)] catch ["KO",.]
"aa"
["KO","string (\"aa\") cannot be searched from"]
strftime("%Y-%m-%dT%H:%M:%SZ")
[2015,2,5,23,51,47,4,63]
"2015-03-05T23:51:47Z"
strftime("%A, %B %d, %Y")
1435677542.822351
"Tuesday, June 30, 2015"
strftime("%Y-%m-%dT%H:%M:%SZ")
[2024,2,15]
"2024-03-15T00:00:00Z"
mktime
[2024,8,21]
1726876800
gmtime
1425599507
[2015,2,5,23,51,47,4,63]
gmtime[5]
1425599507.25
47.25
# test invalid tm input
try strftime("%Y-%m-%dT%H:%M:%SZ") catch .
["a",1,2,3,4,5,6,7]
"strftime/1 requires parsed datetime inputs"
try strflocaltime("%Y-%m-%dT%H:%M:%SZ") catch .
["a",1,2,3,4,5,6,7]
"strflocaltime/1 requires parsed datetime inputs"
try mktime catch .
["a",1,2,3,4,5,6,7]
"mktime requires parsed datetime inputs"
# oss-fuzz #67403: non-string argument with number input fails assert
try ["OK", strftime([])] catch ["KO", .]
0
["KO","strftime/1 requires a string format"]
try ["OK", strflocaltime({})] catch ["KO", .]
0
["KO","strflocaltime/1 requires a string format"]
[strptime("%Y-%m-%dT%H:%M:%SZ")|(.,mktime)]
"2015-03-05T23:51:47Z"
[[2015,2,5,23,51,47,4,63],1425599507]
# Check day-of-week and day of year computations
# (should trip an assert if this fails)
last(range(365 * 67)|("1970-03-01T01:02:03Z"|strptime("%Y-%m-%dT%H:%M:%SZ")|mktime) + (86400 * .)|strftime("%Y-%m-%dT%H:%M:%SZ")|strptime("%Y-%m-%dT%H:%M:%SZ"))
null
[2037,1,11,1,2,3,3,41]
# module system
import "a" as foo; import "b" as bar; def fooa: foo::a; [fooa, bar::a, bar::b, foo::a]
null
["a","b","c","a"]
import "c" as foo; [foo::a, foo::c]
null
[0,"acmehbah"]
include "c"; [a, c]
null
[0,"acmehbah"]
import "data" as $e; import "data" as $d; [$d[].this,$e[].that,$d::d[].this,$e::e[].that]|join(";")
null
"is a test;is too;is a test;is too"
# Regression test for #2000
import "data" as $a; import "data" as $b; def f: {$a, $b}; f
null
{"a":[{"this":"is a test","that":"is too"}],"b":[{"this":"is a test","that":"is too"}]}
include "shadow1"; e
null
2
include "shadow1"; include "shadow2"; e
null
3
import "shadow1" as f; import "shadow2" as f; import "shadow1" as e; [e::e, f::e]
null
[2,3]
%%FAIL
module (.+1); 0
jq: error: Module metadata must be constant at <top-level>, line 1, column 8:
module (.+1); 0
^^^^^
%%FAIL
module []; 0
jq: error: Module metadata must be an object at <top-level>, line 1, column 8:
module []; 0
^^
%%FAIL
include "a" (.+1); 0
jq: error: Module metadata must be constant at <top-level>, line 1, column 13:
include "a" (.+1); 0
^^^^^
%%FAIL
include "a" []; 0
jq: error: Module metadata must be an object at <top-level>, line 1, column 13:
include "a" []; 0
^^
%%FAIL
include "\ "; 0
jq: error: Invalid escape at line 1, column 4 (while parsing '"\ "') at <top-level>, line 1, column 10:
include "\ "; 0
^^
%%FAIL
include "\(a)"; 0
jq: error: Import path must be constant at <top-level>, line 1, column 9:
include "\(a)"; 0
^^^^^^
modulemeta
"c"
{"whatever":null,"deps":[{"as":"foo","is_data":false,"relpath":"a"},{"search":"./","as":"d","is_data":false,"relpath":"d"},{"search":"./","as":"d2","is_data":false,"relpath":"d"},{"search":"./../lib/jq","as":"e","is_data":false,"relpath":"e"},{"search":"./../lib/jq","as":"f","is_data":false,"relpath":"f"},{"as":"d","is_data":true,"relpath":"data"}],"defs":["a/0","c/0"]}
modulemeta | .deps | length
"c"
6
modulemeta | .defs | length
"c"
2
%%FAIL IGNORE MSG
import "syntaxerror" as e; .
jq: error: syntax error, unexpected ';', expecting end of file at tests/modules/syntaxerror/syntaxerror.jq, line 1, column 4:
wat;
^
%%FAIL
%::wat
jq: error: syntax error, unexpected '%', expecting end of file at <top-level>, line 1, column 1:
%::wat
^
import "test_bind_order" as check; check::check
null
true
try -. catch .
"very-long-long-long-long-string"
"string (\"very-long-long-long-long...\") cannot be negated"
try (.-.) catch .
"very-long-long-long-long-string"
"string (\"very-long-long-long-long...\") and string (\"very-long-long-long-long...\") cannot be subtracted"
"x" * range(0; 12; 2) + "☆" * 8 | try -. catch .
null
"string (\"☆☆☆☆☆☆☆☆\") cannot be negated"
"string (\"xx☆☆☆☆☆☆☆☆\") cannot be negated"
"string (\"xxxx☆☆☆☆☆☆...\") cannot be negated"
"string (\"xxxxxx☆☆☆☆☆☆...\") cannot be negated"
"string (\"xxxxxxxx☆☆☆☆☆...\") cannot be negated"
"string (\"xxxxxxxxxx☆☆☆☆...\") cannot be negated"
try (. + "x") catch . == if have_decnum then "number (12345678901234567890123456...) and string (\"x\") cannot be added" else "number (12345678901234568000000000...) and string (\"x\") cannot be added" end
123456789012345678901234567890
true
join(",")
["1",2,true,false,3.4]
"1,2,true,false,3.4"
.[] | join(",")
[[], [null], [null,null], [null,null,null]]
""
""
","
",,"
.[] | join(",")
[["a",null], [null,"a"]]
"a,"
",a"
try join(",") catch .
["1","2",{"a":{"b":{"c":33}}}]
"string (\"1,2,\") and object ({\"a\":{\"b\":{\"c\":33}}}) cannot be added"
try join(",") catch .
["1","2",[3,4,5]]
"string (\"1,2,\") and array ([3,4,5]) cannot be added"
{if:0,and:1,or:2,then:3,else:4,elif:5,end:6,as:7,def:8,reduce:9,foreach:10,try:11,catch:12,label:13,import:14,include:15,module:16}
null
{"if":0,"and":1,"or":2,"then":3,"else":4,"elif":5,"end":6,"as":7,"def":8,"reduce":9,"foreach":10,"try":11,"catch":12,"label":13,"import":14,"include":15,"module":16}
try (1/.) catch .
0
"number (1) and number (0) cannot be divided because the divisor is zero"
try (1/0) catch .
0
"number (1) and number (0) cannot be divided because the divisor is zero"
try (0/0) catch .
0
"number (0) and number (0) cannot be divided because the divisor is zero"
try (1%.) catch .
0
"number (1) and number (0) cannot be divided (remainder) because the divisor is zero"
try (1%0) catch .
0
"number (1) and number (0) cannot be divided (remainder) because the divisor is zero"
# Basic numbers tests: integers, powers of two
[range(-52;52;1)] as $powers | [$powers[]|pow(2;.)|log2|round] == $powers
null
true
[range(-99/2;99/2;1)] as $orig | [$orig[]|pow(2;.)|log2] as $back | ($orig|keys)[]|. as $k | (($orig|.[$k])-($back|.[$k]))|if . < 0 then . * -1 else . end|select(.>.00005)
null
%%FAIL
{
jq: error: syntax error, unexpected end of file at <top-level>, line 1, column 1:
{
^
%%FAIL
}
jq: error: syntax error, unexpected INVALID_CHARACTER, expecting end of file at <top-level>, line 1, column 1:
}
^
(.[{}] = 0)?
null
INDEX(range(5)|[., "foo\(.)"]; .[0])
null
{"0":[0,"foo0"],"1":[1,"foo1"],"2":[2,"foo2"],"3":[3,"foo3"],"4":[4,"foo4"]}
JOIN({"0":[0,"abc"],"1":[1,"bcd"],"2":[2,"def"],"3":[3,"efg"],"4":[4,"fgh"]}; .[0]|tostring)
[[5,"foo"],[3,"bar"],[1,"foobar"]]
[[[5,"foo"],null],[[3,"bar"],[3,"efg"]],[[1,"foobar"],[1,"bcd"]]]
range(5;10)|IN(range(10))
null
true
true
true
true
true
range(5;13)|IN(range(0;10;3))
null
false
true
false
false
true
false
false
false
range(10;12)|IN(range(10))
null
false
false
IN(range(10;20); range(10))
null
false
IN(range(5;20); range(10))
null
true
# Regression test for #1347
(.a as $x | .b) = "b"
{"a":null,"b":null}
{"a":null,"b":"b"}
# Regression test for #1368
(.. | select(type == "object" and has("b") and (.b | type) == "array")|.b) |= .[0]
{"a": {"b": [1, {"b": 3}]}}
{"a": {"b": 1}}
isempty(empty)
null
true
isempty(range(3))
null
false
isempty(1,error("foo"))
null
false
# Regression test for #1815
index("")
""
null
# check that dead code removal occurs after builtin it generation
builtins|length > 10
null
true
"-1"|IN(builtins[] / "/"|.[1])
null
false
all(builtins[] / "/"; .[1]|tonumber >= 0)
null
true
builtins|any(.[:1] == "_")
null
false
## Test ability to use keywords (uncomment after eval is pushed)
#(.[] as $kw | "\"{\($kw)} as $\($kw) | $\($kw) | {$\($kw)} | {\($kw):.\($kw)}\""|eval|empty),null
#["as","def","module","import","include","if","then","else","elif","end","reduce","foreach","and","or","try","catch","label","break","__loc__"]
#null
#
#(.[] as $kw | "\"def f($\($kw)): $\($kw); f(.)\""|eval|empty),null
#["as","def","module","import","include","if","then","else","elif","end","reduce","foreach","and","or","try","catch","label","break","__loc__"]
#null
#
# Tests to cover the new toliteral number functionality
# For an example see #1652 and other linked issues
#
# We are backward and sanity compatible
map(. == 1)
[1, 1.0, 1.000, 100e-2, 1e+0, 0.0001e4]
[true, true, true, true, true, true]
# When no arithmetic is involved jq should preserve the literal value
.[0] | tostring | . == if have_decnum then "13911860366432393" else "13911860366432392" end
[13911860366432393]
true
.x | tojson | . == if have_decnum then "13911860366432393" else "13911860366432392" end
{"x":13911860366432393}
true
(13911860366432393 == 13911860366432392) | . == if have_decnum then false else true end
null
true
# Applying arithmetic to the value will truncate the result to double
. - 10
13911860366432393
13911860366432382
.[0] - 10
[13911860366432393]
13911860366432382
.x - 10
{"x":13911860366432393}
13911860366432382
# Unary negation preserves numerical precision
-. | tojson == if have_decnum then "-13911860366432393" else "-13911860366432392" end
13911860366432393
true
-. | tojson == if have_decnum then "0.12345678901234567890123456789" else "0.12345678901234568" end
-0.12345678901234567890123456789
true
[1E+1000,-1E+1000 | tojson] == if have_decnum then ["1E+1000","-1E+1000"] else ["1.7976931348623157e+308","-1.7976931348623157e+308"] end
null
true
. |= try . catch .
1
1
# decnum to double conversion
.[] as $n | $n+0 | [., tostring, . == $n]
[-9007199254740993, -9007199254740992, 9007199254740992, 9007199254740993, 13911860366432393]
[-9007199254740992,"-9007199254740992",true]
[-9007199254740992,"-9007199254740992",true]
[9007199254740992,"9007199254740992",true]
[9007199254740992,"9007199254740992",true]
[13911860366432392,"13911860366432392",true]
# abs, fabs, length
abs
"abc"
"abc"
map(abs)
[-0, 0, -10, -1.1]
[0,0,10,1.1]
map(fabs)
[-0, 0, -10, -1.1]
[0,0,10,1.1]
map(abs == length) | unique
[-10, -1.1, -1e-1, 1000000000000000002]
[true]
# The following is NOT prescriptive:
map(abs)
[0.1,1000000000000000002]
[1e-1, 1000000000000000002]
[1E+1000,-1E+1000 | abs | tojson] | unique == if have_decnum then ["1E+1000"] else ["1.7976931348623157e+308"] end
null
true
[1E+1000,-1E+1000 | length | tojson] | unique == if have_decnum then ["1E+1000"] else ["1.7976931348623157e+308"] end
null
true
# Using a keyword as variable/label name
123 as $label | $label
null
123
[ label $if | range(10) | ., (select(. == 5) | break $if) ]
null
[0,1,2,3,4,5]
reduce .[] as $then (4 as $else | $else; . as $elif | . + $then * $elif)
[1,2,3]
96
1 as $foreach | 2 as $and | 3 as $or | { $foreach, $and, $or, a }
{"a":4,"b":5}
{"foreach":1,"and":2,"or":3,"a":4}
[ foreach .[] as $try (1 as $catch | $catch - 1; . + $try; .) ]
[10,9,8,7]
[10,19,27,34]
# Object construction
{ a, $__loc__, c }
{"a":[1,2,3],"b":"foo","c":{"hi":"hey"}}
{"a":[1,2,3],"__loc__":{"file":"<top-level>","line":1},"c":{"hi":"hey"}}
1 as $x | "2" as $y | "3" as $z | { $x, as, $y: 4, ($z): 5, if: 6, foo: 7 }
{"as":8}
{"x":1,"as":8,"2":4,"3":5,"if":6,"foo":7}
# nan is parsed as a valid NaN value from JSON
fromjson | isnan
"nan"
true
tojson | fromjson
{"a":nan}
{"a":null}
# NaN with payload is not parsed
.[] | try (fromjson | isnan) catch .
["NaN","-NaN","NaN1","NaN10","NaN100","NaN1000","NaN10000","NaN100000"]
true
true
"Invalid numeric literal at EOF at line 1, column 4 (while parsing 'NaN1')"
"Invalid numeric literal at EOF at line 1, column 5 (while parsing 'NaN10')"
"Invalid numeric literal at EOF at line 1, column 6 (while parsing 'NaN100')"
"Invalid numeric literal at EOF at line 1, column 7 (while parsing 'NaN1000')"
"Invalid numeric literal at EOF at line 1, column 8 (while parsing 'NaN10000')"
"Invalid numeric literal at EOF at line 1, column 9 (while parsing 'NaN100000')"
# calling input/0, or debug/0 in a test doesn't crash jq
try input catch .
null
"break"
debug
1
1
# try/catch catches more than it should #1859
"foo" | try ((try . catch "caught too much") | error) catch "caught just right"
null
"caught just right"
.[]|(try (if .=="hi" then . else error end) catch empty) | "\(.) there!"
["hi","ho"]
"hi there!"
try (["hi","ho"]|.[]|(try . catch (if .=="ho" then "BROKEN"|error else empty end)) | if .=="ho" then error else "\(.) there!" end) catch "caught outside \(.)"
null
"hi there!"
"caught outside ho"
.[]|(try . catch (if .=="ho" then "BROKEN"|error else empty end)) | if .=="ho" then error else "\(.) there!" end
["hi","ho"]
"hi there!"
try (try error catch "inner catch \(.)") catch "outer catch \(.)"
"foo"
"inner catch foo"
try ((try error catch "inner catch \(.)")|error) catch "outer catch \(.)"
"foo"
"outer catch inner catch foo"
# Also #1859, but from #1885
first(.?,.?)
null
null
# Also #1859, but from #2140
{foo: "bar"} | .foo |= .?
null
{"foo": "bar"}
# Also #1859, but from #2220
. |= try 2
1
2
. |= try 2 catch 3
1
2
.[] |= try tonumber
["1", "2a", "3", " 4", "5 ", "6.7", ".89", "-876", "+5.43", 21]
[1, 3, 6.7, 0.89, -876, 5.43, 21]
# Also 1859, but from 2073
any(keys[]|tostring?;true)
{"a":"1","b":"2","c":"3"}
true
# explode/implode
# test replacement character (65533) for outside codepoint range and 0xd800 (55296) - 0xdfff (57343) utf16 surrogate pair range
# 1.1 and 1.9 to test round down of non-ints
implode|explode
[-1,0,1,2,3,1114111,1114112,55295,55296,57343,57344,1.1,1.9]
[65533,0,1,2,3,1114111,65533,55295,65533,65533,57344,1,1]
map(try implode catch .)
[123,["a"],[nan]]
["implode input must be an array","string (\"a\") can't be imploded, unicode codepoint needs to be numeric","number (null) can't be imploded, unicode codepoint needs to be numeric"]
try 0[implode] catch .
[]
"Cannot index number with string (\"\")"
# walk
walk(.)
{"x":0}
{"x":0}
walk(1)
{"x":0}
1
# The following is a regression test, not a requirement:
[walk(.,1)]
{"x":0}
[{"x":0},1]
# Issue #2584
walk(select(IN({}, []) | not))
{"a":1,"b":[]}
{"a":1}
# #2815
[range(10)] | .[1.2:3.5]
null
[1,2,3]
[range(10)] | .[1.5:3.5]
null
[1,2,3]
[range(10)] | .[1.7:3.5]
null
[1,2,3]
[range(10)] | .[1.7:4294967295]
null
[1,2,3,4,5,6,7,8,9]
[range(10)] | .[1.7:-4294967296]
null
[]
[[range(10)] | .[1.1,1.5,1.7]]
null
[1,1,1]
[range(5)] | .[1.1] = 5
null
[0,5,2,3,4]
[range(3)] | .[nan:1]
null
[0]
[range(3)] | .[1:nan]
null
[1,2]
[range(3)] | .[nan]
null
null
try ([range(3)] | .[nan] = 9) catch .
null
"Cannot set array element at NaN index"
try ("foobar" | .[1.5:3.5] = "xyz") catch .
null
"Cannot update string slices"
try ([range(10)] | .[1.5:3.5] = ["xyz"]) catch .
null
[0,"xyz",4,5,6,7,8,9]
try ("foobar" | .[1.5]) catch .
null
"Cannot index string with number (1.5)"
# setpath/2 does not leak the input after an invalid get #2970
try ["ok", setpath([1]; 1)] catch ["ko", .]
{"hi":"hello"}
["ko","Cannot index object with number (1)"]
try fromjson catch .
"{'a': 123}"
"Invalid string literal; expected \", but got ' at line 1, column 5 (while parsing '{'a': 123}')"
# ltrimstr/1 rtrimstr/1 don't leak on invalid input #2977
try ltrimstr(1) catch "x", try rtrimstr(1) catch "x" | "ok"
"hi"
"ok"
"ok"
try ltrimstr("x") catch "x", try rtrimstr("x") catch "x" | "ok"
{"hey":[]}
"ok"
"ok"
# ltrimstr/1 and rtrimstr/1 return an error for non-strings. #2969
.[] as [$x, $y] | try ["ok", ($x | ltrimstr($y))] catch ["ko", .]
[["hi",1],[1,"hi"],["hi","hi"],[1,1]]
["ko","startswith() requires string inputs"]
["ko","startswith() requires string inputs"]
["ok",""]
["ko","startswith() requires string inputs"]
.[] as [$x, $y] | try ["ok", ($x | rtrimstr($y))] catch ["ko", .]
[["hi",1],[1,"hi"],["hi","hi"],[1,1]]
["ko","endswith() requires string inputs"]
["ko","endswith() requires string inputs"]
["ok",""]
["ko","endswith() requires string inputs"]
# oss-fuzz #66061: setpath/2 leaks when indexing array with array
try ["OK", setpath([[1]]; 1)] catch ["KO", .]
[]
["KO","Cannot update field at array index of array"]
# regression test for #3227
foreach .[] as $x (0, 1; . + $x)
[1, 2]
1
3
2
4
# regression test for CVE-2025-49014 (use of fmt after free)
# tests with both empty string literal and empty string created by function
# as they seems to behave reference wise differently.
strflocaltime("" | ., @uri)
0
""
""
# regression tests for #3413
# upper range bounds should be in sync with the constants defined at
# src/jv_parse.c:#define MAX_PARSING_DEPTH (N)
# src/jv_print.c:#define MAX_PRINT_DEPTH (N)
# (N-1)
reduce range(9999) as $_ ([];[.]) | tojson | fromjson | flatten
null
[]
# (N)
reduce range(10000) as $_ ([];[.]) | tojson | try (fromjson) catch . | (contains("<skipped: too deep>") | not) and contains("Exceeds depth limit for parsing")
null
true
# (N+1)
reduce range(10001) as $_ ([];[.]) | tojson | contains("<skipped: too deep>")
null
true
# regression test for CVE-2026-33947
setpath([range(10000) | 0]; 0) | flatten
null
[0]
try setpath([range(10001) | 0]; 0) catch .
null
"Path too deep"
getpath([range(10000) | 0])
null
null
try getpath([range(10001) | 0]) catch .
null
"Path too deep"
delpaths([[range(10000) | 0]])
null
null
try delpaths([[range(10001) | 0]]) catch .
null
"Path too deep"
# regression test for CVE-2026-40612
reduce range(10000) as $_ ([]; [.]) | contains([[]])
null
true
try (reduce range(10001) as $_ ([]; [.]) as $x | $x | contains($x)) catch .
null
"Containment check too deep"
# regression test for CVE-2026-43896
reduce range(10000) as $_ ({}; {a: .}) as $x | $x * $x | length
null
1
try (reduce range(10001) as $_ ({}; {a: .}) as $x | $x * $x) catch .
null
"Object merge too deep"
# regression test for deep structural equality recursion
try ((reduce range(10001) as $_ ([]; [.])) as $x | (reduce range(10001) as $_ ([]; [.])) as $y | $x == $y) catch .
null
"Equality check too deep"
# regression tests for deep ordering comparisons
try ((reduce range(10001) as $_ ([]; [.])) as $x | [$x, $x] | sort) catch .
null
"Comparison too deep"
try ((reduce range(10001) as $_ ([]; [.])) as $x | [$x, $x] | unique) catch .
null
"Comparison too deep"
try ((reduce range(10001) as $_ ({}; {a: .})) as $x | [$x, $x] | sort) catch .
null
"Comparison too deep"
try ((reduce range(10001) as $_ ({}; {a: .})) as $x | [$x, $x] | unique) catch .
null
"Comparison too deep"</pblock>
<pblock filename="ARCHITECTURE_compact.md" role="context" path="/mnt/c/Users/barlo/projects/drydock/uat/jq/runs/20260822.044627/workspace/targets/jq/blueprint/ARCHITECTURE_compact.md">
<!-- Compacted from ARCHITECTURE.md sha256=f0e07d8104b7c23be2772201e83dfaf382f4c295492da7b8ec72b2a914af633b on 2026-08-22 by drydock build agent -->
- Executable: `./jq -c '<program>'`; compact JSON lines on stdout.
- Exit codes: `0` success, `3` compile failure, `5` runtime failure; diagnostics on stderr.
- Standard-library Python only; no external jq, dependencies, networking, or persistence.
- Modules: CLI, lexer, parser/AST, evaluator streams, runtime values, builtins, paths/assignment, diagnostics.
- Preserve generator ordering, multiplicity, backtracking, immutable transformations, and partial output.</pblock>
IMPLEMENTS - Authoritative Step Specifications
<pblock label="Implementation recency anchor" kind="section"> The files in this section are the load-bearing specifications for this build block. Build these files exactly. Treat earlier sections as constraints and context.
</pblock>
<pblock filename="FEATURE-PARSE-003.md" role="implements" path="/mnt/c/Users/barlo/projects/drydock/uat/jq/runs/20260822.044627/workspace/targets/jq/blueprint/FEATURE-PARSE-003.md" guidance="Feature Specification">
# FEATURE: jq Filter Expression Grammar
| Field | Value |
|-------------|-------|
| Version | 20260822 V1 |
| Description | Parse jq expressions with precedence, composition, accessors, collections, and operators. |
| Depends On | ARCHITECTURE.md, FEATURE-PARSE-002.md |
| Provides | expression AST, precedence, pipes, commas, indexing, slicing, arrays, objects, operators |
| Consumes | jq lexer and token stream |
## Scope
Implement the expression grammar and AST for pipes, commas, parentheses, unary operators, binary operators, field and index access, iteration, slices, arrays, objects, optional expressions, and assignment syntax. Operator precedence and associativity must follow `sources/parser.y`.
## Programmatic Acceptance
=== AC parse-003-conformance ===
Intent: The executable passes every selected corpus case exercising expression punctuation, accessors, collections, and operators.
Suite: scoped
Requires: executable=python3; scope=test
import json
import os
import subprocess
import sys
selector = r"\.|\[|\{|\+|\-|\*|/|%"
result = subprocess.run(
[sys.executable, "sources/run_conformance.py", "--select", selector, "--json"],
capture_output=True,
text=True,
env={**os.environ, "JQ": f"{os.getcwd()}/jq"},
)
print(result.stdout)
print(result.stderr, file=sys.stderr)
report = json.loads(result.stdout)
summary = report["summary"]
assert sum(summary.values()) > 0
assert summary["fail"] == 0 and summary["error"] == 0
assert result.returncode == 0
=== END AC parse-003-conformance ===
## User Acceptance
- None.
## Guardrails
- Preserve parser precedence and generator-producing expression structure.
- Reject malformed delimiters and invalid object-key expressions at compile time.
- Do not modify the supplied grammar or corpus.</pblock>
<pblock filename="FEATURE-PARSE-004.md" role="implements" path="/mnt/c/Users/barlo/projects/drydock/uat/jq/runs/20260822.044627/workspace/targets/jq/blueprint/FEATURE-PARSE-004.md" guidance="Feature Specification">
# FEATURE: jq Declarations and Control Syntax
| Field | Value |
|-------------|-------|
| Version | 20260822 V1 |
| Description | Parse jq declarations, bindings, control constructs, and module grammar. |
| Depends On | ARCHITECTURE.md, FEATURE-PARSE-003.md |
| Provides | declarations, conditionals, try/catch, reductions, foreach, labels, bindings, modules grammar |
| Consumes | expression AST |
## Scope
Implement parsing for `def`, function parameters, `as` bindings, destructuring patterns, `if`/`elif`/`else`, `try`/`catch`, `reduce`, `foreach`, `label`/`break`, and module directives. Module syntax must be validated sufficiently to reject invalid metadata, interpolation in import paths, invalid escapes, and unsupported tokens without loading module files.
## Programmatic Acceptance
=== AC parse-004-conformance ===
Intent: The executable passes every selected corpus case covering declarations, control syntax, bindings, reductions, labels, and required compile failures.
Suite: scoped
Requires: executable=python3; scope=test
import json
import os
import subprocess
import sys
selector = r"%%FAIL|if |try |reduce |foreach |def | as |label |module|include"
result = subprocess.run(
[sys.executable, "sources/run_conformance.py", "--select", selector, "--json"],
capture_output=True,
text=True,
env={**os.environ, "JQ": f"{os.getcwd()}/jq"},
)
print(result.stdout)
print(result.stderr, file=sys.stderr)
report = json.loads(result.stdout)
summary = report["summary"]
assert sum(summary.values()) > 0
assert summary["fail"] == 0 and summary["error"] == 0
assert result.returncode == 0
=== END AC parse-004-conformance ===
## User Acceptance
- None.
## Guardrails
- Compile failures must exit 3 and runtime failures must remain distinct.
- Do not resolve imports or read excluded module fixtures in this story.
- Preserve lexical scope and syntax needed by later evaluator stories.</pblock>
<pblock label="Build instructions" kind="instructions">
Build instructions for this block
Implement jq filter expression grammar. (PARSE-003)
Build the expression parser and AST for composition, precedence, indexing, slicing, arrays, objects, unary operators, arithmetic, comparison, optional access, and assignment syntax.
Implement declarations and control syntax parsing. (PARSE-004)
Parse defs, bindings, destructuring, conditionals, try/catch, reduce, foreach, labels, breaks, import/include/module grammar, and required compile-time rejection cases without loading excluded module fixtures.
</pblock>
Agent Task
You are a Drydock build agent implementing exactly one build step of a larger plan. The build job block below names the target, the build working directory, and the step. Everything you need is stacked into this prompt under role headings:
compass— the Target's COMPASS.md orientation.implements— the Typed Specification files this step builds. These are
authoritative; implement them exactly.
context— read-only support specifications. Do not reimplement them.stack— enterprise stack and technology rules. Honor them.rules— governance and branding rules. Honor them.
Operating contract:
- Follow the write authorization and protected paths in the stacked
COMPASS.mdexactly.
That persisted guardrail is the sole authority for paths this build may modify.
- Start by inspecting the build working directory. Preserve existing application
files unless this step's specifications require a change. Its sources/ subdirectory holds staged build assets — imported test corpora, conformance harnesses, and fixtures — placed there for you. They are read-only inputs: run them, import them, and write code against them, but never create, rewrite, trim, regenerate, or substitute one, even to make a check pass. A step that modifies a staged asset fails and the asset is restored. If an asset you expect is absent, report that; do not author a replacement.
- Implement only this step. Use
context,stack, andrulesas constraints,
not as additional work to perform.
- Follow the stack and rules for languages, structure, naming, and branding.
- The programmatic acceptance assertions in the
implementsspecifications are
this step's Definition of Done — human-owned, declared before the build, and fixed. Build the story and, in this same step, write the deterministic tests that prove each declared assertion, as a TDD master would; add finer tests for coverage. Every test you write follows the same rule the acceptance assertions do: act on the system, read the state back, compare to expected. The oracle is a return value, parsed JSON, a status code, a stored row, file contents read back, or an exit status — never a substring of captured stdout or stderr, a test-runner tally, or a log line. Write tests in the project's own language using that language's libraries; an in-language HTTP client yields a status code and a parsed body, where curl yields text to scrape. Round-trip anything that stores state: act, then read back through the public interface. Assert declared failure signals on negative paths, never message wording. You may add tests but must never remove, soften, or weaken a declared acceptance assertion. A Suite: full conformance check gates on the entire imported test suite: the step is done only when it passes in full, never on a representative subset — reproduce the standard exactly rather than wrapping a third-party library that approximates it. For a suite, the runner's exit status is the verdict and the whole verdict: print its captured output for diagnosis, never assert on the text of its summary. When an assertion is a static or filesystem scan (import boundary, "X never appears outside Y," grep/AST gate), honor the scope the specification states and never widen it: scan production source only, exclude .venv/, site-packages, and vendored or generated code, and do not flag test doubles or fixtures that use the guarded dependency. Run every declared acceptance assertion before returning. For a conformance suite, use its section or example filters to diagnose coherent root-cause clusters, but rerun the full declared scope before reporting the result. Treat failing examples as a work queue for fixing general behavior; never add example-specific exceptions.
- Grow the project's own test suite as you write the code, and treat it as the project's
real coverage. The acceptance assertions in implements are gates: few, fixed, and written before any code existed, so every expectation in them is a prediction. The tests you write are written beside the finished code, so their expected values are observed rather than predicted — which is why exhaustive coverage belongs here and not there. Extend the suite in the project's established location and runner, keep it runnable by the project's declared test command, and leave it green when you return. Cover, at minimum: every public entry point and every verb it declares, including declared error paths; the boundaries — empty, exactly one, many, absent optional fields, declared maxima; declared idempotence, applied twice; one behavior per test, named for the behavior; and isolation — each test arranges its own data, with a fresh store or explicit teardown, so a run leaves no residue behind in the build directory. Where a staged authoritative suite already covers a surface, that suite is the coverage: run it, and do not restate its cases. Run it only through the invocation this step's acceptance criteria declare. A criterion marked Suite: scoped names the whole of this step's obligation to that suite; running the suite's unscoped entry point instead is not extra rigor, it is a different step's gate executed early. A partial capability fails most of an authoritative corpus by construction and its unimplemented cases exhaust the runner's per-case timeout rather than returning, so the unscoped run costs the most where it teaches the least, and interrupting it forfeits the step. If no criterion in this step invokes the staged suite, do not invoke it. Report the pass/fail counts of the invocations you did run in your SUMMARY so a reader can see coverage moving across steps.
- Treat
User Acceptanceentries as review evidence requirements. Implement
the supporting behavior, but do not claim to have performed human judgment.
- The
implementssection is authoritative and intentionally stacked late in
the prompt as the recency anchor. Build that WHAT exactly; do not substitute generic framework defaults.
- Before adding or installing Python dependencies, verify each package name
against the declared registry. Do not invent package names. If a needed package cannot be verified or appears newly published, fail explicitly instead of installing it.
- Use the stack's required package manager workflow for dependency changes.
When the stack requires uv, update manifests through uv conventions rather than bare pip install.
- Do not claim success unless you actually created or modified project files in
the build working directory. If you cannot write files or cannot complete the step, report failure explicitly.
- Do not run
git add,git commit, create branches, create tags, rewrite
history, or otherwise mutate Git history. Drydock owns the final build directory commit after you return.
- End your response with this exact closing structure:
RESULT: SUCCESS | FAILED
FILES CHANGED:
- relative/path
SUMMARY:
<brief reviewable summary>
BLOCKERS:
- <only if any>Before RESULT, you may emit one optional JSON payload when implementation required a bounded choice not already settled by the owning specification. This records what you did; it does not ask permission, create a questionnaire, or excuse incomplete work:
<blueprint-decisions>
[{"spec":"FEATURE-Example.md","severity":"Material","subject":"Chosen behavior","decision":"Options A and B were available. I implemented B because ... Is that acceptable, or should this change on replan?"}]
</blueprint-decisions>Name only a specification implemented by this build block. Use Low or Material; Build never emits a Blocking decision. Omit the payload when no implementation decision was necessary.
FILES CHANGEDmust list only files actually written in the build working
directory. If no files were written, use RESULT: FAILED.
- On
RESULT: FAILED, append two additional lines so the failure is actionable
without opening logs. FAILURE_SUMMARY is one line naming the cause; FAILURE_DETAIL states what happened, why, and what to change before a rerun. Name concrete conditions when they apply: token or context limit exceeded, could not execute commands in this environment, a required input was missing, or a specific tool or command failed.
FAILURE_SUMMARY: <one line naming the cause>
FAILURE_DETAIL: <what happened, why, and what to change before rerunning>- When a declared acceptance criterion cannot pass no matter how the code is written,
say so with this exact token. You may not edit the criterion — it is staged and restored before grading:
AC_BROKEN: <check-id>[, <check-id>]This is a report, not a verdict, and it stops nothing. A criterion reaches you only when its expected value is one its author could not have invented — a status code, a staged suite's exit status, a value the criterion itself supplied as input — so your claim that the criterion rather than the code is at fault is the less likely explanation, and the budget is spent as it would be for any other failure. A criterion whose expectation was hand-typed already settles DISPUTED on its own, without you naming it. Emit the token only after running the criterion and confirming the underlying command succeeded while the assertion still failed. Name the affected check ids, emit it alongside your normal RESULT line, state the reasoning in FAILURE_DETAIL, and emit it even when RESULT: SUCCESS. Do not use it for a criterion you merely failed to satisfy.
Repair Feedback
Repair pass
A previous build pass wrote partial work into BUILD_DIRECTORY and did not pass every deterministic acceptance check. Iterate the existing files in BUILD_DIRECTORY to make the checks below pass. Do not restart from scratch, do not weaken or remove any declared acceptance assertion, and keep every check that already passes green.
The diagnostic excerpts below are truncated. Before editing, rerun each failing acceptance assertion from its authoritative specification and inspect the complete failure output. For a conformance suite, use its section or example filters to diagnose coherent root-cause clusters, then rerun the full declared scope. Fix the general parser or renderer behavior; do not add example-specific exceptions. Keep working through failing examples while the deterministic tally is improving.
If a check below cannot pass however the code is written — the command it runs succeeds and the assertion still fails — do not keep repairing. Report it with AC_BROKEN: <check-id> and explain the contradiction in FAILURE_DETAIL. That ends the repair budget and sends the criterion back for repair in the Blueprint, which is where a broken assertion has to be fixed.
Still failing
- parse-003-conformance (story PARSE-003): The executable passes every selected corpus case exercising expression punctuation, accessors, collections, and operators.
assertion: assert summary["fail"] == 0 and summary["error"] == 0 → AssertionError output: "\"Comparison too deep\"" ], "actual": [] } ] }
- parse-004-conformance (story PARSE-004): The executable passes every selected corpus case covering declarations, control syntax, bindings, reductions, labels, and required compile failures.
assertion: assert summary["fail"] == 0 and summary["error"] == 0 → AssertionError output: "\"Comparison too deep\"" ], "actual": [] } ] }
Previous agent note
- Scoped conformance criteria still report failures and timeouts.
Parser improvements resolved several grouping and generator bugs, but the existing evaluator lacks enough jq semantics to satisfy both authoritative conformance slices.
Files written so far
- jq_interpreter/evaluator.py
- jq_interpreter/parser.py