Linting non-markdown facts with external sources

Gate facts that live outside markdown – rows in JIRA, pages in Notion, clauses traced in a Prolog knowledge base – with the same rules, dependency graph, and exit codes ctxgrd already applies to your ADRs and PRDs.

What a source is

ctxgrd only reads .md files with frontmatter from disk. A source is how you lint everything else: an executable script that emits document envelopes on stdout. ctxgrd runs it, parses its output, and lints those envelopes alongside your local markdown using the same rules and the same namespace configuration.

Reach for a source when the thing you want to gate cannot be expressed as a markdown file at all – an API-backed system of record, or a fact that exists by construction in some other file format. If the facts already live in .md files, you want a namespace with paths, not a source; see Configuring namespaces.

Prerequisites

• ctxgrd 2.0.0 or later (for the expect_min floor covered below).
• A ctxgrd.toml already at the project root. Run ctxgrd init first if you don’t have one.

1. Lay out the script

A source is a directory under sources/ containing an executable named run. The directory name becomes the source name:

mkdir -p sources/statute
touch sources/statute/run
chmod +x sources/statute/run

2. Emit document envelopes

run writes one JSON object per line to stdout – JSONL, not a JSON array. Each line needs an id, a body, and a location:

{"id": "STATUTE-1", "body": "# Clause 1\n\nTraced to entrepreneur.pl.", "location": "src/entrepreneur.pl#clause-1"}

Here is a real motivating shape: a project encoding a statute in s(CASP) needs to prove that every clause of the law has a citing rule in its knowledge base. The source walks the encoding and emits one envelope per traced clause:

#!/usr/bin/env bash
set -euo pipefail

for clause in $(list_traced_clauses.sh); do
  printf '{"id":"STATUTE-%s","body":"# Clause %s\n\nTraced to entrepreneur.pl.","location":"src/entrepreneur.pl#clause-%s"}\n' \
    "$clause" "$clause" "$clause"
done

The namespace derives from id alone – [STATUTE].paths never enters into it, because source-emitted documents don’t have a path on disk that you control. Configure [STATUTE] rules in ctxgrd.toml exactly as you would for any other namespace.

3. Activate the source

Discovery is automatic; running it is not. A source with no config table never executes:

[sources.statute]

An empty table is valid – the script receives CTXGRD_SOURCE_PARAMS='{}'. Add keys and they become the script’s parameters:

[sources.statute]
regulation = "1290/2002 §6"

4. Test the script directly

Run it standalone before wiring it into ctxgrd, so a malformed line is your problem to catch, not ctxgrd’s:

CTXGRD_SOURCE_NAME=statute CTXGRD_SOURCE_PARAMS='{}' sources/statute/run | jq .

jq . errors loudly on anything that isn’t valid JSON. Once every line parses, run ctxgrd --root . and watch it validate the envelopes with the rules you configured for [STATUTE].

5. Set a floor with expect_min

This is the step worth not skipping. Writing [sources.statute] is a statement that the source is expected to produce something – so ctxgrd holds it to a floor of one document by default, and warns if it comes back under that floor after running to completion:

warning[src.too-few-documents]: source 'statute' emitted 0 of an expected minimum of 1

This exists because a source that exits 0 without writing anything produces a run byte-identical to one where every clause passed – the gate disappears and the summary line says nothing changed. core.min-docs cannot catch this on its own: it counts markdown documents, and a source that emits 3 of 8 expected envelopes still satisfies “at least one.”

Set the floor to the number you actually expect. For the statute encoding above, that’s the clause count – eight clauses in Työttömyysturvalaki §6:

[sources.statute]
expect_min = 8   # one per clause of the statute

Now a partial emit warns by name:

$ ctxgrd --root .
warning[src.too-few-documents]: source 'statute' emitted 3 of an expected minimum of 8
  help: check the `run` script for source 'statute', or lower `expect_min` under [sources.statute] — 0 accepts a source that may legitimately emit nothing
  note: the source exited cleanly, so nothing else reports this — without the floor the run would be indistinguishable from one where every document passed

And a full emit lints clean:

$ ctxgrd --root .
ok: 8 documents · 4 rules · 0 diagnostics

Set expect_min = 0 for a source that may legitimately emit nothing – an optional feed, a system that’s sometimes empty by design. That opts out of the warning entirely:

[sources.optional-feed]
expect_min = 0

expect_min is ctxgrd’s own key. It’s stripped from the parameters before the script runs, so CTXGRD_SOURCE_PARAMS never carries it.

6. Cross-namespace dependencies still work

A source-emitted document participates in the same dependency graph as local files. STATUTE-1 can depends_on a local PRD-001, and core.dep-resolved validates the link exists regardless of which side is on disk and which came from a script.

Troubleshooting

The source never runs. Check for a [sources.<name>] table in ctxgrd.toml – a script on disk with no config entry is inert by design.

error[src.runtime-error] – the script exited non-zero. ctxgrd reports it and keeps linting everything else; fix the script and rerun.

warning[src.too-few-documents] – the script ran to completion but returned fewer envelopes than expect_min. Either the script has a bug (a set -e that aborted an emit loop partway is the usual cause), or the floor is set higher than reality; fix whichever one is wrong.

Next steps

• Writing custom rules – the rule side of the same extension mechanism, for markdown documents.
• Configuring namespaces – paths, rules, and parameters in depth.
• How ctxgrd decides what to lint – why a namespace pointed at non-markdown ingests nothing, and routes you here.