Skill anatomy · shared/ · 2026-09-08

AAT coverage skill — which files run, and when

One invocation of /coverage --mode aat walks ten stages. Every stage is plain bash orchestrated by SKILL.md; only two stages hand work to an LLM sub-agent. Everything a stage reads comes from shared/, resolved once at S0 into $SHARED.

files from shared/ touched by one AAT run

files in shared/ total — 16 belong to other modes or post-run rollups

stages, S0 → S5, with three cache/skip gates before any analysis

2 / 10

stages use an LLM; the other eight are deterministic

◆ Python module — computes or validates ▶ Shell script — gate or side effect ⬢ JSON Schema / policy data ▤ Template or reference doc

Call order

Resolve the target deterministic

Locates the service repo, the AAT suite and the report directory; pins one timestamp and one discovery model for the whole run; hard-stops if any helper is missing rather than failing stage by stage.

resolve-target.sh
service-domains.json
model.env
mode-capabilities.json
coverage.py preflight
ledger.schema.json preflight
claims.schema.json preflight

produces /tmp/coverage-<ns>-env.sh — the state file every later block sources

S0.1

GraphQL / Node.js early exit gate

Apollo subgraphs mis-score near 0% because the reconciler cannot attribute POST /graphql scenarios to label-based resolver triggers. Those services are named in data and stopped here instead of producing a misleading report.

graphql-guard.sh
graphql-skipped-services.json

exits the run, or falls through

S0.2

Cache lookup gate

Looks for a cached ledger and claims for this exact commit pair, then proves they are usable with a trial reconcile to /dev/null. A stale or fabricated document is a cache miss here rather than a crash three stages later.

coverage.py dry run
ledger.schema.json
claims.schema.json

on a hit, skips S1 – S3 entirely

S0.3

Skip if the report already exists gate

Checks the report directory for this service/commit pair. No shared helper involved — pure git and filesystem, so a re-run of an unchanged service costs nothing.

exits early, or falls through to analysis

Skeleton deterministic

Extracts the trigger set with no LLM: Spring and .NET route annotations (including mappings declared on an implemented interface), minimal-API registrations, @KafkaListener/IConsumer listeners with SpEL topic resolution, and scheduled jobs. Also inventories every feature file and its scenario count so S3 can be batched and checked for truncation.

skeleton.py
ledger.schema.json

produces trigger skeleton + feature-file inventory with per-file scenario counts

S1.5

Incremental plan deterministic

Runs only when a full S2 would otherwise be needed. Diffs the service commit against the cached ledger's commit and reuses every effect from an untouched file, so discovery re-analyses changed code only.

incremental.py
ledger_merge.py
coverage.py
ledger.schema.json

narrows S2 to the changed files

Service discovery → the ledger LLM sub-agents

Sub-agents walk the service source and record every observable effect behind each trigger — HTTP responses, published events, persisted rows, outbound calls. The pinned model does this work; the denominator policy that decides which effects are even coverable lives in data, not in the prompt.

ledger.schema.json
mode-capabilities.json
ledger_merge.py
coverage.py dry run

produces ledger.json — the denominator

Test discovery → the claims LLM sub-agents

Batched by feature file, using S1's scenario counts to catch a truncated return. Each claim names the ledger effect a scenario drives and the citation that proves it; ignored and disabled scenarios are detected by documented rules rather than guessed at.

claims.schema.json
ignored-scenario-detection.md
claims_merge.py
coverage.py dry run

produces claims.json — the numerator

Reconcile deterministic

The only place coverage arithmetic happens. Claims are matched against ledger effects, rejected if they cite the wrong branch or a value the scenario never asserted, and the denominator is trimmed by the mode's observation channels. The verifier also records which model actually ran, versus the pin.

verify-model.py
model.env
coverage.py
mode-capabilities.json

produces scored result — headline %, per-trigger breakdown, rejected claims

Render, then commit deterministic

Composes the report paths once, writes machine-readable metrics, renders Markdown to a standalone HTML file with pandoc, injects the chart, then commits and pushes to the coverage repo under reports/<domain>/<service>/.

write-metrics.sh
metrics.schema.json
display-name.py
display-names.json
template-standalone.html
inject-chart.sh

produces <report>.md + <report>.html + metrics.json, committed and pushed

Files reused across stages

Fifteen of the 21 files are read at exactly one stage. These six are the load-bearing ones — coverage.py is consulted at six of the ten stages, because every gate in the run is really "would the reconciler accept this?".

coverage.py 6 stages

ledger.schema.json 5 stages

claims.schema.json 3 stages

mode-capabilities.json 3 stages

model.env 2 stages

ledger_merge.py 2 stages

0 · stages that read the file · 6

In the directory, but not in an AAT run

Contract mode

6 files · used by /contract-coverage

contract.py
contract_claims.schema.json
contract_expectations.schema.json
contract-taxonomy.json
contract-testing-taxonomy.md
spec_expectations.py

Rollups & site

7 files · run after many reports exist

build-site.py
aat-gap-report.py
sync-roster.py
domain-coverage.sh
service-coverage.sh
coverage-summary.sh
domain-managers.json

Maintenance

3 files · one-off or manual

ledger_diff.py
relabel-metrics-schema.py
VERSION

Around all of it: SKILL.md is a single 1,037-line orchestrator (~105 KB) holding every bash block, the report template and the prompts; shared/tests/ adds 13 pytest modules and 31 reference-fixture files that pin the reconciler's behaviour, and scripts/bulk-aat.sh drives the skill across a service list. Neither the tests nor the batch driver are read during a run.