Hold a reworded spec rule until a test that covers it is touched #150

Merged
nectenda-agent merged 1 commit from worktree-spec-delta-gate into main 2026-09-24 15:39:17 +01:00
Collaborator

The conformance join can't see a docs/v1 rule whose meaning changed while an old test still cites its id. CRYPTO-025 (NEC-52, #147) was reworded like that.

  • scripts/agent/spec-delta.mjs compares each rule's whole text before and after. For every reworded rule, a file that cites it with @covers must be among the changed files; rules marked Not testable are exempt. Against real PRs it flags #147 (CRYPTO-025) and passes #144 and #145.
  • gate-local.sh runs it with --check. The merge queue holds anything that slips past, in Review, naming the rule. The skill states the rule.
  • 63 tests; each rule was mutation-checked.

Changelog

NONE

The conformance join can't see a `docs/v1` rule whose meaning changed while an old test still cites its id. CRYPTO-025 (NEC-52, #147) was reworded like that. - `scripts/agent/spec-delta.mjs` compares each rule's **whole text** before and after. For every reworded rule, a file that cites it with `@covers` must be among the changed files; rules marked *Not testable* are exempt. Against real PRs it flags #147 (CRYPTO-025) and passes #144 and #145. - `gate-local.sh` runs it with `--check`. The merge queue holds anything that slips past, in Review, naming the rule. The skill states the rule. - 63 tests; each rule was mutation-checked. ## Changelog NONE
Hold a reworded spec rule until a test that covers it is touched
All checks were successful
Release note / release-note (pull_request) Successful in 13s
CI / e2e (pull_request) Successful in 4m12s
CI / build (pull_request) Successful in 4m53s
CI / promote (pull_request) Has been skipped
CI / e2e (push) Successful in 4m30s
CI / build (push) Successful in 5m4s
CI / promote (push) Successful in 35s
5af2aa227f
check-conformance.mjs joins rule ids to the tests that cite them, and
fails on a rule nobody cites. It cannot see a rule whose meaning changed
while an old test still cites its id: the join stays green and the test
defends a sentence that no longer exists. NEC-52 (PR #147) reworded
CRYPTO-025, "nothing derived from the passphrase is sent", to "no
authentication value derived from it is sent", and left its only test
alone. That test, it turned out, checked neither wording.

spec-delta.mjs compares every rule's whole text before and after, not
its definition line. The line-only version missed #144 rewording
WIRE-021 on a later line. For each rule present on both sides whose
text changed, at least one file that cites it with @covers must be among
the files the change touched; a rule marked Not testable is exempt.
Against the real pull requests it flags #147 for CRYPTO-025 and passes
#144 and #145, whose reworded rules came with their tests.

- gate-local.sh runs it with --check, so a session fixes the test (or
  asks) before a pull request exists.
- The merge queue holds anything that slips past, in Review, with the
  rule named. It uses the same commit-bound approval as every other hold.
- The skill says what the gate expects: reword a rule, update its test.

Each rule was broken once to see its test fail: the exemption, whole-text
comparison, whitespace-insensitive comparison, and the queue's hold.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015k9tkUtbU3wBpCHAHq7Qas
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
Nectenda/nectenda!150
No description provided.