docs: spec + plan for hard-rule enforcement (10 rules + override vocab)
This commit is contained in:
Дмитрий
@@ -0,0 +1,72 @@
|
||||
# Enforce hard rules — implementation plan
|
||||
|
||||
**Spec:** `docs/superpowers/specs/2026-05-25-enforce-hard-rules-design.md`
|
||||
**Branch:** `feat/enforce-hard-rules`
|
||||
**Estimate:** 4-8 hours autonomous (overnight)
|
||||
|
||||
## Tasks (in commit order — each commit standalone testable)
|
||||
|
||||
### T1 — Shared hook helpers + override vocab
|
||||
**Files:** `tools/enforce-hook-helpers.mjs`, `tools/enforce-hook-helpers.test.mjs`, `tools/enforce-override-vocab.json`
|
||||
**Helpers:** readStdinJson, readTranscript, getCoverageFromLastAssistant, hasOverridePhrase, loadVocab, sentinelPath, writeSentinel, readSentinel, expectedBranchPath, getExpectedBranch, setExpectedBranch, readRationalizationFlags, appendRationalizationFlag.
|
||||
**Override vocab content:** initial 6 phrases per spec §9.
|
||||
**Coverage:** skill:superpowers:test-driven-development
|
||||
|
||||
### T2 — Rule #5 memory-sync coverage (PreToolUse)
|
||||
**File:** `tools/enforce-memory-coverage.mjs` + test.
|
||||
Simplest rule, easy validation. RED test: prod-code edit with TDD coverage → block. GREEN: memory edit with memory-sync coverage → allow.
|
||||
|
||||
### T3 — Rule #7 branch-switch detection (PreToolUse Bash)
|
||||
**File:** `tools/enforce-branch-switch.mjs` + test.
|
||||
Reads expected-branch file, runs `git branch --show-current`, compares.
|
||||
|
||||
### T4 — Rule #4 verify-before-push (PreToolUse + PostToolUse Bash)
|
||||
**Files:** `tools/enforce-verify-before-push.mjs` (PreToolUse) + `tools/enforce-verify-record.mjs` (PostToolUse to write sentinel) + tests.
|
||||
PostToolUse runs after Bash with vitest/pest pattern. If exit 0 + stdout has PASS marker → write sentinel.
|
||||
PreToolUse on git commit/push checks sentinel age + exists.
|
||||
|
||||
### T5 — Rule #2 coverage-verify (Stop)
|
||||
**File:** `tools/enforce-coverage-verify.mjs` + test.
|
||||
Parses last assistant message for coverage line, checks against transcript tool_use history.
|
||||
|
||||
### T6 — Rule #1 mandatory re-classification injection (UserPromptSubmit)
|
||||
**File:** `tools/enforce-prompt-injection.mjs` + test.
|
||||
Reads classifier output from router-state-*.json, injects mandatory coverage list via stdout JSON.
|
||||
|
||||
### T7 — Rule #3 + Rule #6 TDD + writing-plans gate (PreToolUse Edit/Write/MultiEdit)
|
||||
**File:** `tools/enforce-tdd-gate.mjs` + test.
|
||||
Path-match, transcript-scan for test-edit + vitest-fail-output, OR plan-file-exists.
|
||||
|
||||
### T8 — Rule #8 classifier-mismatch (Stop)
|
||||
**File:** `tools/enforce-classifier-match.mjs` + test.
|
||||
Reads classifier output, checks turn for matching Skill/Task tool_use, gates on confidence threshold.
|
||||
|
||||
### T9 — Rule #10 rationalization flags (PostToolUse Bash + Edit/Write)
|
||||
**File:** `tools/enforce-rationalization-audit.mjs` + test.
|
||||
Scan transcript for rationalization phrases / weak tests; append flag JSONL.
|
||||
|
||||
### T10 — Atomic wire-up
|
||||
**File:** `.claude/settings.json` — add all hooks to PreToolUse/PostToolUse/UserPromptSubmit/Stop.
|
||||
**Critical:** this must be the LAST commit. Pre-wire commits keep hooks inert.
|
||||
|
||||
### T11 — Smoke + push
|
||||
Manual smoke each hook with synthetic stdin. Then `git push origin feat/enforce-hard-rules:main` via FF (or merge-commit if main moved).
|
||||
|
||||
### T12 — Memory + state sync
|
||||
Create `memory/project_enforce_hard_rules.md`, update MEMORY.md index, project_state.md, reference_github.md.
|
||||
|
||||
## Risks identified, mitigations
|
||||
|
||||
- **R1:** Parallel session edits `.claude/settings.json` while I'm working. **Mitigation:** Read settings.json fresh right before T10. Use `git stash` for any concurrent local changes if needed.
|
||||
- **R2:** A rule blocks my own work mid-task. **Mitigation:** Rules inert until T10. If T10 wire-up succeeds and immediately blocks me on T11 push, override-vocab is in place (`recovery` phrase).
|
||||
- **R3:** Hook scripts crash → all subsequent tool calls hang. **Mitigation:** Every hook wraps logic in try/catch, exits 0 with empty {} on internal error (fail-quiet). NEVER exit 2 unless intentional violation found.
|
||||
- **R4:** Override-vocab phrase appears coincidentally in user's normal speech. **Mitigation:** Phrases chosen to be unusual (включают «без скилов» which is unlikely normal).
|
||||
- **R5:** PreToolUse latency on Bash slows every command. **Mitigation:** Hook target deltay <100ms by reading minimum (cached classifier-state, sentinel file, no transcript-parse unless rule triggers).
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- All 10 rules implemented with unit tests
|
||||
- All hooks wired in settings.json
|
||||
- Manual smoke per hook: fake-stdin → expected exit code + stderr
|
||||
- Push to origin/main (or PR if main is unstable)
|
||||
- Memory + project_state synced
|
||||
@@ -0,0 +1,157 @@
|
||||
# Enforce hard rules — design (2026-05-25 night)
|
||||
|
||||
**Status:** In progress (autonomous overnight implementation)
|
||||
**Origin:** End of brain factor-analysis 4-passes session (HEAD `58784b18`). Honest retrospective showed brain-governance / observer / classifier architecture is observe-only — no enforce. Controller (Claude) rationalized 4 skill bypasses + single coverage tag for 6 hours of varied activity without any hook blocking the behaviour.
|
||||
**Goal:** Convert soft warnings to hard `exit 2` blocks at the only enforce-able layer Claude Code exposes — PreToolUse + Stop hooks. Substance-of-skill compliance translates to artifact-checks.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Constraining Claude's text output (impossible by architecture — LLM generation).
|
||||
- Enforcing test quality (substance). Future LLM-judge epic.
|
||||
- Enforcing skill content interpretation. Best-effort via artifact gates.
|
||||
- Replacing the classifier / observer / brain-retro infrastructure. This is enforcement layer on top.
|
||||
|
||||
## Architectural premise
|
||||
|
||||
Claude Code hook surface:
|
||||
- **UserPromptSubmit** — can inject `<system-reminder>` text into the next turn's context. CAN'T block.
|
||||
- **PreToolUse** — `exit 2` blocks the tool call. Stderr returns to Claude.
|
||||
- **PostToolUse** — observes, can write state. CAN'T block (tool already ran).
|
||||
- **Stop** — `exit 2` denies turn completion. Stderr returns to Claude on next continuation.
|
||||
|
||||
This proposal uses all four. Output text remains uncontrolled by design — but every consequential ACTION (tool call, turn completion) passes a gate.
|
||||
|
||||
## The 10 rules (priority + risk ordered)
|
||||
|
||||
### Rule #1 — Mandatory re-classification per prompt
|
||||
|
||||
**Mechanism:** UserPromptSubmit hook (`tools/enforce-prompt-classify.mjs`) runs after the existing classifier, then injects a `<system-reminder>` listing:
|
||||
- Classification + confidence
|
||||
- 1-3 recommended skills/nodes
|
||||
- Forced `coverage:` line requirement (first line of response)
|
||||
|
||||
**Effect:** Each turn starts with explicit coverage expectation visible to Claude in context.
|
||||
|
||||
**Override:** User says one of the override-vocab phrases (see Rule #9). Then injection is suppressed for that prompt.
|
||||
|
||||
### Rule #2 — Coverage tag verified against artifacts
|
||||
|
||||
**Mechanism:** Stop hook (`tools/enforce-coverage-verify.mjs`). Reads the assistant's last response, parses `coverage: <channel>:<id>`. Then:
|
||||
- `channel=skill` → check transcript for `Skill` tool_use with `input.skill === id` in this turn. If absent → `exit 2`.
|
||||
- `channel=node` → check for tool_use matching the node's canonical tool (e.g., #19 frontend-design → check for matching skill or canonical command). If absent → `exit 2`.
|
||||
- `channel=direct` → no artifact check, but classifier-recommendation must align with non-direct fallback (handled by Rule #8).
|
||||
- No `coverage:` line at all → `exit 2`.
|
||||
|
||||
**Override:** Override-vocab phrase in previous user prompt.
|
||||
|
||||
### Rule #3 — TDD-gate on production code
|
||||
|
||||
**Mechanism:** PreToolUse hook on `Edit`/`Write`/`MultiEdit` (`tools/enforce-tdd-gate.mjs`). For paths matching production patterns:
|
||||
- `tools/**/*.mjs` (not `*.test.mjs`)
|
||||
- `app/app/**/*.php` (not `app/tests/**`)
|
||||
- `resources/js/**` (not `**/*.spec.ts`, not `**/*.test.ts`)
|
||||
|
||||
Reads transcript of current turn so far. Requires:
|
||||
1. Earlier `Edit`/`Write` on a corresponding test path within the same turn, OR
|
||||
2. Test artifact already exists (Bash `test -f` could verify, but we read git status)
|
||||
|
||||
AND:
|
||||
3. Earlier `Bash` with `vitest` / `pest` in command, AND
|
||||
4. The `Bash` stdout in transcript contains a "fail" / "FAIL" marker (RED phase confirmed)
|
||||
|
||||
If any check fails → `exit 2` with explanation.
|
||||
|
||||
**Override:** Override-vocab phrase + sentinel file `~/.claude/runtime/tdd-bypass-<session_id>.flag` (auto-created from override).
|
||||
|
||||
### Rule #4 — Git commit/push requires verification artifact
|
||||
|
||||
**Mechanism:** PreToolUse hook on `Bash` (`tools/enforce-verify-before-push.mjs`). Pattern-matches command for `git commit` or `git push`. If matched:
|
||||
- Check for sentinel file `~/.claude/runtime/verify-pass-<session_id>.json`
|
||||
- Sentinel contains `last_full_run_at` timestamp, `result: pass|fail`, `command_run`, `tests_total`, `tests_passed`
|
||||
- Sentinel must be written by Rule's companion PostToolUse hook on Bash, when Bash command matches vitest/pest full-run pattern AND stdout indicates success
|
||||
- Sentinel age < 600s required; missing or stale → `exit 2`
|
||||
|
||||
**Override:** Override-vocab phrase or `RECOVERY-INTENT:` marker in previous response.
|
||||
|
||||
### Rule #5 — Memory write requires memory-sync coverage
|
||||
|
||||
**Mechanism:** PreToolUse hook on `Edit`/`Write` (`tools/enforce-memory-coverage.mjs`). Path-match:
|
||||
- `**/memory/*.md`
|
||||
- `**/MEMORY.md`
|
||||
- `C:\Users\*\.claude\projects\**\memory\*.md`
|
||||
|
||||
Reads last assistant message for `coverage: direct:memory-sync` or `coverage: skill:<memory-related-skill>`.
|
||||
If coverage absent or stale (matches non-memory channel) → `exit 2` with re-announce instruction.
|
||||
|
||||
### Rule #6 — Writing-plans enforce for feature/bugfix/refactor
|
||||
|
||||
**Mechanism:** PreToolUse hook on production-code `Edit`/`Write` (folded into Rule #3 hook). Before first production-code edit of a turn classified as `feature`/`bugfix`/`refactor`:
|
||||
- Either invoke `superpowers:writing-plans` skill (Skill tool_use) in this turn so far, OR
|
||||
- Plan file exists at `docs/superpowers/plans/<date>-<slug>.md` referenced in transcript, OR
|
||||
- Override-vocab phrase
|
||||
|
||||
If none → `exit 2`.
|
||||
|
||||
### Rule #7 — Branch-switch detection before commit
|
||||
|
||||
**Mechanism:** PreToolUse on `Bash` matching `git commit`. Hook runs `git branch --show-current`. Compares to expected branch (from `~/.claude/runtime/expected-branch-<session_id>`, written at session start or when user explicitly mentions a branch).
|
||||
|
||||
If actual ≠ expected → `exit 2`: «Branch switched silently. Verify via `BRANCH-SWITCH-CONFIRMED` or `RECOVERY-INTENT`.»
|
||||
|
||||
### Rule #8 — Classifier-mismatch enforce
|
||||
|
||||
**Mechanism:** Stop hook (chained after Rule #2). Reads classifier output:
|
||||
- If `classifier_output.recommended_node !== null` AND
|
||||
- `confidence >= 0.7` AND
|
||||
- No `Skill`/`Task` tool_use matching the recommendation in this turn AND
|
||||
- No `override: <reason>` line in response
|
||||
|
||||
→ `exit 2`.
|
||||
|
||||
### Rule #9 — Override-vocabulary hard-coded
|
||||
|
||||
**Mechanism:** Config file `tools/enforce-override-vocab.json` with allowed phrases. Hooks read this once at startup. Phrases (initial):
|
||||
- `без скилов` → all skill-related rules suppressed for this prompt
|
||||
- `direct ok` → coverage=direct allowed without justification
|
||||
- `срочно` → Rule #4 (verify) suppressed
|
||||
- `быстрый коммит` → Rule #3+#4 suppressed for one commit
|
||||
- `recovery` → Rule #7 suppressed
|
||||
- `memory dump` → Rule #5 acknowledged auto
|
||||
|
||||
Match is literal substring against user's last prompt. Case-insensitive. Each use logged to `~/.claude/runtime/override-usage.jsonl`.
|
||||
|
||||
### Rule #10 — Rationalization flags (post-fact audit)
|
||||
|
||||
**Mechanism:** PostToolUse on `Bash` (`tools/enforce-rationalization-audit.mjs`). After each prod-code Edit/Write or git commit:
|
||||
- Scan turn so far for indicators: weak test (≤2 expects), commit message lacking TDD evidence, "just this once" / "for now" / "пока без" / "сейчас быстрее" phrases.
|
||||
- Each flag appended to `~/.claude/runtime/rationalization-flags-<session_id>.jsonl`.
|
||||
- Next UserPromptSubmit hook reads this file and injects into context: «Previous turn flagged: X — adjust behavior.»
|
||||
|
||||
Soft (no block), but visible to Claude on next turn.
|
||||
|
||||
## Anti-self-block strategy during development
|
||||
|
||||
Implementing the rules inside the very project they will enforce creates a chicken-and-egg problem. Mitigation:
|
||||
|
||||
1. **Develop on feature branch `feat/enforce-hard-rules`** (already created).
|
||||
2. **Hook scripts are inert until wired into `.claude/settings.json`.** All implementation commits don't trigger them.
|
||||
3. **Final commit atomically wires all hooks** in settings.json.
|
||||
4. **First push and test must happen ON main after wire-up commit** — by then all rules are committed AND satisfied (because each new turn after wire will start under enforced rules naturally).
|
||||
|
||||
## Test strategy per rule
|
||||
|
||||
Per-rule unit tests in `tools/enforce-*.test.mjs`:
|
||||
- Hook receives fake stdin (event JSON)
|
||||
- Hook decision verified by exit code + stderr message
|
||||
- Sentinel file behavior tested with mkdtemp baseDir override
|
||||
- Override-vocab integration tested by injecting phrase in prev-prompt fixture
|
||||
|
||||
Target ~60-100 tests total for all hooks.
|
||||
|
||||
## Out of scope (deferred, may revisit morning)
|
||||
|
||||
- LLM-judge on test quality
|
||||
- Confidence threshold tuning (default 0.7, hand-tune via brain-retro)
|
||||
- Multi-prompt session-level reasoning (each prompt evaluated standalone)
|
||||
- Conflict resolution if multiple override-vocab phrases stack
|
||||
- UI for override-usage retro (just JSONL file; brain-retro will read)
|
||||
Reference in New Issue
Block a user