feat(enforce): hole 8 — override-usage monitor in STATUS.md
Brain-retro #5 candidate C, hole 8: ~/.claude/runtime/override-usage.jsonl logged every override-vocab use but no surface analyzed frequency. 18x recovery in lifetime was hidden until manual inspection. New module tools/enforce-override-monitor.mjs computes per-phrase totals plus today's count; warns (warning) at >=5/day per phrase (configurable). Wired into tools/status-md-generator.mjs as a new '## Использование override-фраз' block. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Дмитрий
+17
-8
@@ -1,6 +1,6 @@
|
||||
# Brain Status (auto-generated)
|
||||
|
||||
Last updated: 2026-05-25T14:59:12.388Z
|
||||
Last updated: 2026-05-26T08:15:38.511Z
|
||||
|
||||
| Контролёр | Состояние | Детали |
|
||||
|---|---|---|
|
||||
@@ -8,13 +8,13 @@ Last updated: 2026-05-25T14:59:12.388Z
|
||||
| C2 Cross-ref consistency | ✅ | [cross-ref-checker] OK — 0 drift in 4 files |
|
||||
| C3 Observer-of-observer | ✅ | [observer-of-observer] OK — last read 0 week(s) ago |
|
||||
| C4 Сигнальный статус | ✅ | This file (self-reference) |
|
||||
| C5 Observer-coverage | ⚠️ | 414 episode(s) this month · Stop-hook + post-commit OK · 21 missed activation(s) — see /brain-retro |
|
||||
| C5 Observer-coverage | ⚠️ | 419 episode(s) this month · Stop-hook + post-commit OK · 21 missed activation(s) — see /brain-retro |
|
||||
| C6 Chain map sync | ✅ | [chain-map-checker] OK — 16 chains in sync |
|
||||
|
||||
## Метрики (информационные, не алерты)
|
||||
|
||||
- Observer evidence: 414 episodes this month, 0 observer_error markers, 59 PII matches before filter
|
||||
- Legacy v1 episodes (not in factor analysis): 275
|
||||
- Observer evidence: 419 episodes this month, 0 observer_error markers, 61 PII matches before filter
|
||||
- Legacy v1 episodes (not in factor analysis): 280
|
||||
- Last /brain-retro: 1 day(s) ago
|
||||
- Использование узлов: см. `/brain-retro` (раз в спринт). missed_activations: 21. **Неиспользованные узлы — не алерт, если профильной задачи не было** (Pravila §16.4 v1.36; capability-readiness; см. memory `feedback_brain_unused_tools_not_problem` — outside-repo memory store).
|
||||
|
||||
@@ -27,14 +27,14 @@ Baseline дисциплины роутера (этап 2 router discipline overh
|
||||
| analysis | 19 | 42.1% | 21.1% |
|
||||
| monitoring | 16 | 0.0% | 0.0% |
|
||||
| feature | 14 | 14.3% | 0.0% |
|
||||
| bugfix | 11 | 36.4% | 45.5% |
|
||||
| bugfix | 12 | 33.3% | 41.7% |
|
||||
| planning | 10 | 20.0% | 20.0% |
|
||||
| refactor | 1 | 0.0% | 0.0% |
|
||||
| cleanup | 1 | 0.0% | 0.0% |
|
||||
|
||||
Router step distribution: 1: 166, 2: 143, 3: 54, 5: 46
|
||||
Router step distribution: 1: 168, 2: 144, 3: 55, 5: 47
|
||||
|
||||
Boundaries applied (ADR / границы): 64 of 409 эпизодов (15.6%).
|
||||
Boundaries applied (ADR / границы): 65 of 414 эпизодов (15.7%).
|
||||
|
||||
## Активные многоэтапные проекты
|
||||
|
||||
@@ -67,9 +67,18 @@ Episodes since last run: 0 / threshold: 10
|
||||
|
||||
## Reviewer: субагент vs fallback
|
||||
|
||||
0 эпизодов проверено из 414.
|
||||
0 эпизодов проверено из 419.
|
||||
|
||||
|
||||
## Использование override-фраз
|
||||
|
||||
⚠️ Превышен порог override-использования сегодня (≥5/день)
|
||||
|
||||
| Фраза | За всё время | За сегодня |
|
||||
|---|---|---|
|
||||
| `recovery` | 24 | 14 ⚠️ |
|
||||
| `без скилов` | 6 | 4 |
|
||||
|
||||
## Алерт-индикаторы
|
||||
|
||||
✅ — норма ・ ⚠️ — внимание ・ 🔴 — действие требуется ・ ⚪ — не запускалось
|
||||
|
||||
@@ -0,0 +1,57 @@
|
||||
// Brain-retro #5 candidate C, hole 8: override-usage monitor.
|
||||
//
|
||||
// Reads override-usage.jsonl (one JSON line per override invocation:
|
||||
// {ts, session_id, rule, phrase}) and produces a STATUS.md block with
|
||||
// per-phrase totals + today's count. Warns when any phrase exceeds
|
||||
// threshold/day (default 5).
|
||||
//
|
||||
// Pure — takes raw log string + opts, returns markdown.
|
||||
|
||||
export function computeOverrideUsageBlock(rawLog, opts = {}) {
|
||||
const now = opts.now ? new Date(opts.now) : new Date();
|
||||
const today = now.toISOString().slice(0, 10);
|
||||
const threshold = opts.threshold ?? 5;
|
||||
|
||||
if (!rawLog || typeof rawLog !== 'string') {
|
||||
return `## Использование override-фраз\n\nНе использовалось.`;
|
||||
}
|
||||
|
||||
const lines = rawLog.split('\n').filter(Boolean);
|
||||
if (lines.length === 0) {
|
||||
return `## Использование override-фраз\n\nНе использовалось.`;
|
||||
}
|
||||
|
||||
const todayCounts = {};
|
||||
const allCounts = {};
|
||||
for (const l of lines) {
|
||||
let e;
|
||||
try { e = JSON.parse(l); } catch { continue; }
|
||||
if (!e || typeof e.phrase !== 'string' || !e.phrase) continue;
|
||||
allCounts[e.phrase] = (allCounts[e.phrase] || 0) + 1;
|
||||
if (typeof e.ts === 'string' && e.ts.slice(0, 10) === today) {
|
||||
todayCounts[e.phrase] = (todayCounts[e.phrase] || 0) + 1;
|
||||
}
|
||||
}
|
||||
|
||||
if (Object.keys(allCounts).length === 0) {
|
||||
return `## Использование override-фраз\n\nНе использовалось.`;
|
||||
}
|
||||
|
||||
const sorted = Object.entries(allCounts).sort((a, b) => b[1] - a[1]);
|
||||
const rows = sorted.map(([phrase, total]) => {
|
||||
const tCount = todayCounts[phrase] || 0;
|
||||
const warn = tCount >= threshold ? ' ⚠️' : '';
|
||||
return `| \`${phrase}\` | ${total} | ${tCount}${warn} |`;
|
||||
}).join('\n');
|
||||
|
||||
const anyWarn = Object.values(todayCounts).some((v) => v >= threshold);
|
||||
const header = anyWarn ? `⚠️ Превышен порог override-использования сегодня (≥${threshold}/день)` : '';
|
||||
|
||||
return `## Использование override-фраз
|
||||
|
||||
${header}
|
||||
|
||||
| Фраза | За всё время | За сегодня |
|
||||
|---|---|---|
|
||||
${rows}`;
|
||||
}
|
||||
@@ -0,0 +1,48 @@
|
||||
import { describe, it, expect } from 'vitest';
|
||||
import { computeOverrideUsageBlock } from './enforce-override-monitor.mjs';
|
||||
|
||||
describe('computeOverrideUsageBlock', () => {
|
||||
const today = '2026-05-26';
|
||||
const entry = (phrase, dt = today) => JSON.stringify({ ts: `${dt}T01:00:00Z`, session_id: 'x', rule: 'r', phrase });
|
||||
|
||||
it('returns placeholder when log empty', () => {
|
||||
expect(computeOverrideUsageBlock('')).toContain('Не использовалось');
|
||||
expect(computeOverrideUsageBlock(null)).toContain('Не использовалось');
|
||||
});
|
||||
|
||||
it('lists phrase frequencies and totals', () => {
|
||||
const log = [entry('recovery'), entry('recovery'), entry('без скилов')].join('\n');
|
||||
const out = computeOverrideUsageBlock(log, { now: `${today}T05:00:00Z` });
|
||||
expect(out).toContain('`recovery`');
|
||||
expect(out).toContain('| 2 |');
|
||||
expect(out).toContain('без скилов');
|
||||
});
|
||||
|
||||
it('warns when any phrase exceeds 5/day', () => {
|
||||
const log = Array.from({ length: 7 }, () => entry('recovery')).join('\n');
|
||||
const out = computeOverrideUsageBlock(log, { now: `${today}T05:00:00Z` });
|
||||
expect(out).toContain('⚠️');
|
||||
expect(out).toContain('recovery');
|
||||
});
|
||||
|
||||
it('only counts today for "сегодня" column', () => {
|
||||
const log = [entry('recovery', '2026-05-25'), entry('recovery', today)].join('\n');
|
||||
const out = computeOverrideUsageBlock(log, { now: `${today}T05:00:00Z` });
|
||||
// total=2, today=1
|
||||
expect(out).toMatch(/`recovery`.*\|\s*2\s*\|\s*1/);
|
||||
});
|
||||
|
||||
it('respects custom threshold', () => {
|
||||
const log = Array.from({ length: 3 }, () => entry('recovery')).join('\n');
|
||||
const flagged = computeOverrideUsageBlock(log, { now: `${today}T05:00:00Z`, threshold: 2 });
|
||||
const notFlagged = computeOverrideUsageBlock(log, { now: `${today}T05:00:00Z`, threshold: 10 });
|
||||
expect(flagged).toContain('⚠️');
|
||||
expect(notFlagged).not.toContain('⚠️');
|
||||
});
|
||||
|
||||
it('skips malformed JSON lines silently', () => {
|
||||
const log = ['not-json', entry('recovery'), '{}'].join('\n');
|
||||
const out = computeOverrideUsageBlock(log, { now: `${today}T05:00:00Z` });
|
||||
expect(out).toContain('recovery');
|
||||
});
|
||||
});
|
||||
@@ -2,10 +2,12 @@
|
||||
import { readFileSync, writeFileSync, existsSync } from 'fs';
|
||||
import { join } from 'path';
|
||||
import { execFileSync } from 'child_process';
|
||||
import { homedir } from 'os';
|
||||
import { runCoverageChecker } from './observer-coverage-checker.mjs';
|
||||
import { analyze } from './brain-retro-analyzer.mjs';
|
||||
import { loadRegistry } from './registry-load.mjs';
|
||||
import { buildClassificationMap, buildDormancyMap } from './registry-to-classification-map.mjs';
|
||||
import { computeOverrideUsageBlock } from './enforce-override-monitor.mjs';
|
||||
|
||||
const PRICING = {
|
||||
sonnet46: { input_per_mtok: 3.0, output_per_mtok: 15.0 },
|
||||
@@ -213,7 +215,7 @@ Last updated: ${now}
|
||||
- Legacy v1 episodes (not in factor analysis): ${observer.v1Episodes || 0}
|
||||
- Last /brain-retro: ${retroLine}
|
||||
- Использование узлов: см. \`/brain-retro\` (раз в спринт). missed_activations: ${missed.totalMissed}. **Неиспользованные узлы — не алерт, если профильной задачи не было** (Pravila §16.4 v1.36; capability-readiness; см. memory \`feedback_brain_unused_tools_not_problem\` — outside-repo memory store).
|
||||
${disciplineBlock}${projectsBlock}${inputs.costBlock ? `\n${inputs.costBlock}\n` : ''}${inputs.anomalyBlock ? `\n${inputs.anomalyBlock}\n` : ''}${inputs.selfRetrospectBlock ? `\n${inputs.selfRetrospectBlock}\n` : ''}${inputs.reviewerBlock ? `\n${inputs.reviewerBlock}\n` : ''}
|
||||
${disciplineBlock}${projectsBlock}${inputs.costBlock ? `\n${inputs.costBlock}\n` : ''}${inputs.anomalyBlock ? `\n${inputs.anomalyBlock}\n` : ''}${inputs.selfRetrospectBlock ? `\n${inputs.selfRetrospectBlock}\n` : ''}${inputs.reviewerBlock ? `\n${inputs.reviewerBlock}\n` : ''}${inputs.overrideUsageBlock ? `\n${inputs.overrideUsageBlock}\n` : ''}
|
||||
## Алерт-индикаторы
|
||||
|
||||
✅ — норма ・ ⚠️ — внимание ・ 🔴 — действие требуется ・ ⚪ — не запускалось
|
||||
@@ -343,11 +345,17 @@ if (process.argv[1] && process.argv[1].replace(/\\/g, '/').endsWith('/status-md-
|
||||
};
|
||||
|
||||
const eps = loadCurrentMonthEpisodes();
|
||||
let costBlock = null, anomalyBlock = null, selfRetrospectBlock = null, reviewerBlock = null;
|
||||
let costBlock = null, anomalyBlock = null, selfRetrospectBlock = null, reviewerBlock = null, overrideUsageBlock = null;
|
||||
try { costBlock = computeCostBlock(eps, PRICING); } catch (err) { console.warn('[status-md-generator] costBlock skipped:', err.message); costBlock = '(нет данных)'; }
|
||||
try { anomalyBlock = computeAnomalyBlock(eps); } catch (err) { console.warn('[status-md-generator] anomalyBlock skipped:', err.message); anomalyBlock = '(нет данных)'; }
|
||||
try { selfRetrospectBlock = computeSelfRetrospectBlock(join('docs', 'observer', '.self-retrospect-counter.json')); } catch (err) { console.warn('[status-md-generator] selfRetrospectBlock skipped:', err.message); selfRetrospectBlock = '(нет данных)'; }
|
||||
try { reviewerBlock = computeReviewerBlock(eps); } catch (err) { console.warn('[status-md-generator] reviewerBlock skipped:', err.message); reviewerBlock = '(нет данных)'; }
|
||||
try {
|
||||
const logPath = join(homedir(), '.claude', 'runtime', 'override-usage.jsonl');
|
||||
const raw = existsSync(logPath) ? readFileSync(logPath, 'utf-8') : '';
|
||||
overrideUsageBlock = computeOverrideUsageBlock(raw);
|
||||
} catch (err) { console.warn('[status-md-generator] overrideUsageBlock skipped:', err.message); overrideUsageBlock = '(нет данных)'; }
|
||||
inputs.overrideUsageBlock = overrideUsageBlock;
|
||||
inputs.costBlock = costBlock;
|
||||
inputs.anomalyBlock = anomalyBlock;
|
||||
inputs.selfRetrospectBlock = selfRetrospectBlock;
|
||||
|
||||
Reference in New Issue
Block a user