# A11 ML / AI Tooling Integration — Design

**Date:** 2026-05-17
**Topic:** Populate the empty `A11 «ML / AI-разработка»` map section.
**Deciders:** Дмитрий
**Method:** `superpowers:brainstorming` (economy 5%). Approach А approved 2026-05-17.

---

## Goal

Populate the empty `A11 «ML / AI-разработка»` section of `docs/automation-graph.html`
with a conflict-minimal ML/AI toolset — so A11 becomes a working playbook, the same
way A6 «Архитектура систем» was closed with adr-kit/mermaid/architecture-patterns and
C9 «Управление проектами» with the CCPM stack.

## Context

- **A11 is empty.** `SECTIONS` in `docs/automation-graph.html:1836` defines
  `A11 = «ML / AI-разработка»` (bucket A «Технические и продуктовые»); `NODE_SECTION`
  tags **zero** nodes to A11.
- **The map is a tooling map** — «карта dev-автоматики» (plugins / skills / MCP /
  hooks Claude uses). A11 must therefore be populated with **ML/AI *development
  tools*** — not with product features.
- **The project has no ML/AI code.** Лидерра is a Laravel 13 + Vue 3 multi-tenant
  CRM. The only scoring artifact, `calc_lead_score` (`db/schema.sql`), is a
  deterministic SQL function — not a model. No dataset, no feature store, no ML/AI
  item in `Открытые_вопросы` or the plan backlog.
- **The machine is runtime-minimal by policy.** Native Windows, no Docker (no
  nested virt on the OpenStack VPS), pg_partman replaced with a cron command. Every
  new runtime passes an explicit weighing. Python is **not** in the stack (PHP +
  Node only).

## Scope decision

A11 covers **both** subcategories — classical ML **and** LLM integration (user
choice 2026-05-17). The **Python runtime is deferred**: the executable classical-ML
part (Jupyter MCP) is a severable/conditional task, not a now-install. Rationale:
no ML model to train, no dataset, a new runtime on a deliberately-minimal machine,
and the Jupyter MCP is experimental (Notebook 6.x only). The classical-ML
*knowledge* layer (Data Scientist skill, zero runtime) **is** installed now, so the
subcategory is not empty — only its executor is gated.

This mirrors the project idiom: C9 made product-management a conditional task; A6
made the lefthook-enforcement task severable.

---

## Architecture — the 6-position toolset (Approach А)

Two subcategories. Three positions are **reuse** (already installed — A11 documents
the coverage, REU1), two are **new + light**, one is **new + deferred**.

| # | Tool | Subcategory | Install mode | Status |
|---|---|---|---|---|
| 1 | **claude-api skill** | LLM integration | already available (plugin skill) | reuse |
| 2 | **context7 MCP** | both (library docs) | already installed (MCP, map node `context7` in E7) | reuse |
| 3 | **Sentry MCP** (Tooling #34) | LLM observability | already installed (MCP, map node `mcp_sentry` in A7) | reuse — AI/LLM monitoring activates on the Sentry deployment (blocked on Б-1) |
| 4 | **promptfoo** | LLM integration | new — Node CLI, project `package.json` devDependency, invoked `npx promptfoo`; MIT (OpenAI-owned, OSS) | new, light |
| 5 | **Data Scientist skill** | classical ML | new — vendored standalone skill into `.claude/skills/data-scientist/` (no plugin, no marketplace, no hooks — the A6 mermaid pattern) | new, light |
| 6 | **Jupyter MCP** (`datalayer/jupyter-mcp-server`) | classical ML | new — MCP server, Python-gated | **severable / conditional** — installed only when a concrete ML model is scoped |

**Reuse layer (positions 1-3).** claude-api skill = building AI features on the
Anthropic SDK (lead qualification, call-summary, email drafts) with prompt caching.
context7 MCP = up-to-date docs for AI/ML libraries and SDKs. Sentry MCP = debugging
AI features in production via Sentry's AI/LLM monitoring (read-only, pending the
Sentry instance — Б-1).

**New layer (positions 4-5).** promptfoo = a test suite for LLM prompts/agents:
assertions, regression catching, LLM-graded eval, red-teaming. Node-based — **needs
no new runtime**. Data Scientist skill = a knowledge-only skill: business
objective → ML task, algorithm selection (Linear Regression … XGBoost), feature
engineering, experiment-tracking guidance, A/B analysis.

**Deferred (position 6).** Jupyter MCP = executable notebooks for real model
training. Gated on a Python ML environment + a concrete ML model to train.

A11 is **fully covered** by positions 1-5 from day one; position 6 is a reserved,
registered-but-not-installed slot.

---

## Conflict audit

Pattern follows the K1–K8 / AK1–CC1 / CP1–NUM1 audits used for claude-mem and the
A6 / C9 / D3 plans. Full audit with locked resolutions is produced in the
implementation plan; the known conflicts:

| # | Tool | Sev | Conflict | Resolution direction |
|---|---|---|---|---|
| ML1 | promptfoo | 🟡 | A real eval run needs an Anthropic API key + costs money (paid LLM calls). | Key via env (PowerShell User scope, like Sentry) — never committed (gitleaks). promptfoo runs **manually / CI-only** — never in a hook, never in pre-commit, never auto. No economy-chain impact. |
| ML2 | promptfoo | 🟢 | promptfoo's red-team module overlaps the D3 audit-security tools. | None — promptfoo red-team tests *LLM prompts* for jailbreak/injection; Trail of Bits / Semgrep (D3 #39, #25) are SAST of *code*. Different objects. Boundary documented. |
| ML3 | Data Scientist skill | 🟡 | Vendored `.claude/skills/data-scientist/**/*.md` (third-party English files) is caught by the cspell + markdownlint pre-commit jobs. | Add `.claude/skills/data-scientist/**` to `cspell.json` `ignorePaths` and `.markdownlintignore` (the A6 MK1 pattern). The project's own skills stay linted. |
| ML4 | reuse layer | 🟢 | Re-tagging the existing `context7` (E7) / `mcp_sentry` (A7) nodes to A11 would empty their current sections — `NODE_SECTION` is 1-node→1-section. | Reuse nodes **stay** in their sections; A11 gets its **own** new nodes. The reuse coverage is documented in `docs/ml/README.md`. Same as A6/D3/C9 REU1. |
| ML5 | all | 🟢 | A11 is non-UI tooling. | New off-phase category **ml-ai-tooling**, outside the PSR_v1 UI-pool (no R6.0/R6.1 stack-filter, no R14 pipeline) — same treatment as architecture-tooling (A6) and audit-security (D3). |
| ML6 | Jupyter MCP | 🟡 | Experimental (Notebook 6.x only), would be the 8th `.mcp.json` server, on a Python-less machine. | **Deferred** — severable/conditional task. When triggered: `alpha-substrate-spike-first` — a spike before integration. Registered in the registry/map as a reserved slot now; `.mcp.json` is untouched in the core scope. |
| ML7 | all | 🟡 | Bus-factor — Data Scientist skill is community; Jupyter MCP is community + experimental. | Data Scientist skill is **vendored** → immune to upstream loss (the A6 mermaid pattern). promptfoo is MIT, OpenAI-owned, stable. Jupyter MCP — version-pinned at install time (deferred). Noted in the Tooling entry. |
| ML8 | claude-api skill | 🟡 | claude-api is an available skill but may not be a formalized Tooling-registry position → using it unregistered is a PSR_v1 R0.2/R10 gap. | The plan's pre-flight checks whether claude-api (and its backing plugin) is already in `~/.claude/settings.json` `enabledPlugins` + the Tooling registry; if absent, register it as part of the A11 sync. |
| ML9 | promptfoo | 🟢 | promptfoo install mode — global npm vs project devDep vs vendored binary. | Project `package.json` **devDependency** — version-pinned via `package-lock.json`, no global state, invoked `npx promptfoo`. (Not a `bin/*.exe` — promptfoo is an npm package, not a single binary.) |

---

## File structure

| File | Created / Modified | Responsibility |
|---|---|---|
| `docs/ml/` | Create dir | A11 home — the ML/AI playbook |
| `docs/ml/README.md` | Create | The ML/AI convention: tool boundaries (claude-api skill = *build* AI features / promptfoo = *test* prompts / Data Scientist skill = classical-ML *workflow* / Jupyter MCP = *execute* — when scoped); the reuse-layer map |
| `docs/ml/promptfoo-example/promptfooconfig.yaml` | Create | One seed eval config — a worked example (e.g. a lead-qualification prompt with assertions) |
| `.claude/skills/data-scientist/` | Create (vendored) | The Data Scientist skill — `SKILL.md` + `references/` |
| `docs/adr/ADR-005-ml-ai-tooling.md` | Create (conditional — adr-kit/A6 landed) | Seed ADR documenting the A11 tooling decision + the Python-defer |
| `package.json` | Modify | `promptfoo` devDependency |
| `cspell.json` | Modify | `ignorePaths` += `.claude/skills/data-scientist/**` (ML3) |
| `.markdownlintignore` | Modify | += `.claude/skills/data-scientist/` (ML3) |
| `cspell-words.txt` | Modify (conditional) | New ML/AI vocabulary |
| `docs/Tooling_v8_3.md` | Modify | Прил. Н — new ml-ai-tooling subsection(s) + §0 counter bump |
| `docs/Plugin_stack_rules_v1.md` | Modify | R10.1 — new ml-ai-tooling rows |
| `docs/Pravila_raboty_Claude_v1_1.md` | Modify | §13.2 — ml-ai-tooling category note |
| `CLAUDE.md` | Modify (**via claude-md-management only**) | §3 title count, §1 row 2b count, new §3.3 ml-ai-tooling rows |
| `docs/CHANGELOG_claude_md.md` | Modify | CLAUDE.md version-bump entry |
| `docs/automation-graph.html` | Modify | 3 new A11 nodes (`claude_api`, `promptfoo`, `data_scientist`) → `NODE_SECTION` A11; header metrics |
| `.mcp.json` | **NOT modified in core scope** | Jupyter MCP deferred — `.mcp.json` touched only by the conditional task |

**Map nodes.** A11 gets **3 new nodes** — `claude_api`, `promptfoo`, `data_scientist`.
The reuse nodes `context7` (E7) and `mcp_sentry` (A7) stay put (ML4). A `jupyter_mcp`
node is added only when the conditional task runs (same as C9's conditional
`product_mgmt` node).

---

## Severable scope

**Core A11** = the reuse layer + promptfoo + Data Scientist skill + normative sync +
map closure. This already populates and closes the section.

**Conditional / severable** = the Jupyter MCP install (position 6) — a separate task,
gated on (a) a Python ML environment decision and (b) a concrete ML model to build.
Registered now as a reserved slot; installed later.

A11 adds **no lefthook job** and **no `.mcp.json` change** in the core scope — fewer
conflicts by design (the C9 shape).

---

## Out of scope

- Building actual AI features into the CRM product (AI lead qualification, call
  summaries) — that is product work, tracked separately; A11 only provides the
  *tooling*.
- Training an actual ML model / standing up a Python environment — the Jupyter MCP
  conditional task, triggered later.
- Any LLM-eval gate in the commit pipeline (ML1 — promptfoo stays manual/CI-only).

## Open items resolved by the implementation plan

- **NUM** — A11's Tooling-registry numbers are runtime-resolved: the plan reads the
  **live** `docs/Tooling_v8_3.md` Прил. Н §0 counter (after A6 #36-38, D3 #39-40, and
  whatever C9 added) and assigns A11's numbers sequentially.
- **claude-api registration** (ML8) — the plan's pre-flight checks whether claude-api
  is already a formalized Tooling position; registers it if not.
- **ADR-005 conditional** — written only if adr-kit (A6) has landed in `origin/main`.
- **Sequencing** — A11 forks off `origin/main` after the A6 / D3 / C9 epics land
  (they touch the same shared files: the map, 4 normative docs, the Tooling counter).
  The branch created for this spec is a scratch branch; execution re-forks.
- **Exact tool identity** (promptfoo repo/version, Data Scientist skill source repo,
  Jupyter MCP repo) — verified via WebFetch/WebSearch in the plan's Tool Identity
  section (the A6/C9/D3 pattern).
