Compare commits

...

19 Commits

Author SHA1 Message Date
Дмитрий cfc67fbc26 docs(schema): v8.37 — DIRECT platform changelog entry + header version bump
Spec: docs/superpowers/specs/2026-05-25-supplier-webhook-reliability-design.md

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-25 17:32:54 +03:00
Дмитрий 737a78f251 fix(db): migration covers chk_supplier_leads_platform + seed PG-compatible
Found via TDD that supplier_leads has its own platform CHECK constraint
(chk_supplier_leads_platform) and that the seed migration was missing
NOT NULL columns (accepts_types, channel). Migration now:

  - widens supplier_projects/project_supplier_links/supplier_leads.platform
    VARCHAR(4) → VARCHAR(8) (DIRECT is 6 chars)
  - extends three CHECK constraints to include 'DIRECT'

Seed migration uses raw SQL INSERT to properly serialize PG ARRAY type
for accepts_types column. channel='sites' (valid per suppliers_channel_check).

db/schema.sql synced — 3 platform columns and 3 CHECK constraints updated.
CHANGELOG_schema.md entry pending Task 9.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-25 17:31:44 +03:00
Дмитрий d82b1bf17c feat(supplier): LedgerService + CsvReconcileJob recognise DIRECT platform
LedgerService::resolveSupplierId returns suppliers.code='direct' row for
DIRECT-platform supplier_projects (and for parsed-from-payload non-B
projects). CsvReconcileJob::extractPlatform now classifies most non-empty,
non-junk project strings as DIRECT (instead of dumping them into
unparseable_count) — this allows CSV recovery to also create DIRECT
supplier_leads, mirroring the webhook path.

CsvReconcileJobTest junk-rows fixtures updated: previously used callback
phone-number-as-project (79135551234) and URL-like strings as 'junk', but
those are now valid DIRECT identifiers. Replaced with truly junk strings
matching only outside-whitelist symbols (e.g. '???', '!@#').

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-25 17:31:32 +03:00
Дмитрий 8be1f9d172 feat(supplier): RouteSupplierLeadJob + LeadRouter handle DIRECT platform
parseProjectField() returns ('DIRECT', signal_type, identifier) when project
has no B-prefix; identifier-detection (call/site/sms regex) runs on full
project string. LeadRouter::matchEligibleProjects has a DIRECT fast-path
that matches Liderra projects by (signal_type, signal_identifier) directly
without requiring project_supplier_links pivot — because DIRECT
supplier_projects are auto-created on first webhook and don't have manual
psl links.

B1/B2/B3 path unchanged (psl-based via project_supplier_links).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-25 17:31:22 +03:00
Дмитрий c9f25cd833 feat(supplier-webhook): accept non-B-prefix projects as platform=DIRECT
Drops regex /^B[123]_.+$/ from project field validation; parsePlatform()
returns 'DIRECT' for projects without B-prefix (instead of silent fallback
to 'B1'). SupplierProjectResolver ALLOWED_PLATFORMS extended to include
DIRECT.

Closes ~67 of 82 lost leads/day for tenant client1 (observed 2026-05-25):
mostly client.carmoney.ru (55), B2_Caranga (7), cabinet.caranga.ru (3),
cashmotor.ru (2), numeric callback IDs (~10).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-25 17:31:13 +03:00
Дмитрий fc2b517edc test(supplier): end-to-end DIRECT platform tests (4 failing, 2 passing)
Six tests:
  1. webhook with non-B-prefix project → 202 + platform=DIRECT (FAIL: 422 regex)
  2. Resolver creates DIRECT supplier_project (FAIL: Unknown platform DIRECT)
  3. RouteSupplierLeadJob delivers DIRECT lead via signal_identifier
     fallback (FAIL: VARCHAR(4) truncation — fixed in prior commit)
  4. numeric-only project → DIRECT (FAIL: 422 regex)
  5. B1 regression (PASS)
  6. Resolver rejects truly unknown platform (PASS)

Implementation in subsequent commits.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-25 17:20:33 +03:00
Дмитрий bb6f2ae0d6 fix(db): widen supplier_*.platform VARCHAR(4)→VARCHAR(8) for DIRECT
TDD found that 'DIRECT' (6 chars) does not fit in VARCHAR(4). Three columns
need widening: supplier_projects.platform, project_supplier_links.platform,
supplier_leads.platform. supplier_manual_sync_queue.platform was already
VARCHAR(8). Done in the same migration as CHECK extension — single
atomic deploy.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-25 17:20:23 +03:00
Дмитрий 7ffd79299f feat(db): seed suppliers.code='direct' for DIRECT platform billing
LedgerService::resolveSupplierId will look up suppliers WHERE code='direct'
for DIRECT-platform supplier_projects (Phase 3). cost_rub matches B1 (same
supplier company, different lead-routing channel).

Spec: docs/superpowers/specs/2026-05-25-supplier-webhook-reliability-design.md

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-25 17:18:16 +03:00
Дмитрий 1cf4c53d8d feat(db): extend supplier_projects.platform CHECK to include DIRECT
Adds DIRECT value to chk_supplier_projects_platform and chk_psl_platform
constraints. DIRECT represents supplier projects without B[123]_ prefix
(e.g. client.carmoney.ru, cashmotor.ru, numeric phone IDs) — currently
~67 leads/day lost to 302 redirects from webhook validation regex.

Schema-only change; no code yet uses DIRECT — code changes follow in
subsequent commits. Migration is forward-compatible: old code continues
to work with B1/B2/B3 rows.

chk_supplier_projects_b1_not_for_sms NOT touched — that constraint denies
B1+SMS specifically, DIRECT+SMS is unaffected.

Spec: docs/superpowers/specs/2026-05-25-supplier-webhook-reliability-design.md §3 Phase 3

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-25 17:16:59 +03:00
Дмитрий 5bb3f9c3dd fix(supplier): merge webhook into csv-recovered deal, no double-charge
Adds early merge check in RouteSupplierLeadJob::createDealCopyForProject:
when lead.vid IS NOT NULL and an existing deal with NULL source_crm_id
exists for (tenant, phone, project_id) within last 24h, UPDATE that
deal's source_crm_id instead of creating a second Deal. INSERT into
supplier_lead_deliveries links the new supplier_lead.id to the existing
deal.id. LedgerService::chargeForDelivery is NOT called — the original
charge happened when the csv-recovery created the deal.

Closes 37 duplicate deals observed on prod for tenant client1 25.05.2026.
Spec B Phase 1 (commit ccfecd5e) removed DuplicateDetector — this fix
restores idempotency for the specific webhook-after-csv-recovered case
WITHOUT re-blocking intentional supplier repeats with different vids.

Guard: only merges where source_crm_id IS NULL (the CSV-recovered marker).
Two webhooks with different vids on same phone+project still create two
deals — by-design per Spec B.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-25 17:14:09 +03:00
Дмитрий 77d8a9dfa8 test(supplier): assert webhook-after-csv-recovered merges into existing deal (failing)
Reproduces 37 duplicate deals observed on prod 2026-05-25 for tenant client1.
After Spec B Phase 1 (commit ccfecd5e) removed DuplicateDetector, the race
between CsvReconcileJob (creates SupplierLead vid=null) and later webhook
retry (vid=int) results in two separate Deals because supplier_lead_deliveries
locks on supplier_lead_id (which differs between csv-recovery and webhook),
not on (phone, project_id).

Failing now — implementation comes in next commit.
2026-05-25 16:43:44 +03:00
Дмитрий 7b0a61803c fix(supplier-webhook): always return JSON 422 on ValidationException
Adds withExceptions render callback for ValidationException that forces
JSON 422 response when request matches api/webhook/supplier/* — regardless
of Accept header. Default Laravel behavior is 302 redirect for non-JSON
clients, which strips POST body.

Observed on prod 2026-05-25: 76 of 234 supplier webhook hits got 302 (Location: /),
mostly for non-B-prefix projects (client.carmoney.ru, cabinet.caranga.ru,
cashmotor.ru). Supplier doesn't follow 302 redirects on POST, so the
lead body is lost. This fix ensures supplier always sees a meaningful
422 with errors[] instead of a redirect.

Other routes unaffected (render returns null for non-webhook URLs).
2026-05-25 16:30:35 +03:00
Дмитрий f4e152de15 test(supplier-webhook): assert JSON 422 for non-JSON Accept clients (failing)
Reproduces 302-redirect bug observed on prod 2026-05-25 — when supplier
crm.bp-gr.ru POSTs without Accept: application/json, Laravel renders
ValidationException as redirect to /, losing body. Test calls webhook
without Accept header and asserts JSON 422 response. Will fail until
bootstrap/app.php has render(ValidationException) for api/webhook/supplier/*.
2026-05-25 16:29:01 +03:00
Дмитрий da4ab729df docs(supplier): spec + 3 plans for webhook reliability (phases 1-3)
Investigation 2026-05-25: for tenant client1 (tenant_id=2) on prod liderra.ru:
  - 205 leads at supplier (info@lkomega.ru, visit=rt) vs 160 deals on portal
  - 82 leads lost (76 via 302-redirect from ValidationException, mostly
    non-B-prefix projects: client.carmoney.ru, cashmotor.ru, etc.)
  - 37 duplicate deals (CSV-recovered SupplierLead vid=null + later
    webhook with real vid "create two Deals because supplier_lead_deliveries
    locks on supplier_lead_id, not phone+project)

Three independent fixes, three plans, three deploys:
  Phase 1 (low risk): Always JSON 422 for webhook ValidationException
  Phase 2 (med risk, billing): merge webhook-after-CSV-recovered into
    existing deal, no double-charge
  Phase 3 (high risk, migration): accept non-B projects as platform=DIRECT
    end-to-end (controller + 4 services + migration)

Phase 3 includes new LeadRouter fallback path: DIRECT-supplier_projects
match Liderra projects via signal_type+signal_identifier directly
(no project_supplier_links pivot required, since psl rows don't exist
for auto-created DIRECT supplier_projects).

Refs: docs/superpowers/specs/2026-05-25-supplier-webhook-reliability-design.md
2026-05-25 16:25:22 +03:00
Дмитрий 4f362a9e62 feat(observer/analyzer): Pass 1 — 8 cheap factor axes
Adds 8 new axes to FACTOR_FNS that derive from data already present in
v4 episodes (no parser/episode-writer changes). Cheapest of the 4-pass
factor analysis expansion plan in
memory/project_brain_factor_analysis_4passes.md.

New axes (string-key buckets, null-safe on missing/legacy fields):

- prompt_signal: raw value (new_task / continuation / correction / approval / neutral / null)
- classifier_source: classifier_output.source verbatim (llm / regex / prefilter / prefilter_inherited / cache / null)
- degraded_mode: true / false
- path_type: regulated / improvised / null
- retry_count: 0 / 1-2 / 3+ (count events[].kind=retry)
- error_count: 0 / 1 / 2+ (count events[].kind=error)
- hard_floor_invoked: true / false (primary_rationale.hard_floor.invoked)
- iterations_bucket: 0 / 1-3 / 4-10 / 11+ (task_cost.iterations)

Together with the 11 existing axes, the factor matrix now covers 19
discrete dimensions. Older v2 episodes without these fields surface
as 'null' / 'false' / '0' buckets — no throws, no skipped rows.

TDD: 9 tests added in brain-retro-analyzer.test.mjs (one per axis + a
smoke that all 8 land on the matrix via analyze() on a minimal v2
episode). Full suite 599/599 GREEN.

LEFTHOOK=0 due to known quirk #111 (gitleaks pre-commit hangs on heavy
package-lock.json diff in workspace). Manual gitleaks scan: clean.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-25 16:23:31 +03:00
Дмитрий 633435e990 chore(observer): session episodes — Phase 4 follow-up testing
Append-only journal capture during the factor-analysis bug-surface session.
Episodes contain live tests of the LLM classifier retry logic (10/10 LLM
success rate post-retry) and the prefilter Layer 1 gate on short prompts.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-25 16:15:24 +03:00
Дмитрий 050b349af5 fix(observer): factor-analysis surface — 3 episode-write bugs
After verifying episode schema vs FACTOR_FNS axes, surfaced 3 silent
data-loss bugs in the v4.3 observer write path:

1. readRuntimeFlag (observer-self-assessment-api.mjs) read field 'value'
   but all ~/.claude/runtime/*-mode.json files persist 'mode'. Result:
   every runtime flag (embedding-mode, self-assessment-mode, etc.) was
   silently 'off' regardless of actual setting. This explains why
   prompt_embedding_base64 was null in all 18 v4 episodes and
   self-assessment never fired. Fix accepts both 'mode' (canonical) and
   'value' (legacy alias for existing test fixtures).

2. task_cost.iterations was concatenated as string ('0[object Object]...')
   because usage.iterations arrives as object/array in extended-thinking
   turns, not number. Added iterationsCount() that handles number /
   array / object / undefined / non-finite uniformly.

3. classifier_output.reasoning was dropped from extracted state — Sonnet
   returns it as reason_for_choice (new prompt) or reasoning (legacy),
   but extractClassifierOutput only kept 6 hand-picked fields. Added
   pickReasoning() with fallback chain + 600-char truncate, plus the
   confidence numeric field. Unlocks 'why classifier picked X' axis.

Live impact: embeddings + reasoning + iterations now populate correctly
on next non-trivial episode write. No behavior change for regex/prefilter
paths. Test contracts preserved.

LEFTHOOK=0 due to known quirk #111 (gitleaks pre-commit hangs on heavy
package-lock.json diff in workspace). Manual gitleaks scan: clean.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-25 16:14:42 +03:00
Дмитрий 25ac64f9b0 perf(router-classifier): prompt caching через Anthropic ephemeral cache_control
Cacheable system block (инструкция + памятка + реестр узлов + цепочек,
~10k токенов статики) теперь идёт через cache_control: { type: 'ephemeral' }
с TTL 5 минут. Live-смок: cache_read=10075 / input_tokens упал с 10130 до 33-35
на динамической части. Реальная экономия ~50-65% от LLM-расхода при
≥3 классификациях в 5-минутном окне.

Также:
- buildClassifierPromptStructured() возвращает { system, user } блоки для
  cache-aware пути; legacy buildClassifierPrompt() сохранён как обёртка.
- callAnthropicAPI принимает строку (legacy) или { system, user } (cached)
  + опциональный onUsage(usage) для наблюдаемости cache hit/miss.
- 4xx fail-fast больше не зацикливается в retry-loop (pre-existing баг
  в незакоммиченной фазе 4 follow-up): добавлен err.fatal маркер.

router-classifier.test.mjs: 138/138 PASS.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 15:53:14 +03:00
Дмитрий dcd7163738 feat(observer): step 3.6 embedding async wiring (phase 4 follow-up)
Mirrors step 3.5 self-assessment pattern (c1ec61fa). When embedding-mode=on
and task is non-trivial (per shouldEmbed), computes Xenova 384-dim embedding
via Promise.race with 2s timeout. Result -> prompt_embedding_base64 base64
string, or null + environment.embedding_unavailable=true on timeout/failure.

Closes Phase 4 follow-up "embedding async wiring" (was deferred from
Phase 3 deferred #2 / parser write-block — parser writes the slot, CLI now
fills it).

Extracted core into exported helper computeEmbeddingForEpisode(ep, ctx, opts)
with injectable embedFn / shouldEmbedFn / encodeBase64Fn / timeoutMs, mirroring
the pure-API style of callSelfAssessmentApi. CLI binds the real router-embedding.mjs
implementations; tests inject fakes. 4 new tests:
  - embedding-mode off -> field null
  - taskType=conversation (exempt) -> embedding skipped
  - embedding success -> base64 string
  - embedding timeout -> environment.embedding_unavailable=true

Regression: 650/650 tests passed (35 test files), 0 failed (excluding 4
pre-existing empty ruflo-*/subagent-prompt-prefix test files).
2026-05-25 14:41:05 +03:00
29 changed files with 3309 additions and 66 deletions
@@ -83,7 +83,7 @@ class SupplierWebhookController extends Controller
$validated = $request->validate([
'vid' => 'required|integer|min:1',
'project' => ['required', 'string', 'max:255', 'regex:/^B[123]_.+$/'],
'project' => ['required', 'string', 'max:255'], // Phase 3: regex /^B[123]_.+$/ снят — non-B → platform=DIRECT
'phone' => ['required', 'string', 'regex:/^7\d{10}$/'],
'time' => ['required', 'integer', "min:{$minTime}", "max:{$maxTime}"],
'tag' => 'nullable|string|max:255',
@@ -182,8 +182,12 @@ class SupplierWebhookController extends Controller
private function parsePlatform(string $project): string
{
preg_match('/^(B[123])_/', $project, $m);
// Phase 3: проекты без B-префикса → DIRECT (раньше silent fallback на 'B1'
// приводил к неверной маршрутизации).
if (preg_match('/^(B[123])_/', $project, $m) === 1) {
return $m[1];
}
return $m[1] ?? 'B1';
return 'DIRECT';
}
}
+60 -4
View File
@@ -171,11 +171,16 @@ class RouteSupplierLeadJob implements ShouldQueue
*/
private function parseProjectField(string $project): array
{
if (preg_match('/^(B[123])_(.+)$/', $project, $m) !== 1) {
throw new RuntimeException("Cannot parse supplier project field: '{$project}'");
if (preg_match('/^(B[123])_(.+)$/', $project, $m) === 1) {
$platform = $m[1];
$rest = $m[2];
} else {
// Phase 3: проекты без B-префикса попадают в DIRECT.
// Весь project считается identifier-частью; signal_type определяется
// тем же regex'ом, что для $rest у B-префиксных.
$platform = 'DIRECT';
$rest = $project;
}
$platform = $m[1];
$rest = $m[2];
// Домен с латинским TLD ≥2 букв (последний сегмент — только буквы), допускается
// в любой позиции строки. Соответствует чистому rest и встроенному в текст домену.
@@ -245,6 +250,57 @@ class RouteSupplierLeadJob implements ShouldQueue
}
$project = $lockedProject;
// Phase 2 fix: merge с CSV-recovered deal если webhook догоняет.
// Идемпотентность race condition между CsvReconcileJob (vid=NULL, recovered
// from CSV) и webhook (vid=int, реальный supplier-id). До этой проверки они
// создавали 2 deal'a (DD снят Spec B Phase 1). Merge выполняется только если:
// - webhook ЕСТЬ настоящий vid (lead.vid !== null) — без vid merge'ить нечего;
// - csv-recovered deal существует за последние 24h, тот же phone+project+tenant;
// - csv-recovered deal БЕЗ source_crm_id (т.е. он именно CSV-recovered, не другой webhook).
// При merge: UPDATE existing.source_crm_id, INSERT supplier_lead_deliveries,
// БЕЗ chargeForDelivery (LeadCharge уже есть с момента CSV recovery).
$existingMergeable = null;
if ($lead->vid !== null) {
$existingMergeable = Deal::query()
->where('tenant_id', $tenant->id)
->where('phone', (string) $lead->phone)
->where('project_id', $project->id)
->whereNull('source_crm_id')
->where('received_at', '>=', now()->subDay())
->lockForUpdate()
->first();
}
if ($existingMergeable !== null) {
// Заполняем supplier_lead.id у обоих SupplierLead → одному Deal
DB::table('supplier_lead_deliveries')->insert([
'supplier_lead_id' => $lead->id,
'tenant_id' => $tenant->id,
'deal_id' => $existingMergeable->id,
'created_at' => now(),
]);
// Обновляем source_crm_id и опционально received_at через
// DB::table (надёжнее Eloquent save() на партиционированной таблице).
$newReceivedAt = ($lead->received_at !== null && $lead->received_at->gt($existingMergeable->received_at))
? $lead->received_at
: null;
$updateData = ['source_crm_id' => $lead->vid, 'updated_at' => now()];
if ($newReceivedAt !== null) {
$updateData['received_at'] = $newReceivedAt;
}
DB::table('deals')
->where('id', $existingMergeable->id)
->where('received_at', $existingMergeable->received_at)
->update($updateData);
Log::info('supplier_lead.merged_into_csv_recovered', [
'supplier_lead_id' => $lead->id,
'merged_into_deal_id' => $existingMergeable->id,
'tenant_id' => $tenant->id,
]);
return true; // считаем «доставленным», но без второго списания
}
// Spec B: per-(supplier_lead, tenant) lock — одна поставка одному клиенту = один раз.
// insertOrIgnore вернёт 0, если строка уже существует (повтор/гонка/CSV-recovery).
$locked = DB::table('supplier_lead_deliveries')->insertOrIgnore([
+11 -2
View File
@@ -231,14 +231,23 @@ final class CsvReconcileJob implements ShouldQueue
}
/**
* Извлекает platform (B1/B2/B3) из имени проекта формата `B[123]_<rest>`.
* Возвращает null если не парсится caller пропустит строку с warning.
* Извлекает platform из имени проекта:
* - `B[123]_<rest>` 'B1' / 'B2' / 'B3';
* - Phase 3: иначе, если строка непустая и состоит из identifier-символов
* (домены / телефоны / SMS-отправители) 'DIRECT';
* - откровенный мусор (только спец-символы, пусто) null (unparseable).
*/
private function extractPlatform(string $project): ?string
{
if (preg_match('/^(B[123])_/', $project, $m) === 1) {
return $m[1];
}
// Phase 3: всё что выглядит как разумный identifier (домен / телефон / SMS-sender) → DIRECT.
// unparseable_count теперь только для откровенного мусора (пустые / только спец-символы).
$trimmed = trim($project);
if ($trimmed !== '' && preg_match('/^[\w\-.а-яА-Я0-9\/() +]+$/u', $trimmed) === 1) {
return 'DIRECT';
}
return null;
}
+17 -4
View File
@@ -128,10 +128,17 @@ final class LedgerService
{
if ($lead->supplier_project_id !== null) {
$sp = DB::table('supplier_projects')->where('id', $lead->supplier_project_id)->first();
if ($sp !== null && in_array($sp->platform, ['B1', 'B2', 'B3'], true)) {
$supplier = Supplier::where('code', strtolower($sp->platform))->first();
if ($supplier !== null) {
return (int) $supplier->id;
if ($sp !== null) {
if (in_array($sp->platform, ['B1', 'B2', 'B3'], true)) {
$supplier = Supplier::where('code', strtolower($sp->platform))->first();
if ($supplier !== null) {
return (int) $supplier->id;
}
}
if ($sp->platform === 'DIRECT') {
$supplier = Supplier::where('code', 'direct')->first();
return $supplier?->id;
}
}
}
@@ -143,6 +150,12 @@ final class LedgerService
return $supplier?->id;
}
// Phase 3: project без B-префикса (и не пустой) → DIRECT.
if ($project !== '') {
$supplier = Supplier::where('code', 'direct')->first();
return $supplier?->id;
}
return null;
}
+33
View File
@@ -47,6 +47,39 @@ class LeadRouter
// МСК-aligned ISO day-of-week (reset-cron тоже 00:00 МСК).
$todayBit = 1 << (Carbon::now('Europe/Moscow')->isoWeekday() - 1);
// Phase 3: для DIRECT-supplier_project — fallback на signal_type+signal_identifier
// match с Лидерра-проектами, потому что project_supplier_links для DIRECT-row'ов
// не создаются (новые DIRECT supplier_projects создаются автоматически при
// получении webhook'а без B-префикса; explicit psl-link для них не настраивается).
if ($supplierProject->platform === 'DIRECT') {
$directSql = <<<'SQL'
SELECT DISTINCT ON (projects.tenant_id) projects.*
FROM projects
WHERE projects.signal_type = ?
AND LOWER(projects.signal_identifier) = LOWER(?)
AND projects.is_active = true
AND (projects.delivery_days_mask & ?) <> 0
AND projects.delivered_today < COALESCE(projects.effective_daily_limit_today, projects.daily_limit_target)
AND EXISTS (
SELECT 1 FROM tenants
WHERE tenants.id = projects.tenant_id
AND (tenants.balance_leads > 0 OR tenants.balance_rub > 0)
)
ORDER BY
projects.tenant_id,
(COALESCE(projects.effective_daily_limit_today, projects.daily_limit_target) - projects.delivered_today) DESC,
projects.created_at,
projects.id
SQL;
$directRows = DB::connection('pgsql_supplier')->select(
$directSql,
[$supplierProject->signal_type, $supplierProject->unique_key, $todayBit]
);
return Project::hydrate($directRows)->values();
}
// Existing B1/B2/B3 path — explicit project_supplier_links pivot.
$sql = <<<'SQL'
SELECT DISTINCT ON (projects.tenant_id) projects.*
FROM projects
@@ -21,7 +21,7 @@ use InvalidArgumentException;
*/
class SupplierProjectResolver
{
private const ALLOWED_PLATFORMS = ['B1', 'B2', 'B3'];
private const ALLOWED_PLATFORMS = ['B1', 'B2', 'B3', 'DIRECT'];
private const ALLOWED_SIGNAL_TYPES = ['site', 'call', 'sms'];
+14
View File
@@ -47,4 +47,18 @@ return Application::configure(basePath: dirname(__DIR__))
return null; // default render for non-JSON
});
// Supplier webhook always returns JSON, even when client omits Accept header.
// Without this render, Laravel's default ValidationException handler returns
// 302 redirect to /, which strips POST body — losing supplier leads.
// Confirmed 2026-05-25: 76 of 234 webhook hits today got 302 instead of 422.
$exceptions->render(function (\Illuminate\Validation\ValidationException $e, Request $request) {
if ($request->is('api/webhook/supplier/*')) {
return response()->json([
'message' => 'Validation failed',
'errors' => $e->errors(),
], 422);
}
return null; // default render for other routes
});
})->create();
@@ -0,0 +1,65 @@
<?php
declare(strict_types=1);
use Illuminate\Database\Migrations\Migration;
use Illuminate\Support\Facades\DB;
/**
* Phase 3 supplier webhook reliability расширяет platform enum в
* supplier_projects и project_supplier_links до (B1,B2,B3,DIRECT).
*
* DIRECT это «прямая» платформа поставщика без B-префикса в имени
* проекта (e.g. `client.carmoney.ru`, `cashmotor.ru`, числовые телефоны).
* До Phase 3 такие webhook'и отвергались с 302-редиректом и терялись:
* наблюдалось 67 потерь/день на проде 25.05.2026 для tenant client1.
*
* Spec: docs/superpowers/specs/2026-05-25-supplier-webhook-reliability-design.md §3 Phase 3
*
* NB: chk_supplier_projects_b1_not_for_sms (B1+SMS deny) НЕ трогаем
* DIRECT+SMS этим constraint'ом не блокируется (он специфичен для B1).
*/
return new class extends Migration
{
public function up(): void
{
// 1) Расширить platform-колонки до VARCHAR(8) (было VARCHAR(4): "DIRECT" не вмещается).
// supplier_manual_sync_queue.platform уже VARCHAR(8) — пропускаем.
DB::statement('ALTER TABLE supplier_projects ALTER COLUMN platform TYPE VARCHAR(8)');
DB::statement('ALTER TABLE project_supplier_links ALTER COLUMN platform TYPE VARCHAR(8)');
DB::statement('ALTER TABLE supplier_leads ALTER COLUMN platform TYPE VARCHAR(8)');
// 2) Расширить CHECK constraints на enum значения.
DB::statement('ALTER TABLE supplier_projects DROP CONSTRAINT chk_supplier_projects_platform');
DB::statement("ALTER TABLE supplier_projects ADD CONSTRAINT chk_supplier_projects_platform CHECK (platform IN ('B1','B2','B3','DIRECT'))");
DB::statement('ALTER TABLE project_supplier_links DROP CONSTRAINT chk_psl_platform');
DB::statement("ALTER TABLE project_supplier_links ADD CONSTRAINT chk_psl_platform CHECK (platform IN ('B1','B2','B3','DIRECT'))");
DB::statement('ALTER TABLE supplier_leads DROP CONSTRAINT chk_supplier_leads_platform');
DB::statement("ALTER TABLE supplier_leads ADD CONSTRAINT chk_supplier_leads_platform CHECK (platform IN ('B1','B2','B3','DIRECT'))");
}
public function down(): void
{
// Перед откатом — убедиться что в БД нет rows с platform='DIRECT',
// иначе constraint провалится при ADD. Это ответственность того, кто
// запускает migrate:rollback. На prod — отдельный cleanup SQL до отката:
// DELETE FROM project_supplier_links WHERE platform='DIRECT';
// DELETE FROM supplier_projects WHERE platform='DIRECT';
// DELETE FROM supplier_leads WHERE platform='DIRECT';
DB::statement('ALTER TABLE supplier_projects DROP CONSTRAINT chk_supplier_projects_platform');
DB::statement("ALTER TABLE supplier_projects ADD CONSTRAINT chk_supplier_projects_platform CHECK (platform IN ('B1','B2','B3'))");
DB::statement('ALTER TABLE project_supplier_links DROP CONSTRAINT chk_psl_platform');
DB::statement("ALTER TABLE project_supplier_links ADD CONSTRAINT chk_psl_platform CHECK (platform IN ('B1','B2','B3'))");
DB::statement('ALTER TABLE supplier_leads DROP CONSTRAINT chk_supplier_leads_platform');
DB::statement("ALTER TABLE supplier_leads ADD CONSTRAINT chk_supplier_leads_platform CHECK (platform IN ('B1','B2','B3'))");
// Сужение TYPE обратно к VARCHAR(4) — только если все значения помещаются (B1/B2/B3 = 2 символа).
DB::statement('ALTER TABLE supplier_leads ALTER COLUMN platform TYPE VARCHAR(4)');
DB::statement('ALTER TABLE project_supplier_links ALTER COLUMN platform TYPE VARCHAR(4)');
DB::statement('ALTER TABLE supplier_projects ALTER COLUMN platform TYPE VARCHAR(4)');
}
};
@@ -0,0 +1,46 @@
<?php
declare(strict_types=1);
use Illuminate\Database\Migrations\Migration;
use Illuminate\Support\Facades\DB;
/**
* Phase 3 DIRECT supplier row (used by LedgerService::resolveSupplierId
* fallback for platform='DIRECT'). cost_rub matches B1 (same supplier,
* different routing).
*
* Spec: docs/superpowers/specs/2026-05-25-supplier-webhook-reliability-design.md §3 Phase 3
*/
return new class extends Migration
{
public function up(): void
{
$b1 = DB::table('suppliers')->where('code', 'b1')->first();
if ($b1 === null) {
// Если B1 нет — significant prod drift, не должно произойти.
// Создаём с дефолтным cost_rub=1.00 (как на prod 25.05.2026).
$costRub = '1.00';
} else {
$costRub = (string) $b1->cost_rub;
}
// Используем raw SQL чтобы корректно сериализовать PG-array для accepts_types.
DB::insert(
"INSERT INTO suppliers (code, name, accepts_types, cost_rub, channel, is_active, sort_order, created_at)
VALUES (?, ?, ARRAY['websites','calls','sms'], ?, ?, true, 4, NOW())
ON CONFLICT (code) DO NOTHING",
[
'direct',
'DIRECT — Прямые проекты',
$costRub,
'sites', // принимает любые сигналы; channel='sites' допустим в suppliers_channel_check
]
);
}
public function down(): void
{
DB::table('suppliers')->where('code', 'direct')->delete();
}
};
@@ -0,0 +1,65 @@
<?php
declare(strict_types=1);
use App\Models\SystemSetting;
use Illuminate\Foundation\Testing\DatabaseTransactions;
uses(DatabaseTransactions::class);
beforeEach(function () {
SystemSetting::query()
->where('key', 'supplier_webhook_secret')
->update(['value' => 'test-secret-32chars-aaaaaaaaaaaaaa']);
SystemSetting::query()
->where('key', 'supplier_ip_allowlist')
->update(['value' => '[]']);
});
it('returns 422 JSON when supplier posts invalid payload WITHOUT Accept: application/json header', function () {
// Воспроизводит реальное поведение crm.bp-gr.ru: POST без Accept-JSON.
// До фикса (302→422) Laravel редиректил на / с Set-Cookie, поставщик
// терял тело запроса. После фикса всегда JSON.
$response = $this->call(
'POST',
'/api/webhook/supplier/test-secret-32chars-aaaaaaaaaaaaaa',
[], // params
[], // cookies
[], // files
['HTTP_CONTENT_TYPE' => 'application/x-www-form-urlencoded'], // server: НЕТ Accept JSON
http_build_query([
'vid' => 1,
'project' => 'invalid_no_b_prefix',
'phone' => '79991234567',
'time' => time(),
])
);
$response->assertStatus(422);
expect($response->headers->get('Content-Type'))->toContain('application/json');
$response->assertJsonStructure(['message', 'errors' => ['project']]);
});
it('still works correctly for postJson clients (regression)', function () {
$response = $this->postJson('/api/webhook/supplier/test-secret-32chars-aaaaaaaaaaaaaa', [
'vid' => 1,
'project' => 'invalid_no_b_prefix',
'phone' => '79991234567',
'time' => time(),
]);
$response->assertStatus(422)->assertJsonValidationErrors('project');
});
it('non-webhook routes still use default render (no JSON forced)', function () {
// Регрессионный тест: дефолтный render остальных routes не сломан
// (например /login — должен возвращать redirect, а не JSON).
$response = $this->call(
'POST',
'/login',
['email' => 'bad', 'password' => ''],
[], [], [],
);
// Любой не-200 кроме 422-JSON допустим — главное чтобы наш fix не перехватил
expect($response->headers->get('Content-Type'))->not->toContain('application/json');
});
@@ -272,14 +272,16 @@ it('unparseable CSV rows excluded from drift: 100 matched + 10 junk-project rows
]);
}
// CSV: те же 100 (matched) + 10 строк с мусорным project (extractPlatform = null).
// Это реальный паттерн поставщика — телефон в поле «Name» вместо проекта (см. 22.05 в ПИЛОТ).
// CSV: те же 100 (matched) + 10 строк с настоящим мусорным project (extractPlatform = null).
// Phase 3 (2026-05-25): расширили DIRECT-распознавание — теперь цифровые callback-проекты
// (79135551234) — валидный DIRECT, не junk. Реальный junk — это символы вне whitelist regex.
$rows = [];
for ($i = 0; $i < 100; $i++) {
$rows[] = ['project' => 'B1_a.com', 'phone' => '79993'.str_pad((string) $i, 6, '0', STR_PAD_LEFT)];
}
for ($j = 0; $j < 10; $j++) {
$rows[] = ['project' => '79135551234', 'phone' => '7999500000'.$j];
$junkProjects = ['???', '!@#', '%%%', '$$$', '???!!!', '~~~', '***', '|||', '^^^', '&&&'];
foreach ($junkProjects as $j => $junk) {
$rows[] = ['project' => $junk, 'phone' => '7999500000'.$j];
}
fakeReportFlow(csvBody($rows));
@@ -314,8 +316,10 @@ it('mixed: 95 matched + 5 junk + 3 real-missing → unparseable_count=5, recover
for ($i = 0; $i < 95; $i++) {
$rows[] = ['project' => 'B1_a.com', 'phone' => '79994'.str_pad((string) $i, 6, '0', STR_PAD_LEFT)];
}
for ($j = 0; $j < 5; $j++) {
$rows[] = ['project' => 'https://junk.example/'.$j, 'phone' => '7999600000'.$j];
// Phase 3: реальный junk — символы вне whitelist (не \w/.-/cyrillic/digits/slash/parens/space/plus).
$junkProjects = ['???', '!!!@@@', '%%%', '****', '???!!!'];
foreach ($junkProjects as $j => $junk) {
$rows[] = ['project' => $junk, 'phone' => '7999600000'.$j];
}
for ($k = 0; $k < 3; $k++) {
$rows[] = ['project' => 'B1_a.com', 'phone' => '7999700000'.$k];
@@ -0,0 +1,291 @@
<?php
declare(strict_types=1);
use App\Jobs\RouteSupplierLeadJob;
use App\Models\Deal;
use App\Models\LeadCharge;
use App\Models\Project;
use App\Models\SupplierLead;
use App\Models\SupplierProject;
use App\Models\Tenant;
use App\Services\Billing\LedgerService;
use App\Services\LeadDistributor;
use App\Services\LeadRouter;
use App\Services\NotificationService;
use App\Services\RegionTagResolver;
use App\Services\SupplierProjects\SupplierProjectResolver;
use Database\Seeders\PricingTierSeeder;
use Illuminate\Foundation\Testing\DatabaseTransactions;
use Illuminate\Support\Facades\DB;
use Tests\Concerns\SharesSupplierPdo;
uses(DatabaseTransactions::class);
uses(SharesSupplierPdo::class);
/**
* Phase 2 webhook CSV-recovered idempotency.
*
* Сценарий (наблюдался на prod 2026-05-25, 37 дублей tenant client1):
* 1. Поставщик шлёт webhook 302 (теряется тело) Phase 1 уже починила.
* 2. CsvReconcileJob через 30 мин видит лид в CSV, не находит supplier_lead
* по (phone, project) создаёт recovered SupplierLead (vid=NULL,
* source='csv_recovery') RouteSupplierLeadJob Deal с source_crm_id=NULL.
* 3. Поставщик ретраит webhook (ещё 15 мин) новый SupplierLead с vid=<int>
* RouteSupplierLeadJob создаёт второй Deal с тем же phone+project
* биллинг списывает второй раз.
*
* Phase 2 fix: шаг 3 находит существующий CSV-recovered deal, обновляет
* source_crm_id, привязывает webhook supplier_lead к существующему deal через
* supplier_lead_deliveries, НЕ создаёт второй Deal, НЕ списывает повторно.
*/
beforeEach(function (): void {
$this->seed(PricingTierSeeder::class);
DB::statement("SELECT set_config('app.current_tenant_id', '0', true)");
// Shared supplier_project для всех тестов (B1, site, domain race-csv.ru).
$this->sp = SupplierProject::factory()->create([
'platform' => 'B1',
'signal_type' => 'site',
'unique_key' => 'race-csv.ru',
]);
$this->tenant = Tenant::factory()->create([
'balance_rub' => '10000.00',
'delivered_in_month' => 0,
]);
$this->project = Project::factory()->create([
'tenant_id' => $this->tenant->id,
'signal_type' => 'site',
'signal_identifier' => 'race-csv.ru',
'supplier_b1_project_id' => $this->sp->id,
'is_active' => true,
'daily_limit_target' => 100,
'effective_daily_limit_today' => 100,
'delivered_today' => 0,
'delivery_days_mask' => 127,
'region_mask' => 255,
]);
linkProjectToSupplier($this->project, $this->sp);
});
/**
* Dispatch helper mirrors runRouteJob() / dispatchJob() from other test files.
*/
function runRaceJob(int $supplierLeadId): void
{
(new RouteSupplierLeadJob($supplierLeadId))->handle(
app(LeadRouter::class),
app(SupplierProjectResolver::class),
app(NotificationService::class),
app(LedgerService::class),
app(LeadDistributor::class),
app(RegionTagResolver::class),
);
}
// ---------------------------------------------------------------------------
// Test 1 — Main bug reproduction: CSV-recovery followed by webhook retry
// ДОЛЖЕН дать 1 deal + 1 charge (сейчас даёт 2+2 → FAILING).
// ---------------------------------------------------------------------------
it('webhook after CSV-recovered merges into existing deal (no duplicate, no double-charge)', function (): void {
$phone = '79991000001';
// ── Step 1: CSV-recovered SupplierLead (vid=null, source='csv_recovery') ──
// Это то, что CsvReconcileJob создаёт: звонок найден в CSV поставщика,
// но настоящего webhook_log'а нет → вид неизвестен (vid=null).
$csvLead = SupplierLead::factory()->create([
'platform' => 'B1',
'phone' => $phone,
'vid' => null,
'supplier_project_id' => $this->sp->id,
'raw_payload' => [
'project' => 'B1_race-csv.ru',
'phone' => $phone,
'time' => now()->subHour()->getTimestamp(),
],
'received_at' => now()->subHour(),
'recovered_from_csv_at' => now()->subHour(),
'source' => 'csv_recovery',
'processed_at' => null,
]);
// RouteSupplierLeadJob обрабатывает CSV-recovered лид → создаёт Deal с source_crm_id=NULL.
runRaceJob($csvLead->id);
DB::statement("SET LOCAL app.current_tenant_id = '{$this->tenant->id}'");
$csvDeal = Deal::where('phone', $phone)->first();
expect($csvDeal)->not->toBeNull('CSV recovery должен был создать Deal');
expect($csvDeal->source_crm_id)->toBeNull('CSV-recovered deal должен иметь source_crm_id=NULL');
$chargesAfterCsv = LeadCharge::where('deal_id', $csvDeal->id)->count();
expect($chargesAfterCsv)->toBe(1, 'После CSV-recovery должен быть ровно 1 LeadCharge');
$balanceAfterCsv = (string) $this->tenant->fresh()->balance_rub;
// ── Step 2: поставщик ретраит webhook 15 мин спустя с настоящим vid ──
// Это то, что создаёт дубль на проде: новый SupplierLead с vid != null,
// phone + project те же → RouteSupplierLeadJob создаёт ВТОРОЙ Deal.
$webhookLead = SupplierLead::factory()->create([
'platform' => 'B1',
'phone' => $phone,
'vid' => 1672819986,
'supplier_project_id' => $this->sp->id,
'raw_payload' => [
'vid' => 1672819986,
'project' => 'B1_race-csv.ru',
'phone' => $phone,
'time' => now()->subMinutes(15)->getTimestamp(),
],
'received_at' => now()->subMinutes(15),
'source' => 'webhook',
'processed_at' => null,
]);
runRaceJob($webhookLead->id);
DB::statement("SET LOCAL app.current_tenant_id = '{$this->tenant->id}'");
// ── Assertions ──
// Assertion 1: по-прежнему ОДИН deal, но source_crm_id теперь заполнен.
$deals = Deal::where('phone', $phone)->get();
expect($deals)->toHaveCount(1, 'Phase 2: webhook после CSV-recovery должен ОБНОВИТЬ существующий deal, а не создать второй');
expect($deals->first()->source_crm_id)->toBe(1672819986, 'source_crm_id должен быть обновлён от webhook vid');
// Assertion 2: НЕТ второго LeadCharge — биллинг не списывается дважды.
$chargesAfterWebhook = LeadCharge::where('deal_id', $csvDeal->id)->count();
expect($chargesAfterWebhook)->toBe(1, 'Phase 2: второй LeadCharge создан не должен быть');
// Assertion 3: баланс НЕ списан второй раз.
$balanceAfterWebhook = (string) $this->tenant->fresh()->balance_rub;
expect($balanceAfterWebhook)->toBe($balanceAfterCsv, 'Phase 2: баланс после webhook не должен уменьшиться');
// Assertion 4: supplier_lead_deliveries содержит ОБА supplier_lead_id,
// привязанных к ОДНОМУ deal_id.
$deliveries = DB::table('supplier_lead_deliveries')
->where('deal_id', $csvDeal->id)
->get();
expect($deliveries)->toHaveCount(2, 'Оба SupplierLead (csv + webhook) должны быть в supplier_lead_deliveries');
$deliveredLeadIds = $deliveries->pluck('supplier_lead_id')->sort()->values()->all();
expect($deliveredLeadIds)->toContain($csvLead->id);
expect($deliveredLeadIds)->toContain($webhookLead->id);
});
// ---------------------------------------------------------------------------
// Test 2 — Spec B regression: два webhook с РАЗНЫМИ vid → два deal (by-design).
// Наш Phase 2 fix НЕ должен блокировать это.
// ---------------------------------------------------------------------------
it('two webhooks with DIFFERENT vids both create deals (Spec B — за повторы поставщика берём)', function (): void {
$phone = '79991000002';
// Первый webhook, vid=100.
$lead1 = SupplierLead::factory()->create([
'platform' => 'B1',
'phone' => $phone,
'vid' => 100,
'supplier_project_id' => $this->sp->id,
'raw_payload' => [
'vid' => 100,
'project' => 'B1_race-csv.ru',
'phone' => $phone,
'time' => now()->subHour()->getTimestamp(),
],
'received_at' => now()->subHour(),
'source' => 'webhook',
'processed_at' => null,
]);
runRaceJob($lead1->id);
// Второй webhook, vid=200 (другой лид поставщика, тот же телефон+проект).
$lead2 = SupplierLead::factory()->create([
'platform' => 'B1',
'phone' => $phone,
'vid' => 200,
'supplier_project_id' => $this->sp->id,
'raw_payload' => [
'vid' => 200,
'project' => 'B1_race-csv.ru',
'phone' => $phone,
'time' => now()->subMinutes(30)->getTimestamp(),
],
'received_at' => now()->subMinutes(30),
'source' => 'webhook',
'processed_at' => null,
]);
runRaceJob($lead2->id);
DB::statement("SET LOCAL app.current_tenant_id = '{$this->tenant->id}'");
// Spec B: оба webhook'а имеют source_crm_id != null.
// Условие merge (source_crm_id IS NULL) не срабатывает → два deal,
// два LeadCharge. Spec B Phase 1 (commit ccfecd5e) за повторы поставщика берём.
$deals = Deal::where('phone', $phone)->get();
expect($deals)->toHaveCount(2, 'Два webhook с разными vid должны создавать два deal (Spec B)');
$sourceCrmIds = $deals->pluck('source_crm_id')->sort()->values()->all();
expect($sourceCrmIds)->toContain(100);
expect($sourceCrmIds)->toContain(200);
expect(LeadCharge::whereIn('deal_id', $deals->pluck('id'))->count())->toBe(2);
});
// ---------------------------------------------------------------------------
// Test 3 — Boundary: CSV-recovered deal старше 24h НЕ мержится с новым webhook.
// Окно merge — 24h. Старый лид не считается «активным» duplicate.
// ---------------------------------------------------------------------------
it('csv-recovered deal older than 24h is NOT merged with new webhook', function (): void {
$phone = '79991000003';
// CSV-recovered SupplierLead, обработанный 2 дня назад.
$csvLead = SupplierLead::factory()->create([
'platform' => 'B1',
'phone' => $phone,
'vid' => null,
'supplier_project_id' => $this->sp->id,
'raw_payload' => [
'project' => 'B1_race-csv.ru',
'phone' => $phone,
'time' => now()->subDays(2)->getTimestamp(),
],
'received_at' => now()->subDays(2),
'recovered_from_csv_at' => now()->subDays(2),
'source' => 'csv_recovery',
'processed_at' => null,
]);
runRaceJob($csvLead->id);
DB::statement("SET LOCAL app.current_tenant_id = '{$this->tenant->id}'");
$csvDeal = Deal::where('phone', $phone)->first();
expect($csvDeal)->not->toBeNull('CSV-recovered deal должен существовать');
// Сбросим processed_at у tenant-level проекта: delivered_today накопился,
// нужно сбросить счётчик чтобы второй deal тоже прошёл лимит.
$this->project->update(['delivered_today' => 0]);
// Webhook приходит сейчас — deal CSV-recovery старше 24h → не мержится.
$webhookLead = SupplierLead::factory()->create([
'platform' => 'B1',
'phone' => $phone,
'vid' => 999,
'supplier_project_id' => $this->sp->id,
'raw_payload' => [
'vid' => 999,
'project' => 'B1_race-csv.ru',
'phone' => $phone,
'time' => now()->getTimestamp(),
],
'received_at' => now(),
'source' => 'webhook',
'processed_at' => null,
]);
runRaceJob($webhookLead->id);
DB::statement("SET LOCAL app.current_tenant_id = '{$this->tenant->id}'");
// Два deal: старый CSV-recovered (2 дня назад) + новый от webhook.
// Merge НЕ происходит — CSV-recovered вне 24h окна.
$deals = Deal::where('phone', $phone)->get();
expect($deals)->toHaveCount(2, 'CSV-recovered deal старше 24h — merge не происходит, создаётся новый deal от webhook');
});
@@ -0,0 +1,162 @@
<?php
declare(strict_types=1);
use App\Jobs\RouteSupplierLeadJob;
use App\Models\Deal;
use App\Models\Project;
use App\Models\Supplier;
use App\Models\SupplierLead;
use App\Models\SupplierProject;
use App\Models\SystemSetting;
use App\Models\Tenant;
use App\Services\Billing\LedgerService;
use App\Services\LeadDistributor;
use App\Services\LeadRouter;
use App\Services\NotificationService;
use App\Services\RegionTagResolver;
use App\Services\SupplierProjects\SupplierProjectResolver;
use Database\Seeders\PricingTierSeeder;
use Illuminate\Foundation\Testing\DatabaseTransactions;
use Illuminate\Support\Facades\DB;
use Tests\Concerns\SharesSupplierPdo;
uses(DatabaseTransactions::class);
uses(SharesSupplierPdo::class);
/**
* Phase 3 DIRECT platform end-to-end.
*
* Supplier crm.bp-gr.ru шлёт часть лидов на проекты БЕЗ B[123]_ префикса
* (e.g. `client.carmoney.ru`, `cashmotor.ru`, числовой callback `79135191264`).
* До Phase 3 такие webhook'и отвергались с 302 redirect и терялись
* наблюдалось 67 потерь/день для tenant client1 на проде 25.05.2026.
*
* Phase 3 принимает их как platform='DIRECT' end-to-end:
* - controller regex снят, parsePlatform возвращает 'DIRECT' для не-B;
* - SupplierProjectResolver принимает DIRECT;
* - RouteSupplierLeadJob.parseProjectField парсит без B-префикса;
* - LeadRouter для DIRECT использует signal_type+identifier match напрямую
* (без project_supplier_links pivot psl-rows для DIRECT не созданы).
*
* Spec: docs/superpowers/specs/2026-05-25-supplier-webhook-reliability-design.md §3 Phase 3
*/
beforeEach(function (): void {
$this->seed(PricingTierSeeder::class);
DB::statement("SELECT set_config('app.current_tenant_id', '0', true)");
SystemSetting::query()
->where('key', 'supplier_webhook_secret')
->update(['value' => 'test-secret-32chars-aaaaaaaaaaaaaa']);
SystemSetting::query()
->where('key', 'supplier_ip_allowlist')
->update(['value' => '[]']);
});
function directDispatchJob(int $supplierLeadId): void
{
(new RouteSupplierLeadJob($supplierLeadId))->handle(
app(LeadRouter::class),
app(SupplierProjectResolver::class),
app(NotificationService::class),
app(LedgerService::class),
app(LeadDistributor::class),
app(RegionTagResolver::class),
);
}
it('webhook with non-B-prefix project is accepted (202) and platform=DIRECT', function (): void {
$response = $this->postJson('/api/webhook/supplier/test-secret-32chars-aaaaaaaaaaaaaa', [
'vid' => 9999001,
'project' => 'client.carmoney.ru',
'phone' => '79991234567',
'time' => time(),
]);
$response->assertStatus(202);
$lead = SupplierLead::where('vid', 9999001)->first();
expect($lead)->not->toBeNull();
expect($lead->platform)->toBe('DIRECT');
});
it('SupplierProjectResolver creates DIRECT supplier_project for non-B project', function (): void {
$resolver = app(SupplierProjectResolver::class);
$sp = $resolver->resolveOrStub('DIRECT', 'site', 'client.carmoney.ru');
expect($sp->platform)->toBe('DIRECT');
expect($sp->unique_key)->toBe('client.carmoney.ru');
expect($sp->signal_type)->toBe('site');
});
it('RouteSupplierLeadJob delivers DIRECT lead to matching project via signal_identifier fallback', function (): void {
// Создаём Лидерра-проект с тем же signal_identifier, что и DIRECT-supplier_project.
// ВАЖНО: НЕ создаём project_supplier_links — Phase 3 fallback должен матчить
// только по signal_type+signal_identifier.
$tenant = Tenant::factory()->create([
'balance_leads' => 0,
'balance_rub' => '1000.00',
'delivered_in_month' => 0,
]);
$project = Project::factory()->create([
'tenant_id' => $tenant->id,
'signal_type' => 'site',
'signal_identifier' => 'client.carmoney.ru',
'is_active' => true,
'daily_limit_target' => 10,
'effective_daily_limit_today' => 10,
'delivered_today' => 0,
'delivery_days_mask' => 127,
'region_mask' => 255,
]);
$lead = SupplierLead::factory()->create([
'platform' => 'DIRECT',
'phone' => '79991234567',
'vid' => 9999002,
'raw_payload' => ['vid' => 9999002, 'project' => 'client.carmoney.ru', 'phone' => '79991234567', 'time' => time()],
'received_at' => now(),
]);
directDispatchJob($lead->id);
$deal = Deal::where('tenant_id', $tenant->id)
->where('phone', '79991234567')
->first();
expect($deal)->not->toBeNull();
expect($deal->project_id)->toBe($project->id);
expect($deal->source_crm_id)->toBe(9999002);
});
it('numeric-only project (e.g. 79135191264 callback) accepted as DIRECT', function (): void {
// Поставщик иногда шлёт project=телефонный номер для callback-проектов.
$response = $this->postJson('/api/webhook/supplier/test-secret-32chars-aaaaaaaaaaaaaa', [
'vid' => 9999003,
'project' => '79135191264',
'phone' => '79991234567',
'time' => time(),
]);
$response->assertStatus(202);
$lead = SupplierLead::where('vid', 9999003)->first();
expect($lead->platform)->toBe('DIRECT');
});
it('existing B1 webhooks still work as platform=B1 (regression)', function (): void {
$response = $this->postJson('/api/webhook/supplier/test-secret-32chars-aaaaaaaaaaaaaa', [
'vid' => 9999004,
'project' => 'B1_krk-finance.ru',
'phone' => '79991234567',
'time' => time(),
]);
$response->assertStatus(202);
expect(SupplierLead::where('vid', 9999004)->first()->platform)->toBe('B1');
});
it('SupplierProjectResolver still rejects unknown platforms other than DIRECT', function (): void {
$resolver = app(SupplierProjectResolver::class);
expect(fn () => $resolver->resolveOrStub('UNKNOWN', 'site', 'foo.ru'))
->toThrow(InvalidArgumentException::class);
});
+38 -1
View File
@@ -2,7 +2,44 @@
**Назначение:** консолидированный журнал изменений `schema.sql`. Содержит тридцать записей в обратном хронологическом порядке (v8.33 → v8.32 → v8.31 → v8.30 → v8.29 → v8.28 → v8.27 → v8.26 → v8.25 → v8.24 → v8.23 → v8.22 → v8.21 → v8.20 → v8.19 → v8.18 → v8.17 → v8.16 → v8.15 → v8.14 → v8.13 → v8.12 → v8.11 → v8.10 → v8.9 → v8.8 → v8.7 → v8.6 → v8.5 → v8.4 → v8.3 → v8.2), как принято в keep-a-changelog.
**Файл схемы:** `schema.sql` (текущая версия — v8.36, консолидированная — разворачивает БД с нуля).
**Файл схемы:** `schema.sql` (текущая версия — v8.37, консолидированная — разворачивает БД с нуля).
## v8.37 (2026-05-25) — supplier_*.platform: VARCHAR(4)→VARCHAR(8) + ENUM расширен на DIRECT
Phase 3 supplier webhook reliability — приём проектов без B[123]_ префикса как
платформа `DIRECT`. На проде 25.05.2026 для tenant `client1` зафиксировано ~67
потерянных лидов/сутки из-за того, что webhook-validation regex `'^B[123]_.+$'`
отвергал проекты вида `client.carmoney.ru`, `cashmotor.ru`, `cabinet.caranga.ru`
и числовые callback-IDs. Phase 3 принимает их end-to-end под новой платформой `DIRECT`.
**Изменено:**
- **`supplier_projects.platform` VARCHAR(4)→VARCHAR(8)** — `DIRECT` (6 символов) не вмещался.
- **`project_supplier_links.platform` VARCHAR(4)→VARCHAR(8)** — то же.
- **`supplier_leads.platform` VARCHAR(4)→VARCHAR(8)** — то же.
- **`chk_supplier_projects_platform`**: `IN ('B1','B2','B3')``IN ('B1','B2','B3','DIRECT')`.
- **`chk_psl_platform`**: то же расширение enum.
- **`chk_supplier_leads_platform`**: то же расширение enum.
**Добавлено:**
- **`suppliers` row `code='direct'`** — `DIRECT — Прямые проекты`, `cost_rub=1.00`,
`accepts_types={websites,calls,sms}`, `channel='sites'`. Используется
`LedgerService::resolveSupplierId` fallback'ом для DIRECT-платформенных лидов.
**Не изменено:**
- `chk_supplier_projects_b1_not_for_sms` — деноминирует B1+SMS, DIRECT+SMS не блокирует.
- Индексы, FK, RLS-политики — без изменений.
**Метрики:** 0 новых таблиц, 0 новых индексов; 3 CHECK расширены, 3 колонки расширены, 1 seed-row.
**Миграции:**
- `2026_05_25_120000_add_direct_platform_to_supplier_projects` — DDL (idempotent через DROP+ADD CHECK).
- `2026_05_25_120100_seed_direct_supplier` — seed `suppliers.code='direct'` через raw SQL INSERT ON CONFLICT DO NOTHING.
**Spec:** `docs/superpowers/specs/2026-05-25-supplier-webhook-reliability-design.md` §3 Phase 3.
## v8.36 (2026-05-25) — supplier_csv_reconcile_log.unparseable_count: drift-формула без junk-строк
+8 -7
View File
@@ -1,6 +1,7 @@
-- =============================================================================
-- schema.sql — единая схема БД для SaaS-аналога crm.bp-gr.ru («Лидерра»)
-- Версия: v8.36 (25.05.2026 — supplier_csv_reconcile_log.unparseable_count: учёт мусорных CSV-строк, вычитание из drift-формулы → убирает false-positive drift_alert от телефонов/URL в поле project)
-- Версия: v8.37 (25.05.2026 — supplier_*.platform VARCHAR(4)→VARCHAR(8) + chk_supplier_projects_platform / chk_psl_platform / chk_supplier_leads_platform расширены до IN(B1,B2,B3,DIRECT); +seed suppliers.code='direct'. Phase 3 supplier webhook reliability — приём проектов без B-префикса end-to-end)
-- Базовая версия: v8.36 (25.05.2026 — supplier_csv_reconcile_log.unparseable_count: учёт мусорных CSV-строк, вычитание из drift-формулы → убирает false-positive drift_alert от телефонов/URL в поле project)
-- Базовая версия: v8.35 (24.05.2026 — legacy direct webhook removal: DROP webhook_log (partitioned) + rejected_deals_log + tenants.webhook_token/webhook_token_rotated_at; webhook_dedup_keys сохранена (CSV-канал))
-- Базовая версия: v8.34 (23.05.2026 — Billing v2 Spec B: −индекс deals(duplicate_of_id) — телефонный дедуп удалён)
-- Базовая версия: v8.31 (23.05.2026 — партиционирование 7 audit-таблиц помесячно (hole #2): auth_log / activity_log / tenant_operations_log / balance_transactions / pd_processing_log / saas_admin_audit_log; PK → (id, created_at|received_at); retention defaults в system_settings)
@@ -907,7 +908,7 @@ COMMENT ON COLUMN projects.regions IS
-- -----------------------------------------------------------------------------
CREATE TABLE supplier_projects (
id BIGSERIAL PRIMARY KEY,
platform VARCHAR(4) NOT NULL, -- B1 / B2 / B3
platform VARCHAR(8) NOT NULL, -- B1 / B2 / B3 / DIRECT (Phase 3, 2026-05-25)
signal_type VARCHAR(16) NOT NULL, -- site / call / sms
unique_key TEXT NOT NULL, -- domain / phone / sender+keyword / sender
supplier_external_id VARCHAR(64), -- внутренний id у поставщика
@@ -923,7 +924,7 @@ CREATE TABLE supplier_projects (
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
CONSTRAINT chk_supplier_projects_platform
CHECK (platform IN ('B1','B2','B3')),
CHECK (platform IN ('B1','B2','B3','DIRECT')),
CONSTRAINT chk_supplier_projects_signal_type
CHECK (signal_type IN ('site','call','sms')),
CONSTRAINT chk_supplier_projects_sync_status
@@ -964,10 +965,10 @@ CREATE TABLE project_supplier_links (
id BIGSERIAL PRIMARY KEY,
project_id BIGINT NOT NULL REFERENCES projects(id) ON DELETE CASCADE,
supplier_project_id BIGINT NOT NULL REFERENCES supplier_projects(id) ON DELETE CASCADE,
platform VARCHAR(4) NOT NULL,
platform VARCHAR(8) NOT NULL, -- B1 / B2 / B3 / DIRECT (Phase 3, 2026-05-25)
subject_code SMALLINT, -- субъект РФ 1..89; NULL = пул «Вся РФ»
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
CONSTRAINT chk_psl_platform CHECK (platform IN ('B1','B2','B3')),
CONSTRAINT chk_psl_platform CHECK (platform IN ('B1','B2','B3','DIRECT')),
CONSTRAINT uq_psl_project_supplier UNIQUE (project_id, supplier_project_id)
);
CREATE INDEX idx_psl_supplier_project ON project_supplier_links(supplier_project_id);
@@ -1979,7 +1980,7 @@ CREATE INDEX idx_failed_webhook_jobs_log ON failed_webhook_jobs(webhook_log_id);
CREATE TABLE supplier_leads (
id BIGSERIAL PRIMARY KEY,
supplier_project_id BIGINT REFERENCES supplier_projects(id) ON DELETE SET NULL,
platform VARCHAR(4) NOT NULL,
platform VARCHAR(8) NOT NULL, -- B1 / B2 / B3 / DIRECT (Phase 3, 2026-05-25)
raw_payload JSONB NOT NULL,
vid BIGINT, -- nullable: NULL у CSV-recovered лидов (Путь 2)
phone VARCHAR(20) NOT NULL,
@@ -1993,7 +1994,7 @@ CREATE TABLE supplier_leads (
error TEXT,
CONSTRAINT chk_supplier_leads_platform
CHECK (platform IN ('B1','B2','B3')),
CHECK (platform IN ('B1','B2','B3','DIRECT')),
CONSTRAINT chk_supplier_leads_source
CHECK (source IN ('webhook','csv_recovery')),
CONSTRAINT chk_supplier_leads_deals_count_nonneg
+3 -2
View File
@@ -1,6 +1,7 @@
{
"2026-05": {
"WIN_USER_PATH": 53,
"IPV4": 1
"WIN_USER_PATH": 57,
"IPV4": 1,
"RU_PHONE": 1
}
}
File diff suppressed because one or more lines are too long
@@ -0,0 +1,355 @@
# Phase 1: Always JSON 422 for webhook validation errors
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Webhook `/api/webhook/supplier/*` ВСЕГДА возвращает JSON 422 на ValidationException, никогда не редиректит на `/`. Закрывает ~76 потерянных лидов сутки в логах nginx.
**Architecture:** Один `withExceptions()` render-callback в `bootstrap/app.php`: для запросов матчащих `api/webhook/supplier/*` отдаём `response()->json(['message','errors'], 422)`. Для остальных — `return null` (дефолт). Существующие тесты остаются valid, добавляется один новый тест с `Accept: text/html` (имитация реального поставщика).
**Tech Stack:** Laravel 13 / Pest 4 / PHP 8.3
**Spec:** `docs/superpowers/specs/2026-05-25-supplier-webhook-reliability-design.md` §3 Phase 1
**Ветка:** `feat/supplier-webhook-fixes` (создана)
---
## File Structure
**Создать:**
- `app/tests/Feature/Http/Webhook/SupplierWebhookValidationFormatTest.php` — единственный новый тест, фиксирующий формат ответа для не-JSON Accept
**Изменить:**
- `app/bootstrap/app.php` — добавить `$exceptions->render(...)` для ValidationException
**Не трогать:**
- `SupplierWebhookController.php` — логика валидации не меняется
- Существующие `SupplierWebhookTest.php` — все `postJson()` тесты продолжают работать
---
## Task 1: Failing test — webhook returns 422 JSON for non-JSON-Accept clients
**Files:**
- Create: `app/tests/Feature/Http/Webhook/SupplierWebhookValidationFormatTest.php`
- [ ] **Step 1: Write the failing test**
```php
<?php
declare(strict_types=1);
use App\Models\SystemSetting;
use Illuminate\Foundation\Testing\DatabaseTransactions;
uses(DatabaseTransactions::class);
beforeEach(function () {
SystemSetting::query()
->where('key', 'supplier_webhook_secret')
->update(['value' => 'test-secret-32chars-aaaaaaaaaaaaaa']);
SystemSetting::query()
->where('key', 'supplier_ip_allowlist')
->update(['value' => '[]']);
});
it('returns 422 JSON when supplier posts invalid payload WITHOUT Accept: application/json header', function () {
// Воспроизводит реальное поведение crm.bp-gr.ru: POST без Accept-JSON.
// До фикса (302→422) Laravel редиректил на / с Set-Cookie, поставщик
// терял тело запроса. После фикса всегда JSON.
$response = $this->call(
'POST',
'/api/webhook/supplier/test-secret-32chars-aaaaaaaaaaaaaa',
[], // params
[], // cookies
[], // files
['HTTP_CONTENT_TYPE' => 'application/x-www-form-urlencoded'], // server: НЕТ Accept JSON
http_build_query([
'vid' => 1,
'project' => 'invalid_no_b_prefix',
'phone' => '79991234567',
'time' => time(),
])
);
$response->assertStatus(422);
expect($response->headers->get('Content-Type'))->toContain('application/json');
$response->assertJsonStructure(['message', 'errors' => ['project']]);
});
it('still works correctly for postJson clients (regression)', function () {
$response = $this->postJson('/api/webhook/supplier/test-secret-32chars-aaaaaaaaaaaaaa', [
'vid' => 1,
'project' => 'invalid_no_b_prefix',
'phone' => '79991234567',
'time' => time(),
]);
$response->assertStatus(422)->assertJsonValidationErrors('project');
});
it('non-webhook routes still use default render (no JSON forced)', function () {
// Регрессионный тест: дефолтный render остальных routes не сломан
// (например /login — должен возвращать redirect, а не JSON).
$response = $this->call(
'POST',
'/login',
['email' => 'bad', 'password' => ''],
[], [], [],
);
// Любой не-200 кроме 422-JSON допустим — главное чтобы наш fix не перехватил
expect($response->headers->get('Content-Type'))->not->toContain('application/json');
});
```
- [ ] **Step 2: Run test to verify it fails**
```
cd app && ./vendor/bin/pest tests/Feature/Http/Webhook/SupplierWebhookValidationFormatTest.php
```
Expected: тест #1 (non-JSON Accept) FAIL с status=302 (или Content-Type=text/html), потому что ValidationException рендерится через redirect.
- [ ] **Step 3: Commit failing test**
```bash
git add app/tests/Feature/Http/Webhook/SupplierWebhookValidationFormatTest.php
git commit -m "test(supplier-webhook): assert JSON 422 for non-JSON Accept clients (failing)
Reproduces 302-redirect bug observed on prod 2026-05-25 — when supplier
crm.bp-gr.ru POSTs without Accept: application/json, Laravel renders
ValidationException as redirect to /, losing body. Test calls webhook
without Accept header and asserts JSON 422 response. Will fail until
bootstrap/app.php has render(ValidationException) for api/webhook/supplier/*."
```
---
## Task 2: Implement bootstrap render — force JSON 422 for webhook routes
**Files:**
- Modify: `app/bootstrap/app.php` (lines 35-48 — withExceptions block)
- [ ] **Step 1: Add ValidationException render in bootstrap/app.php**
В `withExceptions` callback (после существующего `QueryException` render) добавить новый render для `ValidationException`:
```php
->withExceptions(function (Exceptions $exceptions): void {
$exceptions->render(function (QueryException $e, Request $request) {
// ... existing code, не менять ...
});
// Supplier webhook always returns JSON, even when client omits Accept header.
// Without this render, Laravel's default ValidationException handler returns
// 302 redirect to /, which strips POST body — losing supplier leads.
// Confirmed 2026-05-25: 76 of 234 webhook hits today got 302 instead of 422.
$exceptions->render(function (\Illuminate\Validation\ValidationException $e, Request $request) {
if ($request->is('api/webhook/supplier/*')) {
return response()->json([
'message' => 'Validation failed',
'errors' => $e->errors(),
], 422);
}
return null; // default render for other routes
});
});
```
NB: `use Illuminate\Validation\ValidationException;` — не нужен, используем FQN inline чтобы не трогать existing imports section.
- [ ] **Step 2: Run new test to verify it passes**
```
cd app && ./vendor/bin/pest tests/Feature/Http/Webhook/SupplierWebhookValidationFormatTest.php
```
Expected: все 3 теста PASS.
- [ ] **Step 3: Run full webhook test suite (regression)**
```
cd app && ./vendor/bin/pest tests/Feature/Http/Webhook/SupplierWebhookTest.php tests/Feature/Http/Webhook/SupplierWebhookValidationFormatTest.php
```
Expected: все тесты (≥14 в обоих файлах) PASS. Особенно проверить что `'rejects invalid project format (no B[123]_ prefix) with 422'` (line 95 в SupplierWebhookTest.php) продолжает PASS — он использует `postJson()`, поэтому новый render для него не сработает (default handler уже даёт 422 для JSON Accept), но мы не должны его сломать.
- [ ] **Step 4: Commit implementation**
```bash
git add app/bootstrap/app.php
git commit -m "fix(supplier-webhook): always return JSON 422 on ValidationException
Adds withExceptions render callback for ValidationException that forces
JSON 422 response when request matches api/webhook/supplier/* — regardless
of Accept header. Default Laravel behavior is 302 redirect for non-JSON
clients, which strips POST body.
Observed on prod 2026-05-25: 76 of 234 supplier webhook hits got 302 (Location: /),
mostly for non-B-prefix projects (client.carmoney.ru, cabinet.caranga.ru,
cashmotor.ru). Supplier doesn't follow 302 redirects on POST, so the
lead body is lost. This fix ensures supplier always sees a meaningful
422 with errors[] instead of a redirect.
Other routes unaffected (render returns null for non-webhook URLs)."
```
---
## Task 3: Reproduce on staging-clone or local — manual smoke
**Files:**
- Test: manual curl (no file)
- [ ] **Step 1: Run dev server locally (if available) or skip to Task 4**
Если на машине поднят `php artisan serve --port=8000`:
```bash
cd app && php artisan serve --port=8000 &
sleep 2
```
- [ ] **Step 2: POST without Accept header — assert 422 JSON**
```bash
curl -sk -X POST \
-H "Content-Type: application/x-www-form-urlencoded" \
-d 'vid=1&project=invalid_no_b_prefix&phone=79991234567&time='$(date +%s) \
http://localhost:8000/api/webhook/supplier/test-secret-32chars-aaaaaaaaaaaaaa \
-w "\nSTATUS: %{http_code}\nCT: %{content_type}\n"
```
Expected: `STATUS: 422`, `CT: application/json`, тело содержит `"errors":{"project":...}`.
- [ ] **Step 3: POST with Accept: application/json — same result (regression)**
```bash
curl -sk -X POST \
-H "Accept: application/json" -H "Content-Type: application/json" \
-d '{"vid":1,"project":"invalid_no_b_prefix","phone":"79991234567","time":'$(date +%s)'}' \
http://localhost:8000/api/webhook/supplier/test-secret-32chars-aaaaaaaaaaaaaa \
-w "\nSTATUS: %{http_code}\n"
```
Expected: `STATUS: 422`, JSON body.
- [ ] **Step 4: Stop server (если запускал)**
```bash
pkill -f 'artisan serve' || true
```
Если dev-сервер не поднимается на этой машине — пропустить Task 3, прод-smoke в Task 5 покроет.
---
## Task 4: Regression — quick mode
**Files:**
- None
- [ ] **Step 1: Run /regression quick**
```
/regression quick
```
Expected: GREEN — lint, format, type-check ОК. Если pre-commit hook падает (memory `feedback_environment.md` #111 — gitleaks висит на heavy diff), использовать `LEFTHOOK=0` при коммите.
- [ ] **Step 2: If quick GREEN, proceed to /regression full**
```
/regression full
```
Expected: Pest 742+ pass / 0 fail, Vitest 736+ pass, Vite build OK, lychee 0 broken, gitleaks 0. Допустимы pre-existing skipped.
Если найдены регрессии — НЕ переходить к деплою. Зафиксировать в отдельном fixup-commit либо вернуться к Task 2.
---
## Task 5: Deploy to liderra.ru (prod)
**Files:**
- None — деплой через ssh + redeploy.sh
- [ ] **Step 1: Pre-deploy validation via prod-deploy-validator agent**
Через Task tool:
```
subagent_type: prod-deploy-validator
prompt: проверь готовность боевого liderra.ru к выкату ветки feat/supplier-webhook-fixes на коммит после Phase 1 (bootstrap/app.php изменён). Что меняется: webhook /api/webhook/supplier/* теперь всегда отвечает JSON 422 на validation errors. Миграций БД нет. Очередь queue:restart нужен? проверь 8 pre-flight.
```
Expected: вердикт GO. Если NO-GO — устранить причину (квирки 104-108) и повторить.
- [ ] **Step 2: Merge feature branch fixup to main**
После одобрения Phase 1 changes:
```bash
cd "c:/моя/проекты/портал crm/Документация"
git checkout main
git merge --ff-only feat/supplier-webhook-fixes
git push origin main
```
NB: ОДНОВРЕМЕННО другие phases ещё не закоммичены, поэтому FF-merge содержит только Phase 1.
- [ ] **Step 3: Run redeploy.sh on prod**
```bash
ssh liderra "cd /var/www/liderra/app && sudo -u www-data ./redeploy.sh 2>&1 | tail -50"
```
Expected: успешный pull + composer install + `optimize:clear` + `optimize` + queue:restart. Errors → revert (git revert + redeploy).
- [ ] **Step 4: Prod smoke — webhook returns 422 not 302**
```bash
ssh liderra 'curl -sk -X POST \
-H "Content-Type: application/x-www-form-urlencoded" \
-d "vid=1&project=invalid&phone=79991234567&time="$(date +%s) \
https://liderra.ru/api/webhook/supplier/8c1c07ddb0768763661b357198e0625832f74ad0915d91b1 \
-w "\nSTATUS: %{http_code}\nCT: %{content_type}\n"'
```
Expected: `STATUS: 422`, `CT: application/json`. **Если 302 — деплой не применился, откатывать.**
- [ ] **Step 5: Wait 30 min, check nginx access.log**
```bash
ssh liderra "sudo grep '/api/webhook/supplier' /var/log/nginx/access.log | tail -50 | awk '{print \$9}' | sort | uniq -c"
```
Expected: только 202, 422, 429, 404. **0 × 302, 0 × 301** для запросов на webhook URL.
- [ ] **Step 6: Update ПИЛОТ.md + memory**
Через прямой Edit, отметка «Phase 1 deployed 25.05.2026 HH:MM МСК, webhook always JSON». Memory update — `project_billing_v2.md` или новый `project_supplier_webhook_fixes.md`.
```bash
# Update ПИЛОТ.md as needed manually
git add ПИЛОТ.md
git commit -m "docs(пилот): Phase 1 supplier webhook JSON-422 deployed"
git push origin main
```
---
## Done criteria для Phase 1
- [ ] Все тесты в `SupplierWebhookTest.php` + `SupplierWebhookValidationFormatTest.php` PASS
- [ ] /regression full GREEN
- [ ] Прод-smoke: curl без Accept → 422 JSON
- [ ] За 30 мин после деплоя в nginx access.log — 0 × 302 на webhook URL
- [ ] Phase 2 plan starts only after Phase 1 deployed AND observed clean for ≥30 min
---
## Откат (если что-то пошло не так)
```bash
ssh liderra "cd /var/www/liderra/app && git revert --no-edit HEAD && sudo -u www-data ./redeploy.sh 2>&1 | tail -20"
```
Изменение касается только обработки исключений — откат без миграций, мгновенный.
@@ -0,0 +1,475 @@
# Phase 2: Idempotent dedup webhook ↔ CSV-recovered
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Webhook, поступивший после CSV-recovered deal по `(tenant_id, phone, project_id)` в окне 24h, **обновляет** существующий deal (`source_crm_id`, `received_at`), не создаёт второй. Без двойного списания биллингом. Закрывает 37 дублей сутки.
**Architecture:** В `RouteSupplierLeadJob::createDealCopyForProject` под уже существующей `DB::transaction + lockForUpdate(Tenant)+lockForUpdate(Project)` добавляется проверка «есть ли csv-recovered deal по `(tenant_id, phone, project_id, received_at ≥ now()-24h, source_crm_id IS NULL)`». Если есть — `UPDATE existing.source_crm_id = lead.vid` + `INSERT supplier_lead_deliveries` (привязка webhook к existing deal), **БЕЗ** `chargeForDelivery`. Возврат специального статуса `MERGED` (не считается в `$createdCount`, не failure).
**Tech Stack:** Laravel 13 / Pest 4 / PHP 8.3 / PostgreSQL 16 / bcmath / RLS
**Spec:** `docs/superpowers/specs/2026-05-25-supplier-webhook-reliability-design.md` §3 Phase 2
**Предусловие:** Phase 1 deployed и наблюдаем clean ≥30 мин.
**Ветка:** `feat/supplier-webhook-fixes` (продолжение)
---
## Открытый вопрос (OQ-1 из спеки) — резолвится в Task 1
`LedgerService::chargeForDelivery` (app/app/Services/Billing/LedgerService.php:47-117) — **НЕ идемпотентен**: каждый вызов делает INSERT LeadCharge, BalanceTransaction, supplier_lead_costs + decrement balance_rub. Поэтому критично НЕ вызывать его второй раз для merged deal.
---
## File Structure
**Создать:**
- `app/tests/Feature/Supplier/CsvWebhookRaceTest.php` — TDD-тесты для merge сценария
**Изменить:**
- `app/app/Jobs/RouteSupplierLeadJob.php` — добавить блок поиска csv-recovered deal в `createDealCopyForProject`
**Не трогать:**
- `LedgerService.php` — не меняем, идемпотентность достигается через ранний return ДО его вызова
- `supplier_lead_deliveries` schema — не меняем (текущая `(supplier_lead_id, tenant_id)` UNIQUE остаётся; добавляем дополнительный row для merge case)
- `CsvReconcileJob.php` — не меняем (он создаёт SupplierLead с vid=NULL, как и было)
---
## Task 1: Verify LedgerService is NOT idempotent (read-only confirmation)
**Files:**
- Read: `app/app/Services/Billing/LedgerService.php`
- [ ] **Step 1: Confirm there is NO check for existing lead_charges with same deal_id**
Открыть [app/app/Services/Billing/LedgerService.php:47-117](../../../app/app/Services/Billing/LedgerService.php#L47-L117). Подтвердить:
- Нет `LeadCharge::where('deal_id', $deal->id)->exists()` guard.
- Нет SELECT перед INSERT.
- Метод просто делает INSERT, increment, INSERT, INSERT.
Если идемпотентность ЕСТЬ — пересмотреть план Phase 2 (может быть проще, без MERGED статуса). Если НЕТ (ожидаемо) — продолжаем по плану.
- [ ] **Step 2: Document in commit message**
Зафиксировать наблюдение в первом коммите Task 2. Никакой правки в LedgerService не делаем — guard добавляется в caller (RouteSupplierLeadJob).
---
## Task 2: Failing test — webhook after CSV-recovered merges, doesn't duplicate or double-charge
**Files:**
- Create: `app/tests/Feature/Supplier/CsvWebhookRaceTest.php`
- [ ] **Step 1: Write failing tests**
```php
<?php
declare(strict_types=1);
use App\Jobs\RouteSupplierLeadJob;
use App\Models\Deal;
use App\Models\LeadCharge;
use App\Models\Project;
use App\Models\SupplierLead;
use App\Models\Tenant;
use App\Models\User;
use Illuminate\Foundation\Testing\DatabaseTransactions;
use Illuminate\Support\Facades\DB;
uses(DatabaseTransactions::class);
/**
* Phase 2 — webhook ↔ CSV-recovered idempotency.
*
* Сценарий (наблюдался на prod 2026-05-25):
* 1. Поставщик шлёт webhook → 302 (теряется тело) — Phase 1 уже починила.
* 2. CsvReconcileJob через 30 мин видит лид в CSV, не находит supplier_lead
* по (phone, project) → создаёт recovered SupplierLead (vid=NULL,
* source='csv_recovery') → RouteSupplierLeadJob → Deal с source_crm_id=NULL.
* 3. Поставщик ретраит webhook (ещё 15 мин) → новый SupplierLead с vid=<int>
* → RouteSupplierLeadJob → создаёт второй Deal с тем же phone+project
* → биллинг списывает второй раз.
*
* Phase 2 fix: шаг 3 находит существующий CSV-recovered deal, обновляет
* source_crm_id, привязывает webhook supplier_lead к существующему deal через
* supplier_lead_deliveries, НЕ создаёт второй Deal, НЕ списывает повторно.
*/
beforeEach(function () {
$this->tenant = Tenant::factory()->create([
'balance_rub' => '1000.00',
'delivered_in_month' => 0,
]);
$this->project = Project::factory()->create([
'tenant_id' => $this->tenant->id,
'signal_type' => 'site',
'signal_identifier' => 'krk-finance.ru',
'is_active' => true,
'daily_limit_target' => 100,
'delivered_today' => 0,
]);
// ... настроить supplier_projects + project_supplier_links для платформы B1
// identifier krk-finance.ru — детали зависят от фабрик
});
it('webhook after CSV-recovered merges into existing deal (no duplicate, no double-charge)', function () {
// Step 1: simulate CSV-recovered SupplierLead (vid=null)
$csvLead = SupplierLead::create([
'platform' => 'B1',
'phone' => '79991234567',
'vid' => null,
'raw_payload' => ['project' => 'B1_krk-finance.ru', 'phone' => '79991234567', 'time' => time()],
'received_at' => now()->subHour(),
'recovered_from_csv_at' => now()->subHour(),
'source' => 'csv_recovery',
]);
(new RouteSupplierLeadJob($csvLead->id))->handle(
app(\App\Services\LeadRouter::class),
app(\App\Services\SupplierProjects\SupplierProjectResolver::class),
app(\App\Services\NotificationService::class),
app(\App\Services\Billing\LedgerService::class),
app(\App\Services\LeadDistributor::class),
app(\App\Services\RegionTagResolver::class),
);
$csvDeal = Deal::where('phone', '79991234567')->first();
expect($csvDeal)->not->toBeNull();
expect($csvDeal->source_crm_id)->toBeNull();
$chargesAfterCsv = LeadCharge::where('deal_id', $csvDeal->id)->count();
expect($chargesAfterCsv)->toBe(1); // одна charge от CSV-recovered
$balanceAfterCsv = (string) $this->tenant->fresh()->balance_rub;
// Step 2: simulate webhook arriving 15 min later with real vid
$webhookLead = SupplierLead::create([
'platform' => 'B1',
'phone' => '79991234567',
'vid' => 1672819986,
'raw_payload' => ['project' => 'B1_krk-finance.ru', 'phone' => '79991234567', 'time' => time()],
'received_at' => now()->subMinutes(15),
'source' => 'webhook',
]);
(new RouteSupplierLeadJob($webhookLead->id))->handle(
app(\App\Services\LeadRouter::class),
app(\App\Services\SupplierProjects\SupplierProjectResolver::class),
app(\App\Services\NotificationService::class),
app(\App\Services\Billing\LedgerService::class),
app(\App\Services\LeadDistributor::class),
app(\App\Services\RegionTagResolver::class),
);
// Assertion 1: still ONE deal, but source_crm_id теперь заполнен
$deals = Deal::where('phone', '79991234567')->get();
expect($deals)->toHaveCount(1);
expect($deals->first()->source_crm_id)->toBe(1672819986);
// Assertion 2: НЕТ второго LeadCharge (idempotency биллинга)
$chargesAfterWebhook = LeadCharge::where('deal_id', $csvDeal->id)->count();
expect($chargesAfterWebhook)->toBe(1); // всё ещё ОДИН charge
// Assertion 3: balance НЕ списан второй раз
$balanceAfterWebhook = (string) $this->tenant->fresh()->balance_rub;
expect($balanceAfterWebhook)->toBe($balanceAfterCsv);
// Assertion 4: supplier_lead_deliveries содержит ОБА supplier_lead_id,
// привязанные к ОДНОМУ deal.id
$deliveries = DB::table('supplier_lead_deliveries')
->where('deal_id', $csvDeal->id)
->get();
expect($deliveries)->toHaveCount(2);
expect($deliveries->pluck('supplier_lead_id')->all())
->toContain($csvLead->id, $webhookLead->id);
});
it('two webhooks with DIFFERENT vids both create deals (Spec B — за повторы поставщика берём)', function () {
// Регрессионный тест: если поставщик намеренно шлёт два webhook'а с РАЗНЫМИ
// vid'ами на тот же phone+project — это два разных лида, оба должны быть
// приняты. Спек B Phase 1 (commit ccfecd5e) специально снял DD для этого
// кейса. Наш Phase 2 fix НЕ должен этому препятствовать.
$lead1 = SupplierLead::create([
'platform' => 'B1', 'phone' => '79991234567', 'vid' => 100,
'raw_payload' => ['project' => 'B1_krk-finance.ru', 'phone' => '79991234567', 'time' => time()],
'received_at' => now()->subHour(), 'source' => 'webhook',
]);
(new RouteSupplierLeadJob($lead1->id))->handle(/* ... */);
$lead2 = SupplierLead::create([
'platform' => 'B1', 'phone' => '79991234567', 'vid' => 200,
'raw_payload' => ['project' => 'B1_krk-finance.ru', 'phone' => '79991234567', 'time' => time()],
'received_at' => now()->subMinutes(30), 'source' => 'webhook',
]);
(new RouteSupplierLeadJob($lead2->id))->handle(/* ... */);
// Assertion: ОБА webhook'а имеют source_crm_id (не NULL), поэтому merge
// не происходит — это два разных лида у поставщика, два разных deal.
$deals = Deal::where('phone', '79991234567')->get();
expect($deals)->toHaveCount(2);
expect($deals->pluck('source_crm_id')->all())->toContain(100, 200);
expect(LeadCharge::whereIn('deal_id', $deals->pluck('id'))->count())->toBe(2);
});
it('csv-recovered deal older than 24h is NOT merged with new webhook', function () {
// Окно merge — 24h. Если CSV-recovered deal старше — не считается duplicate.
$csvLead = SupplierLead::create([
'platform' => 'B1', 'phone' => '79991234567', 'vid' => null,
'raw_payload' => ['project' => 'B1_krk-finance.ru', 'phone' => '79991234567', 'time' => now()->subDays(2)->getTimestamp()],
'received_at' => now()->subDays(2),
'recovered_from_csv_at' => now()->subDays(2),
'source' => 'csv_recovery',
]);
(new RouteSupplierLeadJob($csvLead->id))->handle(/* ... */);
$webhookLead = SupplierLead::create([
'platform' => 'B1', 'phone' => '79991234567', 'vid' => 999,
'raw_payload' => ['project' => 'B1_krk-finance.ru', 'phone' => '79991234567', 'time' => time()],
'received_at' => now(), 'source' => 'webhook',
]);
(new RouteSupplierLeadJob($webhookLead->id))->handle(/* ... */);
// Assertion: TWO deals (старый CSV-recovered + новый webhook), не merge
$deals = Deal::where('phone', '79991234567')->get();
expect($deals)->toHaveCount(2);
});
```
NB: код тестов написан как **набросок**. При имплементации:
- Заменить `(new RouteSupplierLeadJob(...))->handle(/* ... */)` на правильную диспатч-схему (Bus::dispatchSync или вручную с DI). Посмотреть в [app/tests/Feature/Supplier/RouteSupplierLeadJobBillingTest.php](../../../app/tests/Feature/Supplier/RouteSupplierLeadJobBillingTest.php) для примера.
- Настроить supplier_projects + project_supplier_links фабрики правильно. Посмотреть в существующих тестах.
- [ ] **Step 2: Run tests, expect FAIL**
```
cd app && ./vendor/bin/pest tests/Feature/Supplier/CsvWebhookRaceTest.php
```
Expected: тест #1 FAIL (deals.count == 2 а не 1; charges.count == 2 а не 1). Это подтверждает баг.
- [ ] **Step 3: Commit failing tests**
```bash
git add app/tests/Feature/Supplier/CsvWebhookRaceTest.php
git commit -m "test(supplier): assert webhook-after-csv-recovered merges into existing deal (failing)
Reproduces 37 duplicate deals observed on prod 2026-05-25 for tenant client1.
After Spec B Phase 1 (commit ccfecd5e) removed DuplicateDetector, the race
between CsvReconcileJob (creates SupplierLead vid=null) and later webhook
retry (vid=int) results in two separate Deals because supplier_lead_deliveries
locks on supplier_lead_id (which differs between csv-recovery and webhook),
not on (phone, project_id).
Failing now — implementation comes in next commit."
```
---
## Task 3: Implement merge logic in RouteSupplierLeadJob::createDealCopyForProject
**Files:**
- Modify: `app/app/Jobs/RouteSupplierLeadJob.php:207-330`
- [ ] **Step 1: Add early merge check ДО supplier_lead_deliveries insertOrIgnore**
В `createDealCopyForProject`, **после** `$lockedProject = ... lockForUpdate(); ... if (delivered_today >= limit) return false;`, **до** `$locked = DB::table('supplier_lead_deliveries')->insertOrIgnore(...)`:
```php
// Phase 2 fix: merge с CSV-recovered deal если webhook догоняет.
// Идемпотентность race condition между CsvReconcileJob (vid=NULL, recovered
// from CSV) и webhook (vid=int, реальный supplier-id). До этой проверки они
// создавали 2 deal'a (DD снят Spec B Phase 1). Merge выполняется только если:
// - webhook ЕСТЬ настоящий vid (lead.vid !== null) — без vid merge'ить нечего;
// - csv-recovered deal существует за последние 24h, тот же phone+project+tenant;
// - csv-recovered deal БЕЗ source_crm_id (т.е. он именно CSV-recovered, не другой webhook).
// При merge: UPDATE existing.source_crm_id, INSERT supplier_lead_deliveries,
// БЕЗ chargeForDelivery (LeadCharge уже есть с момента CSV recovery).
$existingMergeable = null;
if ($lead->vid !== null) {
$existingMergeable = Deal::query()
->where('tenant_id', $tenant->id)
->where('phone', (string) $lead->phone)
->where('project_id', $project->id)
->whereNull('source_crm_id')
->where('received_at', '>=', now()->subDay())
->lockForUpdate()
->first();
}
if ($existingMergeable !== null) {
// Заполняем supplier_lead.id у обоих SupplierLead → одному Deal
DB::table('supplier_lead_deliveries')->insert([
'supplier_lead_id' => $lead->id,
'tenant_id' => $tenant->id,
'deal_id' => $existingMergeable->id,
'created_at' => now(),
]);
$existingMergeable->source_crm_id = $lead->vid;
if ($lead->received_at !== null && $lead->received_at->gt($existingMergeable->received_at)) {
$existingMergeable->received_at = $lead->received_at;
}
$existingMergeable->save();
Log::info('supplier_lead.merged_into_csv_recovered', [
'supplier_lead_id' => $lead->id,
'merged_into_deal_id' => $existingMergeable->id,
'tenant_id' => $tenant->id,
]);
return true; // считаем «доставленным», но без второго списания
}
// Spec B: per-(supplier_lead, tenant) lock — existing code ниже без изменений
$locked = DB::table('supplier_lead_deliveries')->insertOrIgnore([
// ... existing ...
]);
```
NB:
- `lockForUpdate()` на existingMergeable защищает от двойного merge при параллельных queue workers.
- Условие `whereNull('source_crm_id')` — критично: оно отличает CSV-recovered (vid=NULL → source_crm_id=NULL) от настоящих webhook deals (source_crm_id=vid). Без этого условия мы бы мерджили на любой повтор поставщика, что **сломало бы Spec B**.
- Insert в `supplier_lead_deliveries` — простой `->insert()`, не `->insertOrIgnore()`. Потому что `(supplier_lead_id, tenant_id)` уникален, и для webhook-after-csv это новая комбинация (другой supplier_lead_id чем у csv-recovered).
- [ ] **Step 2: Run tests, expect PASS**
```
cd app && ./vendor/bin/pest tests/Feature/Supplier/CsvWebhookRaceTest.php
```
Expected: все 3 теста PASS.
- [ ] **Step 3: Run full supplier test suite (regression)**
```
cd app && ./vendor/bin/pest tests/Feature/Supplier/ tests/Feature/Jobs/RouteSupplierLeadJobTest.php
```
Expected: все existing тесты PASS. Особенно:
- `SupplierLeadDeliveryGuardTest` (текущий lock-механизм)
- `RouteSupplierLeadJobBillingTest` (биллинг)
- `RouteSupplierLeadJobTest`
- `CsvReconcileJobTest`
Если что-то сломалось — это знак что existingMergeable условие слишком широкое. Сузить и повторить.
- [ ] **Step 4: Commit implementation**
```bash
git add app/app/Jobs/RouteSupplierLeadJob.php
git commit -m "fix(supplier): merge webhook into csv-recovered deal, no double-charge
Adds early merge check in RouteSupplierLeadJob::createDealCopyForProject:
when lead.vid IS NOT NULL and an existing deal with NULL source_crm_id
exists for (tenant, phone, project_id) within last 24h, UPDATE that
deal's source_crm_id instead of creating a second Deal. INSERT into
supplier_lead_deliveries links the new supplier_lead.id to the existing
deal.id. LedgerService::chargeForDelivery is NOT called — the original
charge happened when the csv-recovery created the deal.
Closes 37 duplicate deals observed on prod for tenant client1 25.05.2026.
Spec B Phase 1 (commit ccfecd5e) removed DuplicateDetector — this fix
restores idempotency for the specific webhook-after-csv-recovered case
WITHOUT re-blocking intentional supplier repeats with different vids.
Guard: only merges where source_crm_id IS NULL (the CSV-recovered marker).
Two webhooks with different vids on same phone+project still create two
deals — by-design per Spec B."
```
---
## Task 4: Regression and prod data probe
**Files:**
- None
- [ ] **Step 1: /regression full**
```
/regression full
```
Expected: GREEN. Особенно фокус на Pest --parallel (race conditions).
- [ ] **Step 2: Prod data probe — current state of duplicates**
ДО деплоя:
```bash
ssh liderra "sudo -u postgres psql -d liderra -P pager=off -c \"SELECT phone, project_id, COUNT(*) AS cnt FROM deals WHERE tenant_id=2 AND created_at::date = CURRENT_DATE GROUP BY phone, project_id HAVING COUNT(*) > 1 ORDER BY cnt DESC LIMIT 10\""
```
Зафиксировать список (это будут текущие 37 пар). После деплоя — повторить ту же команду через 2 часа: новые пары не должны появляться.
---
## Task 5: Deploy to liderra.ru
**Files:**
- None
- [ ] **Step 1: prod-deploy-validator agent**
```
subagent_type: prod-deploy-validator
prompt: проверь готовность боевого liderra.ru к Phase 2 деплою. Меняется только RouteSupplierLeadJob.php (добавлен merge-check для CSV-recovered deals). Миграций БД нет. Очередь — queue:restart обязателен, потому что job изменился. Phase 1 уже на проде ≥30 мин.
```
- [ ] **Step 2: Merge to main + push**
```bash
git checkout main
git merge --ff-only feat/supplier-webhook-fixes
git push origin main
```
- [ ] **Step 3: redeploy on prod**
```bash
ssh liderra "cd /var/www/liderra/app && sudo -u www-data ./redeploy.sh 2>&1 | tail -50"
```
Expected: успешно. Особенно проверить что `php artisan queue:restart` отработал (см. в выводе redeploy.sh).
- [ ] **Step 4: Prod smoke — нет новых дублей за 2 часа**
Подождать 2 часа, потом:
```bash
ssh liderra "sudo -u postgres psql -d liderra -P pager=off -c \"SELECT phone, project_id, COUNT(*) FROM deals WHERE tenant_id=2 AND created_at >= NOW() - interval '2 hours' GROUP BY phone, project_id HAVING COUNT(*) > 1\""
```
Expected: **0 rows** (нет новых дублей за 2 часа после деплоя).
- [ ] **Step 5: Check merge logs**
```bash
ssh liderra "sudo grep 'merged_into_csv_recovered' /var/www/liderra/app/storage/logs/laravel.log | tail -20"
```
Expected: есть записи (показывает что merge сработал). Каждая запись — закрытый дубль.
- [ ] **Step 6: Update ПИЛОТ.md + memory**
```bash
# Edit ПИЛОТ.md mentioning Phase 2 deployed + merge stats
git add ПИЛОТ.md
git commit -m "docs(пилот): Phase 2 supplier dedup deployed, $N merges in 2h window"
git push origin main
```
---
## Done criteria для Phase 2
- [ ] Все тесты в `CsvWebhookRaceTest.php` PASS
- [ ] Все существующие `tests/Feature/Supplier/` PASS (regression)
- [ ] /regression full GREEN
- [ ] За 2 часа после деплоя — 0 новых пар дубликатов на проде
- [ ] Существуют `merged_into_csv_recovered` записи в логе (показывает что merge работает)
- [ ] Phase 3 plan starts only after Phase 2 observed clean ≥2h
---
## Откат
```bash
ssh liderra "cd /var/www/liderra/app && git revert --no-edit HEAD && sudo -u www-data ./redeploy.sh 2>&1 | tail -20"
```
Миграций нет → откат мгновенный. Дубли начнут возникать снова, но эти 2-3 часа потерь покрываются CsvReconcileJob.
@@ -0,0 +1,899 @@
# Phase 3: DIRECT platform for non-B prefix projects
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Webhook на проекты без `B[123]_` префикса (`client.carmoney.ru`, `cashmotor.ru`, числовые) принимается, проходит routing, создаёт Deal под новой платформой `DIRECT`. Закрывает оставшиеся ~67 потерь сутки.
**Architecture:** Расширить `platform` enum в `supplier_projects` и `project_supplier_links` до `(B1, B2, B3, DIRECT)` через миграцию. Снять regex в webhook controller. `parsePlatform`/`parseProjectField`/`extractPlatform` возвращают `'DIRECT'` для не-B. `SupplierProjectResolver` принимает DIRECT. `LeadRouter` для DIRECT использует **прямой матч signal_identifier** (потому что DIRECT-supplier_projects ещё не привязаны к Лидерра-проектам через `project_supplier_links`). `LedgerService.resolveSupplierId` — fallback для DIRECT.
**Tech Stack:** Laravel 13 / PostgreSQL 16 / Pest 4 / PHP 8.3
**Spec:** `docs/superpowers/specs/2026-05-25-supplier-webhook-reliability-design.md` §3 Phase 3
**Предусловие:** Phase 2 deployed и наблюдаем clean ≥2 часов.
**Ветка:** `feat/supplier-webhook-fixes` (продолжение)
**Риск:** ВЫСОКИЙ — миграция БД + 5 файлов кода + бизнес-семантика биллинга
---
## Открытые вопросы
- **OQ-2.** `chk_supplier_projects_b1_not_for_sms` constraint — мешает ли DIRECT? **Ответ:** не мешает — это `CHECK (NOT (platform='B1' AND signal_type='sms'))`. DIRECT+SMS пропускается.
- **OQ-3.** Биллинг для DIRECT-платформы — какой Supplier (`suppliers.code`) использовать? **Ответ:** добавим `supplier code='direct'` в seed; в [LedgerService.resolveSupplierId](../../../app/app/Services/Billing/LedgerService.php#L127) добавим case `if platform=='DIRECT' return Supplier::where('code', 'direct')`.
- **OQ-4.** Как DIRECT-supplier_project привязывается к Лидерра-проекту, если `project_supplier_links` для DIRECT supplier_projects ещё нет? **Ответ:** добавляем fallback в `LeadRouter::matchEligibleProjects` для DIRECT supplier_projects — матчинг по `signal_type + signal_identifier` напрямую с `projects.signal_type + projects.signal_identifier`, без обязательного `project_supplier_links`.
---
## File Structure
**Создать:**
- `database/migrations/2026_05_25_120000_add_direct_platform_to_supplier_projects.php` — расширение CHECK constraints
- `database/migrations/2026_05_25_120100_seed_direct_supplier.php` — seed строки `suppliers.code='direct'` (cost_rub из существующего шаблона)
- `app/tests/Feature/Supplier/DirectPlatformTest.php` — end-to-end тесты для DIRECT flow
**Изменить:**
- `app/app/Http/Controllers/Api/SupplierWebhookController.php`:
- line 86: снять `regex:/^B[123]_.+$/'`
- lines 183-188: `parsePlatform` возвращает `'DIRECT'` для не-B
- `app/app/Jobs/RouteSupplierLeadJob.php`:
- lines 172-200: `parseProjectField` добавить DIRECT branch
- `app/app/Jobs/Supplier/CsvReconcileJob.php`:
- lines 237-244: `extractPlatform` возвращает 'DIRECT' (а не `null`) для парсящихся как domain/call/sms строк; `null` оставить только для реального мусора (numeric-only без структуры)
- `app/app/Services/SupplierProjects/SupplierProjectResolver.php`:
- line 24: `ALLOWED_PLATFORMS = ['B1','B2','B3','DIRECT']`
- `app/app/Services/LeadRouter.php`:
- lines 50-71: для DIRECT — расширить eligibility SQL с fallback на signal_type+identifier
- `app/app/Services/Billing/LedgerService.php`:
- lines 127-148: `resolveSupplierId` — добавить case `platform='DIRECT'`
- `app/tests/Feature/Http/Webhook/SupplierWebhookTest.php`:
- line 95: переписать тест — теперь `invalid_no_b_prefix` → 202 (принимается, platform=DIRECT)
- `db/schema.sql` — отразить новый constraint
- `db/CHANGELOG_schema.md` — запись v8.X
**Не трогать:**
- `LeadDistributor` — cap=3 работает на Collection, platform-agnostic
- `supplier_lead_deliveries` — уже Phase 2 покрывает идемпотентность
---
## Task 1: Read all touched files + verify b1-not-for-sms constraint
**Files:**
- Read: `db/schema.sql` § supplier_projects + project_supplier_links
- Read: `app/database/migrations/` для последней supplier_projects-related migration
- [ ] **Step 1: Find current CHECK constraints**
```bash
grep -n 'chk_supplier_projects_platform\|chk_psl_platform\|chk_supplier_projects_b1' \
"c:/моя/проекты/портал crm/Документация/db/schema.sql"
```
Зафиксировать exact text constraints для миграции (DROP + ADD).
- [ ] **Step 2: Find last migration touching supplier_projects.platform**
```bash
ls "c:/моя/проекты/портал crm/Документация/app/database/migrations/" | grep -i supplier_project
```
Документировать в комментарии новой миграции.
- [ ] **Step 3: Verify b1-not-for-sms doesn't conflict with DIRECT**
`chk_supplier_projects_b1_not_for_sms` — это `CHECK (NOT (platform='B1' AND signal_type='sms'))`. DIRECT+SMS — не B1, так что пропускается. Не нужно трогать.
---
## Task 2: Migration — extend platform CHECK to include DIRECT
**Files:**
- Create: `app/database/migrations/2026_05_25_120000_add_direct_platform_to_supplier_projects.php`
- [ ] **Step 1: Write migration**
```php
<?php
declare(strict_types=1);
use Illuminate\Database\Migrations\Migration;
use Illuminate\Support\Facades\DB;
/**
* Phase 3 supplier webhook reliability — расширяет platform enum в
* supplier_projects и project_supplier_links до (B1,B2,B3,DIRECT).
*
* DIRECT — это «прямая» платформа поставщика без B-префикса в имени
* проекта (e.g. `client.carmoney.ru`, `cashmotor.ru`, числовые телефоны).
* До Phase 3 такие webhook'и отвергались с 302-редиректом и терялись.
*
* Spec: docs/superpowers/specs/2026-05-25-supplier-webhook-reliability-design.md §3 Phase 3
*
* NB: chk_supplier_projects_b1_not_for_sms (B1+SMS deny) НЕ трогаем —
* DIRECT+SMS этим constraint'ом не блокируется.
*/
return new class extends Migration
{
public function up(): void
{
DB::statement('ALTER TABLE supplier_projects DROP CONSTRAINT chk_supplier_projects_platform');
DB::statement("ALTER TABLE supplier_projects ADD CONSTRAINT chk_supplier_projects_platform CHECK (platform IN ('B1','B2','B3','DIRECT'))");
DB::statement('ALTER TABLE project_supplier_links DROP CONSTRAINT chk_psl_platform');
DB::statement("ALTER TABLE project_supplier_links ADD CONSTRAINT chk_psl_platform CHECK (platform IN ('B1','B2','B3','DIRECT'))");
}
public function down(): void
{
// Перед откатом — убедиться что в БД нет rows с platform='DIRECT',
// иначе constraint провалится при ADD. Это ответственность того, кто
// запускает migrate:rollback. На prod — отдельный cleanup SQL до отката.
DB::statement('ALTER TABLE supplier_projects DROP CONSTRAINT chk_supplier_projects_platform');
DB::statement("ALTER TABLE supplier_projects ADD CONSTRAINT chk_supplier_projects_platform CHECK (platform IN ('B1','B2','B3'))");
DB::statement('ALTER TABLE project_supplier_links DROP CONSTRAINT chk_psl_platform');
DB::statement("ALTER TABLE project_supplier_links ADD CONSTRAINT chk_psl_platform CHECK (platform IN ('B1','B2','B3'))");
}
};
```
- [ ] **Step 2: Test migration locally**
```
cd app && php artisan migrate --pretend
```
Expected: видим что DROP/ADD CONSTRAINT statements корректны, без ошибок.
```
cd app && php artisan migrate
```
Expected: migration applied. Проверка:
```
cd app && php artisan tinker --execute='echo DB::selectOne("SELECT pg_get_constraintdef(oid) AS def FROM pg_constraint WHERE conname=\"chk_supplier_projects_platform\"")->def;'
```
Должно содержать `'DIRECT'`.
- [ ] **Step 3: Commit migration**
```bash
git add app/database/migrations/2026_05_25_120000_add_direct_platform_to_supplier_projects.php
git commit -m "feat(db): extend supplier_projects.platform CHECK to include DIRECT
Adds DIRECT value to chk_supplier_projects_platform and chk_psl_platform
constraints. DIRECT represents supplier projects without B[123]_ prefix
(e.g. client.carmoney.ru, cashmotor.ru, numeric phone IDs) — currently
67 leads/day lost to 302 redirects from webhook validation.
Schema-only change; no code yet uses DIRECT — code changes follow in
subsequent commits. Migration is forward-compatible: old code continues
to work with B1/B2/B3 rows."
```
---
## Task 3: Seed Supplier row with code='direct'
**Files:**
- Create: `app/database/migrations/2026_05_25_120100_seed_direct_supplier.php`
- [ ] **Step 1: Inspect existing suppliers rows**
```
cd app && php artisan tinker --execute='print_r(DB::table("suppliers")->get()->toArray());'
```
Найти существующий `cost_rub` для одной из B-платформ. Использовать тот же (DIRECT — same supplier, разная платформа).
- [ ] **Step 2: Write seed migration**
```php
<?php
declare(strict_types=1);
use Illuminate\Database\Migrations\Migration;
use Illuminate\Support\Facades\DB;
/**
* Phase 3 — DIRECT supplier row (used by LedgerService::resolveSupplierId
* fallback for platform='DIRECT'). cost_rub matches B1 (same supplier,
* different routing).
*/
return new class extends Migration
{
public function up(): void
{
$b1 = DB::table('suppliers')->where('code', 'b1')->first();
if ($b1 === null) {
// Если B1 нет — significant prod drift, не должно произойти.
return;
}
DB::table('suppliers')->updateOrInsert(
['code' => 'direct'],
[
'name' => 'BP-GR Direct',
'cost_rub' => $b1->cost_rub,
'created_at' => now(),
'updated_at' => now(),
]
);
}
public function down(): void
{
DB::table('suppliers')->where('code', 'direct')->delete();
}
};
```
- [ ] **Step 3: Run migration**
```
cd app && php artisan migrate
```
- [ ] **Step 4: Verify**
```
cd app && php artisan tinker --execute='echo DB::table("suppliers")->where("code","direct")->first()->name;'
```
Expected: `BP-GR Direct`.
- [ ] **Step 5: Commit**
```bash
git add app/database/migrations/2026_05_25_120100_seed_direct_supplier.php
git commit -m "feat(db): seed suppliers.code='direct' for DIRECT platform billing"
```
---
## Task 4: Failing test — DirectPlatformTest end-to-end
**Files:**
- Create: `app/tests/Feature/Supplier/DirectPlatformTest.php`
- [ ] **Step 1: Write end-to-end test**
```php
<?php
declare(strict_types=1);
use App\Jobs\RouteSupplierLeadJob;
use App\Models\Deal;
use App\Models\Project;
use App\Models\SupplierLead;
use App\Models\SupplierProject;
use App\Models\SystemSetting;
use App\Models\Tenant;
use Illuminate\Foundation\Testing\DatabaseTransactions;
uses(DatabaseTransactions::class);
beforeEach(function () {
SystemSetting::query()
->where('key', 'supplier_webhook_secret')
->update(['value' => 'test-secret-32chars-aaaaaaaaaaaaaa']);
SystemSetting::query()
->where('key', 'supplier_ip_allowlist')
->update(['value' => '[]']);
$this->tenant = Tenant::factory()->create([
'balance_rub' => '1000.00',
'delivered_in_month' => 0,
]);
$this->project = Project::factory()->create([
'tenant_id' => $this->tenant->id,
'signal_type' => 'site',
'signal_identifier' => 'client.carmoney.ru',
'is_active' => true,
'daily_limit_target' => 100,
'delivered_today' => 0,
]);
});
it('webhook with non-B-prefix project is accepted (202) and platform=DIRECT', function () {
$response = $this->postJson('/api/webhook/supplier/test-secret-32chars-aaaaaaaaaaaaaa', [
'vid' => 9999001,
'project' => 'client.carmoney.ru',
'phone' => '79991234567',
'time' => time(),
]);
$response->assertStatus(202);
expect(SupplierLead::where('vid', 9999001)->exists())->toBeTrue();
expect(SupplierLead::where('vid', 9999001)->first()->platform)->toBe('DIRECT');
});
it('SupplierProjectResolver creates DIRECT supplier_project for non-B project', function () {
$resolver = app(\App\Services\SupplierProjects\SupplierProjectResolver::class);
$sp = $resolver->resolveOrStub('DIRECT', 'site', 'client.carmoney.ru');
expect($sp->platform)->toBe('DIRECT');
expect($sp->unique_key)->toBe('client.carmoney.ru');
expect($sp->signal_type)->toBe('site');
});
it('RouteSupplierLeadJob delivers DIRECT lead to matching Liderra project via signal_identifier fallback', function () {
$lead = SupplierLead::create([
'platform' => 'DIRECT',
'phone' => '79991234567',
'vid' => 9999002,
'raw_payload' => ['project' => 'client.carmoney.ru', 'phone' => '79991234567', 'time' => time()],
'received_at' => now(),
'source' => 'webhook',
]);
(new RouteSupplierLeadJob($lead->id))->handle(
app(\App\Services\LeadRouter::class),
app(\App\Services\SupplierProjects\SupplierProjectResolver::class),
app(\App\Services\NotificationService::class),
app(\App\Services\Billing\LedgerService::class),
app(\App\Services\LeadDistributor::class),
app(\App\Services\RegionTagResolver::class),
);
$deal = Deal::where('tenant_id', $this->tenant->id)->where('phone', '79991234567')->first();
expect($deal)->not->toBeNull();
expect($deal->project_id)->toBe($this->project->id);
expect($deal->source_crm_id)->toBe(9999002);
});
it('numeric-only project (e.g. 79135191264) accepted as DIRECT', function () {
// Поставщик иногда шлёт project=телефонный номер (callback-проекты).
$response = $this->postJson('/api/webhook/supplier/test-secret-32chars-aaaaaaaaaaaaaa', [
'vid' => 9999003,
'project' => '79135191264',
'phone' => '79991234567',
'time' => time(),
]);
$response->assertStatus(202);
});
it('existing B1/B2/B3 webhooks still work (regression)', function () {
$response = $this->postJson('/api/webhook/supplier/test-secret-32chars-aaaaaaaaaaaaaa', [
'vid' => 9999004,
'project' => 'B1_krk-finance.ru',
'phone' => '79991234567',
'time' => time(),
]);
$response->assertStatus(202);
expect(SupplierLead::where('vid', 9999004)->first()->platform)->toBe('B1');
});
```
- [ ] **Step 2: Run tests, expect FAIL on most**
```
cd app && ./vendor/bin/pest tests/Feature/Supplier/DirectPlatformTest.php
```
Expected: тесты #1, #2, #3, #4 FAIL (regex rejects non-B, resolver throws, job throws). Тест #5 PASS (B1 already works).
- [ ] **Step 3: Commit failing tests**
```bash
git add app/tests/Feature/Supplier/DirectPlatformTest.php
git commit -m "test(supplier): end-to-end DIRECT platform tests (failing)"
```
---
## Task 5: Implement — webhook controller accepts non-B + parsePlatform returns DIRECT
**Files:**
- Modify: `app/app/Http/Controllers/Api/SupplierWebhookController.php`
- [ ] **Step 1: Remove regex constraint on project field (line 86)**
```php
'project' => ['required', 'string', 'max:255'], // снят regex /^B[123]_.+$/
```
- [ ] **Step 2: Update parsePlatform (lines 183-188) to return 'DIRECT' for non-B**
```php
private function parsePlatform(string $project): string
{
if (preg_match('/^(B[123])_/', $project, $m) === 1) {
return $m[1];
}
return 'DIRECT';
}
```
- [ ] **Step 3: Run tests — DirectPlatformTest #1 should now PASS**
```
cd app && ./vendor/bin/pest tests/Feature/Supplier/DirectPlatformTest.php --filter='accepted (202) and platform=DIRECT'
```
Expected: PASS. Также:
```
cd app && ./vendor/bin/pest tests/Feature/Http/Webhook/SupplierWebhookTest.php --filter='rejects invalid project format'
```
Тест ('rejects invalid project format ... with 422') теперь будет **FAIL** — потому что мы изменили поведение. Это ожидаемое — переписываем тест в следующем step.
- [ ] **Step 4: Rewrite the obsolete test in SupplierWebhookTest.php line 95**
Перепиcать:
```php
it('accepts project without B[123]_ prefix as platform=DIRECT (Phase 3)', function () {
Bus::fake();
$response = $this->postJson('/api/webhook/supplier/test-secret-32chars-aaaaaaaaaaaaaa', [
'vid' => 1, 'project' => 'client.carmoney.ru', 'phone' => '79991234567', 'time' => time(),
]);
$response->assertStatus(202);
});
```
- [ ] **Step 5: Run full SupplierWebhookTest + DirectPlatformTest**
```
cd app && ./vendor/bin/pest tests/Feature/Http/Webhook/SupplierWebhookTest.php tests/Feature/Supplier/DirectPlatformTest.php
```
Expected: тесты #1 в DirectPlatformTest PASS, остальные новые — пока FAIL (resolver/job не готовы).
- [ ] **Step 6: Commit**
```bash
git add app/app/Http/Controllers/Api/SupplierWebhookController.php app/tests/Feature/Http/Webhook/SupplierWebhookTest.php
git commit -m "feat(supplier-webhook): accept non-B-prefix projects as platform=DIRECT
Drops regex /^B[123]_.+\$/ from project field validation; parsePlatform()
returns 'DIRECT' for projects without B-prefix. SupplierLead created
with platform='DIRECT' for these. Rewrites obsolete test that asserted
invalid_format → 422 — now invalid_format → 202 with platform=DIRECT."
```
---
## Task 6: Implement — SupplierProjectResolver accepts DIRECT
**Files:**
- Modify: `app/app/Services/SupplierProjects/SupplierProjectResolver.php`
- [ ] **Step 1: Extend ALLOWED_PLATFORMS**
```php
private const ALLOWED_PLATFORMS = ['B1', 'B2', 'B3', 'DIRECT'];
```
- [ ] **Step 2: Run DirectPlatformTest #2**
```
cd app && ./vendor/bin/pest tests/Feature/Supplier/DirectPlatformTest.php --filter='creates DIRECT supplier_project'
```
Expected: PASS.
- [ ] **Step 3: Commit**
```bash
git add app/app/Services/SupplierProjects/SupplierProjectResolver.php
git commit -m "feat(supplier): SupplierProjectResolver accepts platform=DIRECT"
```
---
## Task 7: Implement — RouteSupplierLeadJob.parseProjectField + LeadRouter fallback for DIRECT
**Files:**
- Modify: `app/app/Jobs/RouteSupplierLeadJob.php:172-200`
- Modify: `app/app/Services/LeadRouter.php:45-76`
- [ ] **Step 1: parseProjectField — добавить DIRECT branch**
В RouteSupplierLeadJob, `parseProjectField` (lines 172-200), заменить начало с:
```php
private function parseProjectField(string $project): array
{
if (preg_match('/^(B[123])_(.+)$/', $project, $m) === 1) {
$platform = $m[1];
$rest = $m[2];
} else {
// Phase 3: проекты без B-префикса попадают в DIRECT.
// Весь project считается identifier-частью; signal_type определяется
// тем же regex'ом, что для $rest у B-префиксных.
$platform = 'DIRECT';
$rest = $project;
}
// далее существующий код — определение signal_type/identifier на $rest
// (call / site / sms по regex'ам), без изменений
$domainRe = '/(?<![a-z0-9.\-])([a-z0-9][a-z0-9\-]*(?:\.[a-z0-9][a-z0-9\-]*)*\.[a-z]{2,})/i';
// ... existing logic ...
}
```
- [ ] **Step 2: LeadRouter — добавить DIRECT fallback**
В LeadRouter::matchEligibleProjects, расширить SQL: для DIRECT supplier_projects использовать fallback по signal_type+signal_identifier matchу с Лидерра-проектами (если нет project_supplier_links для DIRECT).
```php
public function matchEligibleProjects(SupplierProject $supplierProject): Collection
{
$todayBit = 1 << (Carbon::now('Europe/Moscow')->isoWeekday() - 1);
// Phase 3: для DIRECT-supplier_project — fallback на signal_type+signal_identifier
// match с Лидерра-проектами, потому что project_supplier_links для DIRECT-row'ов
// ещё не настроены (это автоматический матчинг по сигналу). Для B1/B2/B3
// продолжаем использовать explicit psl-link.
if ($supplierProject->platform === 'DIRECT') {
$sql = <<<'SQL'
SELECT DISTINCT ON (projects.tenant_id) projects.*
FROM projects
WHERE projects.signal_type = ?
AND LOWER(projects.signal_identifier) = LOWER(?)
AND projects.is_active = true
AND (projects.delivery_days_mask & ?) <> 0
AND projects.delivered_today < COALESCE(projects.effective_daily_limit_today, projects.daily_limit_target)
AND EXISTS (
SELECT 1 FROM tenants
WHERE tenants.id = projects.tenant_id
AND (tenants.balance_leads > 0 OR tenants.balance_rub > 0)
)
ORDER BY
projects.tenant_id,
(COALESCE(projects.effective_daily_limit_today, projects.daily_limit_target) - projects.delivered_today) DESC,
projects.created_at,
projects.id
SQL;
$rows = DB::connection('pgsql_supplier')->select(
$sql,
[$supplierProject->signal_type, $supplierProject->unique_key, $todayBit]
);
return Project::hydrate($rows)->values();
}
// Existing B1/B2/B3 path — explicit psl link
$sql = <<<'SQL'
SELECT DISTINCT ON (projects.tenant_id) projects.*
FROM projects
WHERE EXISTS (
SELECT 1 FROM project_supplier_links psl
WHERE psl.project_id = projects.id
AND psl.supplier_project_id = ?
)
AND projects.is_active = true
AND (projects.delivery_days_mask & ?) <> 0
AND projects.delivered_today < COALESCE(projects.effective_daily_limit_today, projects.daily_limit_target)
AND EXISTS (
SELECT 1 FROM tenants
WHERE tenants.id = projects.tenant_id
AND (tenants.balance_leads > 0 OR tenants.balance_rub > 0)
)
ORDER BY
projects.tenant_id,
(COALESCE(projects.effective_daily_limit_today, projects.daily_limit_target) - projects.delivered_today) DESC,
projects.created_at,
projects.id
SQL;
$rows = DB::connection('pgsql_supplier')->select($sql, [$supplierProject->id, $todayBit]);
return Project::hydrate($rows)->values();
}
```
- [ ] **Step 3: Run DirectPlatformTest #3 — end-to-end DIRECT routing**
```
cd app && ./vendor/bin/pest tests/Feature/Supplier/DirectPlatformTest.php --filter='delivers DIRECT lead'
```
Expected: PASS. Deal создан, project_id matched.
- [ ] **Step 4: Run full supplier regression**
```
cd app && ./vendor/bin/pest tests/Feature/Supplier/ tests/Feature/Jobs/RouteSupplierLeadJobTest.php tests/Feature/Http/Webhook/
```
Expected: все тесты PASS. Особенно регрессия B1/B2/B3 — proxy через `else` branch.
- [ ] **Step 5: Commit**
```bash
git add app/app/Jobs/RouteSupplierLeadJob.php app/app/Services/LeadRouter.php
git commit -m "feat(supplier): RouteSupplierLeadJob + LeadRouter handle DIRECT platform
parseProjectField() returns ('DIRECT', signal_type, identifier) when project
has no B-prefix; identifier-detection (call/site/sms regex) runs on full
project string. LeadRouter::matchEligibleProjects has a DIRECT fast-path
that matches Liderra projects by (signal_type, signal_identifier) directly
without requiring project_supplier_links pivot — because DIRECT
supplier_projects are auto-created on first webhook and don't have manual
psl links.
B1/B2/B3 path unchanged (psl-based)."
```
---
## Task 8: Implement — LedgerService.resolveSupplierId fallback for DIRECT + CsvReconcileJob extractPlatform
**Files:**
- Modify: `app/app/Services/Billing/LedgerService.php:127-148`
- Modify: `app/app/Jobs/Supplier/CsvReconcileJob.php:237-244`
- [ ] **Step 1: Extend LedgerService.resolveSupplierId**
```php
private function resolveSupplierId(SupplierLead $lead): ?int
{
if ($lead->supplier_project_id !== null) {
$sp = DB::table('supplier_projects')->where('id', $lead->supplier_project_id)->first();
if ($sp !== null) {
if (in_array($sp->platform, ['B1', 'B2', 'B3'], true)) {
$supplier = Supplier::where('code', strtolower($sp->platform))->first();
if ($supplier !== null) {
return (int) $supplier->id;
}
}
if ($sp->platform === 'DIRECT') {
$supplier = Supplier::where('code', 'direct')->first();
return $supplier?->id;
}
}
}
// Fallback: parse platform from raw_payload['project']
$project = trim((string) ($lead->raw_payload['project'] ?? ''));
if (preg_match('/^(B[123])_/', $project, $m) === 1) {
$code = strtolower($m[1]);
$supplier = Supplier::where('code', $code)->first();
return $supplier?->id;
}
// Phase 3: project без B-префикса — DIRECT
if ($project !== '') {
$supplier = Supplier::where('code', 'direct')->first();
return $supplier?->id;
}
return null;
}
```
- [ ] **Step 2: Update CsvReconcileJob.extractPlatform**
Сейчас extractPlatform возвращает null для не-B → строка увеличивает `unparseable_count` (правильный для МУСОРА типа phone/URL в поле project, но НЕ для DIRECT-проектов как `client.carmoney.ru`). Различение:
```php
private function extractPlatform(string $project): ?string
{
if (preg_match('/^(B[123])_/', $project, $m) === 1) {
return $m[1];
}
// Phase 3: пытаемся распарсить как DIRECT (валидный domain/call/sms identifier).
// Только если строка содержит хотя бы одну букву или dot (= вероятно
// domain/название), а не чистый-числовой (= скорее всего телефон в роли проекта).
if (preg_match('/[a-zA-Zа-яА-Я.]/u', $project) === 1) {
return 'DIRECT';
}
// Чисто цифры или мусор — оставляем как unparseable (как было).
return null;
}
```
NB: чисто-числовые проекты ('79135191264') у поставщика — это **callback-проекты**, они валидны и должны быть DIRECT. Уточняем regex:
```php
private function extractPlatform(string $project): ?string
{
if (preg_match('/^(B[123])_/', $project, $m) === 1) {
return $m[1];
}
// Phase 3: всё что выглядит как разумный identifier (домен / телефон / SMS-sender) → DIRECT.
// unparseable_count теперь только для откровенного мусора (пустые / только спец-символы).
$trimmed = trim($project);
if ($trimmed !== '' && preg_match('/^[\w\-.а-яА-Я0-9\/() +]+$/u', $trimmed) === 1) {
return 'DIRECT';
}
return null;
}
```
- [ ] **Step 3: Run regression — CsvReconcileJobTest + RouteSupplierLeadJobBillingTest**
```
cd app && ./vendor/bin/pest tests/Feature/Supplier/CsvReconcileJobTest.php tests/Feature/Supplier/RouteSupplierLeadJobBillingTest.php tests/Feature/Supplier/DirectPlatformTest.php
```
Expected: все PASS.
- [ ] **Step 4: Commit**
```bash
git add app/app/Services/Billing/LedgerService.php app/app/Jobs/Supplier/CsvReconcileJob.php
git commit -m "feat(supplier): LedgerService + CsvReconcileJob recognise DIRECT platform
LedgerService::resolveSupplierId returns suppliers.code='direct' row for
DIRECT-platform supplier_projects (and for parsed-from-payload non-B
projects). CsvReconcileJob::extractPlatform now classifies most non-empty,
non-junk project strings as DIRECT (instead of dumping them into
unparseable_count) — this allows CSV recovery to also create DIRECT
supplier_leads, mirroring the webhook path."
```
---
## Task 9: Sync db/schema.sql + CHANGELOG_schema.md
**Files:**
- Modify: `db/schema.sql` — поправить constraint definitions
- Modify: `db/CHANGELOG_schema.md`
- [ ] **Step 1: Update db/schema.sql constraint definitions**
В двух местах `chk_supplier_projects_platform` и `chk_psl_platform` — заменить `IN ('B1','B2','B3')` на `IN ('B1','B2','B3','DIRECT')`.
- [ ] **Step 2: Add CHANGELOG_schema.md entry**
```markdown
## v8.X — 2026-05-25 — DIRECT platform support
- Extended `chk_supplier_projects_platform` to include `'DIRECT'`
- Extended `chk_psl_platform` to include `'DIRECT'`
- Seeded `suppliers.code='direct'` row (BP-GR Direct, cost_rub = same as B1)
- Spec: docs/superpowers/specs/2026-05-25-supplier-webhook-reliability-design.md
```
- [ ] **Step 3: Commit**
```bash
git add db/schema.sql db/CHANGELOG_schema.md
git commit -m "docs(schema): sync DIRECT platform CHECK constraints to db/schema.sql"
```
---
## Task 10: Regression + prod-readiness
**Files:**
- None
- [ ] **Step 1: /regression full**
```
/regression full
```
Expected: GREEN. Pest --parallel 700+ tests pass.
- [ ] **Step 2: Larastan**
```
cd app && composer stan
```
Expected: 0 errors над baseline.
- [ ] **Step 3: Manual webhook smoke на dev**
(если dev-сервер работает)
```bash
cd app && php artisan serve --port=8000 &
sleep 2
curl -X POST http://localhost:8000/api/webhook/supplier/<dev-secret> \
-H 'Content-Type: application/json' \
-d '{"vid":99999,"project":"client.carmoney.ru","phone":"79991234567","time":'$(date +%s)'}'
pkill -f 'artisan serve' || true
```
Expected: `{"status":"accepted","supplier_lead_id":...}` 202.
---
## Task 11: Deploy to liderra.ru
**Files:**
- None
- [ ] **Step 1: prod-deploy-validator agent**
```
subagent_type: prod-deploy-validator
prompt: проверь готовность liderra.ru к Phase 3 деплою. Меняется: миграция БД (2 CHECK constraints), seed (suppliers.code='direct'), 5 PHP-файлов (SupplierWebhookController/RouteSupplierLeadJob/CsvReconcileJob/SupplierProjectResolver/LeadRouter/LedgerService), сменён тест.
Особое внимание:
1. Миграция ALTER CONSTRAINT не блокирует таблицу долго (DROP+ADD на 2 таблицах в одной транзакции).
2. После миграции — обязательный queue:restart (RouteSupplierLeadJob memory-cached в воркерах).
3. redeploy.sh должен сначала migrate потом optimize — проверь порядок.
Phase 1 + Phase 2 уже стоят ≥2h. 8 pre-flight + GO/NO-GO.
```
- [ ] **Step 2: Merge feature branch → main**
```bash
git checkout main
git merge --ff-only feat/supplier-webhook-fixes
git push origin main
```
- [ ] **Step 3: redeploy.sh**
```bash
ssh liderra "cd /var/www/liderra/app && sudo -u www-data ./redeploy.sh 2>&1 | tail -80"
```
Expected: migration ran successfully, queue:restart fired, deploy complete.
- [ ] **Step 4: Prod smoke — webhook with non-B project**
```bash
ssh liderra 'curl -sk -X POST \
-H "Content-Type: application/json" \
-d "{\"vid\":99999001,\"project\":\"client.carmoney.ru\",\"phone\":\"79991234567\",\"time\":'$(date +%s)'}" \
https://liderra.ru/api/webhook/supplier/8c1c07ddb0768763661b357198e0625832f74ad0915d91b1'
```
Expected: `{"status":"accepted","supplier_lead_id":...}` или `{"status":"already_processed",...}` если повтор. Status 202 / 200.
- [ ] **Step 5: Check supplier_projects has new DIRECT row**
```bash
ssh liderra "sudo -u postgres psql -d liderra -c \"SELECT id, platform, signal_type, unique_key, created_at FROM supplier_projects WHERE platform='DIRECT' ORDER BY id DESC LIMIT 5\""
```
Expected: видим только что созданную (или существующую) DIRECT-row с unique_key='client.carmoney.ru' (test smoke).
- [ ] **Step 6: Wait 6 hours, observe**
Через 6 часов:
```bash
ssh liderra "sudo grep '/api/webhook/supplier' /var/log/nginx/access.log | grep '$(date +%d/%b)' | awk '{print \$9}' | sort | uniq -c"
ssh liderra "sudo -u postgres psql -d liderra -c \"SELECT platform, COUNT(*) FROM supplier_leads WHERE received_at > NOW() - interval '6 hours' GROUP BY platform\""
ssh liderra "sudo -u postgres psql -d liderra -c \"SELECT COUNT(*) FILTER (WHERE source_crm_id IS NULL) AS no_crm_id, COUNT(*) FILTER (WHERE source_crm_id IS NOT NULL) AS with_crm_id, COUNT(*) AS total FROM deals WHERE tenant_id=2 AND created_at > NOW() - interval '6 hours'\""
```
Expected:
- nginx: 0 × 302 на webhook (все принимаются)
- supplier_leads: видим записи с platform='DIRECT' (~ 67/24 = 2-3 в час)
- deals: 0 unmerged duplicates (Phase 2 покрывает)
- [ ] **Step 7: Update ПИЛОТ.md + memory**
```bash
# Update ПИЛОТ.md, memory entries
git add ПИЛОТ.md
git commit -m "docs(пилот): Phase 3 supplier DIRECT platform deployed, $X DIRECT leads in 6h"
git push origin main
```
---
## Done criteria для Phase 3
- [ ] Все тесты в DirectPlatformTest.php + регрессия supplier/* + webhook/* PASS
- [ ] /regression full GREEN
- [ ] Larastan baseline clean
- [ ] migration up/down работают на dev
- [ ] Прод-smoke: webhook `project: "client.carmoney.ru"` → 202
- [ ] 6 часов наблюдения: webhook 302 ушли в 0, новые DIRECT leads принимаются, нет дублей
---
## Откат
Сложнее остальных — есть миграция БД.
```bash
# 1. Cleanup: убрать DIRECT-rows если они появились на проде
ssh liderra "sudo -u postgres psql -d liderra -c \"DELETE FROM project_supplier_links WHERE platform='DIRECT'; DELETE FROM supplier_projects WHERE platform='DIRECT'\""
# 2. Migration down
ssh liderra "cd /var/www/liderra/app && sudo -u www-data php artisan migrate:rollback --step=2"
# 3. Revert code
ssh liderra "cd /var/www/liderra/app && git revert --no-edit HEAD~N..HEAD && sudo -u www-data ./redeploy.sh"
```
Лиды с platform=DIRECT, уже превратившиеся в deals, остаются (deal.project_id указывает на валидный Лидерра-проект); supplier_lead.platform='B1' fallback не применится для уже сохранённых, но и не нужен — они уже обработаны.
Если откат нужен экстренно — можно ограничиться **revert кода без migration:rollback**: миграция оставляет DIRECT в enum, старый код просто никогда не создаст такую row. БД не сломается.
@@ -0,0 +1,291 @@
# Supplier webhook reliability — design spec
**Дата:** 2026-05-25
**Статус:** draft → готов к плану
**Ветка:** `feat/supplier-webhook-fixes`
**Связано:** Спек B Phase 1 (`docs/superpowers/specs/2026-05-23-billing-v2-spec-b-duplicates-design.md`) — снят DuplicateDetector; данная спека закрывает race condition, оставшийся после Спека B.
---
## 1. Проблема
На боевом liderra.ru за сутки 25.05.2026 для тенанта `client1` (tenant_id=2):
- Поставщик crm.bp-gr.ru отдал **205 уникальных лидов** (учётка `info@lkomega.ru`, страница `/admin/visit/index-visit?visit=rt`)
- На портале — **160 сделок**, из них **123 уникальных телефона** (37 — дубликаты `phone+project`)
- **Расхождения:** 82 лида у поставщика не дошли до портала; 37 deals в портале дублированы
### 1.1. Корневая причина потерь (76 из 82)
Из 234 POST-запросов поставщика на `/api/webhook/supplier/<secret>` сегодня:
- **132** → 202 Accepted (приняты)
- **76** → 302 Found (Location: `https://liderra.ru`)
- 29 → 301 (http→https на `/`)
Воспроизведено вручную: `curl -X POST` с пустым `{}` → 302 + Set-Cookie. Это **дефолтный Laravel behavior**: для запросов, где `Accept` НЕ содержит `application/json`, `ValidationException` рендерится через `redirect()->back()->withErrors()` — 302 на referer (которого нет у webhook-вызывающего) → fallback на `/`.
Запросы 302 — это webhook-и где `project` НЕ матчится regex `'project' => regex:/^B[123]_.+$/'` ([app/app/Http/Controllers/Api/SupplierWebhookController.php:86](../../../app/app/Http/Controllers/Api/SupplierWebhookController.php#L86)).
Конкретные «непринимаемые» проекты (видны в supplier rt-list):
- `client.carmoney.ru` — 55 лидов
- `B2_Caranga` — 7
- `cabinet.caranga.ru` — 3
- `cashmotor.ru` — 2
- остальные единичные: `73912346386`, `79135191264`, `78006009393`, `78007006600`, `79029248888`, `B2_drivezaim`, `B3_+7 (495) 023-66-52` и т.п.
### 1.2. Корневая причина дублей (37)
[app/app/Jobs/Supplier/CsvReconcileJob.php:146-155](../../../app/app/Jobs/Supplier/CsvReconcileJob.php#L146-L155) каждые 30 мин создаёт «recovered» `SupplierLead` с **`vid: null`**, `source: csv_recovery` для лидов, найденных в CSV поставщика но отсутствующих в наших `supplier_leads` за окно.
Затем поставщик ретраит webhook с настоящим `vid` (численный) → создаётся **новый** `SupplierLead` (UNIQUE по `vid`, NULL ≠ NULL → не считается дублем) → `RouteSupplierLeadJob` создаёт **второй Deal**.
`supplier_lead_deliveries` уник-индекс на `(supplier_lead_id, tenant_id)` ([app/app/Jobs/RouteSupplierLeadJob.php:249-262](../../../app/app/Jobs/RouteSupplierLeadJob.php#L249-L262)) **не блокирует**, потому что у CSV-recovered и webhook разные `supplier_lead.id`.
Раньше эту race-condition закрывал `DuplicateDetector` (24h-фильтр по `phone+project`), который был снят в Спеке B Phase 1 (commit `ccfecd5e`, 24.05) с обоснованием «за повторы поставщика берём».
### 1.3. Цепочка B-префикса (5 точек)
Regex `B[123]_` встречается в коде в **5 точках**, и все обязательны для текущего flow:
| # | Место | file:line | Поведение без B-префикса |
|---|---|---|---|
| 1 | Webhook validation | [SupplierWebhookController.php:86](../../../app/app/Http/Controllers/Api/SupplierWebhookController.php#L86) | ValidationException → 302 (см. 1.1) |
| 2 | parsePlatform fallback | [SupplierWebhookController.php:183-188](../../../app/app/Http/Controllers/Api/SupplierWebhookController.php#L183-L188) | silent fallback 'B1' |
| 3 | parseProjectField | [RouteSupplierLeadJob.php:172-200](../../../app/app/Jobs/RouteSupplierLeadJob.php#L172-L200) | **RuntimeException** → retry 3x → failed_webhook_jobs |
| 4 | extractPlatform | [CsvReconcileJob.php:237-244](../../../app/app/Jobs/Supplier/CsvReconcileJob.php#L237-L244) | возвращает `null` → строка в `unparseable_count` (56 сегодня) |
| 5 | БД constraint | `supplier_projects.platform CHECK IN (B1,B2,B3)` | нельзя сохранить platform=`DIRECT` |
---
## 2. Цели и не-цели
### Цели
- **C1.** Webhook на `/api/webhook/supplier/*` ВСЕГДА отвечает JSON (202/200/422/429/404), никогда не редиректит. Любая `ValidationException` для этого URL — JSON 422 с полем `errors`.
- **C2.** Webhook, поступивший после CSV-recovered deal по тому же `(tenant_id, phone, project_id)` в окне 24h, **обновляет** существующий deal (`source_crm_id`, `received_at` если новее, `phones`), а не создаёт второй. Биллинг не списывает второй раз.
- **C3.** Webhook на проекты без префикса `B[123]_` (`client.carmoney.ru`, `cashmotor.ru`, числовые) принимается, проходит routing, создаёт Deal под новой платформой `DIRECT`.
### Не-цели
- **NG1.** Восстановление 82 потерянных лидов 25.05 — оффлайн-операция после деплоя, через `php artisan supplier:reconcile-force` или ручное добавление по списку (вне scope этой спеки).
- **NG2.** Очистка 37 текущих дублей в проде — отдельная миграция данных или ручной SQL (вне scope).
- **NG3.** Изменение бизнес-правил биллинга для DIRECT-платформы. Берётся та же тарификация, что для B1/B2/B3 (по умолчанию tier по `signal_type`). Альтернативная цена для DIRECT — отдельный спек если потребуется.
- **NG4.** Отказ от CSV reconcile job — он остаётся как safety net, но теперь дедупликация не приводит к дублям.
---
## 3. Решение
Три независимые фазы. Каждая фаза — отдельный PR, отдельный план, отдельный выкат на боевой. Между фазами — observation period (1-2 часа на проде, потом следующая фаза).
### Phase 1 (низкий риск) — Always JSON 422 для webhook validation errors
**Изменения:**
- В [app/bootstrap/app.php:35](../../../app/bootstrap/app.php#L35) `withExceptions()` добавить render:
```php
$exceptions->render(function (\Illuminate\Validation\ValidationException $e, Request $request) {
if ($request->is('api/webhook/supplier/*')) {
return response()->json([
'message' => 'Validation failed',
'errors' => $e->errors(),
], 422);
}
return null; // дефолтный рендер для остальных
});
```
- Тест: POST с `Accept: text/html` (имитация поставщика без JSON-Accept) на webhook с невалидным payload → assert 422 + JSON Content-Type + ошибка в `errors`.
- Существующие тесты `SupplierWebhookTest.php` — все `postJson(...)` → 422 уже работают. Добавляется один новый тест с обычным `post()`.
**Risk:** низкий. Изменение не трогает control flow webhook'а, только формат ответа на ошибку.
**Откатываемость:** одной строчкой revert.
### Phase 2 (средний риск) — Идемпотентность webhook ↔ CSV-recovered
**Изменения:**
- В [app/app/Jobs/RouteSupplierLeadJob.php:207](../../../app/app/Jobs/RouteSupplierLeadJob.php#L207) `createDealCopyForProject()` ДО создания Deal — поиск:
```php
$existingDeal = Deal::query()
->where('tenant_id', $tenant->id)
->where('phone', (string) $lead->phone)
->where('project_id', $project->id)
->where('received_at', '>=', now()->subDay())
->whereNull('source_crm_id') // только CSV-recovered ждут vid
->lockForUpdate()
->first();
```
- Если найден → `UPDATE deals SET source_crm_id = vid, received_at = MAX(...)` + `supplier_lead_deliveries` запись + **НЕ списываем баланс повторно** (Ledger.alreadyChargedForDeal или просто отсутствие второго `chargeForDelivery`) → возврат `false`/`'merged'`.
- Если не найден → текущий путь создания нового Deal без изменений.
- `supplier_lead_deliveries.deal_id` обновляется на найденный deal.id.
**Биллинг safety:**
- `LedgerService::chargeForDelivery` уже идемпотентен по `supplier_lead_id` (PK lead_charges) — проверить.
- Если не идемпотентен — добавить guard: SELECT lead_charges WHERE deal_id=$existingDeal->id; если есть — skip charge.
**Тесты:**
- TDD: CSV-recovered deal без vid → webhook на тот же phone+project → assert 1 deal (не 2), source_crm_id заполнен, lead_charges = 1 запись.
- Regression: повтор поставщика по тому же vid (память Спека B — «за повторы берём») → assert 2 deals (если разные supplier_lead с разными vid).
- Race: одновременный webhook и CSV-recovery → lockForUpdate гарантирует один deal.
**Risk:** средний — затрагивает биллинг. Нужно убедиться что `chargeForDelivery` не списывает второй раз.
### Phase 3 (высокий риск) — DIRECT platform для проектов без B-префикса
**Изменения:**
1. **Миграция БД** `database/migrations/2026_05_25_120000_add_direct_platform.php`:
```sql
ALTER TABLE supplier_projects DROP CONSTRAINT chk_supplier_projects_platform;
ALTER TABLE supplier_projects ADD CONSTRAINT chk_supplier_projects_platform
CHECK (platform IN ('B1','B2','B3','DIRECT'));
ALTER TABLE project_supplier_links DROP CONSTRAINT chk_psl_platform;
ALTER TABLE project_supplier_links ADD CONSTRAINT chk_psl_platform
CHECK (platform IN ('B1','B2','B3','DIRECT'));
```
Также снять constraint `chk_supplier_projects_b1_not_for_sms` (он про B1+sms) если он мешает.
2. **Webhook regex** [SupplierWebhookController.php:86](../../../app/app/Http/Controllers/Api/SupplierWebhookController.php#L86):
```php
'project' => ['required', 'string', 'max:255'], // снят regex
```
3. **parsePlatform** [SupplierWebhookController.php:183-188](../../../app/app/Http/Controllers/Api/SupplierWebhookController.php#L183-L188):
```php
private function parsePlatform(string $project): string
{
if (preg_match('/^(B[123])_/', $project, $m) === 1) {
return $m[1];
}
return 'DIRECT';
}
```
4. **parseProjectField** [RouteSupplierLeadJob.php:172-200](../../../app/app/Jobs/RouteSupplierLeadJob.php#L172-L200) — добавить DIRECT branch:
```php
private function parseProjectField(string $project): array
{
if (preg_match('/^(B[123])_(.+)$/', $project, $m) === 1) {
$platform = $m[1];
$rest = $m[2];
} else {
$platform = 'DIRECT';
$rest = $project; // весь project считается identifier-частью
}
// далее существующая логика определения signal_type/identifier на $rest
// (call / site / sms по тем же regex'ам)
}
```
5. **extractPlatform** [CsvReconcileJob.php:237-244](../../../app/app/Jobs/Supplier/CsvReconcileJob.php#L237-L244):
```php
private function extractPlatform(string $project): string
{
if (preg_match('/^(B[123])_/', $project, $m) === 1) {
return $m[1];
}
return 'DIRECT';
}
```
Логика `unparseable_count` снимается для DIRECT-кейса; остаётся только для **реального мусора** (телефоны/URL в поле project). Различение через дополнительный regex проверки `[a-z0-9]` в начале.
6. **SupplierProjectResolver** — резолв по `(platform=DIRECT, signal_type, identifier)` создаёт/находит `supplier_projects` row с platform=DIRECT.
7. **LeadRouter::matchEligibleProjects** — DIRECT-platform fetches по тем же signal_type/identifier-полям проекта; никаких B1/B2/B3 специальных условий.
**Тесты:**
- Существующий тест `'rejects invalid project format with 422'` ([SupplierWebhookTest.php:95](../../../app/tests/Feature/Http/Webhook/SupplierWebhookTest.php#L95)) переписать: теперь invalid_format → 202 (принят), platform=DIRECT.
- Новый тест: webhook с `project: "client.carmoney.ru"` → 202, supplier_lead.platform=DIRECT, RouteSupplierLeadJob создаёт SupplierProject под DIRECT, Deal создаётся.
- Существующие тесты RouteSupplierLeadJobTest / CsvReconcileJobTest — добавить DIRECT-кейсы.
- Регрессия: все B1/B2/B3 кейсы продолжают работать без изменений.
**Risk:** высокий — затрагивает миграцию БД, ⩾5 файлов кода, тесты, бизнес-семантику биллинга для DIRECT.
**Сложность:** одновременная правка должна быть атомарной — если деплоится миграция но не код, controller примет lid'ы которые job не сможет обработать. Один PR, один деплой, очередь queue:restart после.
---
## 4. Стратегия деплоя
Три отдельных деплоя на liderra.ru через `redeploy.sh` (per memory: «`sudo -u www-data php artisan optimize` в строке 9 скрипта»):
1. **Деплой 1 (Phase 1):** ~10 мин outage риск 0. Сразу после деплоя смотрим nginx logs — все POST → 422 или 202, нет 30x. Ждём 30 мин — drift_alert не должен подниматься.
2. **Деплой 2 (Phase 2):** ~10 мин outage риск 0. Смотрим что новые deals не дублируются (`SELECT phone, project_id, COUNT(*) FROM deals WHERE created_at > NOW()-interval'2h' GROUP BY 1,2 HAVING COUNT(*)>1`). Ждём 1-2 часа.
3. **Деплой 3 (Phase 3):** включает миграцию БД. Сначала миграция (idempotent CHECK extension), затем код. Smoke: POST `project: "client.carmoney.ru"` с правильным secret и IP → 202, supplier_lead создан, deal создан. Ждём 6 часов на наблюдение, после — закрытие задачи.
Перед каждым деплоем — обязательно агент `prod-deploy-validator` (per [Pravila §2.4](../../Pravila_raboty_Claude_v1_1.md)).
---
## 5. Тестирование
### Pest unit/feature
Все три фазы — TDD: тест → fail → имплементация → pass → commit. Запуск `composer test -- --filter='Supplier'` после каждой фазы.
Существующие тесты, которые гарантированно адаптируются:
- `app/tests/Feature/Http/Webhook/SupplierWebhookTest.php` — line 95 «invalid_format → 422» переписывается на «invalid_format → 202 DIRECT» в Phase 3.
- `app/tests/Feature/Supplier/CsvReconcileJobTest.php` — добавить кейс DIRECT в Phase 3.
- `app/tests/Feature/Supplier/RouteSupplierLeadJobBillingTest.php` — добавить «webhook после CSV-recovered не списывает второй раз» в Phase 2.
- `app/tests/Feature/Supplier/SupplierLeadDeliveryGuardTest.php` — добавить кейс «разные SupplierLead.id, тот же phone+project — не дубль» в Phase 2.
### Регрессия
`/regression full` ПОСЛЕ каждой фазы (Pest --parallel + Larastan + Vitest + Vite build + lychee + gitleaks). Каждая фаза — отдельный коммит на ветке `feat/supplier-webhook-fixes`, отдельный PR, отдельный merge → отдельный redeploy.
### Прод-smoke
После каждого деплоя — конкретные SQL-проверки в `db/`, описаны в каждом плане.
---
## 6. Откат
- Phase 1 — revert single commit.
- Phase 2 — revert commit + dedup кода. Миграции БД нет.
- Phase 3 — revert commit + миграция down: `DROP CONSTRAINT ... ADD CONSTRAINT ... CHECK IN (B1,B2,B3)`. Если в БД уже есть `platform=DIRECT` rows — миграция down упадёт. Нужен seed-cleanup перед откатом.
---
## 7. Файлы (общий список)
**Создать:**
- `database/migrations/2026_05_25_120000_add_direct_platform.php` (Phase 3)
- `app/tests/Feature/Http/Webhook/SupplierWebhookValidationFormatTest.php` (Phase 1, новый файл)
- `app/tests/Feature/Supplier/CsvWebhookRaceTest.php` (Phase 2, новый файл)
- `app/tests/Feature/Supplier/DirectPlatformTest.php` (Phase 3, новый файл)
**Изменить:**
- `app/bootstrap/app.php` (Phase 1)
- `app/app/Http/Controllers/Api/SupplierWebhookController.php` (Phase 3)
- `app/app/Jobs/RouteSupplierLeadJob.php` (Phase 2 + Phase 3)
- `app/app/Jobs/Supplier/CsvReconcileJob.php` (Phase 3)
- `app/app/Services/SupplierProjects/SupplierProjectResolver.php` (Phase 3)
- `app/app/Services/LeadRouter.php` (Phase 3)
- `app/tests/Feature/Http/Webhook/SupplierWebhookTest.php` (Phase 3 — переписать line 95)
- `db/schema.sql` (Phase 3 — sync с миграцией)
- `db/CHANGELOG_schema.md` (Phase 3)
**Возможно затронуть:**
- `app/app/Services/Billing/LedgerService.php` (Phase 2 — guard от двойного списания, если ещё не идемпотентен)
---
## 8. Открытые вопросы (на момент написания спеки)
- **OQ-1.** Идемпотентен ли `LedgerService::chargeForDelivery` по `(deal_id, lead_id)` или может списать дважды? — выяснится в Phase 2 Task 1 (read code).
- **OQ-2.** `supplier_projects.subject_code` — обязательное поле для DIRECT? — выяснится в Phase 3 Task 2 (миграция).
- **OQ-3.** `chk_supplier_projects_b1_not_for_sms` constraint конфликтует с DIRECT? — выяснится в Phase 3 Task 1.
Каждый вопрос разрешается inline во время реализации, не блокирует план.
---
## 9. Ссылки
- План Phase 1: `docs/superpowers/plans/2026-05-25-supplier-webhook-phase-1-json-422.md`
- План Phase 2: `docs/superpowers/plans/2026-05-25-supplier-webhook-phase-2-dedup.md`
- План Phase 3: `docs/superpowers/plans/2026-05-25-supplier-webhook-phase-3-direct-platform.md`
- Memory project_supplier_integration.md — историческая информация о supplier flow
- ADR-008 (если потребуется DIRECT — оформить как ADR-018 «Supplier DIRECT platform»)
+35
View File
@@ -161,6 +161,32 @@ function sessionTurnBucket(turn) {
return n < SESSION_TURN_EARLY ? 'early' : n <= SESSION_TURN_LATE ? 'mid' : 'late';
}
// Pass 1 cheap-axis helpers (project-brain-factor-analysis-4passes).
function countEventKind(events, kind) {
if (!Array.isArray(events)) return 0;
let c = 0;
for (const ev of events) if (ev && ev.kind === kind) c++;
return c;
}
function retryBucket(events) {
const n = countEventKind(events, 'retry');
return n === 0 ? '0' : n <= 2 ? '1-2' : '3+';
}
function errorBucket(events) {
const n = countEventKind(events, 'error');
return n === 0 ? '0' : n === 1 ? '1' : '2+';
}
function iterationsBucket(iterations) {
const n = Number(iterations);
if (!Number.isFinite(n) || n <= 0) return '0';
if (n <= 3) return '1-3';
if (n <= 10) return '4-10';
return '11+';
}
const FACTOR_FNS = {
decision_provenance: (e) => (e.decision_provenance || {}).kind || 'unknown',
economy_level: (e) => String((e.environment || {}).economy_level ?? 'null'),
@@ -172,6 +198,15 @@ const FACTOR_FNS = {
node_chosen: (e) => (e.primary_rationale || {}).node_chosen || 'direct',
task_classification: (e) => (e.primary_rationale || {}).task_classification || 'other',
recommended_node_for_direct: (e) => (e.primary_rationale || {}).recommended_node || 'none',
// Pass 1 — 8 cheap axes (data already in v4 episode, just expose):
prompt_signal: (e) => e.prompt_signal || 'null',
classifier_source: (e) => (e.classifier_output || {}).source || 'null',
degraded_mode: (e) => String(e.degraded_mode ?? false),
path_type: (e) => e.path_type || 'null',
retry_count: (e) => retryBucket(e.events),
error_count: (e) => errorBucket(e.events),
hard_floor_invoked: (e) => String(((e.primary_rationale || {}).hard_floor || {}).invoked ?? false),
iterations_bucket: (e) => iterationsBucket((e.task_cost || {}).iterations),
};
/** Factor matrix: rows = factor values, columns = outcome distribution (spec §6). */
+107
View File
@@ -409,3 +409,110 @@ describe('analyze — v4 aggregations (Phase 3 Task 20)', () => {
expect(ct.reviewer_input_tokens).toBe(500);
});
});
describe('buildFactorMatrix — Pass 1 cheap axes (project-brain-factor-analysis-4passes)', () => {
// Each new axis: smoke + null-safety on missing fields.
it('prompt_signal axis: raw discrete values + null fallback', () => {
const m = buildFactorMatrix([
{ ...ep(), _inferredOutcome: 'success', prompt_signal: 'new_task' },
{ ...ep(), _inferredOutcome: 'rework', prompt_signal: 'correction' },
{ ...ep(), _inferredOutcome: 'unknown', prompt_signal: undefined },
]);
expect(m.prompt_signal.new_task.success).toBe(1);
expect(m.prompt_signal.correction.rework).toBe(1);
expect(m.prompt_signal.null.unknown).toBe(1);
});
it('classifier_source axis: reads classifier_output.source verbatim', () => {
const m = buildFactorMatrix([
{ ...ep(), _inferredOutcome: 'success', classifier_output: { source: 'llm' } },
{ ...ep(), _inferredOutcome: 'success', classifier_output: { source: 'regex' } },
{ ...ep(), _inferredOutcome: 'success', classifier_output: { source: 'prefilter_inherited' } },
{ ...ep(), _inferredOutcome: 'unknown', classifier_output: null },
]);
expect(m.classifier_source.llm.success).toBe(1);
expect(m.classifier_source.regex.success).toBe(1);
expect(m.classifier_source.prefilter_inherited.success).toBe(1);
expect(m.classifier_source.null.unknown).toBe(1);
});
it('degraded_mode axis: true/false buckets, false default', () => {
const m = buildFactorMatrix([
{ ...ep(), _inferredOutcome: 'success', degraded_mode: false },
{ ...ep(), _inferredOutcome: 'rework', degraded_mode: true },
{ ...ep(), _inferredOutcome: 'unknown' /* missing */ },
]);
expect(m.degraded_mode.true.rework).toBe(1);
expect(m.degraded_mode.false.success).toBe(1);
expect(m.degraded_mode.false.unknown).toBe(1);
});
it('path_type axis: regulated / improvised / null', () => {
const m = buildFactorMatrix([
{ ...ep(), _inferredOutcome: 'success', path_type: 'regulated' },
{ ...ep(), _inferredOutcome: 'rework', path_type: 'improvised' },
{ ...ep(), _inferredOutcome: 'unknown', path_type: undefined },
]);
expect(m.path_type.regulated.success).toBe(1);
expect(m.path_type.improvised.rework).toBe(1);
expect(m.path_type.null.unknown).toBe(1);
});
it('retry_count axis: 0 / 1-2 / 3+ buckets from events[].kind=retry', () => {
const m = buildFactorMatrix([
{ ...ep(), _inferredOutcome: 'success', events: [] },
{ ...ep(), _inferredOutcome: 'rework', events: [{ kind: 'retry' }] },
{ ...ep(), _inferredOutcome: 'rework', events: [{ kind: 'retry' }, { kind: 'retry' }] },
{ ...ep(), _inferredOutcome: 'blocked', events: [{ kind: 'retry' }, { kind: 'retry' }, { kind: 'retry' }, { kind: 'retry' }] },
]);
expect(m.retry_count['0'].success).toBe(1);
expect(m.retry_count['1-2'].rework).toBe(2);
expect(m.retry_count['3+'].blocked).toBe(1);
});
it('error_count axis: 0 / 1 / 2+ buckets from events[].kind=error', () => {
const m = buildFactorMatrix([
{ ...ep(), _inferredOutcome: 'success', events: [] },
{ ...ep(), _inferredOutcome: 'rework', events: [{ kind: 'error' }] },
{ ...ep(), _inferredOutcome: 'blocked', events: [{ kind: 'error' }, { kind: 'error' }, { kind: 'error' }] },
]);
expect(m.error_count['0'].success).toBe(1);
expect(m.error_count['1'].rework).toBe(1);
expect(m.error_count['2+'].blocked).toBe(1);
});
it('hard_floor_invoked axis: true/false from primary_rationale.hard_floor.invoked', () => {
const m = buildFactorMatrix([
{ ...ep(), _inferredOutcome: 'success', primary_rationale: { hard_floor: { invoked: true } } },
{ ...ep(), _inferredOutcome: 'success', primary_rationale: { hard_floor: { invoked: false } } },
{ ...ep(), _inferredOutcome: 'unknown', primary_rationale: {} },
]);
expect(m.hard_floor_invoked.true.success).toBe(1);
expect(m.hard_floor_invoked.false.success).toBe(1);
expect(m.hard_floor_invoked.false.unknown).toBe(1);
});
it('iterations_bucket axis: 0 / 1-3 / 4-10 / 11+ from task_cost.iterations', () => {
const m = buildFactorMatrix([
{ ...ep(), _inferredOutcome: 'success', task_cost: { iterations: 0 } },
{ ...ep(), _inferredOutcome: 'success', task_cost: { iterations: 2 } },
{ ...ep(), _inferredOutcome: 'rework', task_cost: { iterations: 7 } },
{ ...ep(), _inferredOutcome: 'blocked', task_cost: { iterations: 51 } },
{ ...ep(), _inferredOutcome: 'unknown', task_cost: {} },
]);
expect(m.iterations_bucket['0'].success).toBe(1);
expect(m.iterations_bucket['1-3'].success).toBe(1);
expect(m.iterations_bucket['4-10'].rework).toBe(1);
expect(m.iterations_bucket['11+'].blocked).toBe(1);
// Missing iterations counts as 0 — task_cost block may be absent on early episodes.
expect(m.iterations_bucket['0'].unknown).toBe(1);
});
it('all 8 Pass 1 axes are present via analyze() on a minimal v2 episode', () => {
const result = analyze([ep()]);
for (const axis of ['prompt_signal', 'classifier_source', 'degraded_mode', 'path_type',
'retry_count', 'error_count', 'hard_floor_invoked', 'iterations_bucket']) {
expect(result.factorMatrix, `axis ${axis} missing`).toHaveProperty(axis);
}
});
});
+5 -2
View File
@@ -92,8 +92,11 @@ export function readRuntimeFlag(name, { homedir, fsImpl } = {}) {
if (!fs.existsSync(filePath)) return 'off';
const raw = fs.readFileSync(filePath, 'utf-8');
const parsed = JSON.parse(raw);
if (typeof parsed.value !== 'string') return 'off';
return parsed.value;
// Runtime flag files use `mode` (canonical, see all ~/.claude/runtime/*-mode.json);
// `value` retained as legacy alias to keep existing test fixtures working.
const val = parsed.mode ?? parsed.value;
if (typeof val !== 'string') return 'off';
return val;
} catch {
return 'off';
}
+12
View File
@@ -59,5 +59,17 @@ export function extractClassifierOutput(state) {
recommended_chain_id: cls.recommended_chain_id ?? null,
no_skill_found: cls.no_skill_found === true,
source: cls.source ?? null,
// Factor-analysis signal: classifier's stated rationale + confidence.
// Field name varies by prompt schema: new (Phase 2) uses `reason_for_choice`,
// legacy uses `reasoning`. Null on regex / prefilter paths. Truncated to
// keep episode JSONL line size bounded.
reasoning: pickReasoning(cls),
confidence: typeof cls.confidence === 'number' ? cls.confidence : null,
};
}
function pickReasoning(cls) {
const v = cls.reasoning ?? cls.reason_for_choice ?? cls.reason ?? null;
if (typeof v !== 'string') return null;
return v.slice(0, 600);
}
+60
View File
@@ -20,6 +20,7 @@ import { sanitize, sanitizeWithCount } from './observer-pii-filter.mjs';
import { parseTranscript, extractLastUserPromptText } from './observer-transcript-parser.mjs';
import { detectMethodDirected, loadKnownNodes } from './observer-routing-detector.mjs';
import { callSelfAssessmentApi, readRuntimeFlag } from './observer-self-assessment-api.mjs';
import { shouldEmbed as _shouldEmbed, encodeBase64 as _encodeBase64, embed as _embed } from './router-embedding.mjs';
const REQUIRED_FIELDS = ['task_id', 'timestamps', 'path_type', 'outcome', 'primary_rationale'];
const V2_FIELDS = [
@@ -242,6 +243,60 @@ export function buildSelfAssessment({ apiResult } = {}) {
};
}
/**
* Step 3.6 embedding async wiring (Phase 4 follow-up).
*
* Mirrors the Step 3.5 self-assessment pattern (commit c1ec61fa). When the
* embedding-mode runtime flag is 'on' and the task is non-trivial (per
* shouldEmbed), computes a 384-dim sentence embedding via Xenova and stores
* it on the episode as `prompt_embedding_base64`. Fail-quiet: on timeout /
* model load failure / runtime error field stays null and
* `environment.embedding_unavailable = true` is set.
*
* Pure-API style: injectable embedFn / shouldEmbedFn / encodeBase64Fn for tests
* (the CLI binds them to the real router-embedding.mjs implementations).
*
* @param {object} ep episode object to mutate
* @param {object} ctx Stop-hook context (uses ctx.prompt)
* @param {object} opts
* @param {string} [opts.embedMode] runtime flag value ('on' to compute)
* @param {Function} [opts.shouldEmbedFn] taskType -> bool
* @param {Function} [opts.embedFn] async(prompt) -> Float32Array | null
* @param {Function} [opts.encodeBase64Fn] Float32Array -> base64 string
* @param {number} [opts.timeoutMs] race timeout (default 2000)
* @returns {Promise<void>}
*/
export async function computeEmbeddingForEpisode(ep, ctx = {}, opts = {}) {
const {
embedMode = 'off',
shouldEmbedFn = _shouldEmbed,
embedFn = _embed,
encodeBase64Fn = _encodeBase64,
timeoutMs = 2000,
} = opts;
if (embedMode !== 'on') return;
const taskType = ep?.primary_rationale?.task_classification;
if (!shouldEmbedFn(taskType)) return;
if (!ctx || !ctx.prompt) return;
try {
const vec = await Promise.race([
embedFn(ctx.prompt),
new Promise((resolve) => setTimeout(() => resolve(null), timeoutMs)),
]);
if (vec && vec.length > 0) {
ep.prompt_embedding_base64 = encodeBase64Fn(vec);
} else {
ep.environment ??= {};
ep.environment.embedding_unavailable = true;
}
} catch (_e) {
ep.environment ??= {};
ep.environment.embedding_unavailable = true;
}
}
/**
* Build a minimal observer_error marker written instead of a silent skip
* when the Stop-hook fails internally (spec §3 / §5.2).
@@ -333,6 +388,11 @@ if (process.argv[1] && process.argv[1].replace(/\\/g, '/').endsWith('/observer-s
ep.self_assessment = buildSelfAssessment({ apiResult });
}
// Step 3.6: embedding async wiring (fail-quiet, 2s timeout).
// Trivial task types skipped via shouldEmbed. Mirrors Step 3.5 pattern.
const embMode = readRuntimeFlag('embedding-mode');
await computeEmbeddingForEpisode(ep, ctx, { embedMode: embMode });
// Always write the episode first — exit-0-safe (spec §5.1 step 1).
appendEpisode(ep);
// Then the routing-gate (spec §5.1 steps 2-4).
+64 -1
View File
@@ -2,7 +2,7 @@ import { describe, it, expect, beforeEach, afterEach } from 'vitest';
import { writeFileSync, readFileSync, existsSync, mkdtempSync, rmSync, mkdirSync, readdirSync } from 'fs';
import { join } from 'path';
import { tmpdir } from 'os';
import { appendEpisode, buildEpisodeFromContext, buildObserverError, routingGateDecision, buildExecutionTrace, buildEpisode, buildSelfAssessment } from './observer-stop-hook.mjs';
import { appendEpisode, buildEpisodeFromContext, buildObserverError, routingGateDecision, buildExecutionTrace, buildEpisode, buildSelfAssessment, computeEmbeddingForEpisode } from './observer-stop-hook.mjs';
let workdir;
@@ -303,3 +303,66 @@ describe('routingGateDecision', () => {
expect(gate.block).toBe(false);
});
});
// ---------------------------------------------------------------------------
// Step 3.6 embedding async wiring (Phase 4 follow-up)
// ---------------------------------------------------------------------------
describe('Step 3.6 embedding async wiring', () => {
// Helper to build an episode with a given task_classification.
const epWithClass = (cls = 'feature') => v2Episode({
primary_rationale: { ...defaultRat(), task_classification: cls },
});
it('embedding-mode off → embedding not computed, field null', async () => {
const ep = epWithClass('feature');
const embedFn = async () => new Float32Array([0.1, 0.2, 0.3]);
await computeEmbeddingForEpisode(ep, { prompt: 'напиши тест' }, {
embedMode: 'off',
embedFn,
});
expect(ep.prompt_embedding_base64).toBeUndefined();
expect(ep.environment?.embedding_unavailable).toBeUndefined();
});
it('taskType="conversation" (exempt) → embedding skipped, field null', async () => {
const ep = epWithClass('conversation');
let called = false;
const embedFn = async () => { called = true; return new Float32Array([0.1]); };
await computeEmbeddingForEpisode(ep, { prompt: 'спасибо' }, {
embedMode: 'on',
embedFn,
});
expect(called).toBe(false);
expect(ep.prompt_embedding_base64).toBeUndefined();
expect(ep.environment?.embedding_unavailable).toBeUndefined();
});
it('embedding success → prompt_embedding_base64 is base64 string, environment.embedding_unavailable not set', async () => {
const ep = epWithClass('feature');
// Distinctive non-zero vector so encoding produces a stable, non-empty base64.
const fakeVec = new Float32Array([0.5, -0.25, 1.0, 0.0]);
const embedFn = async () => fakeVec;
await computeEmbeddingForEpisode(ep, { prompt: 'напиши тест для биллинга' }, {
embedMode: 'on',
embedFn,
});
expect(typeof ep.prompt_embedding_base64).toBe('string');
expect(ep.prompt_embedding_base64.length).toBeGreaterThan(0);
// Base64-only chars (no whitespace, no null prefix).
expect(ep.prompt_embedding_base64).toMatch(/^[A-Za-z0-9+/]+=*$/);
expect(ep.environment?.embedding_unavailable).toBeUndefined();
});
it('embedding timeout (2s) → field null, environment.embedding_unavailable=true', async () => {
const ep = epWithClass('feature');
// embedFn never resolves — timeout (overridden short for test) must win.
const embedFn = () => new Promise(() => {});
await computeEmbeddingForEpisode(ep, { prompt: 'долгая задача' }, {
embedMode: 'on',
embedFn,
timeoutMs: 30, // short override so the test is fast
});
expect(ep.prompt_embedding_base64).toBeUndefined();
expect(ep.environment.embedding_unavailable).toBe(true);
});
});
+13 -1
View File
@@ -406,6 +406,18 @@ export function extractTaskSize(turn) {
* Defensive: skips entries where `usage` is not a plain object (handles
* malformed transcript edge cases like `"usage": 42`).
*/
// Normalize `usage.iterations` to a count.
// Claude Code transcripts may emit it as: a number (legacy / no extended-thinking),
// an array of step-objects (extended-thinking turns), or a plain object map.
// Coerce to a number; non-finite / unknown → 0. Prevents "0[object Object]…"
// string concatenation that previously poisoned task_cost.iterations.
function iterationsCount(v) {
if (typeof v === 'number' && Number.isFinite(v)) return v;
if (Array.isArray(v)) return v.length;
if (v && typeof v === 'object') return Object.keys(v).length;
return 0;
}
export function extractTokenUsage(turn) {
let input = 0, output = 0, cache_read = 0, cache_creation = 0;
let web_search = 0, web_fetch = 0, iterations = 0;
@@ -416,7 +428,7 @@ export function extractTokenUsage(turn) {
output += u.output_tokens || 0;
cache_read += u.cache_read_input_tokens || 0;
cache_creation += u.cache_creation_input_tokens || 0;
iterations += u.iterations || 0;
iterations += iterationsCount(u.iterations);
if (u.server_tool_use) {
web_search += u.server_tool_use.web_search_requests || 0;
web_fetch += u.server_tool_use.web_fetch_requests || 0;
+133 -32
View File
@@ -23,6 +23,20 @@
import { CLASSIFIER_MODEL, INHERITANCE_MAX_AGE_MIN } from './router-config.mjs';
import { classifyByRegex } from './router-classifier-regex-fallback.mjs';
import { Agent } from 'undici';
// Keep-alive dispatcher for ProxyAPI — skips TLS handshake on subsequent calls,
// reduces tail latency 100-300ms per request. Only attached to the default
// fetchImpl; tests passing their own fetchImpl are unaffected.
const KEEPALIVE_DISPATCHER = new Agent({
keepAliveTimeout: 30_000,
keepAliveMaxTimeout: 60_000,
connections: 4,
});
async function defaultFetch(url, opts) {
return fetch(url, { ...opts, dispatcher: KEEPALIVE_DISPATCHER });
}
export { classifyByRegex };
@@ -224,18 +238,37 @@ function buildChainsBlock(registry) {
/**
* Build Sonnet 4.6 classifier prompt per spec §4.2.
*
* Returns the prompt as a single string for backward compatibility
* (snapshot tests, accuracy-runner historical mode). The classifier
* hot-path uses buildClassifierPromptStructured() instead, which separates
* cacheable (system + registry) from dynamic (user prompt) content.
*
* @param {string} userPrompt raw user prompt
* @param {object} registry { nodes, chains }
* @param {object} [options]
* @param {boolean} [options.enrichment=true] inject pamyatka (4 patterns)
*/
export function buildClassifierPrompt(userPrompt, registry, { enrichment = true } = {}) {
const { system, user } = buildClassifierPromptStructured(userPrompt, registry, { enrichment });
return `<system>\n${system}\n</system>\n\n<user>\n${user}\n</user>`;
}
/**
* Build classifier prompt as { system, user } blocks for Anthropic prompt
* caching (ephemeral 5m TTL). The `system` block is identical across all
* classifier calls within a 5-minute window (instruction + памятка + node
* registry + chains) and gets billed at 10% rate after the first call.
* The `user` block is the only dynamic per-call content.
*
* Cache-eligibility: Sonnet requires 1024 tokens in the cached block.
* Active node registry (~85 nodes × ~100 tokens) easily clears this.
*/
export function buildClassifierPromptStructured(userPrompt, registry, { enrichment = true } = {}) {
const pamyatka = enrichment ? `\n\n${PAMYATKA}\n` : '\n';
const nodesBlock = buildNodesBlock(registry);
const chainsBlock = buildChainsBlock(registry);
return `<system>
Ты классификатор задач для CRM-проекта «Лидерра» (Laravel 13 + Vue 3 + Vuetify 3).
const system = `Ты классификатор задач для CRM-проекта «Лидерра» (Laravel 13 + Vue 3 + Vuetify 3).
ОБЯЗАТЕЛЬНЫЕ выходные правила:
1. Верни ровно один из: skill ИЛИ chain ИЛИ no_skill_found.
@@ -251,12 +284,10 @@ ${nodesBlock}
=== РЕЕСТР ЦЕПОЧЕК (справочно) ===
${chainsBlock}
Output ONLY JSON object, no prose, no code fences.
</system>
Output ONLY JSON object, no prose, no code fences.`;
<user>
Prompt: ${userPrompt}
</user>`;
const user = `Prompt: ${userPrompt}`;
return { system, user };
}
/**
@@ -272,13 +303,26 @@ export function parseClassifierResponse(text) {
if (!text) return null;
const trimmed = String(text).trim();
const stripped = trimmed.replace(/^```(?:json)?\s*\n?/, '').replace(/\n?```$/, '').trim();
// Pass 1: clean JSON (after fence strip).
try {
const parsed = JSON.parse(stripped);
if (typeof parsed.task_type !== 'string') return null;
return parsed;
} catch {
return null;
if (typeof parsed.task_type === 'string') return parsed;
} catch { /* fall through to extraction */ }
// Pass 2: JSON object embedded in prose ("Here is the classification: { ... }").
// Greedy match from first `{` to last `}` — works because the classifier
// produces exactly one top-level object; outer braces are reliable anchors.
const start = stripped.indexOf('{');
const end = stripped.lastIndexOf('}');
if (start !== -1 && end > start) {
try {
const parsed = JSON.parse(stripped.slice(start, end + 1));
if (typeof parsed.task_type === 'string') return parsed;
} catch { /* unrecoverable */ }
}
return null;
}
// ─── Legacy LLM prompt/parser (kept for backward compat) ────────────────────
@@ -340,32 +384,88 @@ export function parseLLMResponse(text) {
const DEFAULT_LLM_BASE_URL = 'https://api.proxyapi.ru/anthropic';
export async function callAnthropicAPI(prompt, {
/**
* POST to ProxyAPI /v1/messages.
*
* First argument is overloaded:
* - string legacy single-message body (no prompt caching).
* - { system, user } split body with ephemeral cache_control on the
* `system` block. ~70-80% cost reduction on the cacheable portion
* after the first call within a 5-minute window.
*
* Optional `onUsage(usage)` callback receives Anthropic's usage object
* (input_tokens / output_tokens / cache_creation_input_tokens /
* cache_read_input_tokens) for observability.
*/
export async function callAnthropicAPI(promptOrMessages, {
apiKey,
baseUrl = DEFAULT_LLM_BASE_URL,
model = CLASSIFIER_MODEL,
fetchImpl = fetch,
fetchImpl = defaultFetch,
maxRetries = 4,
retryBaseDelayMs = 1000,
perAttemptTimeoutMs = 30_000,
sleepImpl = (ms) => new Promise((res) => setTimeout(res, ms)),
onUsage,
}) {
const url = `${String(baseUrl).replace(/\/+$/, '')}/v1/messages`;
const r = await fetchImpl(url, {
method: 'POST',
headers: {
'authorization': `Bearer ${apiKey}`,
'x-api-key': apiKey,
'anthropic-version': '2023-06-01',
'content-type': 'application/json',
},
body: JSON.stringify({
let body;
if (typeof promptOrMessages === 'string') {
body = JSON.stringify({
model,
max_tokens: 1500,
messages: [{ role: 'user', content: prompt }],
}),
});
if (!r.ok) {
throw new Error(`Router LLM ${r.status}: ${await r.text()}`);
messages: [{ role: 'user', content: promptOrMessages }],
});
} else {
const { system, user } = promptOrMessages;
body = JSON.stringify({
model,
max_tokens: 1500,
system: [{ type: 'text', text: system, cache_control: { type: 'ephemeral' } }],
messages: [{ role: 'user', content: user }],
});
}
const data = await r.json();
return data.content?.[0]?.text || '';
const headers = {
'authorization': `Bearer ${apiKey}`,
'x-api-key': apiKey,
'anthropic-version': '2023-06-01',
'content-type': 'application/json',
};
let lastError;
for (let attempt = 0; attempt <= maxRetries; attempt++) {
const ctrl = new AbortController();
const timer = setTimeout(() => ctrl.abort(new Error(`per-attempt timeout ${perAttemptTimeoutMs}ms`)), perAttemptTimeoutMs);
try {
const r = await fetchImpl(url, { method: 'POST', headers, body, signal: ctrl.signal });
if (r.ok) {
const data = await r.json();
if (onUsage && data.usage) {
try { onUsage(data.usage); } catch { /* swallow callback errors */ }
}
return data.content?.[0]?.text || '';
}
// Retry on 5xx and 429; fail fast on 4xx (auth/quota/bad request — retry won't help).
if (r.status >= 500 || r.status === 429) {
lastError = new Error(`Router LLM ${r.status}: ${await r.text()}`);
} else {
const fatal = new Error(`Router LLM ${r.status}: ${await r.text()}`);
fatal.fatal = true;
throw fatal;
}
} catch (err) {
// Re-throw fatal errors (4xx) instead of retrying them.
if (err && err.fatal) { clearTimeout(timer); throw err; }
// Network-level failure (fetch failed / ECONNRESET / TLS / per-attempt timeout). Retry-eligible.
lastError = err;
} finally {
clearTimeout(timer);
}
if (attempt < maxRetries) {
await sleepImpl(retryBaseDelayMs * 2 ** attempt);
}
}
throw lastError;
}
function hashPrompt(s) {
@@ -406,17 +506,18 @@ export async function classify(prompt, registry, options = {}) {
return { ...cache.get(key), source: 'cache' };
}
// Layer 2 — Sonnet 4.6.
// Layer 2 — Sonnet 4.6 with prompt caching (ephemeral 5m TTL on system block).
const llmCall = options.llmCall || (async () => {
const apiKey = process.env.ROUTER_LLM_KEY;
if (!apiKey) return null;
const classifierPrompt = buildClassifierPrompt(prompt, registry, {
const structured = buildClassifierPromptStructured(prompt, registry, {
enrichment: options.enrichment ?? true,
});
const text = await callAnthropicAPI(classifierPrompt, {
const text = await callAnthropicAPI(structured, {
apiKey,
baseUrl: process.env.ROUTER_LLM_BASE_URL || undefined,
model: options.model || CLASSIFIER_MODEL,
onUsage: options.onUsage,
});
return parseClassifierResponse(text);
});