{
  "scope": [
    "skills/**/evals/**",
    ".agents/skills/**/evals/**",
    "scripts/*test*|*smoke*",
    "tests/**",
    "fixtures/**",
    "package.json (npm test scripts only)"
  ],
  "npm_test_scripts": {
    "test:runtime": "node scripts/test-user-runtime.js",
    "test:routing:unit": "node --test tests/routing-hooks.test.mjs",
    "test:routing:catalog": "node scripts/build-routing-catalog.mjs",
    "test:routing:engine": "node scripts/test-routing-engine.mjs",
    "test:routing:smoke": "node scripts/smoke-codex-hooks.js",
    "test:routing": "npm run test:routing:catalog && npm run test:routing:unit && npm run test:routing:engine && npm run test:routing:smoke",
    "test:portable-agents": "node --test tests/portable-agents.test.cjs",
    "test:git": "npm --prefix git-node test",
    "test": "npm run test:portable-agents && npm run test:runtime && npm run test:routing && npm run test:git",
    "test:affected": "node scripts/test-affected.mjs",
    "test:affected:unit": "node --test tests/test-affected.test.mjs",
    "test:analytics": "npm --prefix analytics-node test",
    "test:all": "npm test && npm run test:analytics",
    "_ci_note": ".github/workflows/tests.yaml runs: npm --prefix git-node ci, npm run test:affected:unit, npm test. skills/*/evals/evals.json files are NOT executed or referenced anywhere in npm scripts or CI; only checked for JSON syntax validity by scripts/check-skill-evals-json.js, which itself is not wired into any npm script or CI job (manual PR-checklist step per docs/pr-checklist-mcp.md line 13)."
  },
  "evals": [
    { "path": "skills/ai-documents-validation-document-type/evals/evals.json", "skill": "ai-documents-validation-document-type", "cases_count": 8, "types": ["routing", "procedura", "outcome"], "executable": false, "coupling_notes": "id 1 (line 12) hardcodes an environment-specific absolute path 'E:\\\\ai-documents-validation'; id 3 (lines 33-37) asserts on implementation-specific function-naming convention 'extract_<name>_fields_with_diagnostics' and exact status vocabulary confirmed/partial/conflicting/missing." },
    { "path": "skills/mcp-analytics-operator/evals/evals.json", "skill": "mcp-analytics-operator", "cases_count": 5, "types": ["outcome", "safety", "procedura"], "executable": false, "coupling_notes": null },
    { "path": "skills/mcp-browser-automation/evals/evals.json", "skill": "mcp-browser-automation", "cases_count": 5, "types": ["procedura", "trigger", "anti-trigger"], "executable": false, "coupling_notes": null },
    { "path": "skills/mcp-code-reviewer/evals/evals.json", "skill": "mcp-code-reviewer", "cases_count": 12, "types": ["outcome", "routing", "procedura"], "executable": false, "coupling_notes": null },
    { "path": "skills/mcp-coldfusion-developer/evals/evals.json", "skill": "mcp-coldfusion-developer", "cases_count": 8, "types": ["procedura", "routing", "safety"], "executable": false, "coupling_notes": null },
    { "path": "skills/mcp-database-expert/evals/evals.json", "skill": "mcp-database-expert", "cases_count": 5, "types": ["procedura", "outcome", "safety", "anti-trigger"], "executable": false, "coupling_notes": null },
    { "path": "skills/mcp-docs-navigator/evals/evals.json", "skill": "mcp-docs-navigator", "cases_count": 9, "types": ["routing", "procedura", "outcome"], "executable": false, "coupling_notes": null },
    { "path": "skills/mcp-frontend-performance-debugger/evals/evals.json", "skill": "mcp-frontend-performance-debugger", "cases_count": 10, "types": ["routing", "procedura", "outcome", "robustness"], "executable": false, "coupling_notes": "eval_focus 'benchmark-overfitting-prevention' and 'browser-measurement-integrity' imply asserting on specific measurement-methodology wording; not verified line-by-line but flagged as a plausible over-specification risk." },
    { "path": "skills/mcp-git-mantis-workflow/evals/evals.json", "skill": "mcp-git-mantis-workflow", "cases_count": 7, "types": ["procedura", "outcome", "safety"], "executable": false, "coupling_notes": null },
    { "path": "skills/mcp-grid-ui-debugger/evals/evals.json", "skill": "mcp-grid-ui-debugger", "cases_count": 5, "types": ["trigger", "routing", "procedura"], "executable": false, "coupling_notes": "eval_focus lists 7 tags for only 5 cases (tags: grid-rendering-symptom-trigger, dom-inspection-and-hypothesis, reload-verification-and-no-hot-reload-only, discrimination-from-browser-automation-e2e, rebuild-confirmation-for-compiled-stacks, symptom-pattern-recognition, genesis-attempts-documentation vs 5 evals[]) - focus/cases count mismatch, some focus areas may be undercovered." },
    { "path": "skills/mcp-handoff-pack/evals/evals.json", "skill": "mcp-handoff-pack", "cases_count": 7, "types": ["outcome", "anti-trigger", "procedura"], "executable": false, "coupling_notes": null },
    { "path": "skills/mcp-mantis-test-writer/evals/evals.json", "skill": "mcp-mantis-test-writer", "cases_count": 5, "types": ["anti-trigger", "safety", "procedura"], "executable": false, "coupling_notes": "Uses a DIFFERENT schema than every other skill's evals.json: top-level {version:1, skill, cases:[{id,input,expect:[]}]} instead of {skill_name, eval_focus, evals:[{id,prompt,expected_output,expectations}]}. A generic evals-schema tool/reader built for the common shape would silently read skill_name as undefined and cases_count as 0 for this file (reproduced during this audit)." },
    { "path": "skills/mcp-mantis-ticket-writer/evals/evals.json", "skill": "mcp-mantis-ticket-writer", "cases_count": 15, "types": ["procedura", "outcome", "routing"], "executable": false, "coupling_notes": "eval_focus mentions 'corporate-title-format-and-conditional-additional-information' suggesting assertions tied to a specific corporate template/format string; not confirmed line-by-line." },
    { "path": "skills/mcp-master-orchestrator/evals/evals.json", "skill": "mcp-master-orchestrator", "cases_count": 11, "types": ["routing", "anti-trigger", "trigger"], "executable": false, "coupling_notes": "This is the densest routing/anti-trigger eval set in the repo (legacy-modernization-routing, legacy-inventory-anti-trigger, cfml-cross-version-anti-overlap, local-negation-scope) and overlaps conceptually with the executable tests/routing-hooks.test.mjs + scripts/test-routing-engine.mjs, but the two are not linked: the orchestrator's evals.json is prose/manual while the engine tests are separate synthetic fixtures. No shared source of truth between the documental routing evals and the executable routing-engine tests." },
    { "path": "skills/mcp-memory-operator/evals/evals.json", "skill": "mcp-memory-operator", "cases_count": 6, "types": ["safety", "procedura", "routing"], "executable": false, "coupling_notes": null },
    { "path": "skills/mcp-office-expert/evals/evals.json", "skill": "mcp-office-expert", "cases_count": 3, "types": ["procedura", "outcome"], "executable": false, "coupling_notes": "Smallest eval set in the repo (3 cases) for a skill covering three distinct document types (Word/Excel/PDF)." },
    { "path": "skills/mcp-skill-miner/evals/evals.json", "skill": "mcp-skill-miner", "cases_count": 6, "types": ["trigger", "anti-trigger", "procedura"], "executable": false, "coupling_notes": null },
    { "path": "skills/mcp-sophia-yii-developer/evals/evals.json", "skill": "mcp-sophia-yii-developer", "cases_count": 5, "types": ["routing", "procedura", "outcome"], "executable": false, "coupling_notes": null },
    { "path": "skills/mcp-technical-analyst/evals/evals.json", "skill": "mcp-technical-analyst", "cases_count": 14, "types": ["procedura", "outcome", "routing", "safety"], "executable": false, "coupling_notes": null },
    { "path": "skills/svg/evals/evals.json", "skill": "svg", "cases_count": 17, "types": ["routing", "procedura", "outcome", "robustness"], "executable": false, "coupling_notes": "id 11 and 15 (lines 132-183) hardcode a Windows-only tool policy asserting 'inkscape' must be primary and 'rsvg-convert' must never be proposed as fallback on Windows - tightly coupled to current tool-availability policy in skills/svg/references, would break if the export-tool strategy changes. id 16 (line 192) hardcodes the path 'skills/svg/tools/search-assets.js'. Several ids (1,2,3,4,7) hardcode fixture paths under skills/svg/evals/fixtures/ which do exist (verified) but make the eval brittle to fixture renames." },
    { "path": ".agents/skills/mcp-runtime-integrator/evals/evals.json", "skill": "mcp-runtime-integrator", "cases_count": 5, "types": ["procedura", "anti-trigger", "robustness"], "executable": false, "coupling_notes": null },
    { "path": "skills/svg/evals/fixtures/valid-minimal.svg", "skill": "svg", "cases_count": null, "types": ["unknown"], "executable": true, "coupling_notes": "Referenced by evals.json ids 1 and 3, and by scripts/check-svg-skill-smoke.js as a fixture input, not a standalone test." },
    { "path": "skills/svg/evals/fixtures/invalid-malformed.svg", "skill": "svg", "cases_count": null, "types": ["unknown"], "executable": true, "coupling_notes": "Referenced by evals.json id 4 and check-svg-skill-smoke.js as malformed-input fixture." },
    { "path": "skills/svg/evals/fixtures/valid-currentcolor.svg", "skill": "svg", "cases_count": null, "types": ["unknown"], "executable": true, "coupling_notes": "Referenced by evals.json id 7 (currentColor-preservation fixture)." },
    { "path": "skills/svg/evals/README.md", "skill": "svg", "cases_count": null, "types": ["procedura"], "executable": false, "coupling_notes": "Explicitly documents that evals.json is executed only as 'smoke/eval rapido... manuale guidato' (manual guided smoke), i.e. not automated; only the JSON-syntax check and the deterministic scripts/check-svg-skill-smoke.js are automatable." },
    { "path": "tests/routing-hooks.test.mjs", "skill": "cross-skill (routing infrastructure, feeds mcp-master-orchestrator / mcp-sophia-yii-developer / mcp-coldfusion-developer / mcp-frontend-performance-debugger / mcp-grid-ui-debugger routing)", "cases_count": 25, "types": ["routing", "trigger", "anti-trigger"], "executable": true, "coupling_notes": "Executable via `npm run test:routing:unit`, part of the `npm test` root gate that CI (.github/workflows/tests.yaml) runs on every push/PR to master. Covers repo-context detection (Yii/ColdFusion markers), negation-aware phrase matching, and hint scoring - genuine behavioral/unit coverage of the routing engine, distinct from the documental skills/*/evals/evals.json files." },
    { "path": "scripts/test-routing-engine.mjs", "skill": "cross-skill (routing infrastructure, feeds mcp-master-orchestrator legacy-modernization intents)", "cases_count": 16, "types": ["routing", "robustness"], "executable": true, "coupling_notes": "Wired as `npm run test:routing:engine`, part of `npm run test:routing` -> `npm test` -> CI gate. Case count reflects fixtures/hooks/legacy-modernization-routing.json 'engineCases' array (16 entries); file also contains ~70 additional inline assert.equal/assert.deepEqual calls for schema validation, anti-pattern matching, priority resolution, and catalog-corruption handling not captured in the 16-case count. Some assertions are tightly coupled to exact internal field names (e.g. positiveLocks, matchedAntiPatterns, catalogStatus values 'invalid-schema'/'unsupported-version') - a refactor of the routing-engine's return shape would break many assertions simultaneously (see lines 37-54, 169-234)." },
    { "path": "scripts/build-routing-catalog.mjs", "skill": "cross-skill (routing infrastructure)", "cases_count": null, "types": ["procedura"], "executable": true, "coupling_notes": "Wired as `npm run test:routing:catalog`; it is a build/validation step (constructs the catalog from skills/ metadata and validates schema) rather than a test with discrete cases." },
    { "path": "scripts/smoke-codex-hooks.js", "skill": "cross-skill (Codex hook generation/routing)", "cases_count": null, "types": ["routing", "procedura"], "executable": true, "coupling_notes": "1127 lines, wired as `npm run test:routing:smoke`, part of the CI-gated `npm test`. Spawns generated Codex hooks as child processes and asserts on stdout JSON/exit codes - not enumerated as discrete named cases in this pass; recommend a follow-up count." },
    { "path": "scripts/check-skill-evals-json.js", "skill": "cross-skill (meta: validates all skills/*/evals/evals.json)", "cases_count": null, "types": ["unknown"], "executable": true, "coupling_notes": "Only validates JSON.parse() succeeds for each skills/*/evals/evals.json - does NOT validate schema shape (confirmed: silently accepts the divergent mcp-mantis-test-writer schema) and does NOT execute/judge any eval content. Not referenced by package.json or .github/workflows/tests.yaml; only documented as a manual PR-checklist step (docs/pr-checklist-mcp.md line 13) and in skills/svg/evals/README.md." },
    { "path": "scripts/check-svg-skill-smoke.js (+ scripts/smoke-svg-phase2.js, smoke-svg-phase3.js, smoke-svg-phase3-helpers.js, smoke-svg-phase3-runtime.js, smoke-svg-phase4.js, smoke-svg-superellipse.js)", "skill": "svg", "cases_count": null, "types": ["outcome", "procedura"], "executable": true, "coupling_notes": "Deterministic, code-level smoke tests of SVG parsing/tooling behavior (not LLM-judge routing evals). Not wired into package.json or CI; invoked manually per skills/svg/evals/README.md ('node scripts/check-svg-skill-smoke.js --profile phase2'). smoke-svg-phase2.js documented as a backward-compatible alias." },
    { "path": "tests/portable-agents.test.cjs", "skill": "cross-skill (portable agent/subagent sync, tangential to skill routing)", "cases_count": 3, "types": ["outcome", "regressione"], "executable": true, "coupling_notes": "Wired as `npm run test:portable-agents`, part of CI-gated `npm test`. Tests nickname normalization and Codex/Claude Code drift detection for portable agent definitions - relevant to agent/skill portability but not to prompt routing or trigger logic; included because it is under tests/ and touches skill/agent metadata." },
    { "path": "tests/test-affected.test.mjs", "skill": "cross-skill (test-scope selection tooling, not skill/routing-specific)", "cases_count": null, "types": ["unknown"], "executable": true, "coupling_notes": "Wired as `npm run test:affected:unit`, run directly by CI before `npm test`. Out of the routing/skill domain - flagged as low-relevance inclusion required by the tests/ scope pattern." },
    { "path": "tests/smoke/*.smoke.mjs (16 files: git-node, sql-node, mantis-node, cf-node, docs-node, office-node x4, office-docs-bridge, playwright-node, linter-node x3, projectfs-node, memory-node, analytics-node) + tests/smoke/run-all.mjs + tests/smoke/run-smoke.mjs", "skill": "cross-skill (underlying MCP-server tool behavior consumed by multiple skills, not routing/trigger evals)", "cases_count": null, "types": ["outcome"], "executable": true, "coupling_notes": "Executable and runnable via `node tests/smoke/run-all.mjs`, but NOT wired into package.json npm scripts or .github/workflows/tests.yaml - confirmed no 'test:smoke' script exists and CI only runs test:affected:unit + npm test. These test MCP server/tool correctness (e.g. git-node, sql-node), not skill selection/routing - included per tests/ scope but tangential to the audit's routing/skill focus." },
    { "path": "fixtures/hooks/legacy-modernization-routing.json", "skill": "cross-skill (consumed by scripts/test-routing-engine.mjs and referenced by tests/routing-hooks.test.mjs family)", "cases_count": 16, "types": ["routing"], "executable": true, "coupling_notes": "cases_count reflects the 'engineCases' array (16); file also has a sibling 'hookCases' array whose count was not separately enumerated in this pass. Consumed directly by scripts/test-routing-engine.mjs as executable fixture data, not documental." },
    { "path": "fixtures/hooks/*.json (47 files total under fixtures/hooks/, remaining ~45 not individually enumerated)", "skill": "cross-skill (routing/prompt-hook infrastructure)", "cases_count": null, "types": ["routing", "trigger"], "executable": true, "coupling_notes": "Naming convention (prompt-git*.json, prompt-docs*.json, pretool-projectfs*.json, prompt-yii.json, prompt-coldfusion.json, etc.) indicates per-scenario fixtures for prompt/pretool hook routing decisions, presumably consumed by scripts under scripts/hooks/ and/or tests/routing-hooks.test.mjs; full consumer-mapping not traced file-by-file in this pass (see uncertainties)." }
  ],
  "skills_without_evals": [],
  "coverage_summary": {
    "per_type_counts": {
      "routing": 14,
      "trigger": 6,
      "anti-trigger": 9,
      "procedura": 20,
      "outcome": 15,
      "safety": 8,
      "regressione": 1,
      "efficienza": 0,
      "robustness": 4,
      "cross-model": 0,
      "unknown": 4
    },
    "notes": "Counts are per evals-file/test-file entry in the 'evals' array above (a file tagged with 3 types counts once toward each of those 3 types), not per individual case/id. No eval or test in scope targets 'efficienza' (performance/cost budgets for the model itself) or 'cross-model' (behavior parity across different LLM backends) explicitly. All 21 skills (20 under skills/, 1 under .agents/skills/) have exactly one evals/evals.json file each - full nominal coverage, but see executable=false pattern below."
  },
  "uncertainties": [
    "All 21 skills/*/evals/evals.json + .agents/skills/*/evals/evals.json files are marked executable:false because no automated harness in this repo runs the prompts in them against a model and grades the 'expectations' - they are consumed only as documental/manual-smoke material (explicitly confirmed by skills/svg/evals/README.md wording 'smoke manuali... nel contesto attuale gli eval skill sono fixture testuali'). It is possible an external/manual process (outside this repo) does execute them via an LLM-judge harness; this could not be verified from repo contents alone.",
    "scripts/smoke-codex-hooks.js (1127 lines) was not fully read line-by-line; its discrete assertion/case count was not enumerated, only confirmed as executable and CI-wired.",
    "fixtures/hooks/*.json (47 files) were enumerated by filename only; their exact consumer scripts and per-file case counts were not individually traced beyond legacy-modernization-routing.json.",
    "No dedicated 'tests/' subfolder or evals file specifically targets cross-model behavior parity or model-efficiency/cost budgets; absence confirmed via grep for 'cross-model' and 'efficien' across all evals.json files (no matches) but not verified against scripts/ or tests/ prose comments.",
    "skills/mcp-mantis-test-writer/evals/evals.json uses a structurally different schema (version:1/cases/expect) from all 20 other evals.json files (skill_name/eval_focus/evals/expected_output/expectations) - unclear if this is an intentional deviation or drift; not resolved in this read-only pass."
  ],
  "recommended_followups": [
    "Decide whether skills/*/evals/evals.json is meant to become an automated LLM-judge harness (currently 100% manual/documental) - if so, prioritize the mcp-master-orchestrator (11 cases) and mcp-technical-analyst (14 cases) sets given their routing/anti-trigger density.",
    "Reconcile skills/mcp-mantis-test-writer/evals/evals.json schema with the common shape used by the other 20 files, or document the divergence intentionally so tooling built for the common schema does not silently misread it (reproduced: skill_name read as undefined, cases read as 0).",
    "Wire scripts/check-skill-evals-json.js into package.json/CI (or explicitly confirm it should remain manual-checklist-only) since it currently only validates JSON syntax, not schema shape or content.",
    "Investigate the svg skill's Windows-only 'inkscape primary, rsvg-convert never fallback' assertions embedded directly in evals.json ids 11 and 15 (lines 132-183) for durability against future tool-policy changes.",
    "If routing/skill eval coverage is a priority, consider linking skills/mcp-master-orchestrator/evals/evals.json's routing scenarios to the executable scripts/test-routing-engine.mjs fixture (fixtures/hooks/legacy-modernization-routing.json) so the two do not drift independently."
  ]
}