bdfinst avatar

security-assessment

Deep security assessment + adversarial ML red-team: SARIF-first tool orchestration, narrowly-scoped LLM agents, FP-reduction with fallback banner, compliance mapping, service-comm diagramming, and a self-owned-target red-team harness. Companion plugin to dev-team. Previously published as agentic-security-assessment.

version
3.7.6
GitHub stars
304

Install

/plugin install security-assessment@bfinster

Run it inside Claude Code after adding the marketplace with /plugin marketplace add bdfinst/agentic-dev-team.

Contents

Discovered from source

Skills 3

Compliance Mapping (pattern-first)

Purpose

Map unified findings to regulatory control citations so executive-audience reports can name the specific regulations at risk. Designed to be deterministic first — a pattern table handles the bulk; LLM judgment is invoked ONLY for cases the table flags as ambiguous.

This is explicitly informational, not audit-grade. No report produced by this pipeline substitutes for a certified auditor's opinion. The disclaimer is mandatory and exact-wording.

Inputs

  • Disposition register (post-fp-reduction unified findings with verdicts)
  • knowledge/compliance-patterns.yaml (this plugin) — the pattern table
  • Optional: target organization's pre-declared control scope (which regulations apply)

Output

A compliance-annotations.json file per the following shape:

{
  "schema_version": "1.0",
  "generated_at": "2026-04-21T10:00:00Z",
  "disclaimer": "<verbatim text from knowledge/disclaimers.md § Compliance mapping disclaimer>",
  "annotations": [
    {
      "finding_rule_id": "semgrep.python.hardcoded-password",
      "finding_file": "config/prod.py",
      "finding_line": 42,
      "regulations": [
        {"regulation": "PCI-DSS", "control_id": "3.2.1", "citation": "PCI-DSS v4.0 §3.2.1"},
        {"regulation": "GDPR", "control_id": "Art. 32", "citation": "GDPR Article 32 — Security of Processing"}
      ],
      "annotator": "pattern-table"   // or "llm" if llm_review_trigger fired
    }
  ]
}

Pattern table schema (knowledge/compliance-patterns.yaml)

Each row maps a finding pattern to one or more regulatory citations.

# plugins/security-assessment/knowledge/compliance-patterns.yaml
patterns:
  - id: pan-at-log
    pattern_regex: 'log\.(debug|info).*(pan|card_number|primary_account)'
    field_type: "pii.pan"
    applies_to_rules:
      - "semgrep.*.pii-log"
      - "business-logic.fraud.tokenization-skip-under-flag"
    regulations:
      - regulation: "PCI-DSS"
        control_id: "3.4"
        citation: "PCI-DSS v4.0 §3.4 — PAN must be rendered unreadable in storage and in logs"
      - regulation: "PCI-DSS"
        control_id: "10.2"
        citation: "PCI-DSS v4.0 §10.2 — Audit logs must not contain sensitive authentication data"
    llm_review_trigger: false

  - id: unencrypted-db-transit
    pattern_regex: "(mongodb|postgres|mysql)://[^@]+@[^/]+/[^?]+($|(?!.*ssl|.*tls))"
    field_type: "db.transit"
    applies_to_rules:
      - "semgrep.*.unencrypted-database-connection"
      - "trivy.iac.*"
    regulations:
      - regulation: "PCI-DSS"
        control_id: "4.1"
        citation: "PCI-DSS v4.0 §4.1 — Strong cryptography and security protocols on open public networks"
      - regulation: "GDPR"
        control_id: "Art. 32"
        citation: "GDPR Article 32 — Security of Processing"
    llm_review_trigger: false

  - id: auth-bypass-admin
    pattern_regex: ".*admin.*"
    field_type: "auth.missing"
    applies_to_rules:
      - "semgrep.*.missing-csrf"
      - "semgrep.*.unauthenticated-endpoint"
      - "business-logic.fraud.*"
    regulations:
      - regulation: "PCI-DSS"
        control_id: "8.3"
        citation: "PCI-DSS v4.0 §8.3 — Multi-factor authentication for admin access"
      - regulation: "SOC2"
        control_id: "CC6.1"
        citation: "SOC2 Trust Services Criteria CC6.1 — Logical access controls"
    llm_review_trigger: true   # rules in this class often need case-specific judgment

  - id: insecure-random
    pattern_regex: ".*"
    field_type: "crypto.random"
    applies_to_rules:
      - "semgrep.*.insecure-random"
    regulations:
      - regulation: "PCI-DSS"
        control_id: "3.6.1"
        citation: "PCI-DSS v4.0 §3.6.1 — Cryptographic key generation practices"
    llm_review_trigger: false

Schema fields per row:

FieldRequiredTypePurpose
idyesstringStable identifier
pattern_regexyesstringRegex applied to the finding's message / file / context
field_typeyesstringCategorical tag (pii.pan, auth.missing, crypto.random, etc.) for audit traceability
applies_to_rulesyeslist[string]Rule-id glob patterns this row matches
regulationsyeslist[object]One or more citation objects
llm_review_triggernobool (default false)If true, LLM edge annotator is invoked for findings matched by this row

Procedure

1. Load and validate the pattern table

Parse YAML. Every row MUST have the required fields. An invalid row fails the run with a named error pointing to the row id.

2. For each finding in the disposition register

Apply rows in file-declaration order:

  1. Check applies_to_rules globs against finding.rule_id. No match → skip row.
  2. Check pattern_regex against finding.message + finding.file (concatenated). No match → skip row.
  3. Match: record annotation with the row's regulations. Set annotator: "pattern-table".
  4. If llm_review_trigger: true, invoke the compliance-edge-annotator agent for this finding only. The agent can refine citations, add or remove regulations, or add a judgment note. Set annotator: "llm" if the agent modified anything.

A finding with zero matching rows gets no annotation and does NOT appear in the output.

3. LLM call counting (for eval)

The skill exposes an optional LLMCallCounter interface for eval assertions:

class LLMCallCounter:
    def invoke(self, prompt: str, context: dict) -> str: ...
    def count(self) -> int: ...
    def reset(self) -> None: ...

Production dispatches go through a real LLM. Eval runs inject a mock counter and assert the count matches the expected number of llm_review_trigger: true matches in the fixture.

4. Write output

Write memory/compliance-<slug>.json with the disclaimer at the root. The exec-report-generator includes the disclaimer verbatim in the report header.

Disclaimer

Use the exact wording in knowledge/disclaimers.md § "Compliance mapping disclaimer". Required on every report.

Invariants

  • Pattern table is the source of truth. LLM never invents citations; it only refines or annotates table-matched citations.
  • LLM call count is bounded: one call per finding with llm_review_trigger: true. No loops, no recursion.
  • Schema-invalid pattern rows fail the run. No silent skipping.
  • Every annotation carries annotator: "pattern-table" | "llm" for audit.
  • The disclaimer appears verbatim in every compliance output. Downstream must not strip it.

Graph-assisted discovery

If the target repo has .codegraph/ (CodeGraph MCP server — codegraph_explore/get_symbol for fast callers/callees/impact lookups) and/or a Repowise MCP server (get_context/search_codebase for semantic search, get_risk for modification-risk context), prefer them over raw Grep/Read — useful when the compliance-edge-annotator agent needs to inspect the finding's surrounding code to refine a citation. Never assume either tool is present — fall back to Read/Grep/Glob when neither is present; the tools are simply unavailable (no error) on repos without an index.

Related

  • agents/compliance-edge-annotator.md — the sonnet-tier agent invoked for llm_review_trigger: true matches
  • knowledge/compliance-patterns.yaml — the pattern table itself
  • plugins/dev-team/knowledge/security-primitives-contract.md — unified finding envelope

False-Positive Reduction (hybrid joern + LLM)

Purpose

Transform a stream of unified findings into a disposition register that the exec-report-generator can trust. Every finding gets a verdict (true_positive | likely_true_positive | uncertain | likely_false_positive | false_positive), a reachability trace, an exploitability score, and a reachability_source tag (joern-cpg or llm-fallback).

The skill's job is to remove noise without suppressing real issues. False positives waste analyst attention; missed true positives get someone fired.

Six-stage rubric (applied in order; each stage can downgrade severity or change verdict)

Lifted from the opus_repo_scan_test reference's § analyze-11 framework with extensions for the disposition-register output format. Stage 0 is new: a self-adversarial pre-pass that sharpens Stage 1 and strengthens the audit trail.

Stage 0 — Devil's advocate

Question: What is the strongest argument that this finding is NOT a vulnerability?

The agent generates a counter-argument before applying the rubric. This is not a skip gate — all five subsequent stages still run. The purpose is twofold:

  1. Sharpen Stage 1: a strong counter-argument gives Stage 1 a concrete hypothesis to test (is the path actually dead / test-only?) rather than an open-ended search.
  2. Strengthen the audit trail: a true_positive that explicitly refuted a counter-argument is more trustworthy than one that never examined the counter-case. A well-reasoned false_positive is more trustworthy than a silent discard.

Counter-argument prompts:

  • Framework/runtime protection: does the tech stack have a built-in prevention for this class (ORM parameterization, template auto-escaping, TLS termination at the LB)?
  • Trusted caller: is this code only reachable from internal, trusted, or admin-only paths?
  • Non-production context: is the file a migration, test fixture, seed script, or utility that RECON's entry_points don't include?
  • Rule pattern noise: does this rule commonly fire on intentional non-exploitable configurations?

Disposition rules:

  • Strong counter-argument → da_strong: true; Stage 1 tests the hypothesis
  • Weak / no counter-argument → da_strong: false; Stage 1 performs open-ended reachability search
  • da_strong: true + Stage 1 confirms (unreachable) → false_positive; both arguments cited in rationale
  • da_strong: true + Stage 1 disproves (reachable) → rejected counter-argument cited in true_positive rationale

Stage 1 — Reachability

Question: Is this code executed in production at all?

Disposition rules:

  • Dead code (no inbound call graph from any entry point) → verdict: false_positive, severity → info presentational.
  • Test-only paths (only reached from test code) → verdict: likely_false_positive, severity → one level down (CRITICAL → HIGH, HIGH → MEDIUM).
  • Feature-flagged-off in all production configs → one level down, verdict stays true_positive.
  • Reached from production entry point → no change; record the entry point in reachability.rationale.

Joern-present mode: reachability is computed from the CPG by tracing back from the finding location to HTTP/CLI/lambda/cron entry points.

Joern-absent mode: the agent reasons from RECON's entry_points and security_surface fields, plus grep over the call sites. Tag each entry with reachability_source: llm-fallback.

Stage 2 — Environment context

Question: Could deployed configuration override the committed value, making the finding inert?

Disposition rules:

  • Confirmed override at deploy time (e.g. env var in values.yaml or Helm chart overrides a committed default) → one level down, verdict: likely_true_positive (the committed value is still a weak default).
  • No override found → full severity, verdict unchanged.

The agent consults docker-compose*.yml, values.yaml, helmfile.yaml, k8s/*.yaml, and any CI-scoped env vars discoverable in .github/workflows/* or GitLab equivalents.

Stage 3 — Compensating controls

Question: Is there a control in the repo that mitigates this finding's impact?

Disposition rules:

  • Confirmed in-repo control (WAF rule, rate limiter, input validation layer upstream of the finding, idempotency key check, etc.) → one level down, verdict likely_true_positive with the control's file:line in the rationale.
  • Assumed-only ("we have a WAF in prod" — not verifiable from the repo) → no change.
  • Absent → no change.

Stage 4 — Deduplication

Question: Is this the same root cause as another finding already in the register?

Disposition rules:

  • Same rule_id + same value (e.g. same secret SHA-256) across multiple files → collapse to ONE finding with a locations array; emit one disposition entry referencing the primary finding.
  • Same rule_id + different values (e.g. 14 different hardcoded passwords) → separate findings, NOT deduplicated.
  • Different rule_ids that describe the same root cause (e.g. semgrep.python.hardcoded-password + gitleaks.generic.aws-access-key firing on the same line) → dedupe keeping the higher-priority source per the static-analysis skill's priority order.

Stage 5 — Severity calibration

Question: Is the severity consistent across similar findings?

Disposition rules:

  • Ensure two findings with identical exploitability profiles receive identical presentational severity across the run.
  • If a finding falls between two severity levels, prefer the higher — better to over-flag than miss. Use exploitability score (0–10) to break ties deterministically per the severity-mapping table in the primitives contract.

Graph-assisted discovery

If the target repo has .codegraph/ (CodeGraph MCP server — codegraph_explore/get_symbol for fast callers/callees/impact lookups) and/or a Repowise MCP server (get_context/search_codebase for semantic search, get_risk for modification-risk context), prefer them over raw Grep/Read when tracing call graphs. This benefits Stage 1 reachability analysis most directly: following inbound calls from a finding location back to production entry points is exactly the callers/impact query these tools are built for, and is especially useful in Joern-absent (LLM-fallback) mode where reachability is otherwise reasoned from grep over call sites. Never assume either tool is present — fall back to Read/Grep/Glob when neither is present; the tools are simply unavailable (no error) on repos without an index.

Exploitability scoring (0–10)

Per-finding score determines presentational severity bucket (see primitives contract § Severity mapping). Factors:

FactorWeightExample
Network reachability+3Finding is in an HTTP handler on a public route
Authentication bypass+3Finding bypasses an auth check (not merely missing one)
Credential exposure+2Finding leaks a credential an attacker could use elsewhere
Input-controlled+2An attacker can influence the vulnerable value via request parameters
Persistent+1Finding creates persistent state (stored XSS, stored credentials)
Privileged context+1Finding runs in an elevated context (root, admin route)
Cascading+1A successful exploit unlocks further access (lateral movement)

Rationale field is mandatory (min 20 chars per schema). Summarize which factors applied and why.

Joern integration (when present)

If joern is on PATH, invoke via ${CLAUDE_PLUGIN_ROOT}/skills/false-positive-reduction/tools/reachability.sh (build commands + CPG cache details are in the script). The wrapper detects the target's primary language (C#/.NET, JS/TS, Python, Java, Go — first match wins) and passes the matching joern-parse --language, so non-default frontends like C# build a CPG instead of silently degrading to LLM-fallback; the CPG cache is keyed <sha>.<language>.cpg. The C# frontend needs joern ≥ 2.0.0 plus the dotnetastgen tool — the install scripts provision and verify it (scripts/verify-joern-csharp.sh). Stage 1 reachability queries the CPG for paths from the finding location back to entry points; cite the entry point path in reachability.rationale.

LLM-fallback mode (joern absent)

Stages 1–3 use judgment rather than CPG data; Stages 4–5 work unchanged. Every entry in fallback mode carries reachability_source: llm-fallback. The exec-report-generator detects this and emits a banner — see agents/exec-report-generator.md § Section 0 banners.

Output

A DispositionRegister object per plugins/dev-team/knowledge/schemas/disposition-register-v1.json. Required entry fields and required envelope fields (schema_version, generated_at, dispositioner, reachability_tool, entries[]) are defined in the schema. Written to memory/disposition-<assessment-slug>.json.

Related

  • agents/fp-reduction.md — the opus agent that implements this skill
  • plugins/dev-team/knowledge/security-primitives-contract.md — disposition register schema + severity mapping
  • plugins/dev-team/knowledge/schemas/disposition-register-v1.json — JSON Schema
  • tools/reachability.sh — joern wrapper (installed if joern is on PATH)

Security Assessment Pipeline

The /security-assessment command executes this multi-phase pipeline over one or more target repos. This skill is authoritative for phase order, dependencies, artifacts, and failure semantics.

Phase graph

Key properties:

  • Multi-target: each target's Phase 0 → Phase 2b runs independently in parallel. Only Phase 4 (cross-repo) and Phase 5 (exec report) join across targets.
  • Phase 4 runs concurrently with Phase 1b – Phase 3 once Phase 0 finishes — it only depends on Phase 0 output.
  • Phase timing: every phase is bracketed with scripts/phase-timer.sh start/end so memory/phase-timings-<slug>.jsonl records actual wall times. exec-report-generator surfaces this data and flags drift from intended parallelism.
                               ┌─────────────────────┐
                               │ Phase 0: Recon      │
                               │ (parallel per       │
                               │  target)            │
                               └──────────┬──────────┘
                                          │
                 ┌────────────────────────┼────────────────────────┐
                 │                        │                        │
                 ▼                        ▼                        ▼
     ┌─────────────────────┐   ┌──────────────────┐   ┌──────────────────────┐
     │ Phase 1+1b          │   │ Phase 4 (cross-  │   │ (anything else       │
     │ (parallel across    │   │  repo: service-  │   │  Phase-0-dependent)  │
     │  tools AND agents)  │   │  comm parser;    │   └──────────────────────┘
     └──────────┬──────────┘   │  multi-target    │
                │              │  only)           │
                ▼              └──────────────────┘
     ┌─────────────────────┐
     │ Phase 1c: SEQUENTIAL│
     │ ACCEPTED-RISKS      │
     └──────────┬──────────┘
                ▼
     ┌─────────────────────┐
     │ Phase 2: SEQUENTIAL │
     │ fp-reduction        │
     └──────────┬──────────┘
                ▼
     ┌─────────────────────┐
     │ Phase 2b: severity  │
     │ floors              │
     └──────────┬──────────┘
                ▼
     ┌─────────────────────┐
     │ Phase 3: narratives │
     │ + compliance        │
     │ (parallel)          │
     └──────────┬──────────┘
                ▼
     ┌─────────────────────┐
     │ Phase 5: exec report│
     │ (single agent)      │
     └─────────────────────┘

Per-phase spec

Phase 0: Reconnaissance
  agent:     codebase-recon (opus)
  produces:  memory/recon-<slug>.{json,md}
  parallelism: parallel across targets

Phase 1: Tool-first detection (parallel across tools)
  skill:     static-analysis-integration
  produces:  memory/findings-<slug>.jsonl (unified finding stream)
  requires:  Phase 0
  parallelism: all tool × target × ruleset combinations run concurrently

Phase 1b: Judgment-layer detection (parallel across agents)
  agents:    security-review, business-logic-domain-review, deep-code-reasoning,
             authorization-logic-review, recon-driven-scan (opus all five)
  produces:  adds unified findings to memory/findings-<slug>.jsonl
  requires:  Phase 0, Phase 1
  parallelism: all five agents dispatched in a single Agent tool message,
               repeated per target
  adapters:  see knowledge/phase-1b-adapters.md

Phase 1c: ACCEPTED-RISKS suppression (sequential gate, mandatory)
  procedure: scripts/apply-accepted-risks.sh parses the first fenced
             ```json block in ACCEPTED-RISKS.md, matches findings by
             (rule_id exact, source_ref_glob), and suppresses matches
  produces:  memory/accepted-risks-<slug>.jsonl (suppressed + expired
             entries) + rewrites memory/findings-<slug>.jsonl in place
  requires:  Phase 1 + Phase 1b
  enforced:  exec-report-generator's Appendix C expects
             memory/accepted-risks-<slug>.jsonl when ACCEPTED-RISKS.md
             was present at target root

Phase 2: FP-reduction (sequential)
  agent:     fp-reduction (opus)
  produces:  memory/disposition-<slug>.json
  requires:  Phase 1 + Phase 1b + Phase 1c
  optional:  skipped when --fp-reduce=no

Phase 2b: Domain-class severity floors (sequential, deterministic)
  script:    ${CLAUDE_PLUGIN_ROOT}/scripts/apply-severity-floors.sh
  produces:  memory/severity-floors-log-<slug>.jsonl; rewrites
             memory/disposition-<slug>.json with floor-adjusted scores
  requires:  Phase 2
  optional:  no-op when Phase 2 was skipped

Phase 3: Narrative + compliance (parallel across agents)
  agent:     tool-finding-narrative-annotator (sonnet)
  skill:     compliance-mapping
  produces:  memory/narratives-<slug>.md, memory/compliance-<slug>.json
  requires:  Phase 2b (or Phase 1+1b if fp-reduction skipped)
  parallelism: dispatched together in one Agent tool message

Phase 4: Service-communication (runs concurrently with Phase 1b-3)
  tools:     ${CLAUDE_PLUGIN_ROOT}/harness/tools/service-comm-parser.py
             ${CLAUDE_PLUGIN_ROOT}/harness/tools/shared-cred-hash-match.py (multi-target only)
  produces:  memory/service-comm-<slug>.mermaid,
             memory/shared-creds-<slug>.sarif
  requires:  Phase 0 only

Phase 5: Report generation (sequential)
  agent:     exec-report-generator (opus)
  produces:  memory/report-<slug>.md (+ memory/cross-repo-summary-<slug>.md for multi-repo)
  requires:  Phase 2b + Phase 3 + Phase 4 (joined)

Phase 5b: Severity-consistency check (multi-target only, deterministic)
  script:    ${CLAUDE_PLUGIN_ROOT}/scripts/check-severity-consistency.sh
  produces:  memory/severity-consistency-<combined-slug>.txt
  requires:  Phase 5 + Phase 2 disposition registers
  condition: skipped for single-target runs

Phase 5c: Report verification (per target, deterministic)
  script:    ${CLAUDE_PLUGIN_ROOT}/scripts/verify-report.sh
  produces:  memory/verify-report-<slug>.txt
  requires:  Phase 5
  on-fail:   logs to audit trail; does NOT block publication

Canonical phase names for phase-timer.sh: phase-0-recon, phase-1-tool-first, phase-1b-judgment, phase-1c-accepted-risks, phase-2-fp-reduction, phase-2b-severity-floors, phase-3-narrative-compliance, phase-4-cross-repo, phase-5-report.

Graph-assisted discovery

If a target repo has .codegraph/ (CodeGraph MCP server — codegraph_explore/get_symbol for fast callers/callees/impact lookups) and/or a Repowise MCP server (get_context/search_codebase for semantic search, get_risk for modification-risk context), prefer them over raw Grep/Read when locating candidate files and tracing call paths. This benefits the recon-driven scanning step (Phase 1b's recon-driven-scan agent, bridging RECON narrative claims to concrete file:line evidence) most directly — graph-backed callers/impact lookups and semantic search surface exactly the file:line evidence that step needs. Never assume either tool is present — fall back to Read/Grep/Glob when neither is present; the tools are simply unavailable (no error) on repos without an index.

Helper-script invocation contract

Deterministic phase helpers carry strict ordering requirements. The orchestrator must not reorder these invocations:

Invocation paths (discoverable in an installed plugin). Every helper below ships inside this plugin and MUST be invoked by its plugin-root path so it resolves regardless of the caller's working directory — the bare names in the table are shorthand for these:

  • helper scripts → ${CLAUDE_PLUGIN_ROOT}/scripts/<name> (e.g. ${CLAUDE_PLUGIN_ROOT}/scripts/phase-timer.sh)
  • harness tools → ${CLAUDE_PLUGIN_ROOT}/harness/tools/<name>
ScriptMust run afterMust run beforeArtifacts
${CLAUDE_PLUGIN_ROOT}/scripts/phase-timer.sh start/endany phase boundary—phase-timings-<slug>.jsonl
${CLAUDE_PLUGIN_ROOT}/scripts/find-ci-files.shphase-0-reconphase-1-tool-firststdout (Phase 1 tool fan-out)
${CLAUDE_PLUGIN_ROOT}/scripts/apply-accepted-risks.shphase-1b-judgmentphase-2-fp-reductionrewrites findings-<slug>.jsonl; writes accepted-risks-<slug>.jsonl
${CLAUDE_PLUGIN_ROOT}/scripts/apply-severity-floors.shphase-2-fp-reductionphase-3-narrative-compliancerewrites disposition-<slug>.json in place; writes severity-floors-log-<slug>.jsonl

Concurrency: each script assumes a single writer on its input file. .tmp+mv handles the single-writer-but-crash case. Concurrent writers are a contract violation. Callers MUST serialize invocations against the same slug.

Idempotency: apply-severity-floors.sh and apply-accepted-risks.sh are both idempotent (marker-based and atomic-rewrite-based respectively) — safe to re-run from --start.

Invocation

/security-assessment <path>                    # single-repo (default)
/security-assessment <path1> <path2> [...]     # multi-repo
/security-assessment <path> --start 3          # resume from Phase 3
/security-assessment <path> --agents 0 1b      # run only listed phases
/security-assessment <path> --fp-reduce=no     # skip Phase 2

Flag semantics

  • --start <phase>: resume from 0|1|1b|2|3|4|5. Skill validates required artifacts exist; dependency check is SKIPPED (operator asserts preconditions).
  • --agents <phase-list>: run only listed phases. Dependency check skipped.
  • --fp-reduce=no: skip Phase 2. Phases 3 and 5 run on the raw finding stream; exec report carries banner "FP-reduction skipped; findings may contain false positives. Review Appendix B before acting." Default: yes.

Failure handling

Per-phase best-effort continuation:

  • Phase 0: stops the pipeline. RECON is the precondition.
  • Phase 1 / 1b: failed tool/agent logged; pipeline continues to Phase 2 with partial stream. Phase 5 names the failed detectors.
  • Phase 2: continue with raw stream; tag fp-reduce: failed. Phase 5 banner names the failure.
  • Phase 3: narratives + compliance marked "unavailable"; Phase 5 continues.
  • Phase 4: service-comm diagram replaced with a one-line note; Phase 5 continues.
  • Phase 5: failure reported; all produced artifacts remain in memory/ for manual assembly.

Artifacts

Every phase writes to memory/<kind>-<slug>.<ext>. Slug is derived from target repo name (or --joined for multi-repo).

KindProducerConsumer
recon-<slug>.jsonPhase 0Phase 1, 1b, 2, 3, 5
findings-<slug>.jsonlPhase 1, 1bPhase 2, 3
deep-reasoning-<slug>.jsonPhase 1bPhase 1b append step
authz-review-<slug>.jsonPhase 1bPhase 1b append step
recon-driven-<slug>.jsonPhase 1bPhase 1b append step
disposition-<slug>.jsonPhase 2Phase 3, 5
narratives-<slug>.mdPhase 3Phase 5
compliance-<slug>.jsonPhase 3Phase 5
service-comm-<slug>.mermaidPhase 4Phase 5, /cross-repo-analysis
report-<slug>.mdPhase 5Final output
cross-repo-summary-<slug>.mdPhase 5 (multi-repo)Final output
severity-consistency-<combined>.txtPhase 5bexec-report Section 6
verify-report-<slug>.txtPhase 5cAudit trail; surfaced on FAIL

Invariants

  • Every phase is idempotent within a run — same inputs → same outputs.
  • Artifact writes are atomic — partial writes produce an .incomplete suffix so the pipeline can detect and reject stale half-written artifacts.
  • The pipeline never deletes prior artifacts. --start resumes from existing; a new run without --start archives to memory/archive/<timestamp>/.
  • Audit log at memory/audit-<slug>.jsonl is append-only; records phase start/end, artifact produced, and failures.

Not covered

  • Red-team pipeline — harness/redteam/orchestrator.py.
  • Cross-repo analysis as a standalone command — /cross-repo-analysis.
  • PDF rendering — /export-pdf.

Commands 5

/cross-repo-analysis

You have been invoked with the /cross-repo-analysis command.

Role

Orchestrator for cross-repo security analysis. Composes deterministic tool outputs (service-comm-parser, shared-cred-hash-match) with the cross-repo-synthesizer agent's narrative synthesis.

Constraints

  1. Minimum two target repos. Below that, this command is a no-op — use /security-assessment for a single repo.
  2. No detection. This command assumes /security-assessment has already run per repo (or RECON has been generated per repo). It does not re-scan.
  3. Mermaid passthrough. The diagram from service-comm-parser.py flows through unchanged to the final report.

Parse arguments

Arguments: $ARGUMENTS

Positional: two or more directory paths. Each must contain memory/recon-<slug>.json (produced by an earlier codebase-recon run) or the command refuses.

Steps

1. Verify RECON per repo

For each path, locate memory/recon-*.json. If any target path lacks a RECON artifact, stop and tell the user: "Target has no RECON artifact — run codebase-recon or /security-assessment <path> first."

2. Run service-comm-parser.py

python3 "${CLAUDE_PLUGIN_ROOT}/harness/tools/service-comm-parser.py" <path1> <path2> ... > memory/service-comm-<slug>.mermaid

Where <slug> is a dash-joined concatenation of target repo names.

3. Run shared-cred-hash-match.py

python3 "${CLAUDE_PLUGIN_ROOT}/harness/tools/shared-cred-hash-match.py" <path1> <path2> ... > memory/shared-cred-<slug>.sarif

The SARIF output is consumed by the synthesizer agent; it is NOT passed through the unified-finding SARIF parser at this stage (findings are shared-credential hashes, not code-level issues).

4. Dispatch cross-repo-synthesizer

Using the Agent tool, dispatch the cross-repo-synthesizer agent with inputs:

  • List of RECON artifact paths (one per repo)
  • Path to the Mermaid file from step 2
  • Path to the shared-cred SARIF from step 3

The agent produces memory/cross-repo-analysis-<slug>.md. Wait for completion.

5. Present summary

Print to stdout:

Cross-repo analysis complete.

  Target repos: <list>
  Mermaid diagram: memory/service-comm-<slug>.mermaid
  Shared credentials: memory/shared-cred-<slug>.sarif (<N> groups, <M> total occurrences)
  Named attack chains: <count from the synthesizer output>
  Full report: memory/cross-repo-analysis-<slug>.md

Integration

  • Typically run after /security-assessment has completed per repo (so disposition registers exist).
  • The exec-report-generator reads memory/cross-repo-analysis-<slug>.md when producing the cross-repo summary report (one of the four reference outputs).

Escalation

Stop and ask the user when:

  • Only one target path is passed (requires at least two).
  • Any target path lacks a RECON artifact.
  • service-comm-parser.py or shared-cred-hash-match.py fails with a non-zero exit code — surface the error; do not silently swallow.

/export-pdf

You have been invoked with the /export-pdf command.

Role

Converter from Markdown to PDF. Uses pandoc when available (cleaner tables, wider Markdown support); falls back to weasyprint when pandoc is absent; skips with a warning when neither is installed.

Used for:

  • Executive reports from /security-assessment (memory/report-<slug>.md)
  • Red-team reports from /redteam-model (harness/redteam/results/adversarial-report.md)
  • Cross-repo summaries from /cross-repo-analysis
  • Any Markdown produced by this plugin's agents

Parse arguments

Arguments: $ARGUMENTS

Positional: <report.md> — path to the input Markdown file.

Flags:

  • --output <path>: output PDF path. Default: same basename with .pdf extension.
  • --css <path>: custom CSS stylesheet. Default: plugins/security-assessment/templates/report-css/default.css.

Steps

1. Validate inputs

  • Input file exists and is readable.
  • If --output is passed, its parent directory exists.
  • If --css is passed, the CSS file exists.

2. Detect available converter

Check in order:

  1. pandoc + wkhtmltopdf (or pandoc with --pdf-engine=weasyprint if weasyprint is installed).
  2. weasyprint with a small markdown-to-html wrapper.
  3. Neither → skip.

3. Convert

Preferred (pandoc):

pandoc "$INPUT" \
  --pdf-engine=weasyprint \
  --css="$CSS" \
  --standalone \
  --metadata title="$(basename "$INPUT" .md)" \
  -o "$OUTPUT"

Fallback (weasyprint + python-markdown):

The input Markdown is derived from a scanned repo / probed model (see probe_08_report_generator.py, service-comm-parser.py) — treat it as untrusted. Rendered HTML/CSS can reference remote resources (<img src="https://...">, CSS url(...)), and weasyprint fetches those by default, which is a plausible SSRF vector at PDF-conversion time. Pass a url_fetcher that blocks every non-local scheme so conversion never makes an outbound network request:

python3 -c "
import sys, markdown
from weasyprint import HTML, CSS

def _no_remote_fetch(url, *args, **kwargs):
    if not url.startswith(('file://', 'data:')):
        raise ValueError(f'blocked remote resource fetch during PDF export: {url}')
    from weasyprint import default_url_fetcher
    return default_url_fetcher(url, *args, **kwargs)

with open('$INPUT') as f:
    md_text = f.read()
html_body = markdown.markdown(md_text, extensions=['tables', 'fenced_code', 'codehilite'])
html = f'<html><head><meta charset=\"utf-8\"></head><body>{html_body}</body></html>'
HTML(string=html, url_fetcher=_no_remote_fetch).write_pdf(
    '$OUTPUT', stylesheets=[CSS(filename='$CSS')]
)
"

4. Report

On success:

PDF written: /absolute/path/to/report.pdf

Print the absolute path — some downstream tooling expects an absolute path.

On graceful skip (neither tool):

SKIP: pandoc and weasyprint both absent; PDF not produced.
      Install one:
        brew install pandoc weasyprint   (macOS)
        apt install pandoc               (Linux)
        pip install weasyprint           (any)
      The markdown report remains at: $INPUT

Exit 0 in both cases (skip is not a failure).

Escalation

Stop and ask the user when:

  • Input file does not exist.
  • Output directory is not writable.
  • CSS file specified by --css does not exist (default CSS should always exist; if missing, report as a plugin install issue).

CSS stylesheet

The bundled CSS at plugins/security-assessment/templates/report-css/default.css is modelled on the opus_repo_scan_test reference's utilities/convert_report.py CSS:

  • A4 page size, 2cm × 1.5cm margins
  • Page footer: confidentiality notice (center) + page N of M (right)
  • Red accent (#c00) on H1 underline and blockquote border
  • Zebra-striped tables
  • Monospace code blocks; syntax highlighting via codehilite
  • Muted color scheme, no decorative images

Override via --css <path> for organization-specific branding.

Integration

  • Runs after /security-assessment produces memory/report-<slug>.md.
  • Runs after /redteam-model produces harness/redteam/results/adversarial-report.md.
  • Not run automatically — user invokes at their discretion.

/redteam-model

Orchestrator entry point for the adversarial ML red-team harness. Perform scope + consent checks, then dispatch the Python orchestrator at ${CLAUDE_PLUGIN_ROOT}/harness/redteam/orchestrator.py. Interpret probe artifacts via the four analyzer agents after the Python passes complete.

Full runtime spec — including bash invocations, exact refusal wording, flag-interaction matrix, and step output formats — is in ${CLAUDE_PLUGIN_ROOT}/harness/redteam/ORCHESTRATION.md. Read it before dispatching.

Safety constraints (non-negotiable)

  1. Scope-enforced by default. Targets must resolve to a self-owned CIDR (127.0.0.0/8, 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, ::1). Public targets require --self-certify-owned <path>.
  2. Self-certification is logged. Artifact's SHA-256 is written to harness/redteam/results/audit_log.jsonl with the run timestamp.
  3. Rate-limit + budget enforced. All HTTP goes through http_client which enforces RATE_LIMIT req/sec and QUERY_BUDGET total queries. Exhaustion halts the pipeline with budget_exhausted status.
  4. Harness-direct invocation blocked. The redteam-guard.sh hook refuses direct python orchestrator.py calls unless REDTEAM_AUTHORIZED=1 is in env. This command sets it after the scope/consent checks pass; scoped to the child process only.

Parse arguments

Arguments: $ARGUMENTS

Positional: <target-url> (base URL, no trailing slash).

Flags:

  • --self-certify-owned <path>: authorization artifact for public targets. Format: knowledge/redteam-authorization.md.
  • --dry-run: validate config + scope + consent; print step graph; zero HTTP requests. Scope enforcement still runs.
  • --agents <id> [<id> ...]: run only these probe IDs (01-08).
  • --start <id>: resume from this probe ID (token from prior run's result_store.resume_message()).

Flag interactions are in ${CLAUDE_PLUGIN_ROOT}/harness/redteam/ORCHESTRATION.md § "Flag interactions".

Procedure

Execute the steps in ${CLAUDE_PLUGIN_ROOT}/harness/redteam/ORCHESTRATION.md:

  1. Resolve and validate scope (Step 1)
  2. Validate config (Step 2)
  3. Dispatch the Python orchestrator (Step 3)
  4. Dispatch analyzer agents (Step 4) — three in parallel, then redteam-report-generator last
  5. Present summary (Step 5)

Escalation

Stop and ask the user per ${CLAUDE_PLUGIN_ROOT}/harness/redteam/ORCHESTRATION.md § "Escalation".

Integration

  • Paired with /security-assessment for static analysis (independent pipelines; can run against the same or different targets).
  • Paired with /export-pdf for the final report.
  • redteam-guard.sh PreToolUse hook enforces the authorization gate.
  • knowledge/redteam-authorization.md declares --self-certify-owned artifact format.

/security-assessment

Orchestrator for the static-analysis pipeline. Execute the phase graph defined in skills/security-assessment-pipeline/SKILL.md. Phases dispatch to agents/scripts; artifacts pass through memory/. Produces a publication-ready exec report per target (plus a cross-repo summary for multi-target runs).

Constraints

  1. Follow the pipeline exactly. The skill is authoritative. Do not reorder phases or skip deterministic gates.
  2. Never silently drop findings. Every input flows through to either the published report or a suppression appendix with reason.
  3. Artifacts are the source of truth. Every phase writes to memory/; --start and failure-recovery rely on that.
  4. Informational-not-audit-grade. Every produced report carries the compliance disclaimer (verbatim per knowledge/disclaimers.md) at the header.

Parse arguments

Arguments: $ARGUMENTS

Positional: one or more directory paths (target repos).

Flags:

  • --start <phase>: resume from phase (0 / 1 / 1b / 2 / 3 / 4 / 5). Requires prior phase artifacts in memory/.
  • --agents <phase> [<phase> ...]: run only listed phases. Dependency check skipped.
  • --fp-reduce=yes|no: skip Phase 2 FP-reduction when no. Default yes.

Steps

1. Validate arguments

  • ≥ 1 target path required.
  • Each target must be a directory (not a file).
  • For each target: consult ACCEPTED-RISKS.md if present at the target root.

2. Initialize run

For each target repo, derive a slug from the directory name (kebab-cased, lowercased). Multi-repo runs use dash-joined slug for cross-repo artifacts.

Create memory/audit-<slug>.jsonl (append-only). Record run start with targets, flags, and contract version. If prior artifacts exist AND --start is NOT set, archive them to memory/archive/<timestamp>/ before overwriting.

3. Execute the phase graph

Follow skills/security-assessment-pipeline/SKILL.md § "Phase graph". The skill defines per-phase inputs, outputs, agents/scripts, and parallelism. Three parallelism rules MUST be observed:

  1. Multi-target fan-out — each target's Phase 0 → Phase 2b runs as an independent pipeline; dispatch as parallel Agent tool calls in the SAME message.
  2. Intra-phase fan-out — Phase 1b dispatches its five agents (security-review, business-logic-domain-review, deep-code-reasoning, authorization-logic-review, recon-driven-scan) in one message; Phase 3 dispatches narrative + compliance in one message; Phase 1's static-analysis-integration runs every tool concurrently.
  3. Phase 4 runs concurrently with Phase 1b–3 once Phase 0 finishes. Phase 4 depends only on Phase 0.

Per-phase agent / script dispatch + adapter wiring is documented in the skill. Do not reorder. Helper-script invocation contract (which script runs after which phase boundary) is in the skill § "Helper-script invocation contract".

4. Phase timing

Every phase is bracketed with ${CLAUDE_PLUGIN_ROOT}/scripts/phase-timer.sh start <phase> <slug> / ${CLAUDE_PLUGIN_ROOT}/scripts/phase-timer.sh end <phase> <slug>. Writes accumulate in memory/phase-timings-<slug>.jsonl. The exec-report-generator reads this file to compute actual parallelism vs. sequential execution.

5. Surface summary

Print:

Security assessment complete.

  Target(s): <list>
  Phases run: <list of phase numbers>
  Artifacts:
    memory/recon-<slug>.{json,md}
    memory/findings-<slug>.jsonl (<N> unified findings)
    memory/disposition-<slug>.json (<N> entries, <X>% true/likely-true)
    memory/narratives-<slug>.md
    memory/compliance-<slug>.json (<N> annotations, <M> LLM-triggered)
    memory/service-comm-<slug>.mermaid
    memory/report-<slug>.md (CRITICAL: <N>, HIGH: <N>, MEDIUM: <N>, LOW: <N>)
    (+ memory/cross-repo-summary-<slug>.md for multi-repo)

  Run `/export-pdf <report>.md` for PDF.
  Run `/cross-repo-analysis <path1> <path2>` for cross-repo chain analysis
  if not already included.

Escalation

Stop and ask the user when:

  • No target paths are provided.
  • Any target path is not a directory.
  • --start is set but required precondition artifacts are missing.
  • Phase 0 (recon) fails on any target — no meaningful downstream without it.
  • More than 3 phases fail in a single target — pipeline output is no longer trustworthy; escalate rather than emit a misleading report.

Integration

  • Phase graph: skills/security-assessment-pipeline/SKILL.md
  • Cross-repo: pair with /cross-repo-analysis
  • PDF: pair with /export-pdf
  • Primitives: plugins/dev-team/knowledge/security-primitives-contract.md

Upgrade

Role: worker. This command updates the security-assessment plugin to the latest version and ensures its marketplace is set to auto-update going forward.

You have been invoked with the /upgrade command.

Steps

1. Read current version

claude plugin list

Parse the output to find security-assessment and its current version. Also read the installed plugin.json directly:

~/.claude/plugins/cache/*/security-assessment/*/.claude-plugin/plugin.json

Report:

Current version: security-assessment v{version} (installed from {marketplace})

2. Check auto-update status and ask the user

First, check the current auto-update status by running the script below and reporting it to the user.

python3 - <<'PY'
import json, os

PLUGIN = "security-assessment"
home = os.path.expanduser("~")
cwd = os.getcwd()

def load(path):
    try:
        with open(path) as f:
            return json.load(f)
    except FileNotFoundError:
        return None
    except json.JSONDecodeError as e:
        print(f"unknown ({path} is not valid JSON)")
        raise SystemExit(0)

installed = (load(os.path.join(home, ".claude", "plugins", "installed_plugins.json")) or {}).get("plugins", {})
market = next((pid.split("@", 1)[1] for pid in installed if pid.split("@", 1)[0] == PLUGIN and "@" in pid), None)
if not market:
    print("unknown (marketplace not found)")
    raise SystemExit(0)

candidates = [
    os.path.join(cwd, ".claude", "settings.json"),
    os.path.join(cwd, ".claude", "settings.local.json"),
    os.path.join(home, ".claude", "settings.json"),
]
for path in candidates:
    data = load(path)
    if data and market in (data.get("extraKnownMarketplaces") or {}):
        flag = data["extraKnownMarketplaces"][market].get("autoUpdate")
        print("enabled" if flag is True else "disabled")
        raise SystemExit(0)

print("disabled")
PY

Then ask the user:

Auto-update for the {marketplace} marketplace is currently {enabled/disabled}. Would you like to enable auto-update so future releases install automatically? (yes/no)

Note: this flag is marketplace-level and affects every plugin published from {marketplace} — including the companion dev-team plugin if you have it installed.

Wait for the user's answer before continuing. If they say yes, run the enable block below. If they say no, skip to step 3.

Enable block (run only if the user consents):

python3 - <<'PY'
import json, os

PLUGIN = "security-assessment"
home = os.path.expanduser("~")
cwd = os.getcwd()

def load(path):
    try:
        with open(path) as f:
            return json.load(f)
    except FileNotFoundError:
        return None
    except json.JSONDecodeError as e:
        print(f"  ! {path} is not valid JSON ({e}); skipping")
        return None

installed = (load(os.path.join(home, ".claude", "plugins", "installed_plugins.json")) or {}).get("plugins", {})
market = next((pid.split("@", 1)[1] for pid in installed if pid.split("@", 1)[0] == PLUGIN and "@" in pid), None)
if not market:
    print(f"  ! Could not resolve the marketplace for '{PLUGIN}'; skipping.")
    raise SystemExit(0)

candidates = [
    os.path.join(cwd, ".claude", "settings.json"),
    os.path.join(cwd, ".claude", "settings.local.json"),
    os.path.join(home, ".claude", "settings.json"),
]
target = None
for path in candidates:
    data = load(path)
    if data and market in (data.get("extraKnownMarketplaces") or {}):
        target = [path, data]
        break

if target is None:
    reg = load(os.path.join(home, ".claude", "plugins", "known_marketplaces.json")) or {}
    entry = reg.get(market)
    if not entry:
        print(f"  ! Marketplace '{market}' not found in settings or registry; skipping.")
        raise SystemExit(0)
    path = os.path.join(home, ".claude", "settings.json")
    data = load(path) or {}
    data.setdefault("extraKnownMarketplaces", {})[market] = {"source": entry["source"]}
    target = [path, data]

path, data = target
mk = data["extraKnownMarketplaces"][market]
if mk.get("autoUpdate") is True:
    print(f"  auto-update already enabled for '{market}' ({path})")
else:
    mk["autoUpdate"] = True
    with open(path, "w") as f:
        json.dump(data, f, indent=2)
        f.write("\n")
    print(f"  enabled auto-update for '{market}' in {path}")
    print("  (takes effect on next Claude Code launch / plugin operation)")
PY

Report the one-line result to the user, then continue to step 3.

3. Run the update

First, determine the install scope from claude plugin list output (the Scope: line for security-assessment). It will be one of: user, project, local, managed.

claude plugin update --scope {scope} security-assessment@{marketplace}

Where {scope} is the detected install scope (e.g., project) and {marketplace} is the marketplace name (e.g., bfinster). The --scope flag is required — the CLI defaults to user, which will fail if the plugin is installed at a different scope.

If the command succeeds with a version change, proceed to step 4.

If the output indicates already up to date:

Already running the latest version (v{version}).

Exit.

If the command fails, report the error and suggest:

Update failed. You can try a manual reinstall:

claude plugin uninstall --scope {scope} security-assessment@{marketplace}
claude plugin install --scope {scope} security-assessment@{marketplace}

Exit.

4. Check companion dependency

If dev-team@{marketplace} is not installed, surface the dependency:

WARNING: dev-team@{marketplace} is not installed but is required by security-assessment (primitives contract, codebase-recon agent, SARIF parser). Install it with:

claude plugin install --scope {scope} dev-team@{marketplace}

This is informational — the security-assessment update has already succeeded. The user can act on it after restart.

5. Confirm the update

claude plugin list

Report:

## Upgrade Complete

Previous: security-assessment v{old_version}
Updated:  security-assessment v{new_version}

Restart Claude Code to apply the update.

Notes

  • The claude plugin update command handles fetching, caching, and version management. Previous versions are kept for 7 days so active sessions continue working.
  • A restart of Claude Code is required for the new version to take effect.
  • Step 2 enables marketplace-level auto-update by writing extraKnownMarketplaces.<marketplace>.autoUpdate: true to settings.json — the same flag the /plugin UI toggles. With it on, routine releases for both plugins in the marketplace (dev-team and security-assessment) land without running /upgrade again.

Agents 13

Authorization Logic Review

Cites: [severity-floors]

Map the authorization model first, then check consistent enforcement. Report the gap between stated policy and observed implementation — not single suspicious lines.

Context needs: project-structure

Inputs

  1. Target repo files — RECON-scoped or grep-discovered
  2. memory/recon-<slug>.json — entry points + security surface
  3. knowledge/authz-review-categories.yaml — rule_id + CWE assignments

Output

  • memory/authz-review-<slug>.json — JSON array of unified findings. Schema: plugins/dev-team/knowledge/schemas/unified-finding-v1.json.

Required metadata: source: "llm-reasoning", cwe[], confidence (high|medium only), secondary_locations[] (≥ 1 policy + gap pairing), reasoning (2-3 sentences).

Procedure

1. Map the authorization model

Discover how the application declares and enforces access control:

  • Route decorators / middleware annotations (@require_auth, @roles_allowed, [Authorize(Roles=...)], router.use(authMiddleware))
  • Permission constants and role definitions (permissions.py, roles.js, AuthorizationPolicy.cs, scopes.go)
  • Middleware stacks (Express chain, Django middleware list, ASP.NET Core pipeline)
  • Tenancy models (tenant_id, organization_id, account_id in models or query builders)

Classify as RBAC, ABAC, ACL, tenancy-scoped, or mixed. Note where the predominant pattern is declared.

2. Identify enforcement points

Per route or operation class, note where authorization is enforced:

  • Controller / handler — checked before business logic runs
  • Service layer — checked inside business logic
  • Repository / data-access — enforced in the query (e.g. .where(tenant_id=current_tenant))
  • Not found — no enforcement located

Gaps in this {operation → enforcement} map are findings.

3. Tenant isolation consistency

If a tenancy model is present:

  • Grep direct object-load patterns that could return cross-tenant data: findById, getById, SELECT ... WHERE id = ? without a tenant filter.
  • Check whether ORM base query builder / repository base class enforces tenancy (global scope = OK; ad-hoc per-query = risky).
  • Note admin or superuser paths that bypass tenancy for legitimate reasons — acknowledged bypasses, not findings.

4. Role/permission escalation paths

  • Can a low-privilege user update fields that determine their own role/permissions?
  • Are role assignments validated server-side on every mutation, or only at creation time?
  • Is admin promotion / impersonation gated on a separate high-privilege check, not just "is authenticated"?

5. Cross-service authorization propagation

If RECON identifies inter-service calls (S2S HTTP, gRPC, message queue):

  • Does the receiving service re-verify authorization?
  • Are S2S credentials separate from user credentials?
  • Can a user indirectly trigger privileged S2S operations via user-facing inputs?

6. Minimum evidence bar

A finding requires at least two specific code locations — the policy declaration and the location where it is violated or absent. Single- location suspicions are discarded.

Categories

Use rule_ids from knowledge/authz-review-categories.yaml. Severity and confidence calibration are in the same file.

An empty array [] is valid — not every codebase has authorization gaps.

Output discipline

Emit only the JSON file at § Output. No chat preamble or summary.

Rationale & provenance

See docs/agents/authorization-logic-review.md for the top-down vs. bottom-up split and the two-location-evidence rule.

Business-Logic Domain Review

Read the codebase with fraud-detection domain knowledge. Surface issues that require cross-file reasoning — bugs in WHAT the code does, not HOW. Static tools catch syntax; this agent catches business logic.

Context needs: project-structure

Inputs

  • Target repo (walked on demand)
  • memory/recon-<slug>.json — scoping + entry points + security surface
  • knowledge/domain-logic-patterns.md — full catalogue of 9 patterns with grep cues, exploit scenarios, and remediation pointers. Read before scanning.

Output

  • memory/business-logic-findings-<slug>.json — unified findings. Schema: plugins/dev-team/knowledge/schemas/unified-finding-v1.json.

Required fields per finding:

  • rule_id in the form business-logic.fraud.<category> (e.g. business-logic.fraud.fail-open-scoring, business-logic.fraud.messaging-subject-injection, business-logic.fraud.messaging-subscriber-poisoning, business-logic.fraud.training-data-inference)
  • metadata.source: "business-logic-domain-review"
  • metadata.confidence: high | medium only — never low
  • metadata.exploitability when reasoning supports it
  • metadata.cwe[] when applicable (CWE-754, CWE-841, CWE-840, CWE-200, etc.)
  • metadata.attack_scenario — 2-3 sentences
  • metadata.source_ref.path_class: "test" | "production" | "unknown" (downstream uses this; do not filter on it here)

Detection patterns

Nine patterns catalogued in knowledge/domain-logic-patterns.md. Summary:

  1. Fail-open scoring
  2. Score manipulation / client-controlled features
  3. Emulation-mode bypass
  4. Model-endpoint confusion
  5. Tokenization / PII-masking skip
  6. Feature poisoning
  7. Missing replay idempotency
  8. Messaging attack surface (subject injection / subscriber poisoning / missing replay protection)
  9. Training data inference from metrics/logs

The knowledge file is authoritative for grep cues, exploit scenarios, remediation pointers, and CWE assignments per pattern.

Procedure

  1. Read RECON. Focus on files under security_surface.auth_paths, security_surface.network_egress, and any files whose path contains score, fraud, predict, model, risk, decision.
  2. For each in-scope file, grep for the cues in knowledge/domain-logic-patterns.md. Read ±30 lines of context.
  3. For each confirmed instance, emit a unified finding. Message = one-line summary; full attack scenario goes in metadata.attack_scenario. Set confidence based on context verification.
  4. Tag metadata.source_ref.path_class (test/production/unknown). Do not filter test-only paths — that is fp-reduction's job.

Invariants

  • Every finding traces to a code location (file + line).
  • Every finding has a 2-3 sentence attack scenario in metadata.attack_scenario.
  • confidence: high only when the pattern is present AND there is no obvious mitigation in a ±30-line window. Otherwise medium.
  • Never emit low confidence — if not confident, do not emit.

Output discipline

Emit only the JSON file at § Output. No chat preamble or summary.

Rationale & provenance

See docs/agents/business-logic-domain-review.md for the why-static-tools- miss-this argument and the messaging-pattern split rationale.

Compliance Edge Annotator

Single-turn. No loops. Invoked once per triggering finding by the compliance-mapping skill when a matched pattern row has llm_review_trigger: true.

Context needs: full-file

Inputs (passed by the skill)

  • finding — full unified finding being annotated
  • pattern_row — matched compliance-patterns.yaml row (with base citations)
  • matched_code_context — ±20 lines at the finding location
  • ff_context — RECON's relevant sections (endpoints, auth paths)

Output

Strict JSON, no markdown fences, no prose outside the note field:

{
  "finding_rule_id": "<finding.rule_id>",
  "refinements": {
    "keep": ["PCI-DSS:8.3", "SOC2:CC6.1"],
    "remove": ["PCI-DSS:3.4"],
    "add": []
  },
  "note": "<1-2 sentence judgment: why keep/remove/add>",
  "confidence": "high | medium | low"
}

The skill merges this with the pattern row's base citations to produce the final annotation.

Rules

  • Never invent a citation. The add list draws ONLY from the closed regulation set: PCI-DSS, GDPR, HIPAA, SOC2, NIST (SP 800-*, AI RMF), EU-AI-Act, OWASP, CWE. If none of these clearly apply beyond what the pattern row gave, add is empty.
  • Removal requires a reason in note. If you cannot articulate why a base citation does not apply, keep it.
  • confidence: low when the judgment is guesswork. The skill treats low-confidence annotations as "defer to pattern table" — remove/add are ignored; keep is taken as-is.
  • Strict JSON output. Schema-invalid output fails the run; do not try to recover.

When to invoke (skill-side)

The skill invokes this agent when a pattern row matches a finding AND the row has llm_review_trigger: true. Typical triggering classes:

  • admin-endpoint-unauth: context matters (/debug behind VPN ≠ public /admin)
  • pii-in-response-body: field-level judgment (email might be the user's own; ssn rarely is)
  • model-integrity: whether the integrity gap crosses a compliance line depends on the model's role

Rate / cost

One LLM call per triggering finding. The skill caps total calls at the number of triggering findings; overage fails the run and logs the discrepancy.

For eval runs, LLMCallCounter counts exact invocations so tests can assert "exactly 1 LLM call for this fixture".

Invariants

  • Single-turn. No follow-up queries to the user.
  • Output validates as JSON. Schema-invalid output fails the run.
  • Untouched base citations (keep only, no remove, no add) appear in the final annotation unchanged — the safe path.
  • confidence is always set.

Disclaimer

Pipeline-wide compliance disclaimer (verbatim per knowledge/disclaimers.md § "Compliance mapping disclaimer") is applied by the skill to every output containing annotations.

Output discipline

Emit only the JSON object at § Output. No chat preamble or summary.

Cross-Repo Synthesizer

Read aggregated data from multiple repos. Name cross-repo attack chains explicitly. Cite findings; do not invent.

Invoked by /cross-repo-analysis after service-comm-parser.py and shared-cred-hash-match.py have produced their outputs.

Context needs: artifact-stream

Inputs

  • One RECON artifact per target repo (memory/recon-<slug>.json)
  • Per-repo disposition register (memory/disposition-<slug>.json) if /security-assessment ran per repo; else unified findings directly
  • Service-comm Mermaid diagram (stdout from service-comm-parser.py)
  • Shared-cred SARIF (stdout from shared-cred-hash-match.py)

Output

memory/cross-repo-analysis-<assessment-slug>.md:

  1. Overview — 1 paragraph naming target repos and analysis scope.
  2. Shared credentials — table: hash prefix, count of repos affected, SHA-256 log reference.
  3. Inter-service communication diagram — Mermaid block from service-comm-parser.py embedded byte-identical.
  4. Named attack chains — 3-10 chains, each with:
    • Short name (e.g. "Credential reuse → privileged NATS → model endpoint")
    • Findings cited by rule_id + file:line
    • Step-by-step walk (attacker action → state → next action)
    • Repos involved + each service's role
  5. Systemic patterns — 1-3 paragraphs on organizational tendencies (e.g. "no repo uses a secrets manager; all use env-file credentials").
  6. Defensive priorities — ranked, with cross-repo coordination notes.

Procedure

  1. Load every RECON, the Mermaid (read node + edge labels), and shared- cred SARIF.

  2. Build the cross-index:

    • Credential hash → repos + files + lines
    • Service → role (publisher / subscriber / both) per NATS subject
    • Service → entry points with auth status
    • Service → service dependencies (from Mermaid edges)
  3. Name attack chains. Candidate patterns: credential-reuse, unauthenticated- messaging, model-confusion, feature-poisoning. A chain is meaningful only when ≥ 2 repos are involved AND the chain advances the attacker's position. If none qualify, say so — do not invent.

  4. Identify systemic patterns (≥ 2 repos exhibiting the same tendency).

  5. Rank defensive priorities:

    • P1: fixes that break ≥ 2 named chains
    • P2: fixes that address a systemic pattern
    • P3: per-repo fixes

    Each priority names affected chains/patterns + a role-level owner.

Invariants

  • Every chain cites findings by rule_id + file:line. Uncited chains are not emitted.
  • Embedded Mermaid is byte-identical to service-comm-parser.py output — no reformatting, no re-rendering.
  • Shared-credential SHA-256 values are printed as first 12 hex chars only (full hash in audit log).
  • Recommendations name roles, never individuals.
  • Do not detect findings — synthesize from existing only.

Output discipline

Emit only the markdown file at § Output. No chat preamble or summary.

Rationale & provenance

See docs/agents/cross-repo-synthesizer.md for the chain-meaningfulness threshold, citation discipline, and no-individual-ownership rule.

Deep Code Reasoning

Phase 1b peer agent. Catches vulnerabilities that require cross-file reasoning — IDOR, confused deputy, TOCTOU, indirect privilege escalation, workflow bypass. Scope discipline is mandatory: read only RECON- identified surface.

Context needs: project-structure

Inputs

  1. memory/recon-<slug>.json — entry_points, security_surface.auth_paths, security_surface.sensitive_data_flows (required)
  2. Target repo files at RECON-identified paths (on demand; load scoped files + immediate callers/callees only)

Output

  • memory/deep-reasoning-<slug>.json — JSON array of unified findings. Schema: plugins/dev-team/knowledge/schemas/unified-finding-v1.json.

Required metadata: source: "llm-reasoning", cwe[] (≥ 1), confidence (high|medium only — never low), secondary_locations[] (≥ 1), reasoning (2-3 sentences tracing the attack path).

Scope extraction

Read memory/recon-<slug>.json:

  • entry_points[] — HTTP handlers, CLI, cron, event consumers
  • security_surface.auth_paths[]
  • security_surface.sensitive_data_flows[]

If security_surface is missing, grep for common auth indicators (@require_auth, hasPermission, isAuthorized, checkRole, verify_token, @login_required, [Authorize]) and use the matching files as the working surface. Document the fallback in the first entry's metadata.

Detection categories (rule_id prefixes)

  • llm-reasoning.idor.<descriptor> — object-level authorization bypass
  • llm-reasoning.function-level-authz.<descriptor> — function-level auth gap
  • llm-reasoning.confused-deputy.<descriptor> — confused deputy / SSRF via delegation
  • llm-reasoning.toctou.<descriptor> — time-of-check-time-of-use
  • llm-reasoning.privilege-escalation.<descriptor> — indirect privilege escalation
  • llm-reasoning.workflow-bypass.<descriptor> — business logic / state machine bypass

OWASP cross-reference: broken access control (A01:2021), confused deputy, TOCTOU across service boundaries, indirect privilege escalation, business logic bypass.

Procedure

1. Load and bound the surface

Extract from RECON or grep fallback. If the surface exceeds 30 files, priority order: auth_paths → entry_points → sensitive_data_flows. Process the top 30 only; note the truncation.

2. Read and trace each surface item

For each file in scope:

  1. Read the file.
  2. Find callers (grep exported names; read the top 3 by reference count).
  3. Find security-sensitive callees: for data-access, permission, or state-transition ops, read one level deeper.

Do not recurse further without a specific reason tied to an active hypothesis.

3. Apply minimum evidence bar

Advance to output only when you can cite at least two specific code locations (file:line) that together constitute the vulnerability. See docs/agents/deep-code-reasoning.md for paired-evidence examples.

4. Severity

  • error — reachable from a public entry point; directly enables privilege escalation or data access bypass
  • warning — reachable but requires additional conditions, or only reachable from authenticated paths
  • info — pattern present but exploit viability unclear without runtime context

5. Confidence (mandatory; two values only)

  • high — full attack path traceable with no gaps; every step has a citation
  • medium — path is plausible but one step requires an assumption (note it explicitly in reasoning)

Never emit low.

6. Write output

JSON array. Empty [] is valid and expected when the scoped surface yields no confirmed findings — do not manufacture findings to fill the file. Validate each entry carries all required metadata before writing.

Output discipline

Emit only the JSON file at § Output. No chat preamble or summary.

Rationale & provenance

See docs/agents/deep-code-reasoning.md for scope-discipline rationale, paired-evidence examples, and the no-low-confidence policy.

Executive Report Generator

Final agent in /security-assessment. Synthesizes narrative from upstream artifacts. Does not detect, disposition, or score. Maps presentational severity (CRITICAL/HIGH/MEDIUM/LOW) per primitives contract v1.0.0.

Context needs: artifact-stream

Inputs (per target repo)

  • memory/recon-<slug>.{json,md} — Phase 0
  • memory/findings-<slug>.jsonl — Phase 1 + 1b raw findings
  • memory/disposition-<slug>.json — Phase 2 disposition register (absent when --fp-reduce=no)
  • memory/narratives-<slug>.md — Phase 3 narratives
  • memory/compliance-<slug>.json — Phase 3 compliance annotations
  • memory/service-comm-<slug>.mermaid — Phase 4 diagram

Multi-repo: also memory/cross-repo-analysis-<combined-slug>.md (if produced).

Outputs

  • memory/report-<slug>.md (filename: <repo-name>-security-assessment.md)
  • memory/cross-repo-summary-<combined-slug>.md (multi-repo only; filename: cross-repository-security-summary.md)

Seven-section per-repo structure

Section 0 — Executive Summary

One page, business terms. Written LAST (Top 3 Actions depend on §2 and §5).

  • 2-3 sentence overview: scope, dominant risk category, recommended next step.
  • Top 3 Actions table: action / owner (role) / effort (S/M/L) / blocking-id.
  • Presentational severity tally: CRITICAL: N HIGH: N MEDIUM: N LOW: N.
  • Banners (verbatim text required):
    • FP-reduction skipped: "FP-reduction skipped; findings may contain false positives. Review Appendix B before acting."
    • LLM-fallback reachability (any disposition entry with reachability_source: llm-fallback): "Reachability stage used LLM reasoning instead of call-graph analysis; dead-code paths may be less accurate. Stages 2–5 unaffected."

Section 1 — Findings Dashboard

One table, all findings (post-disposition), grouped by presentational severity. Columns: ID, Rule, File:Line, Category, Severity, Verdict, Confidence.

  • Confidence: read from disposition.confidence. null confidence (likely_false_positive / false_positive) entries are excluded.
  • CWE format (dashboard): number-only. Single: CWE-NNN. Multiple: CWE-NNN + CWE-MMM. Separator is always +, never /.

Section 2 — CRITICAL and HIGH Findings

One detail block per finding. Contains: Summary (1 sentence), Location (file:line), CWE (invariant — every C/H must have CWE), Confidence, Reachability trace (from disposition.reachability.rationale), Attack scenario (2-3 sentences), Remediation (2-4 sentences), Compliance citations.

CWE format (Section 2): CWE-NNN — Full CWE Name. Multiple: CWE-NNN — Name + CWE-MMM — Name (em dash before name; + between).

Section 3 — MEDIUM and LOW Findings

One row per finding: summary sentence + remediation pointer.

CWE format (Section 3): number-only. **CWE-NNN** or **CWE-NNN + CWE-MMM**.

Section 3b — Cross-Cutting Concerns (multi-repo only)

Omit for single-repo. Multi-repo: ≤ 1 page summarizing for THIS repo:

  1. Shared credentials: read memory/shared-creds-<combined-slug>.sarif; list entries where one of locations belongs to this slug. Format: secret type, value (truncated to 4 chars + …), other repos involved.
  2. Attack chains involving this repo: read memory/cross-repo-analysis-<combined-slug>.md; include only chains whose step list names this slug. Format: name, one-line summary, severity.
  3. Regulatory gaps specific to this repo: from compliance-<slug>.json, any annotation with regulation_risk: high not already in §2 or §3.

Point to the cross-repo summary for depth.

Section 4 — Service Communication Diagram

Embed the Mermaid block from service-comm-parser.py byte-identical (post-CRLF-normalize). Do not re-render.

Section 5 — Remediation Roadmap

P1 / P2 / P3 / P4 buckets. Each entry: owner (role), effort estimate, blocking finding IDs. P1 = do today; P4 = informational.

Section 6 — Methodology and Scope

Brief statement of what was and was not assessed. Detailed spec (timing-table assembly, agent→phase mapping, drift-detection messages, coverage-gap callouts, cross-repo severity calibration) is in knowledge/exec-report-section6-spec.md — read it on demand.

Section 7 — Appendices

  • Appendix A — Secrets inventory (gitleaks + entropy-check)
  • Appendix B — Findings missing CWE or reachability (invariant violations)
  • Appendix C — Suppressed findings (ACCEPTED-RISKS matches). Read memory/accepted-risks-<slug>.jsonl. If ACCEPTED-RISKS.md existed at target root but the file is absent or empty, emit: "ACCEPTED-RISKS.md was present at the target root but the Phase 1c suppression gate did not run — findings flow through without suppression. Re-run the pipeline or investigate why Phase 1c was skipped."
  • Appendix D — Compliance annotations (full list; disclaimer at top verbatim per knowledge/disclaimers.md § "Compliance mapping disclaimer")
  • Appendix E — File inventory (from RECON)

Cross-repo summary structure

When multiple targets assessed, generate cross-repo-summary-<slug>.md:

  1. Top 3 cross-repo actions
  2. Shared risk patterns (issues appearing in ≥ 2 repos)
  3. Cross-repo attack chains (from /cross-repo-analysis if available, else synthesized inline)
  4. Aggregated inter-service Mermaid (embed verbatim from service-comm-parser.py)
  5. Compliance roll-up
  6. Consolidated risk matrix

Invariants (violations → Appendix B, not silently dropped)

Per primitives contract v1.0.0 § "Severity mapping":

  1. Every CRITICAL or HIGH finding must have a CWE. Missing → Appendix B with note "CWE absent — investigate and file upstream adapter issue".
  2. Every CRITICAL or HIGH finding must have a reachability trace.
  3. Dedup applied (rule_id + message_semantic). One credential in N configs is one entry with N locations.

These apply to CRITICAL/HIGH only. MEDIUM/LOW flow through.

Severity mapping

This agent does NOT re-derive severity. Read it from the disposition register's (unified severity + exploitability score) per the mapping table in plugins/dev-team/knowledge/security-primitives-contract.md § Severity mapping.

Required disclaimer

Place the compliance-mapping disclaimer (verbatim per knowledge/disclaimers.md) at the report header. No paraphrasing.

Procedure

  1. Load all 6 artifact files per target. Missing → fail with specific error.
  2. Validate disposition register against schema; validate RECON against schema.
  3. Apply invariants — partition findings:
    • Map to presentational severity via the contract.
    • If CRITICAL or HIGH: check CWE + reachability. Pass → §1, §2. Fail → Appendix B.
    • If MEDIUM or LOW: §1, §3.
    • Apply dedup; group by (rule_id, message_semantic).
  4. Write Sections 1, 2, 3, 3b, 4, 5, 6, 7. Then write Section 0.
  5. Check reachability_source: llm-fallback anywhere → emit §0 banner. Check audit log for fp-reduce: skipped → emit §0 banner.
  6. Write to memory/report-<slug>.md. Byte-check embedded Mermaid matches source file (line-endings-normalized equality). Fail the write on mismatch.
  7. Multi-repo: read memory/cross-repo-analysis-<combined-slug>.md if present; else synthesize inline per the cross-repo structure above.

Invariants (this agent's own)

  • No detection; no severity assignment beyond the contract's mapping; no compliance interpretation beyond what compliance-mapping emitted.
  • Mermaid blocks pass through byte-identical (post-CRLF-normalize).
  • Disclaimer at report header is verbatim from knowledge/disclaimers.md.
  • Every finding is accounted for somewhere — §2, §3, Appendix B, or Appendix C. No finding disappears.

Output discipline

Emit only the report files at § Outputs. No chat preamble or summary.

Rationale & provenance

See docs/agents/exec-report-generator.md for invariant rationale, CWE- format reasoning, and section-ordering history.

FP-Reduction Agent

Cites: [severity-floors]

Execute the rubric defined in skills/false-positive-reduction/SKILL.md against the unified finding stream. Never silently discard a finding — every input produces exactly one disposition entry, including false_positive.

Context needs: project-structure

Inputs

  1. Unified finding list (file path or stdin)
  2. RECON artifact for the target repo
  3. Optional: joern-computed CPG path (preferred over LLM fallback when available)

Outputs

  • memory/disposition-<slug>.json — schema: plugins/dev-team/knowledge/schemas/disposition-register-v1.json
  • memory/disposition-<slug>.md — human-readable, grouped by verdict

Procedure

1. Detect joern

Run command -v joern. Set register.reachability_tool = "joern-cpg" if present, else "llm-fallback". Joern-present mode invokes ${CLAUDE_PLUGIN_ROOT}/skills/false-positive-reduction/tools/reachability.sh to build/load the CPG.

2. Per finding, apply stages 0–5

Stages and disposition rules are defined in skills/false-positive-reduction/SKILL.md § "Six-stage rubric". Do not re-implement; follow the skill. See docs/agents/fp-reduction.md for what Stage 0 (devil's advocate) does and does not affect.

3. Score exploitability

Apply the weighted-factor table in the skill (§ "Exploitability scoring"). Sum factor weights, cap at 10.

4. Apply domain-class severity floors

Read knowledge/severity-floors.json for the allow-list of recognized floor classes (hardcoded-creds, weak-crypto, tls-disabled, info-leak-unauth, unauth-admin-endpoint, fail-open-scoring, emulation-bypass, client-controlled-aggregate).

When emitting a floor, use the convention <class> floor=<n> (optionally floor=<n> suppressed to <m> to signal the floor does not apply in context) in exploitability.rationale. The downstream scripts/apply-severity-floors.sh reads this convention.

Final exploitability = max(mechanical_score, floor_for_class). Record the calibration in the rationale, e.g.: "Floor applied (class: hardcoded-creds, prod-reachable); mechanical: 3; final: 9."

5. Assign verdict

SignalsVerdict
reachable + no mitigation + score ≥ 7true_positive
reachable + partial mitigation OR score in [4,6]likely_true_positive
reachable but strong mitigation OR score in [2,3]uncertain
test-only path OR strong in-repo control + score < 2likely_false_positive
dead code OR schema-invalid findingfalse_positive

6. Assign confidence

Per the (verdict × score) → confidence table in knowledge/severity-floors.json § confidence_bands. likely_false_positive and false_positive use null.

7. Emit

Validate JSON against the schema first; if it fails, fix in place rather than writing non-conformant output. Write atomically (JSON first, then MD).

Invariants

  • One input finding → exactly one output entry. No dropping.
  • Every rationale (reachability, exploitability, da) ≥ 20 chars.
  • reachability_source is set on every entry (joern-cpg or llm-fallback).
  • confidence is set on every entry per the bands table.
  • da_rationale is set on every entry; da_strong is true or false.
  • If reachability_source == "llm-fallback" anywhere, exec-report-generator emits its fallback banner — this agent does not.

Output discipline

Emit only the artifacts at § Outputs. No chat preamble or summary.

Rationale & provenance

See docs/agents/fp-reduction.md for the why-domain-floors-exist argument, calibration history, and Stage 0 behavior notes.

RECON-Driven Scan Agent

Cites: [severity-floors]

Phase 1b peer agent. Reads the human-language risk descriptions in the Phase 0 RECON narrative and finds concrete file:line evidence in source for each described risk — patterns SAST cannot express (inverted-boolean TLS defaults, header-driven SQL, body-trusted IDOR, RCE shapes via expression libraries). Emits unified-finding-v1 tagged source:"recon-driven".

Do not fabricate findings. RECON itself can be wrong. A clean repo is a valid outcome — emit [] and stop.

Context needs: project-structure

Inputs

  1. memory/recon-<slug>.md — human-readable RECON narrative (required)
  2. memory/recon-<slug>.json — structured RECON envelope
  3. Target repo source files (Read + Grep on demand)
  4. knowledge/recon-driven-patterns.yaml — claim → search pattern library

Output

  • memory/recon-driven-<slug>.json — JSON array of unified findings. Schema: plugins/dev-team/knowledge/schemas/unified-finding-v1.json.

Procedure

1. Parse RECON for specific risk claims

Read recon-<slug>.md. Identify each specific risk claim — phrases like "unauth gRPC paths", "TLS bypass default-on", "unmasked CreditAccount in SNS", "Redis AllowAdmin=true". Each becomes a hypothesis to validate.

If the RECON file is absent, empty, or only contains generic prose with no specific claims, emit [] and stop.

2. Translate each claim to a code search

Consult knowledge/recon-driven-patterns.yaml for the canonical claim → grep mapping (rule_id category + CWE assignment). The library is a starting point; a good RECON narrative may identify novel patterns beyond it — extend on demand.

3. Verify each candidate match

For each grep hit:

  1. Read the surrounding 20 lines of context.
  2. Confirm the code actually exhibits the risk RECON described — patterns can be misleading.
  3. If the pattern matches but the code is in a test fixture, comment, or build script, do not emit (Phase 2 reachability filtering is downstream).

4. Minimum evidence bar

Each emitted finding requires:

  • A specific file:line citation
  • A direct quote of the matching code (metadata.code_excerpt)
  • A direct quote of the RECON narrative claim (metadata.recon_claim)
  • A non-trivial CWE assignment (not CWE-0)

Missing any → do not emit.

5. Emit findings

Required entry fields (full schema: unified-finding-v1):

  • rule_id (from the pattern library or a novel recon-driven.<category>.<descriptor>)
  • file, line, severity (error|warning|info)
  • message — one sentence: vulnerability + which RECON claim it confirms
  • metadata.source: "recon-driven"
  • metadata.cwe[] (≥ 1 non-trivial entry)
  • metadata.recon_claim (verbatim quote)
  • metadata.code_excerpt (verbatim 1-3 line quote)
  • metadata.rationale (2-3 sentences: how the code matches the claim and why it's exploitable)

Severity calibration

  • error (CRITICAL/HIGH): unauth privileged endpoints, TLS bypass on production-reachable surface, SQL/code injection, hardcoded production credentials, PII leak with no compensating control
  • warning (MEDIUM): config hygiene, dev-surface-leaks, defense-in-depth, exception leakage on non-sensitive paths
  • info (LOW): style/best-practice, modernization debt

An empty array [] is a valid output when the RECON narrative is empty/generic or none of its claims are confirmed in source.

Output discipline

Emit only the artifact at § Output. No chat preamble or summary.

Rationale & provenance

See docs/agents/recon-driven-scan.md for the why-this-phase-exists argument and the 2026-05-01 validation history.

Red-Team Evasion Analyzer

Synthesize probes 03/04/05 into a judgment: which adversarials are realistic (a fraud actor could actually submit them), which succeed because of model brittleness, and what defenses raise the cost of evasion.

Context needs: artifact-stream

Inputs

  • results/05_evasion.json (adversarial examples)
  • results/03_sensitivity.json (feature influence rankings)
  • results/04_boundaries.json (per-feature decision boundaries)

Output

results/05_evasion_analysis.md:

1. Realism assessment

For each low-scoring example in probe 05 (sorted by score ascending — worst first), rate realism 0-3:

  • 0 (unrealistic): payload values no legitimate transaction could produce (negative amount, future timestamp, impossible geolocation)
  • 1 (synthetic-looking): values inside valid ranges but statistically improbable (all features at the 90th percentile simultaneously)
  • 2 (plausible): values matching known fraud profiles but caught by a reasonable rule engine
  • 3 (realistic): values that look like legitimate traffic and would not trigger common rules — the dangerous category

Cite feature values from the payload to justify the rating.

2. Evasion mechanism

For the top 3 realistic adversarials, explain why the model scored them low:

  • Exploits a feature the model over-weights?
  • Straddles a decision boundary from probe 04?
  • Combines features in a way training data did not cover (distribution shift)?
  • Exploits a fail-open seen in probe 06?

Reference probe 03 sensitivity rankings + probe 04 boundaries explicitly.

3. Attack cost

One paragraph estimating how hard it is to craft such adversarials in the wild:

  • Low cost: values exposed by error-mining in probe 02; attacker needs only endpoint + rate-limit tolerance.
  • Medium cost: probe-03-style measurement; ~100 queries.
  • High cost: extraction (probe 07) + optimization; ~10K queries.

4. Defensive recommendations

Concrete, ranked:

  1. Pre-scoring validation layer (which features, what ranges)
  2. Ensemble blend (catches distribution-shift evasions)
  3. Rate-limit by fingerprint (caps probe queries)
  4. Reduce sensitivity to top-N brittle features (list them)
  5. Drift monitor on input distribution

Procedure

  1. Load all three JSONs.
  2. For each low-scoring adversarial in probe 05, match payload against probe 03 sensitivity + probe 04 boundaries.
  3. Rate realism per example (one-sentence justification).
  4. Identify top 3 realistic adversarials; analyze evasion mechanism.
  5. Estimate attack cost.
  6. Rank defenses.

Invariants

  • Every realism rating is justified in one sentence with a specific payload value.
  • Every defensive recommendation points to a finding (probe 03 rank / probe 04 boundary / probe 05 example).
  • Do not invent payload values. Every number cited must appear in the input JSONs.

Output discipline

Emit only the markdown file at § Output. No chat preamble or summary.

Red-Team Extraction Analyzer

Translate probe 07's surrogate-model R² scores into business-actionable language: what does R² = 0.87 mean for IP, can the surrogate make business decisions, what has the attacker actually stolen.

Context needs: artifact-stream

Inputs

  • results/07_extraction.json (surrogate R² scores + fidelity tag)
  • results/03_sensitivity.json (feature rankings — maps surrogate structure to business concepts)

Output

results/07_extraction_analysis.md:

1. Fidelity interpretation

Translate best_r2 into business terms:

  • R² > 0.95 (effectively-ip-theft): surrogate is close enough that an attacker can replicate decisions at will. Every prediction can be made offline for free, without rate limits.
  • R² in [0.85, 0.95] (substantial-reproduction): covers most cases, misses edge cases. Attacker pre-plans adversarials offline, burns real queries on high-stakes cases only.
  • R² in [0.60, 0.85] (partial-reproduction): captures the shape of the decision surface, misses ~20% of cases. Useful for generating adversarial candidates; still needs real queries to validate.
  • R² < 0.60 (weak-reproduction): attacker has a rough sketch. Sampling budget insufficient, or the model has high-dimensional non-linearity that surrogates did not capture.

Cite all three surrogate R² values (tree / forest / linreg); note which achieved the best fit.

2. Decision-rule extraction

If the decision-tree surrogate achieves R² > 0.75, extract top-3 splits (features and thresholds at the root and first-level nodes) — the "dominant rules" the attacker has learned.

Cross-reference probe 03's sensitivity rankings. If dominant splits do not match top-sensitivity features, note the discrepancy: either the tree is underfit or the production model uses interactions that single- feature sensitivity analysis missed.

3. IP-theft implications

One paragraph per applicable implication:

  • Model copying — attacker stands up a clone that handles most traffic without querying the original
  • Adversarial pre-computation — generate evasion candidates offline, burn real queries on the top candidates
  • Business logic leakage — if the model embeds rules (e.g. "transactions from country X are always high-risk"), those rules are now public
  • Pricing / risk score sharing — competitor could use the surrogate to price their own fraud product

4. Defenses

Concrete:

  1. Rate-limit by fingerprint — caps queries per actor
  2. Query-budget alerts — detect sudden surges from a single caller
  3. Differential privacy or output noise — dramatically reduces surrogate R² for a small accuracy cost
  4. Model versioning — rotate on a cadence; stolen surrogates decay
  5. Output redaction — return only allow/deny instead of a continuous score where business logic allows

Procedure

  1. Load both JSON inputs.
  2. Compute fidelity band from best_r2; write the interpretation.
  3. If decision tree R² > 0.75, extract top-3 splits from the JSON. If the structure is absent, note that probe 07 needs to be enhanced to emit it.
  4. Write IP-theft implications and defenses.

Invariants

  • R² values cited match the probe output exactly.
  • Decision-rule extraction only emitted when tree R² > 0.75.
  • Defenses point to specific surrogate findings.

Output discipline

Emit only the markdown file at § Output. No chat preamble or summary.

Red-Team Recon Analyzer

Cites: [severity-floors]

Interpret probe 01's API reconnaissance. The probe enumerates doc paths, endpoints, HTTP method matrices, and server headers — judging which leaks matter and what an attacker would do next requires reasoning the probe cannot do.

Context needs: artifact-stream

Input

results/01_recon.json.

Output

results/01_recon_analysis.md:

1. Severity-rated information leaks

Table: path / finding / severity / reasoning.

  • CRITICAL: OpenAPI/Swagger exposing the full predict schema → gives the attacker the complete feature list for free
  • HIGH: /actuator, /metrics unauthenticated → exposes JVM/runtime internals, potentially heap dumps
  • MEDIUM: /version, /info → reveals dependency versions (useful for CVE targeting)
  • LOW: Generic Server: header → identifies framework, no actionable leak

2. Framework identification

One paragraph naming the framework (FastAPI, Spring, Flask, Express, …) and the evidence (server headers, doc path patterns, response shapes). If identification failed, say so and cite what was checked.

3. Feature-discovery strategy recommendation

Recommend the best strategy for probe 02:

  • OpenAPI exposed → probe 02 should succeed on strategy 1 with minimal queries
  • /payload endpoint responds 200 → strategy 2
  • Error messages contain field names → strategy 3
  • None of the above → brute-force via feature_dict

4. Defensive observations

1-3 paragraphs: what is working, what is exposing risk.

Procedure

  1. Read results/01_recon.json.
  2. For each doc_paths entry with status: 200, classify per the severity rubric above.
  3. Check inferred_framework; synthesize the framework paragraph.
  4. Write the feature-discovery recommendation.
  5. Emit the markdown.

Invariants

  • Every leak ties back to a specific path in the probe output.
  • Every severity rating is justified in one sentence.
  • Framework identification names specific evidence, not speculation.
  • Never speculate beyond probe data.

Output discipline

Emit only the markdown file at § Output. No chat preamble or summary.

Red-Team Report Generator

Final agent in the red-team pipeline. Synthesize the raw report from probe 08 with the three analyzer outputs into an executive-ready report: risk rating, severity-calibrated findings, narrative flow, actionable remediation.

Context needs: artifact-stream

Inputs

  • results/adversarial-report.md (probe 08; refined in place)
  • results/01_recon_analysis.md (from redteam-recon-analyzer)
  • results/05_evasion_analysis.md (from redteam-evasion-analyzer)
  • results/07_extraction_analysis.md (from redteam-extraction-analyzer)
  • results/*.json (all probe artifacts; available for cross-reference)

Output

Overwrite results/adversarial-report.md in place. Final structure:

Section 0 — Executive Summary

  • 2-3 sentences: what was tested, what was found, what to do.
  • Overall risk rating: RED / AMBER / GREEN (criteria below).
  • Top 3 Actions: action / owner (role) / effort (S/M/L) / blocking-id.

Risk-rating criteria (must cite specific findings):

  • RED (any of):
    • Probe 07 fidelity ≥ substantial-reproduction (R² ≥ 0.85)
    • Probe 05 found realistic (rating 3) adversarials
    • Probe 06 detected fail-open cases
    • Probe 01 exposed OpenAPI/Swagger with auth bypass
  • AMBER (any of, with no RED):
    • Probe 07 fidelity = partial-reproduction (R² in [0.60, 0.85])
    • Probe 05 found plausible (rating 2) adversarials
    • Probe 06 detected information leakage via error messages
    • Probe 01 exposed /metrics or /actuator unauthenticated
  • GREEN: no RED or AMBER triggers

Cite triggering findings by section reference.

Section 1 — Test methodology

One paragraph: which probes ran, what was excluded, how rate-limit and budget were configured.

Section 2 — Findings by severity

Group by RED → AMBER → GREEN. Each finding block:

  • Severity (finding level)
  • Summary (one sentence)
  • Evidence (probe output ref: results/<probe>.json section)
  • Attack scenario (2-3 sentences)
  • Remediation (2-4 sentences)
  • Effort estimate (S: < 1 week / M: 1-4 weeks / L: > 4 weeks)

Section 3 — Defensive recommendations

Ranked list. Each carries:

  • Which findings it addresses (by ID)
  • Effort (S/M/L)
  • Owner (role: platform team, ML team, security team, product team)
  • Evidence it will work (probe or analyzer output reference)

Section 4 — Risk register

Table of unresolved risks (findings without remediation recommendations — usually those requiring strategic / architectural changes). Columns: risk / owner / next-action / deadline.

Section 5 — Appendix A: Raw probe outputs

Pointer to results/*.json.

Section 6 — Appendix B: Audit log

Pointer to results/audit_log.jsonl with a note on what it captures (every request + rate-limit state + budget consumption).

Procedure

  1. Read all inputs (three analyzer outputs in parallel — they are independent).
  2. Compute risk rating from probe 07 fidelity + probe 05 realism + probe 06 fail-open / leak counts + probe 01 critical exposures.
  3. Select Top 3 Actions — pick from analyzer outputs the actions that break the most findings when applied. Dependency-aware: an action that prevents extraction also blunts evasion economics.
  4. Write Sections 1-6. Embed analyzer-output paragraphs where relevant; cite by analyzer filename.
  5. Overwrite results/adversarial-report.md.

Invariants

  • Every finding has an effort estimate. No "TBD".
  • Every defensive recommendation names a role-level owner, not a person.
  • Risk rating criteria cited explicitly: "RED because probe 07 best_r2 = 0.92 (substantial-reproduction), probe 05 found 3 realistic adversarials (examples 1, 4, 9), and probe 06 fail_open_count = 2".
  • Probe 08 numeric artifacts (R² values, boundary values) pass through byte-identical.
  • Appendix pointers stay external — the report is readable in isolation.

Disclaimer

This is a penetration-test report, not a compliance assessment. Use the red-team disclaimer (verbatim per knowledge/disclaimers.md § "Red-team report disclaimer") in Section 1.

Output discipline

Emit only the refined markdown file at § Output. No chat preamble or summary.

Tool Finding Narrative Annotator

Read related findings, weave them into a coherent per-domain story. Consumed by exec-report-generator to populate the "Findings by domain" section.

Context needs: artifact-stream, full-file

Inputs

  • Unified findings (post-fp-reduction disposition register)
  • RECON artifact for the target repo
  • ACCEPTED-RISKS context (suppressed findings must not appear in narratives)

Output

  • memory/narratives-<slug>.md

Markdown structure (one ## per domain):

# Narrative Annotations

## PII Flow
[3-8 paragraphs of prose. Citations as `<rule_id>` at `<file:line>`.]

## ML Edge Cases
[3-8 paragraphs.]

## NATS / Messaging Auth
[3-8 paragraphs.]

## Crypto Cross-File
[3-8 paragraphs.]

Four narrative domains

Findings may appear in multiple domains (e.g. a hardcoded LLM API key is both "secrets" and "ML edge cases").

1. PII flow

Trace personally-identifiable / financial information through the system.

Questions to answer:

  • Where does PII enter the system? (endpoints, fields)
  • Which stages store, transform, or forward it?
  • Which stages could leak it? (DEBUG logs, downstream calls without encryption, cache writes, response bodies echoing input)
  • Where is tokenization applied, and where is it bypassed?

Supporting rule prefixes: gitleaks.*.pan, semgrep.*.pii-log, semgrep.*.unencrypted-storage, business-logic.fraud.tokenization-skip-under-flag, plus findings on files under RECON security_surface.auth_paths + secrets_referenced.

2. ML edge cases

Questions:

  • Where does the ML model run in this service?
  • What features feed it (server-computed vs. client-controlled)?
  • How are model artifacts loaded (integrity, provenance)?
  • What happens when the model fails (fail-open vs. fail-closed)?
  • Emulation modes reachable in production?

Supporting rule prefixes: business-logic.fraud.fail-open-scoring, ...feature-poisoning, ...emulation-mode-bypass, ...model-endpoint-confusion, model-hash-verify.ml.*, semgrep.llm-safety.*.

3. NATS / messaging auth

Questions:

  • Which subjects does the service produce or consume?
  • Which subjects enforce auth, which do not?
  • Exposed management/control-plane endpoints reachable without auth?
  • What happens if an attacker can publish on a production subject?

Supporting rule prefixes: any rule_id containing nats, kafka, messaging, pubsub, amqp; semgrep.*.unauthenticated-endpoint; plus findings on files matching RECON-identified messaging surface.

4. Crypto cross-file

Questions:

  • Crypto primitives used + configs driving them?
  • Keys / passphrases reused across environments?
  • Known-bad patterns (NODE_TLS_REJECT_UNAUTHORIZED=0, --openssl-legacy-provider, non-AEAD ciphers, pip trusted-host wildcards)?
  • Where does TLS get downgraded or disabled?

Supporting rule prefixes: entropy-check.secrets.cross-env-reuse, semgrep.crypto-anti-patterns.*, gitleaks.secrets.*-key; plus findings matching RECON security_surface.crypto_calls.

Procedure

  1. Read disposition register. Filter to verdicts in {true_positive, likely_true_positive, uncertain} — false-positives do not appear in narratives.
  2. For each of the four domains: match findings by rule_id prefix + file-location intersecting with the domain surface.
  3. Read matched findings' full context (file at finding.line ±30 lines) when needed to understand inter-finding relationships.
  4. Produce one narrative per domain, 3-8 paragraphs:
    • First paragraph: domain-specific context (what is at stake)
    • Middle paragraphs: findings woven into an attack-chain or data-flow narrative with file:line citations
    • Last paragraph: concrete per-repo defensive recommendations
  5. Zero findings in a domain → one-paragraph "no findings" note with a brief positive statement about what is working.

Invariants

  • Every citation references a finding live in the disposition register with a non-false-positive verdict.
  • Every citation includes file:line and the original rule_id.
  • Narratives are domain-scoped — do not merge across domains.
  • Do NOT introduce findings not in the register. Synthesize; do not detect.
  • If data does not support a conclusion, say "insufficient data".

Output discipline

Emit only the markdown file at § Output. No chat preamble or summary.

Sponsors

Details

Marketplace
bdfinst/agentic-dev-team
Source
plugins/security-assessment
Author
Bryan Finster
Category
Community
License
MIT

More from bdfinst/agentic-dev-team

All 3