
security-assessment
Deep security assessment + adversarial ML red-team: SARIF-first tool orchestration, narrowly-scoped LLM agents, FP-reduction with fallback banner, compliance mapping, service-comm diagramming, and a self-owned-target red-team harness. Companion plugin to dev-team. Previously published as agentic-security-assessment.
- version
- 3.7.6
- GitHub stars
- 304
Install
/plugin install security-assessment@bfinsterRun it inside Claude Code after adding the marketplace with /plugin marketplace add bdfinst/agentic-dev-team.
Contents
Discovered from sourceSkills 3
Compliance Mapping (pattern-first)
Purpose
Map unified findings to regulatory control citations so executive-audience reports can name the specific regulations at risk. Designed to be deterministic first — a pattern table handles the bulk; LLM judgment is invoked ONLY for cases the table flags as ambiguous.
This is explicitly informational, not audit-grade. No report produced by this pipeline substitutes for a certified auditor's opinion. The disclaimer is mandatory and exact-wording.
Inputs
- Disposition register (post-fp-reduction unified findings with verdicts)
knowledge/compliance-patterns.yaml(this plugin) — the pattern table- Optional: target organization's pre-declared control scope (which regulations apply)
Output
A compliance-annotations.json file per the following shape:
{
"schema_version": "1.0",
"generated_at": "2026-04-21T10:00:00Z",
"disclaimer": "<verbatim text from knowledge/disclaimers.md § Compliance mapping disclaimer>",
"annotations": [
{
"finding_rule_id": "semgrep.python.hardcoded-password",
"finding_file": "config/prod.py",
"finding_line": 42,
"regulations": [
{"regulation": "PCI-DSS", "control_id": "3.2.1", "citation": "PCI-DSS v4.0 §3.2.1"},
{"regulation": "GDPR", "control_id": "Art. 32", "citation": "GDPR Article 32 — Security of Processing"}
],
"annotator": "pattern-table" // or "llm" if llm_review_trigger fired
}
]
}
Pattern table schema (knowledge/compliance-patterns.yaml)
Each row maps a finding pattern to one or more regulatory citations.
# plugins/security-assessment/knowledge/compliance-patterns.yaml
patterns:
- id: pan-at-log
pattern_regex: 'log\.(debug|info).*(pan|card_number|primary_account)'
field_type: "pii.pan"
applies_to_rules:
- "semgrep.*.pii-log"
- "business-logic.fraud.tokenization-skip-under-flag"
regulations:
- regulation: "PCI-DSS"
control_id: "3.4"
citation: "PCI-DSS v4.0 §3.4 — PAN must be rendered unreadable in storage and in logs"
- regulation: "PCI-DSS"
control_id: "10.2"
citation: "PCI-DSS v4.0 §10.2 — Audit logs must not contain sensitive authentication data"
llm_review_trigger: false
- id: unencrypted-db-transit
pattern_regex: "(mongodb|postgres|mysql)://[^@]+@[^/]+/[^?]+($|(?!.*ssl|.*tls))"
field_type: "db.transit"
applies_to_rules:
- "semgrep.*.unencrypted-database-connection"
- "trivy.iac.*"
regulations:
- regulation: "PCI-DSS"
control_id: "4.1"
citation: "PCI-DSS v4.0 §4.1 — Strong cryptography and security protocols on open public networks"
- regulation: "GDPR"
control_id: "Art. 32"
citation: "GDPR Article 32 — Security of Processing"
llm_review_trigger: false
- id: auth-bypass-admin
pattern_regex: ".*admin.*"
field_type: "auth.missing"
applies_to_rules:
- "semgrep.*.missing-csrf"
- "semgrep.*.unauthenticated-endpoint"
- "business-logic.fraud.*"
regulations:
- regulation: "PCI-DSS"
control_id: "8.3"
citation: "PCI-DSS v4.0 §8.3 — Multi-factor authentication for admin access"
- regulation: "SOC2"
control_id: "CC6.1"
citation: "SOC2 Trust Services Criteria CC6.1 — Logical access controls"
llm_review_trigger: true # rules in this class often need case-specific judgment
- id: insecure-random
pattern_regex: ".*"
field_type: "crypto.random"
applies_to_rules:
- "semgrep.*.insecure-random"
regulations:
- regulation: "PCI-DSS"
control_id: "3.6.1"
citation: "PCI-DSS v4.0 §3.6.1 — Cryptographic key generation practices"
llm_review_trigger: false
Schema fields per row:
| Field | Required | Type | Purpose |
|---|---|---|---|
id | yes | string | Stable identifier |
pattern_regex | yes | string | Regex applied to the finding's message / file / context |
field_type | yes | string | Categorical tag (pii.pan, auth.missing, crypto.random, etc.) for audit traceability |
applies_to_rules | yes | list[string] | Rule-id glob patterns this row matches |
regulations | yes | list[object] | One or more citation objects |
llm_review_trigger | no | bool (default false) | If true, LLM edge annotator is invoked for findings matched by this row |
Procedure
1. Load and validate the pattern table
Parse YAML. Every row MUST have the required fields. An invalid row fails the run with a named error pointing to the row id.
2. For each finding in the disposition register
Apply rows in file-declaration order:
- Check
applies_to_rulesglobs againstfinding.rule_id. No match → skip row. - Check
pattern_regexagainstfinding.message+finding.file(concatenated). No match → skip row. - Match: record annotation with the row's regulations. Set
annotator: "pattern-table". - If
llm_review_trigger: true, invoke thecompliance-edge-annotatoragent for this finding only. The agent can refine citations, add or remove regulations, or add a judgment note. Setannotator: "llm"if the agent modified anything.
A finding with zero matching rows gets no annotation and does NOT appear in the output.
3. LLM call counting (for eval)
The skill exposes an optional LLMCallCounter interface for eval assertions:
class LLMCallCounter:
def invoke(self, prompt: str, context: dict) -> str: ...
def count(self) -> int: ...
def reset(self) -> None: ...
Production dispatches go through a real LLM. Eval runs inject a mock counter and assert the count matches the expected number of llm_review_trigger: true matches in the fixture.
4. Write output
Write memory/compliance-<slug>.json with the disclaimer at the root. The exec-report-generator includes the disclaimer verbatim in the report header.
Disclaimer
Use the exact wording in knowledge/disclaimers.md § "Compliance mapping disclaimer". Required on every report.
Invariants
- Pattern table is the source of truth. LLM never invents citations; it only refines or annotates table-matched citations.
- LLM call count is bounded: one call per finding with
llm_review_trigger: true. No loops, no recursion. - Schema-invalid pattern rows fail the run. No silent skipping.
- Every annotation carries
annotator: "pattern-table" | "llm"for audit. - The disclaimer appears verbatim in every compliance output. Downstream must not strip it.
Graph-assisted discovery
If the target repo has .codegraph/ (CodeGraph MCP server — codegraph_explore/get_symbol for fast callers/callees/impact lookups) and/or a Repowise MCP server (get_context/search_codebase for semantic search, get_risk for modification-risk context), prefer them over raw Grep/Read — useful when the compliance-edge-annotator agent needs to inspect the finding's surrounding code to refine a citation. Never assume either tool is present — fall back to Read/Grep/Glob when neither is present; the tools are simply unavailable (no error) on repos without an index.
Related
agents/compliance-edge-annotator.md— the sonnet-tier agent invoked forllm_review_trigger: truematchesknowledge/compliance-patterns.yaml— the pattern table itselfplugins/dev-team/knowledge/security-primitives-contract.md— unified finding envelope
False-Positive Reduction (hybrid joern + LLM)
Purpose
Transform a stream of unified findings into a disposition register that the exec-report-generator can trust. Every finding gets a verdict (true_positive | likely_true_positive | uncertain | likely_false_positive | false_positive), a reachability trace, an exploitability score, and a reachability_source tag (joern-cpg or llm-fallback).
The skill's job is to remove noise without suppressing real issues. False positives waste analyst attention; missed true positives get someone fired.
Six-stage rubric (applied in order; each stage can downgrade severity or change verdict)
Lifted from the opus_repo_scan_test reference's § analyze-11 framework with extensions for the disposition-register output format. Stage 0 is new: a self-adversarial pre-pass that sharpens Stage 1 and strengthens the audit trail.
Stage 0 — Devil's advocate
Question: What is the strongest argument that this finding is NOT a vulnerability?
The agent generates a counter-argument before applying the rubric. This is not a skip gate — all five subsequent stages still run. The purpose is twofold:
- Sharpen Stage 1: a strong counter-argument gives Stage 1 a concrete hypothesis to test (is the path actually dead / test-only?) rather than an open-ended search.
- Strengthen the audit trail: a
true_positivethat explicitly refuted a counter-argument is more trustworthy than one that never examined the counter-case. A well-reasonedfalse_positiveis more trustworthy than a silent discard.
Counter-argument prompts:
- Framework/runtime protection: does the tech stack have a built-in prevention for this class (ORM parameterization, template auto-escaping, TLS termination at the LB)?
- Trusted caller: is this code only reachable from internal, trusted, or admin-only paths?
- Non-production context: is the file a migration, test fixture, seed script, or utility that RECON's
entry_pointsdon't include? - Rule pattern noise: does this rule commonly fire on intentional non-exploitable configurations?
Disposition rules:
- Strong counter-argument →
da_strong: true; Stage 1 tests the hypothesis - Weak / no counter-argument →
da_strong: false; Stage 1 performs open-ended reachability search da_strong: true+ Stage 1 confirms (unreachable) →false_positive; both arguments cited in rationaleda_strong: true+ Stage 1 disproves (reachable) → rejected counter-argument cited intrue_positiverationale
Stage 1 — Reachability
Question: Is this code executed in production at all?
Disposition rules:
- Dead code (no inbound call graph from any entry point) →
verdict: false_positive, severity →infopresentational. - Test-only paths (only reached from test code) →
verdict: likely_false_positive, severity → one level down (CRITICAL → HIGH, HIGH → MEDIUM). - Feature-flagged-off in all production configs → one level down, verdict stays
true_positive. - Reached from production entry point → no change; record the entry point in
reachability.rationale.
Joern-present mode: reachability is computed from the CPG by tracing back from the finding location to HTTP/CLI/lambda/cron entry points.
Joern-absent mode: the agent reasons from RECON's entry_points and security_surface fields, plus grep over the call sites. Tag each entry with reachability_source: llm-fallback.
Stage 2 — Environment context
Question: Could deployed configuration override the committed value, making the finding inert?
Disposition rules:
- Confirmed override at deploy time (e.g. env var in
values.yamlor Helm chart overrides a committed default) → one level down,verdict: likely_true_positive(the committed value is still a weak default). - No override found → full severity, verdict unchanged.
The agent consults docker-compose*.yml, values.yaml, helmfile.yaml, k8s/*.yaml, and any CI-scoped env vars discoverable in .github/workflows/* or GitLab equivalents.
Stage 3 — Compensating controls
Question: Is there a control in the repo that mitigates this finding's impact?
Disposition rules:
- Confirmed in-repo control (WAF rule, rate limiter, input validation layer upstream of the finding, idempotency key check, etc.) → one level down, verdict
likely_true_positivewith the control's file:line in the rationale. - Assumed-only ("we have a WAF in prod" — not verifiable from the repo) → no change.
- Absent → no change.
Stage 4 — Deduplication
Question: Is this the same root cause as another finding already in the register?
Disposition rules:
- Same rule_id + same value (e.g. same secret SHA-256) across multiple files → collapse to ONE finding with a
locationsarray; emit one disposition entry referencing the primary finding. - Same rule_id + different values (e.g. 14 different hardcoded passwords) → separate findings, NOT deduplicated.
- Different rule_ids that describe the same root cause (e.g.
semgrep.python.hardcoded-password+gitleaks.generic.aws-access-keyfiring on the same line) → dedupe keeping the higher-priority source per the static-analysis skill's priority order.
Stage 5 — Severity calibration
Question: Is the severity consistent across similar findings?
Disposition rules:
- Ensure two findings with identical exploitability profiles receive identical presentational severity across the run.
- If a finding falls between two severity levels, prefer the higher — better to over-flag than miss. Use exploitability score (0–10) to break ties deterministically per the severity-mapping table in the primitives contract.
Graph-assisted discovery
If the target repo has .codegraph/ (CodeGraph MCP server — codegraph_explore/get_symbol for fast callers/callees/impact lookups) and/or a Repowise MCP server (get_context/search_codebase for semantic search, get_risk for modification-risk context), prefer them over raw Grep/Read when tracing call graphs. This benefits Stage 1 reachability analysis most directly: following inbound calls from a finding location back to production entry points is exactly the callers/impact query these tools are built for, and is especially useful in Joern-absent (LLM-fallback) mode where reachability is otherwise reasoned from grep over call sites. Never assume either tool is present — fall back to Read/Grep/Glob when neither is present; the tools are simply unavailable (no error) on repos without an index.
Exploitability scoring (0–10)
Per-finding score determines presentational severity bucket (see primitives contract § Severity mapping). Factors:
| Factor | Weight | Example |
|---|---|---|
| Network reachability | +3 | Finding is in an HTTP handler on a public route |
| Authentication bypass | +3 | Finding bypasses an auth check (not merely missing one) |
| Credential exposure | +2 | Finding leaks a credential an attacker could use elsewhere |
| Input-controlled | +2 | An attacker can influence the vulnerable value via request parameters |
| Persistent | +1 | Finding creates persistent state (stored XSS, stored credentials) |
| Privileged context | +1 | Finding runs in an elevated context (root, admin route) |
| Cascading | +1 | A successful exploit unlocks further access (lateral movement) |
Rationale field is mandatory (min 20 chars per schema). Summarize which factors applied and why.
Joern integration (when present)
If joern is on PATH, invoke via ${CLAUDE_PLUGIN_ROOT}/skills/false-positive-reduction/tools/reachability.sh (build commands + CPG cache details are in the script). The wrapper detects the target's primary language (C#/.NET, JS/TS, Python, Java, Go — first match wins) and passes the matching joern-parse --language, so non-default frontends like C# build a CPG instead of silently degrading to LLM-fallback; the CPG cache is keyed <sha>.<language>.cpg. The C# frontend needs joern ≥ 2.0.0 plus the dotnetastgen tool — the install scripts provision and verify it (scripts/verify-joern-csharp.sh). Stage 1 reachability queries the CPG for paths from the finding location back to entry points; cite the entry point path in reachability.rationale.
LLM-fallback mode (joern absent)
Stages 1–3 use judgment rather than CPG data; Stages 4–5 work unchanged. Every entry in fallback mode carries reachability_source: llm-fallback. The exec-report-generator detects this and emits a banner — see agents/exec-report-generator.md § Section 0 banners.
Output
A DispositionRegister object per plugins/dev-team/knowledge/schemas/disposition-register-v1.json. Required entry fields and required envelope fields (schema_version, generated_at, dispositioner, reachability_tool, entries[]) are defined in the schema. Written to memory/disposition-<assessment-slug>.json.
Related
agents/fp-reduction.md— the opus agent that implements this skillplugins/dev-team/knowledge/security-primitives-contract.md— disposition register schema + severity mappingplugins/dev-team/knowledge/schemas/disposition-register-v1.json— JSON Schematools/reachability.sh— joern wrapper (installed if joern is on PATH)
Security Assessment Pipeline
The /security-assessment command executes this multi-phase pipeline over
one or more target repos. This skill is authoritative for phase order,
dependencies, artifacts, and failure semantics.
Phase graph
Key properties:
- Multi-target: each target's Phase 0 → Phase 2b runs independently in parallel. Only Phase 4 (cross-repo) and Phase 5 (exec report) join across targets.
- Phase 4 runs concurrently with Phase 1b – Phase 3 once Phase 0 finishes — it only depends on Phase 0 output.
- Phase timing: every phase is bracketed with
scripts/phase-timer.sh start/endsomemory/phase-timings-<slug>.jsonlrecords actual wall times. exec-report-generator surfaces this data and flags drift from intended parallelism.
┌─────────────────────┐
│ Phase 0: Recon │
│ (parallel per │
│ target) │
└──────────┬──────────┘
│
┌────────────────────────┼────────────────────────┐
│ │ │
▼ ▼ ▼
┌─────────────────────┐ ┌──────────────────┐ ┌──────────────────────┐
│ Phase 1+1b │ │ Phase 4 (cross- │ │ (anything else │
│ (parallel across │ │ repo: service- │ │ Phase-0-dependent) │
│ tools AND agents) │ │ comm parser; │ └──────────────────────┘
└──────────┬──────────┘ │ multi-target │
│ │ only) │
▼ └──────────────────┘
┌─────────────────────┐
│ Phase 1c: SEQUENTIAL│
│ ACCEPTED-RISKS │
└──────────┬──────────┘
▼
┌─────────────────────┐
│ Phase 2: SEQUENTIAL │
│ fp-reduction │
└──────────┬──────────┘
▼
┌─────────────────────┐
│ Phase 2b: severity │
│ floors │
└──────────┬──────────┘
▼
┌─────────────────────┐
│ Phase 3: narratives │
│ + compliance │
│ (parallel) │
└──────────┬──────────┘
▼
┌─────────────────────┐
│ Phase 5: exec report│
│ (single agent) │
└─────────────────────┘
Per-phase spec
Phase 0: Reconnaissance
agent: codebase-recon (opus)
produces: memory/recon-<slug>.{json,md}
parallelism: parallel across targets
Phase 1: Tool-first detection (parallel across tools)
skill: static-analysis-integration
produces: memory/findings-<slug>.jsonl (unified finding stream)
requires: Phase 0
parallelism: all tool × target × ruleset combinations run concurrently
Phase 1b: Judgment-layer detection (parallel across agents)
agents: security-review, business-logic-domain-review, deep-code-reasoning,
authorization-logic-review, recon-driven-scan (opus all five)
produces: adds unified findings to memory/findings-<slug>.jsonl
requires: Phase 0, Phase 1
parallelism: all five agents dispatched in a single Agent tool message,
repeated per target
adapters: see knowledge/phase-1b-adapters.md
Phase 1c: ACCEPTED-RISKS suppression (sequential gate, mandatory)
procedure: scripts/apply-accepted-risks.sh parses the first fenced
```json block in ACCEPTED-RISKS.md, matches findings by
(rule_id exact, source_ref_glob), and suppresses matches
produces: memory/accepted-risks-<slug>.jsonl (suppressed + expired
entries) + rewrites memory/findings-<slug>.jsonl in place
requires: Phase 1 + Phase 1b
enforced: exec-report-generator's Appendix C expects
memory/accepted-risks-<slug>.jsonl when ACCEPTED-RISKS.md
was present at target root
Phase 2: FP-reduction (sequential)
agent: fp-reduction (opus)
produces: memory/disposition-<slug>.json
requires: Phase 1 + Phase 1b + Phase 1c
optional: skipped when --fp-reduce=no
Phase 2b: Domain-class severity floors (sequential, deterministic)
script: ${CLAUDE_PLUGIN_ROOT}/scripts/apply-severity-floors.sh
produces: memory/severity-floors-log-<slug>.jsonl; rewrites
memory/disposition-<slug>.json with floor-adjusted scores
requires: Phase 2
optional: no-op when Phase 2 was skipped
Phase 3: Narrative + compliance (parallel across agents)
agent: tool-finding-narrative-annotator (sonnet)
skill: compliance-mapping
produces: memory/narratives-<slug>.md, memory/compliance-<slug>.json
requires: Phase 2b (or Phase 1+1b if fp-reduction skipped)
parallelism: dispatched together in one Agent tool message
Phase 4: Service-communication (runs concurrently with Phase 1b-3)
tools: ${CLAUDE_PLUGIN_ROOT}/harness/tools/service-comm-parser.py
${CLAUDE_PLUGIN_ROOT}/harness/tools/shared-cred-hash-match.py (multi-target only)
produces: memory/service-comm-<slug>.mermaid,
memory/shared-creds-<slug>.sarif
requires: Phase 0 only
Phase 5: Report generation (sequential)
agent: exec-report-generator (opus)
produces: memory/report-<slug>.md (+ memory/cross-repo-summary-<slug>.md for multi-repo)
requires: Phase 2b + Phase 3 + Phase 4 (joined)
Phase 5b: Severity-consistency check (multi-target only, deterministic)
script: ${CLAUDE_PLUGIN_ROOT}/scripts/check-severity-consistency.sh
produces: memory/severity-consistency-<combined-slug>.txt
requires: Phase 5 + Phase 2 disposition registers
condition: skipped for single-target runs
Phase 5c: Report verification (per target, deterministic)
script: ${CLAUDE_PLUGIN_ROOT}/scripts/verify-report.sh
produces: memory/verify-report-<slug>.txt
requires: Phase 5
on-fail: logs to audit trail; does NOT block publication
Canonical phase names for phase-timer.sh: phase-0-recon,
phase-1-tool-first, phase-1b-judgment, phase-1c-accepted-risks,
phase-2-fp-reduction, phase-2b-severity-floors,
phase-3-narrative-compliance, phase-4-cross-repo, phase-5-report.
Graph-assisted discovery
If a target repo has .codegraph/ (CodeGraph MCP server — codegraph_explore/get_symbol for fast callers/callees/impact lookups) and/or a Repowise MCP server (get_context/search_codebase for semantic search, get_risk for modification-risk context), prefer them over raw Grep/Read when locating candidate files and tracing call paths. This benefits the recon-driven scanning step (Phase 1b's recon-driven-scan agent, bridging RECON narrative claims to concrete file:line evidence) most directly — graph-backed callers/impact lookups and semantic search surface exactly the file:line evidence that step needs. Never assume either tool is present — fall back to Read/Grep/Glob when neither is present; the tools are simply unavailable (no error) on repos without an index.
Helper-script invocation contract
Deterministic phase helpers carry strict ordering requirements. The orchestrator must not reorder these invocations:
Invocation paths (discoverable in an installed plugin). Every helper below ships inside this plugin and MUST be invoked by its plugin-root path so it resolves regardless of the caller's working directory — the bare names in the table are shorthand for these:
- helper scripts →
${CLAUDE_PLUGIN_ROOT}/scripts/<name>(e.g.${CLAUDE_PLUGIN_ROOT}/scripts/phase-timer.sh) - harness tools →
${CLAUDE_PLUGIN_ROOT}/harness/tools/<name>
| Script | Must run after | Must run before | Artifacts |
|---|---|---|---|
${CLAUDE_PLUGIN_ROOT}/scripts/phase-timer.sh start/end | any phase boundary | — | phase-timings-<slug>.jsonl |
${CLAUDE_PLUGIN_ROOT}/scripts/find-ci-files.sh | phase-0-recon | phase-1-tool-first | stdout (Phase 1 tool fan-out) |
${CLAUDE_PLUGIN_ROOT}/scripts/apply-accepted-risks.sh | phase-1b-judgment | phase-2-fp-reduction | rewrites findings-<slug>.jsonl; writes accepted-risks-<slug>.jsonl |
${CLAUDE_PLUGIN_ROOT}/scripts/apply-severity-floors.sh | phase-2-fp-reduction | phase-3-narrative-compliance | rewrites disposition-<slug>.json in place; writes severity-floors-log-<slug>.jsonl |
Concurrency: each script assumes a single writer on its input file.
.tmp+mv handles the single-writer-but-crash case. Concurrent writers
are a contract violation. Callers MUST serialize invocations against the
same slug.
Idempotency: apply-severity-floors.sh and apply-accepted-risks.sh
are both idempotent (marker-based and atomic-rewrite-based respectively)
— safe to re-run from --start.
Invocation
/security-assessment <path> # single-repo (default)
/security-assessment <path1> <path2> [...] # multi-repo
/security-assessment <path> --start 3 # resume from Phase 3
/security-assessment <path> --agents 0 1b # run only listed phases
/security-assessment <path> --fp-reduce=no # skip Phase 2
Flag semantics
--start <phase>: resume from0|1|1b|2|3|4|5. Skill validates required artifacts exist; dependency check is SKIPPED (operator asserts preconditions).--agents <phase-list>: run only listed phases. Dependency check skipped.--fp-reduce=no: skip Phase 2. Phases 3 and 5 run on the raw finding stream; exec report carries banner "FP-reduction skipped; findings may contain false positives. Review Appendix B before acting." Default:yes.
Failure handling
Per-phase best-effort continuation:
- Phase 0: stops the pipeline. RECON is the precondition.
- Phase 1 / 1b: failed tool/agent logged; pipeline continues to Phase 2 with partial stream. Phase 5 names the failed detectors.
- Phase 2: continue with raw stream; tag
fp-reduce: failed. Phase 5 banner names the failure. - Phase 3: narratives + compliance marked "unavailable"; Phase 5 continues.
- Phase 4: service-comm diagram replaced with a one-line note; Phase 5 continues.
- Phase 5: failure reported; all produced artifacts remain in
memory/for manual assembly.
Artifacts
Every phase writes to memory/<kind>-<slug>.<ext>. Slug is derived from
target repo name (or --joined for multi-repo).
| Kind | Producer | Consumer |
|---|---|---|
recon-<slug>.json | Phase 0 | Phase 1, 1b, 2, 3, 5 |
findings-<slug>.jsonl | Phase 1, 1b | Phase 2, 3 |
deep-reasoning-<slug>.json | Phase 1b | Phase 1b append step |
authz-review-<slug>.json | Phase 1b | Phase 1b append step |
recon-driven-<slug>.json | Phase 1b | Phase 1b append step |
disposition-<slug>.json | Phase 2 | Phase 3, 5 |
narratives-<slug>.md | Phase 3 | Phase 5 |
compliance-<slug>.json | Phase 3 | Phase 5 |
service-comm-<slug>.mermaid | Phase 4 | Phase 5, /cross-repo-analysis |
report-<slug>.md | Phase 5 | Final output |
cross-repo-summary-<slug>.md | Phase 5 (multi-repo) | Final output |
severity-consistency-<combined>.txt | Phase 5b | exec-report Section 6 |
verify-report-<slug>.txt | Phase 5c | Audit trail; surfaced on FAIL |
Invariants
- Every phase is idempotent within a run — same inputs → same outputs.
- Artifact writes are atomic — partial writes produce an
.incompletesuffix so the pipeline can detect and reject stale half-written artifacts. - The pipeline never deletes prior artifacts.
--startresumes from existing; a new run without--startarchives tomemory/archive/<timestamp>/. - Audit log at
memory/audit-<slug>.jsonlis append-only; records phase start/end, artifact produced, and failures.
Not covered
- Red-team pipeline —
harness/redteam/orchestrator.py. - Cross-repo analysis as a standalone command —
/cross-repo-analysis. - PDF rendering —
/export-pdf.
Commands 5
/cross-repo-analysis
You have been invoked with the /cross-repo-analysis command.
Role
Orchestrator for cross-repo security analysis. Composes deterministic tool outputs (service-comm-parser, shared-cred-hash-match) with the cross-repo-synthesizer agent's narrative synthesis.
Constraints
- Minimum two target repos. Below that, this command is a no-op — use
/security-assessmentfor a single repo. - No detection. This command assumes
/security-assessmenthas already run per repo (or RECON has been generated per repo). It does not re-scan. - Mermaid passthrough. The diagram from
service-comm-parser.pyflows through unchanged to the final report.
Parse arguments
Arguments: $ARGUMENTS
Positional: two or more directory paths. Each must contain memory/recon-<slug>.json (produced by an earlier codebase-recon run) or the command refuses.
Steps
1. Verify RECON per repo
For each path, locate memory/recon-*.json. If any target path lacks a RECON artifact, stop and tell the user: "Target has no RECON artifact — run codebase-recon or /security-assessment <path> first."
2. Run service-comm-parser.py
python3 "${CLAUDE_PLUGIN_ROOT}/harness/tools/service-comm-parser.py" <path1> <path2> ... > memory/service-comm-<slug>.mermaid
Where <slug> is a dash-joined concatenation of target repo names.
3. Run shared-cred-hash-match.py
python3 "${CLAUDE_PLUGIN_ROOT}/harness/tools/shared-cred-hash-match.py" <path1> <path2> ... > memory/shared-cred-<slug>.sarif
The SARIF output is consumed by the synthesizer agent; it is NOT passed through the unified-finding SARIF parser at this stage (findings are shared-credential hashes, not code-level issues).
4. Dispatch cross-repo-synthesizer
Using the Agent tool, dispatch the cross-repo-synthesizer agent with inputs:
- List of RECON artifact paths (one per repo)
- Path to the Mermaid file from step 2
- Path to the shared-cred SARIF from step 3
The agent produces memory/cross-repo-analysis-<slug>.md. Wait for completion.
5. Present summary
Print to stdout:
Cross-repo analysis complete.
Target repos: <list>
Mermaid diagram: memory/service-comm-<slug>.mermaid
Shared credentials: memory/shared-cred-<slug>.sarif (<N> groups, <M> total occurrences)
Named attack chains: <count from the synthesizer output>
Full report: memory/cross-repo-analysis-<slug>.md
Integration
- Typically run after
/security-assessmenthas completed per repo (so disposition registers exist). - The exec-report-generator reads
memory/cross-repo-analysis-<slug>.mdwhen producing the cross-repo summary report (one of the four reference outputs).
Escalation
Stop and ask the user when:
- Only one target path is passed (requires at least two).
- Any target path lacks a RECON artifact.
service-comm-parser.pyorshared-cred-hash-match.pyfails with a non-zero exit code — surface the error; do not silently swallow.
/export-pdf
You have been invoked with the /export-pdf command.
Role
Converter from Markdown to PDF. Uses pandoc when available (cleaner tables, wider Markdown support); falls back to weasyprint when pandoc is absent; skips with a warning when neither is installed.
Used for:
- Executive reports from
/security-assessment(memory/report-<slug>.md) - Red-team reports from
/redteam-model(harness/redteam/results/adversarial-report.md) - Cross-repo summaries from
/cross-repo-analysis - Any Markdown produced by this plugin's agents
Parse arguments
Arguments: $ARGUMENTS
Positional: <report.md> — path to the input Markdown file.
Flags:
--output <path>: output PDF path. Default: same basename with.pdfextension.--css <path>: custom CSS stylesheet. Default:plugins/security-assessment/templates/report-css/default.css.
Steps
1. Validate inputs
- Input file exists and is readable.
- If
--outputis passed, its parent directory exists. - If
--cssis passed, the CSS file exists.
2. Detect available converter
Check in order:
pandoc+wkhtmltopdf(or pandoc with--pdf-engine=weasyprintif weasyprint is installed).weasyprintwith a small markdown-to-html wrapper.- Neither → skip.
3. Convert
Preferred (pandoc):
pandoc "$INPUT" \
--pdf-engine=weasyprint \
--css="$CSS" \
--standalone \
--metadata title="$(basename "$INPUT" .md)" \
-o "$OUTPUT"
Fallback (weasyprint + python-markdown):
The input Markdown is derived from a scanned repo / probed model (see
probe_08_report_generator.py, service-comm-parser.py) — treat it as
untrusted. Rendered HTML/CSS can reference remote resources (<img src="https://...">, CSS url(...)), and weasyprint fetches those by
default, which is a plausible SSRF vector at PDF-conversion time. Pass a
url_fetcher that blocks every non-local scheme so conversion never makes
an outbound network request:
python3 -c "
import sys, markdown
from weasyprint import HTML, CSS
def _no_remote_fetch(url, *args, **kwargs):
if not url.startswith(('file://', 'data:')):
raise ValueError(f'blocked remote resource fetch during PDF export: {url}')
from weasyprint import default_url_fetcher
return default_url_fetcher(url, *args, **kwargs)
with open('$INPUT') as f:
md_text = f.read()
html_body = markdown.markdown(md_text, extensions=['tables', 'fenced_code', 'codehilite'])
html = f'<html><head><meta charset=\"utf-8\"></head><body>{html_body}</body></html>'
HTML(string=html, url_fetcher=_no_remote_fetch).write_pdf(
'$OUTPUT', stylesheets=[CSS(filename='$CSS')]
)
"
4. Report
On success:
PDF written: /absolute/path/to/report.pdf
Print the absolute path — some downstream tooling expects an absolute path.
On graceful skip (neither tool):
SKIP: pandoc and weasyprint both absent; PDF not produced.
Install one:
brew install pandoc weasyprint (macOS)
apt install pandoc (Linux)
pip install weasyprint (any)
The markdown report remains at: $INPUT
Exit 0 in both cases (skip is not a failure).
Escalation
Stop and ask the user when:
- Input file does not exist.
- Output directory is not writable.
- CSS file specified by
--cssdoes not exist (default CSS should always exist; if missing, report as a plugin install issue).
CSS stylesheet
The bundled CSS at plugins/security-assessment/templates/report-css/default.css is modelled on the opus_repo_scan_test reference's utilities/convert_report.py CSS:
- A4 page size, 2cm × 1.5cm margins
- Page footer: confidentiality notice (center) + page N of M (right)
- Red accent (
#c00) on H1 underline and blockquote border - Zebra-striped tables
- Monospace code blocks; syntax highlighting via
codehilite - Muted color scheme, no decorative images
Override via --css <path> for organization-specific branding.
Integration
- Runs after
/security-assessmentproducesmemory/report-<slug>.md. - Runs after
/redteam-modelproducesharness/redteam/results/adversarial-report.md. - Not run automatically — user invokes at their discretion.
/redteam-model
Orchestrator entry point for the adversarial ML red-team harness. Perform
scope + consent checks, then dispatch the Python orchestrator at
${CLAUDE_PLUGIN_ROOT}/harness/redteam/orchestrator.py. Interpret probe artifacts via the four
analyzer agents after the Python passes complete.
Full runtime spec — including bash invocations, exact refusal wording,
flag-interaction matrix, and step output formats — is in
${CLAUDE_PLUGIN_ROOT}/harness/redteam/ORCHESTRATION.md. Read it before dispatching.
Safety constraints (non-negotiable)
- Scope-enforced by default. Targets must resolve to a self-owned
CIDR (
127.0.0.0/8,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,::1). Public targets require--self-certify-owned <path>. - Self-certification is logged. Artifact's SHA-256 is written to
harness/redteam/results/audit_log.jsonlwith the run timestamp. - Rate-limit + budget enforced. All HTTP goes through
http_clientwhich enforcesRATE_LIMITreq/sec andQUERY_BUDGETtotal queries. Exhaustion halts the pipeline withbudget_exhaustedstatus. - Harness-direct invocation blocked. The
redteam-guard.shhook refuses directpython orchestrator.pycalls unlessREDTEAM_AUTHORIZED=1is in env. This command sets it after the scope/consent checks pass; scoped to the child process only.
Parse arguments
Arguments: $ARGUMENTS
Positional: <target-url> (base URL, no trailing slash).
Flags:
--self-certify-owned <path>: authorization artifact for public targets. Format:knowledge/redteam-authorization.md.--dry-run: validate config + scope + consent; print step graph; zero HTTP requests. Scope enforcement still runs.--agents <id> [<id> ...]: run only these probe IDs (01-08).--start <id>: resume from this probe ID (token from prior run'sresult_store.resume_message()).
Flag interactions are in ${CLAUDE_PLUGIN_ROOT}/harness/redteam/ORCHESTRATION.md § "Flag
interactions".
Procedure
Execute the steps in ${CLAUDE_PLUGIN_ROOT}/harness/redteam/ORCHESTRATION.md:
- Resolve and validate scope (Step 1)
- Validate config (Step 2)
- Dispatch the Python orchestrator (Step 3)
- Dispatch analyzer agents (Step 4) — three in parallel, then
redteam-report-generatorlast - Present summary (Step 5)
Escalation
Stop and ask the user per ${CLAUDE_PLUGIN_ROOT}/harness/redteam/ORCHESTRATION.md § "Escalation".
Integration
- Paired with
/security-assessmentfor static analysis (independent pipelines; can run against the same or different targets). - Paired with
/export-pdffor the final report. redteam-guard.shPreToolUse hook enforces the authorization gate.knowledge/redteam-authorization.mddeclares--self-certify-ownedartifact format.
/security-assessment
Orchestrator for the static-analysis pipeline. Execute the phase graph
defined in skills/security-assessment-pipeline/SKILL.md. Phases dispatch
to agents/scripts; artifacts pass through memory/. Produces a
publication-ready exec report per target (plus a cross-repo summary for
multi-target runs).
Constraints
- Follow the pipeline exactly. The skill is authoritative. Do not reorder phases or skip deterministic gates.
- Never silently drop findings. Every input flows through to either the published report or a suppression appendix with reason.
- Artifacts are the source of truth. Every phase writes to
memory/;--startand failure-recovery rely on that. - Informational-not-audit-grade. Every produced report carries the
compliance disclaimer (verbatim per
knowledge/disclaimers.md) at the header.
Parse arguments
Arguments: $ARGUMENTS
Positional: one or more directory paths (target repos).
Flags:
--start <phase>: resume from phase (0 / 1 / 1b / 2 / 3 / 4 / 5). Requires prior phase artifacts inmemory/.--agents <phase> [<phase> ...]: run only listed phases. Dependency check skipped.--fp-reduce=yes|no: skip Phase 2 FP-reduction whenno. Defaultyes.
Steps
1. Validate arguments
- ≥ 1 target path required.
- Each target must be a directory (not a file).
- For each target: consult
ACCEPTED-RISKS.mdif present at the target root.
2. Initialize run
For each target repo, derive a slug from the directory name (kebab-cased, lowercased). Multi-repo runs use dash-joined slug for cross-repo artifacts.
Create memory/audit-<slug>.jsonl (append-only). Record run start with
targets, flags, and contract version. If prior artifacts exist AND
--start is NOT set, archive them to memory/archive/<timestamp>/
before overwriting.
3. Execute the phase graph
Follow skills/security-assessment-pipeline/SKILL.md § "Phase graph".
The skill defines per-phase inputs, outputs, agents/scripts, and
parallelism. Three parallelism rules MUST be observed:
- Multi-target fan-out — each target's Phase 0 → Phase 2b runs as an independent pipeline; dispatch as parallel Agent tool calls in the SAME message.
- Intra-phase fan-out — Phase 1b dispatches its five agents
(
security-review,business-logic-domain-review,deep-code-reasoning,authorization-logic-review,recon-driven-scan) in one message; Phase 3 dispatches narrative + compliance in one message; Phase 1's static-analysis-integration runs every tool concurrently. - Phase 4 runs concurrently with Phase 1b–3 once Phase 0 finishes. Phase 4 depends only on Phase 0.
Per-phase agent / script dispatch + adapter wiring is documented in the skill. Do not reorder. Helper-script invocation contract (which script runs after which phase boundary) is in the skill § "Helper-script invocation contract".
4. Phase timing
Every phase is bracketed with
${CLAUDE_PLUGIN_ROOT}/scripts/phase-timer.sh start <phase> <slug> /
${CLAUDE_PLUGIN_ROOT}/scripts/phase-timer.sh end <phase> <slug>. Writes accumulate in
memory/phase-timings-<slug>.jsonl. The exec-report-generator reads this
file to compute actual parallelism vs. sequential execution.
5. Surface summary
Print:
Security assessment complete.
Target(s): <list>
Phases run: <list of phase numbers>
Artifacts:
memory/recon-<slug>.{json,md}
memory/findings-<slug>.jsonl (<N> unified findings)
memory/disposition-<slug>.json (<N> entries, <X>% true/likely-true)
memory/narratives-<slug>.md
memory/compliance-<slug>.json (<N> annotations, <M> LLM-triggered)
memory/service-comm-<slug>.mermaid
memory/report-<slug>.md (CRITICAL: <N>, HIGH: <N>, MEDIUM: <N>, LOW: <N>)
(+ memory/cross-repo-summary-<slug>.md for multi-repo)
Run `/export-pdf <report>.md` for PDF.
Run `/cross-repo-analysis <path1> <path2>` for cross-repo chain analysis
if not already included.
Escalation
Stop and ask the user when:
- No target paths are provided.
- Any target path is not a directory.
--startis set but required precondition artifacts are missing.- Phase 0 (recon) fails on any target — no meaningful downstream without it.
- More than 3 phases fail in a single target — pipeline output is no longer trustworthy; escalate rather than emit a misleading report.
Integration
- Phase graph:
skills/security-assessment-pipeline/SKILL.md - Cross-repo: pair with
/cross-repo-analysis - PDF: pair with
/export-pdf - Primitives:
plugins/dev-team/knowledge/security-primitives-contract.md
Upgrade
Role: worker. This command updates the security-assessment plugin to the latest version and ensures its marketplace is set to auto-update going forward.
You have been invoked with the /upgrade command.
Steps
1. Read current version
claude plugin list
Parse the output to find security-assessment and its current version.
Also read the installed plugin.json directly:
~/.claude/plugins/cache/*/security-assessment/*/.claude-plugin/plugin.json
Report:
Current version: security-assessment v{version} (installed from {marketplace})
2. Check auto-update status and ask the user
First, check the current auto-update status by running the script below and reporting it to the user.
python3 - <<'PY'
import json, os
PLUGIN = "security-assessment"
home = os.path.expanduser("~")
cwd = os.getcwd()
def load(path):
try:
with open(path) as f:
return json.load(f)
except FileNotFoundError:
return None
except json.JSONDecodeError as e:
print(f"unknown ({path} is not valid JSON)")
raise SystemExit(0)
installed = (load(os.path.join(home, ".claude", "plugins", "installed_plugins.json")) or {}).get("plugins", {})
market = next((pid.split("@", 1)[1] for pid in installed if pid.split("@", 1)[0] == PLUGIN and "@" in pid), None)
if not market:
print("unknown (marketplace not found)")
raise SystemExit(0)
candidates = [
os.path.join(cwd, ".claude", "settings.json"),
os.path.join(cwd, ".claude", "settings.local.json"),
os.path.join(home, ".claude", "settings.json"),
]
for path in candidates:
data = load(path)
if data and market in (data.get("extraKnownMarketplaces") or {}):
flag = data["extraKnownMarketplaces"][market].get("autoUpdate")
print("enabled" if flag is True else "disabled")
raise SystemExit(0)
print("disabled")
PY
Then ask the user:
Auto-update for the
{marketplace}marketplace is currently {enabled/disabled}. Would you like to enable auto-update so future releases install automatically? (yes/no)Note: this flag is marketplace-level and affects every plugin published from
{marketplace}— including the companion dev-team plugin if you have it installed.
Wait for the user's answer before continuing. If they say yes, run the enable block below. If they say no, skip to step 3.
Enable block (run only if the user consents):
python3 - <<'PY'
import json, os
PLUGIN = "security-assessment"
home = os.path.expanduser("~")
cwd = os.getcwd()
def load(path):
try:
with open(path) as f:
return json.load(f)
except FileNotFoundError:
return None
except json.JSONDecodeError as e:
print(f" ! {path} is not valid JSON ({e}); skipping")
return None
installed = (load(os.path.join(home, ".claude", "plugins", "installed_plugins.json")) or {}).get("plugins", {})
market = next((pid.split("@", 1)[1] for pid in installed if pid.split("@", 1)[0] == PLUGIN and "@" in pid), None)
if not market:
print(f" ! Could not resolve the marketplace for '{PLUGIN}'; skipping.")
raise SystemExit(0)
candidates = [
os.path.join(cwd, ".claude", "settings.json"),
os.path.join(cwd, ".claude", "settings.local.json"),
os.path.join(home, ".claude", "settings.json"),
]
target = None
for path in candidates:
data = load(path)
if data and market in (data.get("extraKnownMarketplaces") or {}):
target = [path, data]
break
if target is None:
reg = load(os.path.join(home, ".claude", "plugins", "known_marketplaces.json")) or {}
entry = reg.get(market)
if not entry:
print(f" ! Marketplace '{market}' not found in settings or registry; skipping.")
raise SystemExit(0)
path = os.path.join(home, ".claude", "settings.json")
data = load(path) or {}
data.setdefault("extraKnownMarketplaces", {})[market] = {"source": entry["source"]}
target = [path, data]
path, data = target
mk = data["extraKnownMarketplaces"][market]
if mk.get("autoUpdate") is True:
print(f" auto-update already enabled for '{market}' ({path})")
else:
mk["autoUpdate"] = True
with open(path, "w") as f:
json.dump(data, f, indent=2)
f.write("\n")
print(f" enabled auto-update for '{market}' in {path}")
print(" (takes effect on next Claude Code launch / plugin operation)")
PY
Report the one-line result to the user, then continue to step 3.
3. Run the update
First, determine the install scope from claude plugin list output
(the Scope: line for security-assessment). It will be one of:
user, project, local, managed.
claude plugin update --scope {scope} security-assessment@{marketplace}
Where {scope} is the detected install scope (e.g., project) and
{marketplace} is the marketplace name (e.g., bfinster). The
--scope flag is required — the CLI defaults to user, which will
fail if the plugin is installed at a different scope.
If the command succeeds with a version change, proceed to step 4.
If the output indicates already up to date:
Already running the latest version (v{version}).
Exit.
If the command fails, report the error and suggest:
Update failed. You can try a manual reinstall:
claude plugin uninstall --scope {scope} security-assessment@{marketplace} claude plugin install --scope {scope} security-assessment@{marketplace}
Exit.
4. Check companion dependency
If dev-team@{marketplace} is not installed, surface the dependency:
WARNING:
dev-team@{marketplace}is not installed but is required bysecurity-assessment(primitives contract, codebase-recon agent, SARIF parser). Install it with:claude plugin install --scope {scope} dev-team@{marketplace}
This is informational — the security-assessment update has already succeeded. The user can act on it after restart.
5. Confirm the update
claude plugin list
Report:
## Upgrade Complete
Previous: security-assessment v{old_version}
Updated: security-assessment v{new_version}
Restart Claude Code to apply the update.
Notes
- The
claude plugin updatecommand handles fetching, caching, and version management. Previous versions are kept for 7 days so active sessions continue working. - A restart of Claude Code is required for the new version to take effect.
- Step 2 enables marketplace-level auto-update by writing
extraKnownMarketplaces.<marketplace>.autoUpdate: truetosettings.json— the same flag the/pluginUI toggles. With it on, routine releases for both plugins in the marketplace (dev-teamandsecurity-assessment) land without running/upgradeagain.
Agents 13
Authorization Logic Review
Cites: [severity-floors]
Map the authorization model first, then check consistent enforcement. Report the gap between stated policy and observed implementation — not single suspicious lines.
Context needs: project-structure
Inputs
- Target repo files — RECON-scoped or grep-discovered
memory/recon-<slug>.json— entry points + security surfaceknowledge/authz-review-categories.yaml— rule_id + CWE assignments
Output
memory/authz-review-<slug>.json— JSON array of unified findings. Schema:plugins/dev-team/knowledge/schemas/unified-finding-v1.json.
Required metadata: source: "llm-reasoning", cwe[], confidence
(high|medium only), secondary_locations[] (≥ 1 policy + gap pairing),
reasoning (2-3 sentences).
Procedure
1. Map the authorization model
Discover how the application declares and enforces access control:
- Route decorators / middleware annotations (
@require_auth,@roles_allowed,[Authorize(Roles=...)],router.use(authMiddleware)) - Permission constants and role definitions (
permissions.py,roles.js,AuthorizationPolicy.cs,scopes.go) - Middleware stacks (Express chain, Django middleware list, ASP.NET Core pipeline)
- Tenancy models (
tenant_id,organization_id,account_idin models or query builders)
Classify as RBAC, ABAC, ACL, tenancy-scoped, or mixed. Note where the predominant pattern is declared.
2. Identify enforcement points
Per route or operation class, note where authorization is enforced:
- Controller / handler — checked before business logic runs
- Service layer — checked inside business logic
- Repository / data-access — enforced in the query
(e.g.
.where(tenant_id=current_tenant)) - Not found — no enforcement located
Gaps in this {operation → enforcement} map are findings.
3. Tenant isolation consistency
If a tenancy model is present:
- Grep direct object-load patterns that could return cross-tenant data:
findById,getById,SELECT ... WHERE id = ?without a tenant filter. - Check whether ORM base query builder / repository base class enforces tenancy (global scope = OK; ad-hoc per-query = risky).
- Note admin or superuser paths that bypass tenancy for legitimate reasons — acknowledged bypasses, not findings.
4. Role/permission escalation paths
- Can a low-privilege user update fields that determine their own role/permissions?
- Are role assignments validated server-side on every mutation, or only at creation time?
- Is admin promotion / impersonation gated on a separate high-privilege check, not just "is authenticated"?
5. Cross-service authorization propagation
If RECON identifies inter-service calls (S2S HTTP, gRPC, message queue):
- Does the receiving service re-verify authorization?
- Are S2S credentials separate from user credentials?
- Can a user indirectly trigger privileged S2S operations via user-facing inputs?
6. Minimum evidence bar
A finding requires at least two specific code locations — the policy declaration and the location where it is violated or absent. Single- location suspicions are discarded.
Categories
Use rule_ids from knowledge/authz-review-categories.yaml. Severity and
confidence calibration are in the same file.
An empty array [] is valid — not every codebase has authorization gaps.
Output discipline
Emit only the JSON file at § Output. No chat preamble or summary.
Rationale & provenance
See docs/agents/authorization-logic-review.md for the top-down vs.
bottom-up split and the two-location-evidence rule.
Business-Logic Domain Review
Read the codebase with fraud-detection domain knowledge. Surface issues that require cross-file reasoning — bugs in WHAT the code does, not HOW. Static tools catch syntax; this agent catches business logic.
Context needs: project-structure
Inputs
- Target repo (walked on demand)
memory/recon-<slug>.json— scoping + entry points + security surfaceknowledge/domain-logic-patterns.md— full catalogue of 9 patterns with grep cues, exploit scenarios, and remediation pointers. Read before scanning.
Output
memory/business-logic-findings-<slug>.json— unified findings. Schema:plugins/dev-team/knowledge/schemas/unified-finding-v1.json.
Required fields per finding:
rule_idin the formbusiness-logic.fraud.<category>(e.g.business-logic.fraud.fail-open-scoring,business-logic.fraud.messaging-subject-injection,business-logic.fraud.messaging-subscriber-poisoning,business-logic.fraud.training-data-inference)metadata.source: "business-logic-domain-review"metadata.confidence:high | mediumonly — neverlowmetadata.exploitabilitywhen reasoning supports itmetadata.cwe[]when applicable (CWE-754, CWE-841, CWE-840, CWE-200, etc.)metadata.attack_scenario— 2-3 sentencesmetadata.source_ref.path_class: "test" | "production" | "unknown"(downstream uses this; do not filter on it here)
Detection patterns
Nine patterns catalogued in knowledge/domain-logic-patterns.md. Summary:
- Fail-open scoring
- Score manipulation / client-controlled features
- Emulation-mode bypass
- Model-endpoint confusion
- Tokenization / PII-masking skip
- Feature poisoning
- Missing replay idempotency
- Messaging attack surface (subject injection / subscriber poisoning / missing replay protection)
- Training data inference from metrics/logs
The knowledge file is authoritative for grep cues, exploit scenarios, remediation pointers, and CWE assignments per pattern.
Procedure
- Read RECON. Focus on files under
security_surface.auth_paths,security_surface.network_egress, and any files whose path containsscore,fraud,predict,model,risk,decision. - For each in-scope file, grep for the cues in
knowledge/domain-logic-patterns.md. Read ±30 lines of context. - For each confirmed instance, emit a unified finding. Message = one-line
summary; full attack scenario goes in
metadata.attack_scenario. Setconfidencebased on context verification. - Tag
metadata.source_ref.path_class(test/production/unknown). Do not filter test-only paths — that is fp-reduction's job.
Invariants
- Every finding traces to a code location (file + line).
- Every finding has a 2-3 sentence attack scenario in
metadata.attack_scenario. confidence: highonly when the pattern is present AND there is no obvious mitigation in a ±30-line window. Otherwisemedium.- Never emit
lowconfidence — if not confident, do not emit.
Output discipline
Emit only the JSON file at § Output. No chat preamble or summary.
Rationale & provenance
See docs/agents/business-logic-domain-review.md for the why-static-tools-
miss-this argument and the messaging-pattern split rationale.
Compliance Edge Annotator
Single-turn. No loops. Invoked once per triggering finding by the
compliance-mapping skill when a matched pattern row has
llm_review_trigger: true.
Context needs: full-file
Inputs (passed by the skill)
finding— full unified finding being annotatedpattern_row— matched compliance-patterns.yaml row (with base citations)matched_code_context— ±20 lines at the finding locationff_context— RECON's relevant sections (endpoints, auth paths)
Output
Strict JSON, no markdown fences, no prose outside the note field:
{
"finding_rule_id": "<finding.rule_id>",
"refinements": {
"keep": ["PCI-DSS:8.3", "SOC2:CC6.1"],
"remove": ["PCI-DSS:3.4"],
"add": []
},
"note": "<1-2 sentence judgment: why keep/remove/add>",
"confidence": "high | medium | low"
}
The skill merges this with the pattern row's base citations to produce the final annotation.
Rules
- Never invent a citation. The
addlist draws ONLY from the closed regulation set: PCI-DSS, GDPR, HIPAA, SOC2, NIST (SP 800-*, AI RMF), EU-AI-Act, OWASP, CWE. If none of these clearly apply beyond what the pattern row gave,addis empty. - Removal requires a reason in
note. If you cannot articulate why a base citation does not apply, keep it. confidence: lowwhen the judgment is guesswork. The skill treats low-confidence annotations as "defer to pattern table" —remove/addare ignored;keepis taken as-is.- Strict JSON output. Schema-invalid output fails the run; do not try to recover.
When to invoke (skill-side)
The skill invokes this agent when a pattern row matches a finding AND the
row has llm_review_trigger: true. Typical triggering classes:
admin-endpoint-unauth: context matters (/debugbehind VPN ≠ public/admin)pii-in-response-body: field-level judgment (emailmight be the user's own;ssnrarely is)model-integrity: whether the integrity gap crosses a compliance line depends on the model's role
Rate / cost
One LLM call per triggering finding. The skill caps total calls at the number of triggering findings; overage fails the run and logs the discrepancy.
For eval runs, LLMCallCounter counts exact invocations so tests can
assert "exactly 1 LLM call for this fixture".
Invariants
- Single-turn. No follow-up queries to the user.
- Output validates as JSON. Schema-invalid output fails the run.
- Untouched base citations (
keeponly, noremove, noadd) appear in the final annotation unchanged — the safe path. confidenceis always set.
Disclaimer
Pipeline-wide compliance disclaimer (verbatim per
knowledge/disclaimers.md § "Compliance mapping disclaimer") is applied
by the skill to every output containing annotations.
Output discipline
Emit only the JSON object at § Output. No chat preamble or summary.
Cross-Repo Synthesizer
Read aggregated data from multiple repos. Name cross-repo attack chains explicitly. Cite findings; do not invent.
Invoked by /cross-repo-analysis after service-comm-parser.py and
shared-cred-hash-match.py have produced their outputs.
Context needs: artifact-stream
Inputs
- One RECON artifact per target repo (
memory/recon-<slug>.json) - Per-repo disposition register (
memory/disposition-<slug>.json) if/security-assessmentran per repo; else unified findings directly - Service-comm Mermaid diagram (stdout from
service-comm-parser.py) - Shared-cred SARIF (stdout from
shared-cred-hash-match.py)
Output
memory/cross-repo-analysis-<assessment-slug>.md:
- Overview — 1 paragraph naming target repos and analysis scope.
- Shared credentials — table: hash prefix, count of repos affected, SHA-256 log reference.
- Inter-service communication diagram — Mermaid block from
service-comm-parser.pyembedded byte-identical. - Named attack chains — 3-10 chains, each with:
- Short name (e.g. "Credential reuse → privileged NATS → model endpoint")
- Findings cited by rule_id + file:line
- Step-by-step walk (attacker action → state → next action)
- Repos involved + each service's role
- Systemic patterns — 1-3 paragraphs on organizational tendencies (e.g. "no repo uses a secrets manager; all use env-file credentials").
- Defensive priorities — ranked, with cross-repo coordination notes.
Procedure
-
Load every RECON, the Mermaid (read node + edge labels), and shared- cred SARIF.
-
Build the cross-index:
- Credential hash → repos + files + lines
- Service → role (publisher / subscriber / both) per NATS subject
- Service → entry points with auth status
- Service → service dependencies (from Mermaid edges)
-
Name attack chains. Candidate patterns: credential-reuse, unauthenticated- messaging, model-confusion, feature-poisoning. A chain is meaningful only when ≥ 2 repos are involved AND the chain advances the attacker's position. If none qualify, say so — do not invent.
-
Identify systemic patterns (≥ 2 repos exhibiting the same tendency).
-
Rank defensive priorities:
- P1: fixes that break ≥ 2 named chains
- P2: fixes that address a systemic pattern
- P3: per-repo fixes
Each priority names affected chains/patterns + a role-level owner.
Invariants
- Every chain cites findings by rule_id + file:line. Uncited chains are not emitted.
- Embedded Mermaid is byte-identical to
service-comm-parser.pyoutput — no reformatting, no re-rendering. - Shared-credential SHA-256 values are printed as first 12 hex chars only (full hash in audit log).
- Recommendations name roles, never individuals.
- Do not detect findings — synthesize from existing only.
Output discipline
Emit only the markdown file at § Output. No chat preamble or summary.
Rationale & provenance
See docs/agents/cross-repo-synthesizer.md for the chain-meaningfulness
threshold, citation discipline, and no-individual-ownership rule.
Deep Code Reasoning
Phase 1b peer agent. Catches vulnerabilities that require cross-file reasoning — IDOR, confused deputy, TOCTOU, indirect privilege escalation, workflow bypass. Scope discipline is mandatory: read only RECON- identified surface.
Context needs: project-structure
Inputs
memory/recon-<slug>.json—entry_points,security_surface.auth_paths,security_surface.sensitive_data_flows(required)- Target repo files at RECON-identified paths (on demand; load scoped files + immediate callers/callees only)
Output
memory/deep-reasoning-<slug>.json— JSON array of unified findings. Schema:plugins/dev-team/knowledge/schemas/unified-finding-v1.json.
Required metadata: source: "llm-reasoning", cwe[] (≥ 1),
confidence (high|medium only — never low), secondary_locations[]
(≥ 1), reasoning (2-3 sentences tracing the attack path).
Scope extraction
Read memory/recon-<slug>.json:
entry_points[]— HTTP handlers, CLI, cron, event consumerssecurity_surface.auth_paths[]security_surface.sensitive_data_flows[]
If security_surface is missing, grep for common auth indicators
(@require_auth, hasPermission, isAuthorized, checkRole,
verify_token, @login_required, [Authorize]) and use the matching
files as the working surface. Document the fallback in the first entry's
metadata.
Detection categories (rule_id prefixes)
llm-reasoning.idor.<descriptor>— object-level authorization bypassllm-reasoning.function-level-authz.<descriptor>— function-level auth gapllm-reasoning.confused-deputy.<descriptor>— confused deputy / SSRF via delegationllm-reasoning.toctou.<descriptor>— time-of-check-time-of-usellm-reasoning.privilege-escalation.<descriptor>— indirect privilege escalationllm-reasoning.workflow-bypass.<descriptor>— business logic / state machine bypass
OWASP cross-reference: broken access control (A01:2021), confused deputy, TOCTOU across service boundaries, indirect privilege escalation, business logic bypass.
Procedure
1. Load and bound the surface
Extract from RECON or grep fallback. If the surface exceeds 30 files, priority order: auth_paths → entry_points → sensitive_data_flows. Process the top 30 only; note the truncation.
2. Read and trace each surface item
For each file in scope:
- Read the file.
- Find callers (grep exported names; read the top 3 by reference count).
- Find security-sensitive callees: for data-access, permission, or state-transition ops, read one level deeper.
Do not recurse further without a specific reason tied to an active hypothesis.
3. Apply minimum evidence bar
Advance to output only when you can cite at least two specific code
locations (file:line) that together constitute the vulnerability. See
docs/agents/deep-code-reasoning.md for paired-evidence examples.
4. Severity
error— reachable from a public entry point; directly enables privilege escalation or data access bypasswarning— reachable but requires additional conditions, or only reachable from authenticated pathsinfo— pattern present but exploit viability unclear without runtime context
5. Confidence (mandatory; two values only)
high— full attack path traceable with no gaps; every step has a citationmedium— path is plausible but one step requires an assumption (note it explicitly inreasoning)
Never emit low.
6. Write output
JSON array. Empty [] is valid and expected when the scoped surface
yields no confirmed findings — do not manufacture findings to fill the
file. Validate each entry carries all required metadata before writing.
Output discipline
Emit only the JSON file at § Output. No chat preamble or summary.
Rationale & provenance
See docs/agents/deep-code-reasoning.md for scope-discipline rationale,
paired-evidence examples, and the no-low-confidence policy.
Executive Report Generator
Final agent in /security-assessment. Synthesizes narrative from upstream
artifacts. Does not detect, disposition, or score. Maps presentational
severity (CRITICAL/HIGH/MEDIUM/LOW) per primitives contract v1.0.0.
Context needs: artifact-stream
Inputs (per target repo)
memory/recon-<slug>.{json,md}— Phase 0memory/findings-<slug>.jsonl— Phase 1 + 1b raw findingsmemory/disposition-<slug>.json— Phase 2 disposition register (absent when--fp-reduce=no)memory/narratives-<slug>.md— Phase 3 narrativesmemory/compliance-<slug>.json— Phase 3 compliance annotationsmemory/service-comm-<slug>.mermaid— Phase 4 diagram
Multi-repo: also memory/cross-repo-analysis-<combined-slug>.md (if produced).
Outputs
memory/report-<slug>.md(filename:<repo-name>-security-assessment.md)memory/cross-repo-summary-<combined-slug>.md(multi-repo only; filename:cross-repository-security-summary.md)
Seven-section per-repo structure
Section 0 — Executive Summary
One page, business terms. Written LAST (Top 3 Actions depend on §2 and §5).
- 2-3 sentence overview: scope, dominant risk category, recommended next step.
- Top 3 Actions table: action / owner (role) / effort (S/M/L) / blocking-id.
- Presentational severity tally:
CRITICAL: N HIGH: N MEDIUM: N LOW: N. - Banners (verbatim text required):
- FP-reduction skipped: "FP-reduction skipped; findings may contain false positives. Review Appendix B before acting."
- LLM-fallback reachability (any disposition entry with
reachability_source: llm-fallback): "Reachability stage used LLM reasoning instead of call-graph analysis; dead-code paths may be less accurate. Stages 2–5 unaffected."
Section 1 — Findings Dashboard
One table, all findings (post-disposition), grouped by presentational severity. Columns: ID, Rule, File:Line, Category, Severity, Verdict, Confidence.
- Confidence: read from
disposition.confidence.nullconfidence (likely_false_positive / false_positive) entries are excluded. - CWE format (dashboard): number-only. Single:
CWE-NNN. Multiple:CWE-NNN + CWE-MMM. Separator is always+, never/.
Section 2 — CRITICAL and HIGH Findings
One detail block per finding. Contains: Summary (1 sentence), Location
(file:line), CWE (invariant — every C/H must have CWE), Confidence,
Reachability trace (from disposition.reachability.rationale), Attack
scenario (2-3 sentences), Remediation (2-4 sentences), Compliance
citations.
CWE format (Section 2): CWE-NNN — Full CWE Name. Multiple:
CWE-NNN — Name + CWE-MMM — Name (em dash before name; + between).
Section 3 — MEDIUM and LOW Findings
One row per finding: summary sentence + remediation pointer.
CWE format (Section 3): number-only. **CWE-NNN** or
**CWE-NNN + CWE-MMM**.
Section 3b — Cross-Cutting Concerns (multi-repo only)
Omit for single-repo. Multi-repo: ≤ 1 page summarizing for THIS repo:
- Shared credentials: read
memory/shared-creds-<combined-slug>.sarif; list entries where one oflocationsbelongs to this slug. Format: secret type, value (truncated to 4 chars +…), other repos involved. - Attack chains involving this repo: read
memory/cross-repo-analysis-<combined-slug>.md; include only chains whose step list names this slug. Format: name, one-line summary, severity. - Regulatory gaps specific to this repo: from
compliance-<slug>.json, any annotation withregulation_risk: highnot already in §2 or §3.
Point to the cross-repo summary for depth.
Section 4 — Service Communication Diagram
Embed the Mermaid block from service-comm-parser.py byte-identical
(post-CRLF-normalize). Do not re-render.
Section 5 — Remediation Roadmap
P1 / P2 / P3 / P4 buckets. Each entry: owner (role), effort estimate, blocking finding IDs. P1 = do today; P4 = informational.
Section 6 — Methodology and Scope
Brief statement of what was and was not assessed. Detailed spec
(timing-table assembly, agent→phase mapping, drift-detection messages,
coverage-gap callouts, cross-repo severity calibration) is in
knowledge/exec-report-section6-spec.md — read it on demand.
Section 7 — Appendices
- Appendix A — Secrets inventory (gitleaks + entropy-check)
- Appendix B — Findings missing CWE or reachability (invariant violations)
- Appendix C — Suppressed findings (ACCEPTED-RISKS matches). Read
memory/accepted-risks-<slug>.jsonl. IfACCEPTED-RISKS.mdexisted at target root but the file is absent or empty, emit: "ACCEPTED-RISKS.md was present at the target root but the Phase 1c suppression gate did not run — findings flow through without suppression. Re-run the pipeline or investigate why Phase 1c was skipped." - Appendix D — Compliance annotations (full list; disclaimer at top
verbatim per
knowledge/disclaimers.md§ "Compliance mapping disclaimer") - Appendix E — File inventory (from RECON)
Cross-repo summary structure
When multiple targets assessed, generate
cross-repo-summary-<slug>.md:
- Top 3 cross-repo actions
- Shared risk patterns (issues appearing in ≥ 2 repos)
- Cross-repo attack chains (from
/cross-repo-analysisif available, else synthesized inline) - Aggregated inter-service Mermaid (embed verbatim from
service-comm-parser.py) - Compliance roll-up
- Consolidated risk matrix
Invariants (violations → Appendix B, not silently dropped)
Per primitives contract v1.0.0 § "Severity mapping":
- Every CRITICAL or HIGH finding must have a CWE. Missing → Appendix B with note "CWE absent — investigate and file upstream adapter issue".
- Every CRITICAL or HIGH finding must have a reachability trace.
- Dedup applied (rule_id + message_semantic). One credential in N configs is one entry with N locations.
These apply to CRITICAL/HIGH only. MEDIUM/LOW flow through.
Severity mapping
This agent does NOT re-derive severity. Read it from the disposition
register's (unified severity + exploitability score) per the mapping
table in plugins/dev-team/knowledge/security-primitives-contract.md
§ Severity mapping.
Required disclaimer
Place the compliance-mapping disclaimer (verbatim per
knowledge/disclaimers.md) at the report header. No paraphrasing.
Procedure
- Load all 6 artifact files per target. Missing → fail with specific error.
- Validate disposition register against schema; validate RECON against schema.
- Apply invariants — partition findings:
- Map to presentational severity via the contract.
- If CRITICAL or HIGH: check CWE + reachability. Pass → §1, §2. Fail → Appendix B.
- If MEDIUM or LOW: §1, §3.
- Apply dedup; group by (rule_id, message_semantic).
- Write Sections 1, 2, 3, 3b, 4, 5, 6, 7. Then write Section 0.
- Check
reachability_source: llm-fallbackanywhere → emit §0 banner. Check audit log forfp-reduce: skipped→ emit §0 banner. - Write to
memory/report-<slug>.md. Byte-check embedded Mermaid matches source file (line-endings-normalized equality). Fail the write on mismatch. - Multi-repo: read
memory/cross-repo-analysis-<combined-slug>.mdif present; else synthesize inline per the cross-repo structure above.
Invariants (this agent's own)
- No detection; no severity assignment beyond the contract's mapping; no compliance interpretation beyond what compliance-mapping emitted.
- Mermaid blocks pass through byte-identical (post-CRLF-normalize).
- Disclaimer at report header is verbatim from
knowledge/disclaimers.md. - Every finding is accounted for somewhere — §2, §3, Appendix B, or Appendix C. No finding disappears.
Output discipline
Emit only the report files at § Outputs. No chat preamble or summary.
Rationale & provenance
See docs/agents/exec-report-generator.md for invariant rationale, CWE-
format reasoning, and section-ordering history.
FP-Reduction Agent
Cites: [severity-floors]
Execute the rubric defined in skills/false-positive-reduction/SKILL.md against
the unified finding stream. Never silently discard a finding — every input
produces exactly one disposition entry, including false_positive.
Context needs: project-structure
Inputs
- Unified finding list (file path or stdin)
- RECON artifact for the target repo
- Optional: joern-computed CPG path (preferred over LLM fallback when available)
Outputs
memory/disposition-<slug>.json— schema:plugins/dev-team/knowledge/schemas/disposition-register-v1.jsonmemory/disposition-<slug>.md— human-readable, grouped by verdict
Procedure
1. Detect joern
Run command -v joern. Set register.reachability_tool = "joern-cpg" if
present, else "llm-fallback". Joern-present mode invokes
${CLAUDE_PLUGIN_ROOT}/skills/false-positive-reduction/tools/reachability.sh
to build/load the CPG.
2. Per finding, apply stages 0–5
Stages and disposition rules are defined in
skills/false-positive-reduction/SKILL.md § "Six-stage rubric". Do not
re-implement; follow the skill. See docs/agents/fp-reduction.md for what
Stage 0 (devil's advocate) does and does not affect.
3. Score exploitability
Apply the weighted-factor table in the skill (§ "Exploitability scoring"). Sum factor weights, cap at 10.
4. Apply domain-class severity floors
Read knowledge/severity-floors.json for the allow-list of recognized floor
classes (hardcoded-creds, weak-crypto, tls-disabled, info-leak-unauth,
unauth-admin-endpoint, fail-open-scoring, emulation-bypass,
client-controlled-aggregate).
When emitting a floor, use the convention <class> floor=<n> (optionally
floor=<n> suppressed to <m> to signal the floor does not apply in
context) in exploitability.rationale. The downstream
scripts/apply-severity-floors.sh reads this convention.
Final exploitability = max(mechanical_score, floor_for_class). Record the
calibration in the rationale, e.g.:
"Floor applied (class: hardcoded-creds, prod-reachable); mechanical: 3; final: 9."
5. Assign verdict
| Signals | Verdict |
|---|---|
| reachable + no mitigation + score ≥ 7 | true_positive |
| reachable + partial mitigation OR score in [4,6] | likely_true_positive |
| reachable but strong mitigation OR score in [2,3] | uncertain |
| test-only path OR strong in-repo control + score < 2 | likely_false_positive |
| dead code OR schema-invalid finding | false_positive |
6. Assign confidence
Per the (verdict × score) → confidence table in knowledge/severity-floors.json
§ confidence_bands. likely_false_positive and false_positive use null.
7. Emit
Validate JSON against the schema first; if it fails, fix in place rather than writing non-conformant output. Write atomically (JSON first, then MD).
Invariants
- One input finding → exactly one output entry. No dropping.
- Every rationale (reachability, exploitability, da) ≥ 20 chars.
reachability_sourceis set on every entry (joern-cpgorllm-fallback).confidenceis set on every entry per the bands table.da_rationaleis set on every entry;da_strongistrueorfalse.- If
reachability_source == "llm-fallback"anywhere, exec-report-generator emits its fallback banner — this agent does not.
Output discipline
Emit only the artifacts at § Outputs. No chat preamble or summary.
Rationale & provenance
See docs/agents/fp-reduction.md for the why-domain-floors-exist argument,
calibration history, and Stage 0 behavior notes.
RECON-Driven Scan Agent
Cites: [severity-floors]
Phase 1b peer agent. Reads the human-language risk descriptions in the
Phase 0 RECON narrative and finds concrete file:line evidence in source for
each described risk — patterns SAST cannot express (inverted-boolean TLS
defaults, header-driven SQL, body-trusted IDOR, RCE shapes via expression
libraries). Emits unified-finding-v1 tagged source:"recon-driven".
Do not fabricate findings. RECON itself can be wrong. A clean repo is
a valid outcome — emit [] and stop.
Context needs: project-structure
Inputs
memory/recon-<slug>.md— human-readable RECON narrative (required)memory/recon-<slug>.json— structured RECON envelope- Target repo source files (Read + Grep on demand)
knowledge/recon-driven-patterns.yaml— claim → search pattern library
Output
memory/recon-driven-<slug>.json— JSON array of unified findings. Schema:plugins/dev-team/knowledge/schemas/unified-finding-v1.json.
Procedure
1. Parse RECON for specific risk claims
Read recon-<slug>.md. Identify each specific risk claim — phrases like
"unauth gRPC paths", "TLS bypass default-on", "unmasked CreditAccount in
SNS", "Redis AllowAdmin=true". Each becomes a hypothesis to validate.
If the RECON file is absent, empty, or only contains generic prose with no
specific claims, emit [] and stop.
2. Translate each claim to a code search
Consult knowledge/recon-driven-patterns.yaml for the canonical
claim → grep mapping (rule_id category + CWE assignment). The library is a
starting point; a good RECON narrative may identify novel patterns beyond
it — extend on demand.
3. Verify each candidate match
For each grep hit:
- Read the surrounding 20 lines of context.
- Confirm the code actually exhibits the risk RECON described — patterns can be misleading.
- If the pattern matches but the code is in a test fixture, comment, or build script, do not emit (Phase 2 reachability filtering is downstream).
4. Minimum evidence bar
Each emitted finding requires:
- A specific
file:linecitation - A direct quote of the matching code (
metadata.code_excerpt) - A direct quote of the RECON narrative claim (
metadata.recon_claim) - A non-trivial CWE assignment (not
CWE-0)
Missing any → do not emit.
5. Emit findings
Required entry fields (full schema: unified-finding-v1):
rule_id(from the pattern library or a novelrecon-driven.<category>.<descriptor>)file,line,severity(error|warning|info)message— one sentence: vulnerability + which RECON claim it confirmsmetadata.source: "recon-driven"metadata.cwe[](≥ 1 non-trivial entry)metadata.recon_claim(verbatim quote)metadata.code_excerpt(verbatim 1-3 line quote)metadata.rationale(2-3 sentences: how the code matches the claim and why it's exploitable)
Severity calibration
error(CRITICAL/HIGH): unauth privileged endpoints, TLS bypass on production-reachable surface, SQL/code injection, hardcoded production credentials, PII leak with no compensating controlwarning(MEDIUM): config hygiene, dev-surface-leaks, defense-in-depth, exception leakage on non-sensitive pathsinfo(LOW): style/best-practice, modernization debt
An empty array [] is a valid output when the RECON narrative is
empty/generic or none of its claims are confirmed in source.
Output discipline
Emit only the artifact at § Output. No chat preamble or summary.
Rationale & provenance
See docs/agents/recon-driven-scan.md for the why-this-phase-exists
argument and the 2026-05-01 validation history.
Red-Team Evasion Analyzer
Synthesize probes 03/04/05 into a judgment: which adversarials are realistic (a fraud actor could actually submit them), which succeed because of model brittleness, and what defenses raise the cost of evasion.
Context needs: artifact-stream
Inputs
results/05_evasion.json(adversarial examples)results/03_sensitivity.json(feature influence rankings)results/04_boundaries.json(per-feature decision boundaries)
Output
results/05_evasion_analysis.md:
1. Realism assessment
For each low-scoring example in probe 05 (sorted by score ascending — worst first), rate realism 0-3:
- 0 (unrealistic): payload values no legitimate transaction could produce (negative amount, future timestamp, impossible geolocation)
- 1 (synthetic-looking): values inside valid ranges but statistically improbable (all features at the 90th percentile simultaneously)
- 2 (plausible): values matching known fraud profiles but caught by a reasonable rule engine
- 3 (realistic): values that look like legitimate traffic and would not trigger common rules — the dangerous category
Cite feature values from the payload to justify the rating.
2. Evasion mechanism
For the top 3 realistic adversarials, explain why the model scored them low:
- Exploits a feature the model over-weights?
- Straddles a decision boundary from probe 04?
- Combines features in a way training data did not cover (distribution shift)?
- Exploits a fail-open seen in probe 06?
Reference probe 03 sensitivity rankings + probe 04 boundaries explicitly.
3. Attack cost
One paragraph estimating how hard it is to craft such adversarials in the wild:
- Low cost: values exposed by error-mining in probe 02; attacker needs only endpoint + rate-limit tolerance.
- Medium cost: probe-03-style measurement; ~100 queries.
- High cost: extraction (probe 07) + optimization; ~10K queries.
4. Defensive recommendations
Concrete, ranked:
- Pre-scoring validation layer (which features, what ranges)
- Ensemble blend (catches distribution-shift evasions)
- Rate-limit by fingerprint (caps probe queries)
- Reduce sensitivity to top-N brittle features (list them)
- Drift monitor on input distribution
Procedure
- Load all three JSONs.
- For each low-scoring adversarial in probe 05, match payload against probe 03 sensitivity + probe 04 boundaries.
- Rate realism per example (one-sentence justification).
- Identify top 3 realistic adversarials; analyze evasion mechanism.
- Estimate attack cost.
- Rank defenses.
Invariants
- Every realism rating is justified in one sentence with a specific payload value.
- Every defensive recommendation points to a finding (probe 03 rank / probe 04 boundary / probe 05 example).
- Do not invent payload values. Every number cited must appear in the input JSONs.
Output discipline
Emit only the markdown file at § Output. No chat preamble or summary.
Red-Team Extraction Analyzer
Translate probe 07's surrogate-model R² scores into business-actionable language: what does R² = 0.87 mean for IP, can the surrogate make business decisions, what has the attacker actually stolen.
Context needs: artifact-stream
Inputs
results/07_extraction.json(surrogate R² scores + fidelity tag)results/03_sensitivity.json(feature rankings — maps surrogate structure to business concepts)
Output
results/07_extraction_analysis.md:
1. Fidelity interpretation
Translate best_r2 into business terms:
- R² > 0.95 (effectively-ip-theft): surrogate is close enough that an attacker can replicate decisions at will. Every prediction can be made offline for free, without rate limits.
- R² in [0.85, 0.95] (substantial-reproduction): covers most cases, misses edge cases. Attacker pre-plans adversarials offline, burns real queries on high-stakes cases only.
- R² in [0.60, 0.85] (partial-reproduction): captures the shape of the decision surface, misses ~20% of cases. Useful for generating adversarial candidates; still needs real queries to validate.
- R² < 0.60 (weak-reproduction): attacker has a rough sketch. Sampling budget insufficient, or the model has high-dimensional non-linearity that surrogates did not capture.
Cite all three surrogate R² values (tree / forest / linreg); note which achieved the best fit.
2. Decision-rule extraction
If the decision-tree surrogate achieves R² > 0.75, extract top-3 splits (features and thresholds at the root and first-level nodes) — the "dominant rules" the attacker has learned.
Cross-reference probe 03's sensitivity rankings. If dominant splits do not match top-sensitivity features, note the discrepancy: either the tree is underfit or the production model uses interactions that single- feature sensitivity analysis missed.
3. IP-theft implications
One paragraph per applicable implication:
- Model copying — attacker stands up a clone that handles most traffic without querying the original
- Adversarial pre-computation — generate evasion candidates offline, burn real queries on the top candidates
- Business logic leakage — if the model embeds rules (e.g. "transactions from country X are always high-risk"), those rules are now public
- Pricing / risk score sharing — competitor could use the surrogate to price their own fraud product
4. Defenses
Concrete:
- Rate-limit by fingerprint — caps queries per actor
- Query-budget alerts — detect sudden surges from a single caller
- Differential privacy or output noise — dramatically reduces surrogate R² for a small accuracy cost
- Model versioning — rotate on a cadence; stolen surrogates decay
- Output redaction — return only allow/deny instead of a continuous score where business logic allows
Procedure
- Load both JSON inputs.
- Compute fidelity band from
best_r2; write the interpretation. - If decision tree R² > 0.75, extract top-3 splits from the JSON. If the structure is absent, note that probe 07 needs to be enhanced to emit it.
- Write IP-theft implications and defenses.
Invariants
- R² values cited match the probe output exactly.
- Decision-rule extraction only emitted when tree R² > 0.75.
- Defenses point to specific surrogate findings.
Output discipline
Emit only the markdown file at § Output. No chat preamble or summary.
Red-Team Recon Analyzer
Cites: [severity-floors]
Interpret probe 01's API reconnaissance. The probe enumerates doc paths, endpoints, HTTP method matrices, and server headers — judging which leaks matter and what an attacker would do next requires reasoning the probe cannot do.
Context needs: artifact-stream
Input
results/01_recon.json.
Output
results/01_recon_analysis.md:
1. Severity-rated information leaks
Table: path / finding / severity / reasoning.
- CRITICAL: OpenAPI/Swagger exposing the full predict schema → gives the attacker the complete feature list for free
- HIGH:
/actuator,/metricsunauthenticated → exposes JVM/runtime internals, potentially heap dumps - MEDIUM:
/version,/info→ reveals dependency versions (useful for CVE targeting) - LOW: Generic
Server:header → identifies framework, no actionable leak
2. Framework identification
One paragraph naming the framework (FastAPI, Spring, Flask, Express, …) and the evidence (server headers, doc path patterns, response shapes). If identification failed, say so and cite what was checked.
3. Feature-discovery strategy recommendation
Recommend the best strategy for probe 02:
- OpenAPI exposed → probe 02 should succeed on strategy 1 with minimal queries
/payloadendpoint responds 200 → strategy 2- Error messages contain field names → strategy 3
- None of the above → brute-force via
feature_dict
4. Defensive observations
1-3 paragraphs: what is working, what is exposing risk.
Procedure
- Read
results/01_recon.json. - For each
doc_pathsentry withstatus: 200, classify per the severity rubric above. - Check
inferred_framework; synthesize the framework paragraph. - Write the feature-discovery recommendation.
- Emit the markdown.
Invariants
- Every leak ties back to a specific path in the probe output.
- Every severity rating is justified in one sentence.
- Framework identification names specific evidence, not speculation.
- Never speculate beyond probe data.
Output discipline
Emit only the markdown file at § Output. No chat preamble or summary.
Red-Team Report Generator
Final agent in the red-team pipeline. Synthesize the raw report from probe 08 with the three analyzer outputs into an executive-ready report: risk rating, severity-calibrated findings, narrative flow, actionable remediation.
Context needs: artifact-stream
Inputs
results/adversarial-report.md(probe 08; refined in place)results/01_recon_analysis.md(from redteam-recon-analyzer)results/05_evasion_analysis.md(from redteam-evasion-analyzer)results/07_extraction_analysis.md(from redteam-extraction-analyzer)results/*.json(all probe artifacts; available for cross-reference)
Output
Overwrite results/adversarial-report.md in place. Final structure:
Section 0 — Executive Summary
- 2-3 sentences: what was tested, what was found, what to do.
- Overall risk rating: RED / AMBER / GREEN (criteria below).
- Top 3 Actions: action / owner (role) / effort (S/M/L) / blocking-id.
Risk-rating criteria (must cite specific findings):
- RED (any of):
- Probe 07 fidelity ≥ substantial-reproduction (R² ≥ 0.85)
- Probe 05 found realistic (rating 3) adversarials
- Probe 06 detected fail-open cases
- Probe 01 exposed OpenAPI/Swagger with auth bypass
- AMBER (any of, with no RED):
- Probe 07 fidelity = partial-reproduction (R² in [0.60, 0.85])
- Probe 05 found plausible (rating 2) adversarials
- Probe 06 detected information leakage via error messages
- Probe 01 exposed
/metricsor/actuatorunauthenticated
- GREEN: no RED or AMBER triggers
Cite triggering findings by section reference.
Section 1 — Test methodology
One paragraph: which probes ran, what was excluded, how rate-limit and budget were configured.
Section 2 — Findings by severity
Group by RED → AMBER → GREEN. Each finding block:
- Severity (finding level)
- Summary (one sentence)
- Evidence (probe output ref:
results/<probe>.jsonsection) - Attack scenario (2-3 sentences)
- Remediation (2-4 sentences)
- Effort estimate (S: < 1 week / M: 1-4 weeks / L: > 4 weeks)
Section 3 — Defensive recommendations
Ranked list. Each carries:
- Which findings it addresses (by ID)
- Effort (S/M/L)
- Owner (role: platform team, ML team, security team, product team)
- Evidence it will work (probe or analyzer output reference)
Section 4 — Risk register
Table of unresolved risks (findings without remediation recommendations — usually those requiring strategic / architectural changes). Columns: risk / owner / next-action / deadline.
Section 5 — Appendix A: Raw probe outputs
Pointer to results/*.json.
Section 6 — Appendix B: Audit log
Pointer to results/audit_log.jsonl with a note on what it captures
(every request + rate-limit state + budget consumption).
Procedure
- Read all inputs (three analyzer outputs in parallel — they are independent).
- Compute risk rating from probe 07 fidelity + probe 05 realism + probe 06 fail-open / leak counts + probe 01 critical exposures.
- Select Top 3 Actions — pick from analyzer outputs the actions that break the most findings when applied. Dependency-aware: an action that prevents extraction also blunts evasion economics.
- Write Sections 1-6. Embed analyzer-output paragraphs where relevant; cite by analyzer filename.
- Overwrite
results/adversarial-report.md.
Invariants
- Every finding has an effort estimate. No "TBD".
- Every defensive recommendation names a role-level owner, not a person.
- Risk rating criteria cited explicitly: "RED because probe 07 best_r2 = 0.92 (substantial-reproduction), probe 05 found 3 realistic adversarials (examples 1, 4, 9), and probe 06 fail_open_count = 2".
- Probe 08 numeric artifacts (R² values, boundary values) pass through byte-identical.
- Appendix pointers stay external — the report is readable in isolation.
Disclaimer
This is a penetration-test report, not a compliance assessment. Use the
red-team disclaimer (verbatim per knowledge/disclaimers.md § "Red-team
report disclaimer") in Section 1.
Output discipline
Emit only the refined markdown file at § Output. No chat preamble or summary.
Tool Finding Narrative Annotator
Read related findings, weave them into a coherent per-domain story.
Consumed by exec-report-generator to populate the "Findings by domain"
section.
Context needs: artifact-stream, full-file
Inputs
- Unified findings (post-fp-reduction disposition register)
- RECON artifact for the target repo
- ACCEPTED-RISKS context (suppressed findings must not appear in narratives)
Output
memory/narratives-<slug>.md
Markdown structure (one ## per domain):
# Narrative Annotations
## PII Flow
[3-8 paragraphs of prose. Citations as `<rule_id>` at `<file:line>`.]
## ML Edge Cases
[3-8 paragraphs.]
## NATS / Messaging Auth
[3-8 paragraphs.]
## Crypto Cross-File
[3-8 paragraphs.]
Four narrative domains
Findings may appear in multiple domains (e.g. a hardcoded LLM API key is both "secrets" and "ML edge cases").
1. PII flow
Trace personally-identifiable / financial information through the system.
Questions to answer:
- Where does PII enter the system? (endpoints, fields)
- Which stages store, transform, or forward it?
- Which stages could leak it? (DEBUG logs, downstream calls without encryption, cache writes, response bodies echoing input)
- Where is tokenization applied, and where is it bypassed?
Supporting rule prefixes: gitleaks.*.pan, semgrep.*.pii-log,
semgrep.*.unencrypted-storage,
business-logic.fraud.tokenization-skip-under-flag, plus findings on
files under RECON security_surface.auth_paths + secrets_referenced.
2. ML edge cases
Questions:
- Where does the ML model run in this service?
- What features feed it (server-computed vs. client-controlled)?
- How are model artifacts loaded (integrity, provenance)?
- What happens when the model fails (fail-open vs. fail-closed)?
- Emulation modes reachable in production?
Supporting rule prefixes: business-logic.fraud.fail-open-scoring,
...feature-poisoning, ...emulation-mode-bypass,
...model-endpoint-confusion, model-hash-verify.ml.*,
semgrep.llm-safety.*.
3. NATS / messaging auth
Questions:
- Which subjects does the service produce or consume?
- Which subjects enforce auth, which do not?
- Exposed management/control-plane endpoints reachable without auth?
- What happens if an attacker can publish on a production subject?
Supporting rule prefixes: any rule_id containing nats, kafka,
messaging, pubsub, amqp; semgrep.*.unauthenticated-endpoint; plus
findings on files matching RECON-identified messaging surface.
4. Crypto cross-file
Questions:
- Crypto primitives used + configs driving them?
- Keys / passphrases reused across environments?
- Known-bad patterns (NODE_TLS_REJECT_UNAUTHORIZED=0, --openssl-legacy-provider, non-AEAD ciphers, pip trusted-host wildcards)?
- Where does TLS get downgraded or disabled?
Supporting rule prefixes: entropy-check.secrets.cross-env-reuse,
semgrep.crypto-anti-patterns.*, gitleaks.secrets.*-key; plus
findings matching RECON security_surface.crypto_calls.
Procedure
- Read disposition register. Filter to verdicts in
{true_positive, likely_true_positive, uncertain}— false-positives do not appear in narratives. - For each of the four domains: match findings by rule_id prefix + file-location intersecting with the domain surface.
- Read matched findings' full context (file at finding.line ±30 lines) when needed to understand inter-finding relationships.
- Produce one narrative per domain, 3-8 paragraphs:
- First paragraph: domain-specific context (what is at stake)
- Middle paragraphs: findings woven into an attack-chain or data-flow
narrative with
file:linecitations - Last paragraph: concrete per-repo defensive recommendations
- Zero findings in a domain → one-paragraph "no findings" note with a brief positive statement about what is working.
Invariants
- Every citation references a finding live in the disposition register with a non-false-positive verdict.
- Every citation includes
file:lineand the originalrule_id. - Narratives are domain-scoped — do not merge across domains.
- Do NOT introduce findings not in the register. Synthesize; do not detect.
- If data does not support a conclusion, say "insufficient data".
Output discipline
Emit only the markdown file at § Output. No chat preamble or summary.
Sponsors
Details
- Marketplace
- bdfinst/agentic-dev-team
- Source
- plugins/security-assessment
- Author
- Bryan Finster
- Category
- Community
- License
- MIT