Formal verification workflows and mechanical anti-reward-hacking gates for coding agents.

version
0.108.0
GitHub stars
77

Install

/plugin install vstack@hao-ai-lab-vstack

Run it inside Claude Code after adding the marketplace with /plugin marketplace add hao-ai-lab/vstack.

Contents

Discovered from source

Skills 10

vstack mode

Read Layer 0: shared operating rules. Select one playbook from the request and keep its stage exits visible. The coordinator owns the route, approval records, frozen inputs and final independent review. A worker reads only Layer 0, its own stage and assigned inputs.

PlaybookRoute and purpose
full-verificationRun S0 through S8; select the proof tier for each property.
hunt-system-bugsRun S0 through S6 to find and confirm defects with a calibrated model.
build-provable-coreRun S0 through S5, then S7 and S8, for new code committed to deductive proof.
live-regressionRun the project-local verify-<case> produced in S8 and route drift or failures.

A route from S0 through S5 includes every intervening stage. S4 and S5 cannot be omitted. S6 can be omitted only for newly written code going to deductive proof. If a prerequisite is missing, perform it or report BLOCKED rather than calling the downstream stage complete.

Record each task's property, source revision, tier, input hashes, writable paths, exit checklist, budget and evidence locations. Use the case playbook. Read working principles when scoping or reviewing experiments.

Assign work to vstack-worker, independent review to vstack-redteam, and keep authority with vstack-main. Delegate only with available host tools and user authorization. Role documents provide neither scheduling nor sandboxing.

Coordinating units. Every route runs its units through the coordination loop. The coordinator generates each unit's task brief from a fields file with scripts/make_brief.py, never from another brief, and sends it only after scripts/check_brief.py passes, checks every reply's hand-back block with scripts/parse_handback.py, and keeps the unit table as the coverage matrix, checked against the plan by scripts/check_unit_table.py. It writes briefs, accepts and merges rows; it does not census, model or write specifications itself. Each unit gets one fresh worker with its own output directory and row copy; workers never share a writable file or see each other's directories. Drain every row to accepted or blocked, re-spawning a unit once before recording a gap. A row is complete only with evidence paths in its output directory.

One unit first. Send one unit of each batch through brief, work, hand-back and acceptance before fanning out the rest, so that a flaw in the brief costs one worker instead of the batch. The unit table marks that unit in first_of_batch; its check fails when another unit of the batch leaves queued before the first is accepted.

Key units are written twice. For a unit marked key, two workers write it independently from the same brief and frozen inputs, in separate directories. Compare the writings row by row (writer lists, guards, candidate findings) in a writing-compare table checked by scripts/check_writings.py, pick a base and graft the other's extra rows before either writing is used.

An error seen twice becomes a check or a template field. At the end of each batch, read its hand-backs and lessons. An error class seen twice gets a failing check or a required template field in the same round, registered in the rule registry; text alone is not a fix (Layer 0).

At a failed upstream assumption, issue a rollback ticket, preserving frozen artifacts. A property change needs the property owner's explicit decision and coordinator re-pinning. Deliver PASS, ISSUES or BLOCKED with evidence; incomplete or unknown checks never count as PASS.

The gate interface guide describes actual scripts and their limits. Read it before configuring a project runner; the package does not include a generic solver harness.

Concrete Examples

Use PD KV slot lifecycle regression to see how a slot-release promise becomes pins, drift checks, model checks, trace conformance and mutation regression. This illustrates the workflow, without supplying a runnable case bundle.

S0 — Reconstruct and separate the system

Read Layer 0: shared operating rules, this stage and the assigned input bundle.

Role and write scope

Assigned investigators own separate maps and candidate files; a verifier and coordinator check evidence. Target source is read-only.

1. Input

A fixed source revision, existing anchors for deduplication, and existing maps and catalogues when available. Project-specific catalogues stay in the project.

2. Procedure

  1. Build a control map of mutable state, owners and all writers. Label every in-scope subsystem control, data or both, stating each slice for both; use the plane definitions. Write the subsystem list as an ownership table of rules by path, class and function, the most specific rule winning, and check before any census that it partitions the in-scope code: every file, and every class or function of a file that subsystems split, maps to exactly one subsystem or is marked shared with the sharing subsystems named. Settle overlaps in this table, so that two censuses attribute one site to the same subsystem.

  2. Fill the resource matrix: one row per shared mutable resource and subsystem that touches it, with code sites and the feature or switch conditions. Set access to write (changes it), hold (keeps it so others must not change or free it) or read. The matrix is the main table of the census; units are derived from it resource by resource in S1. Attribute sites by these rules:

    • The subsystem whose own code performs the write owns the site, by the ownership table. A subsystem that only calls another subsystem's method is a caller, not a writer, and gets no write row for that call. Exception: when the obligation is about what every holder must do, such as returning what it took exactly once, list every holder with access = hold.
    • Count a constructor write by the lifetime of the object it initialises. It is shared state only when the object is not created for one request or one call, lives across requests or steps (a server-wide pool, cache, scheduler or backend, or a batch object reused from step to step), and another party later reads or writes the field. A constructor of a per-request or per-call object (a request's output parser or stop detector, a per-forward metadata builder) is not shared state; later writes to such an object by another party are ordinary write sites. Record the lifetime (service, request or call) of every constructor site you classify.
    • Search all assignments before saying a field has one writer: search each field and method by its bare name (.slots =, free(), not only a qualified form, across the whole package rather than the files read for a property.

    For Python sources, use check_anchors.py with the anchor table and review its coverage limits.

  3. Map data and computation consumers and their unchecked preconditions. At each boundary, map the producer, consumer, execution context, overwrite time and synchronization. Classify every in-scope file. Classify each boundary as a state or parameter edge, recording the criterion and any value range used.

  4. Run the writer census as a task of its own: a census worker does not also write property clauses or unit rows in that task. Have two workers independently perform each resource's writer census and boundary classification without seeing each other's output. The coordinator lists and settles every difference, classifying each as a missed shared write, not shared state, or one site attributed to different subsystems (settled by the ownership table); a count of extra sites in one census is not a count of missed writers. Retain both censuses and the settlement record. Every later correction of a census row (by the settlement, a second pass or a review) goes into a census-correction record (scripts/check_census_fixes.py --template): who, which row, the wrong and the right value, its kind and path:line evidence; then apply it to the table.

  5. Record facts with path:line@commit and short source excerpts. Keep unsupported inferences explicitly unverified.

  6. Search the target and related systems by component. Record every examined candidate, rejection reason and search coverage. Confirm buggy triggers and fixed behavior on the stated revisions; a closed issue is not evidence that a fix merged.

  7. Test critical physical and synchronization assumptions on the real system before treating them as axioms, using authorized resources.

  8. For the full adequacy readout, freeze each slice's historical-fix denominator I_U with its exclusion reason codes and SHA-256. The coordinator then draws the calibration set by a published seed and seals the blind set H, removing every trace of H from the maps handed downstream.

3. Exit checks

  • Require zero evidence schema errors and a positive checked-item count. The source KB checker is repository-specific and is not bundled.
  • Have the verifier and coordinator review original source and bind review marks to its content. This review is manual.
  • Classify every file, label unsupported claims and deduplicate upstream identifiers. These checks have no generic automation.
  • Documented-only: review every subsystem's plane slices, the ownership table's partition, resource accesses and conditions, constructor lifetimes, boundary kind and criterion, and the two independent censuses with all differences settled and classified. No generic completeness or partition checker is bundled.
  • Partly manual: run check_anchors.py for Python sources and review the resource matrix against source; its scan does not establish a complete writer census or correct edge classification.
  • Mechanical: scripts/check_census_fixes.py FIXES.tsv --table NAME=MATRIX --source SOURCE refuses a correction without its evidence or one the table does not hold yet; scripts/check_names.py SOURCE MATRIX --column ... refuses a row naming a class, function or field absent from the pinned source (name check).
  • Keep disproved anchors. Rename an unsupported historical Bug* switch to Mut*; do not count it as historical calibration.

4. Rollback

An invalid physical assumption reopens the boundary map. A downstream report of a missing state writer returns here with the pinned source location. Use a rollback ticket.

Concrete Examples

For a PD KV slot, map every writer of ownership and in-flight state, then record the producer-to-consumer handoff. An unsupported historical-bug claim stays a mutation hypothesis until pinned code supports it.

S1 — Split obligations and select methods

Read Layer 0: shared operating rules, this stage and the assigned input bundle.

Role and write scope

The coordinator owns the design and obligation matrix; the property owner approves the scope.

1. Input

Control, data and boundary maps, the settled resource matrix and verified anchors from S0, with hashes.

2. Procedure

  1. Read each unit from a resource-matrix row group. Record it in the unit table: resource, writing or holding subsystems, their distinct count k, plane, property clauses, pillars, formal classes and methods. Start from the resources, never from combinations of subsystems: a resource with n writers or holders has n(n-1)(n-2)/6 triples, and a census that reaches three-subsystem resources does not call for a model of every triple. Choose each resource's units by steps 3 and 4.
  2. Use the method-selection table by plane and formal class. Split by property, not module or processor type: route control questions to a state machine and model checker; route data questions to SMT over dispatch descriptors, an exhaustive sweep or a differential test on real functions. Give every boundary a contract card.
  3. Plan k <= 3 by default. For resources with more writers, split by deployment tier: default-on writers form the unit; declare the rest as environment or separate units adding one feature each, so a resource yields units per feature added, not per pair or triple of writers. A unit with k >= 4 is allowed under the search budget. If exhaustive search at the smallest scale reaching the concurrency witness does not finish within that budget, record over_budget and do not report an exhaustive result.
  4. Order units by the two danger axes, recording entry_kind and deployment tier; ordinary client traffic under the default configuration comes first.
  5. Record each obligation's difficulty, finite or unbounded scale, code ownership, change rate and failure severity.
  6. Map ledger, protocol, computation, alignment and fallback guarantees. Identify which row supports each promise and which is weakest.
  7. Record each property's assurance tier, rationale, available tools and skipped stages. Require S4 and S5. Skip S6 only for new code destined for deductive proof.

3. Exit checks

  • Manually check that every hazardous transition and consumer precondition has an obligation.
  • Documented-only: review unit rows against the settled resource matrix, including k, plane, property/method routing, boundary contracts, deployment splits and priority order. Record the search budget and within / over_budget / not_run result in the case playbook; keep the unit table consistent with run evidence. scripts/check_unit_table.py checks status, evidence, one unit first and gaps against the plan; no budget checker is bundled.
  • Account for every gap in the guarantee matrix; no generic checker is bundled.
  • Record scope and design approval. Limit each guarantee to its weakest supporting evidence.

4. Rollback

Missing maps return to S0. If no small clean proof core exists, propose the smallest interface separation before proving tangled orchestration code. Use a rollback ticket.

Concrete Examples

PD KV slot reuse is a state-machine obligation: finite exploration finds interleavings; an arbitrary-scale claim needs a corresponding design-model theorem plus explicit model/code alignment.

S2 — Freeze specifications and contracts

Read Layer 0: shared operating rules, this stage and the assigned input bundle.

Role and write scope

The coordinator writes top-level promises and pins; the property owner approves them. Workers supply anchored evidence and staged proposals.

1. Input

The approved S1 design and obligation matrix.

2. Procedure

  1. Write a short, reviewable statement of observable consequences and explicit non-goals. Use contract discovery and the card template. Name each property clause's pillar and formal class from property classes; the class decides the checker and whether fairness is needed.
  2. Require top-level decomposition plus historical bugs or consumer requirements; complete one-step induction for ledger and protocol families. Scan all seven consumer-precondition categories.
  3. Calibrate negative examples, legal edge behavior, witnesses and minimum scale before freezing. S4 repeats calibration on the complete model. A clause that will be checked on real code carries the four parts of a clause check: its input dimensions, a second oracle, refusal of a documented operation counted as failure, and triage before counting.
  4. Expose environment assumptions and ambiguous semantics as sourced switches. State which interpretation each claim requires.
  5. Have the coordinator pin the approved text, ideal model and check configurations: S2 freezes the property clauses and the ideal reference model of the three artifacts; S3 builds the code-anchored model against them. Before deductive proof, also freeze the complete inductive invariant and root statement, including explicit parameter ranges and configuration premises. Check content hashes before every counted run.

3. Exit checks

  • Check approval, provenance, consumer scans and calibration records for completeness. This is partly manual.
  • Require scripts/check_spec_pins.sh ROOT PINS to succeed; it compares pinned bytes mechanically.
  • Mechanical: before freezing, run scripts/check_names.py SOURCE CLAUSES.tsv --column SENTENCE_COLUMN on every table whose clauses or instance sentences name code; it refuses a sentence naming a class, function or field that does not exist in the pinned source, such as an abbreviated class name (name check). Correct the sentence from the source, never the other way round.
  • Run the approved bad-implementation suite against the frozen ideal: bad cases must fail and controls must pass. These project-specific runs do not grant approval.

4. Rollback

Report an overly strong property or missing assumption with a rollback ticket; a finding reviewed as spec_defect (review verdicts) arrives the same way. Stage the change, compare full regressions, obtain a new property-owner decision, then let the coordinator independently rerun and re-pin. An invariant change requires new Init and affected preservation checks in the same tree, then fresh acceptance. Old proof results do not carry over.

Concrete Examples

For a PD KV slot, a posted write must not simply disappear from the ideal model. Freeze that promise and the configurations that check it; revise them only through the property owner and coordinator.

S3 — Build an anchored model

Read Layer 0: shared operating rules, this stage and the assigned input bundle.

Role and write scope

The worker owns a new model version and append-only notes, never upstream pins or gold expectations.

1. Input

Frozen contracts from S2 and maps from S0, with hashes.

2. Procedure

  1. Fill the case playbook. Bind every action, descriptor and ledger method to source lines and revision. On both adequacy tracks, write each modeled guard and state update as a named operator in the anchor map; also list every write site of in-scope state and its status in anchors.tsv.
  2. Extract constants from pinned source when possible; do not let a correct verdict hide a transcribed constant error.
  3. Preserve atomicity: two separately visible shared writes are two model steps. Name actions; make Next their disjunction; keep deadlock checking and an explicit legitimate terminal action.
  4. Use switches for historical bugs and repair components in one model. Bug* needs a real historic revision; use Mut* otherwise. Check complete switch combinations against the source revision.
  5. Give critical paths reachability witnesses (W_*, see anti-vacuity checks). When a property concerns concurrent requests or actors, one witness must show two in flight at once (W_Concurrent_<what>); if no passing configuration reaches it, report the result as serial only. Declare unchecked mapping dependencies in one ASSUMES header and ensure every using configuration checks them; this comment is not permission to add proof axioms.
  6. Finish invariant discovery and have the coordinator pin its full definition before deductive proof. Prove this calibrated model; prove bridges from proof-layer helpers to its pinned definitions. Do not replace the model.
  7. Model the environment by the rules below.

Open environment

Model each external stimulus E (cancel, disconnect, session close, admin pause, cache flush, weight update, timeout expiry) as Env_* actions and the engine's responses as Engine_* actions; one action may carry both when the code dispatches them together.

RuleRequirement
Input enabledFor each E, define InputEnabled_<E> == EnvPremise_<E> => ENABLED Handle_<E> and check it as an invariant. EnvPremise_<E> holds while the environment's own finite budget lasts and the object exists from the client's side (for example nAbort < MaxAbort and the request is still open for its client). Handle_<E> is any action that takes E: handles it now, records it as pending, or rejects it by a code branch. TLC evaluates ENABLED in invariants; with a checker that cannot, cover every branch explicitly or give each branch a reachability witness.
Guard ruleEvery enabling guard of every action, Engine_* included, is either the condition the code evaluates at its file:line@commit anchor or a registered finite bound or \* ASSUMES: premise. Anything else is illegal pruning; keep real failure paths, such as a full allocator raising or asserting.
Caller contractsRestrict a call order only when the engine's documentation forbids it and the code rejects it. Name the restriction EnvContract_<name>, declare it in the \* ASSUMES: header with the documentation quoted verbatim and its source, and run a robustness control without it. Report a violation in that control as a robustness finding outside the documented contract.
FairnessPut WF/SF on Engine_* actions only. Environment actions get no fairness except in a named control. Every liveness check registers EnvEventuallyQuiet: a finite total of arrivals, or a named constant after which the environment stops.
RestrictionsNamed environment timing restrictions follow the asymmetric rule: hunt and control configurations only, never a configuration reported as passing.

3. Exit checks

  • Mechanical: scripts/check_anchors.py validates citations in anchors.tsv and, with --fields and --paths, lists Python writes missing from the reverse map. Writer completeness outside its supported syntax remains partly manual; review the documented limits.
  • Review action boundaries, named Next, termination and deadlock settings manually.
  • Check every InputEnabled_<E> invariant and every W_* witness in each delivered configuration. Review each guard against its anchor or registered bound; this is partly manual.
  • scripts/assumptions_check.sh MANIFEST checks the supported header/config syntax. scripts/op_mirror.sh checks only its documented record-update dialect; neither is a semantic TLA+ parser.
  • Confirm that frozen inputs remain unchanged.

4. Rollback

A missing writer returns to S0 with its source location. A contract defect returns to S2. Do not patch upstream artifacts to fit the new model. Use a rollback ticket.

Concrete Examples

In a PD KV slot lifecycle, dequeue accounting and the following status check may be separately visible. Preserve that interleaving instead of merging both into one atomic model step.

S4 — Calibrate and align traces

Read Layer 0: shared operating rules, this stage and the assigned input bundle.

Role and write scope

Model and trace workers own distinct outputs; the coordinator consolidates the checklist and independently reruns representatives.

1. Input

Models, configurations, source anchors and witnesses from S3.

2. Procedure

  1. Timestamp predictions. Run four calibration checks: the historical bug violates the expected property by the expected mechanism; the fix completes within bounds; removing each repair component causes a violation; reachability witnesses are reached and their negations fail. Aggregated over the frozen denominator I_U, these results give the numerators Cov_expr and Cov_emergent of the adequacy readout.
  2. Repeat two-sided contract calibration at the minimum scale and one increment above it. A component outside the abstraction is unverified, not a pass.
  3. Apply trace-conformance checklist to real traces. Keep observation, instrumentation, completion and safety evidence separate. Report readout 1: TAR with its negative-mutant rejection rate (or the agreement rule's stateful-test readout or labeled model-only exception), and clean-baseline counterexamples as Confirmed Bug, Model Defect and Unjudged (pending repro) counts.
  4. Classify every NOT_CONSUMED at the first divergence; never edit a trace to make it count as accepted.
  5. Complete the agreement record for the pinned baseline and every calibration mutant, clause by clause. Apply each mutant to an unmutated row that passes at the same scale, with the known repairs switched on when the pinned baseline already violates, and confirm that its counterexample takes the mutated action. Resolve differing verdicts before S6 counts anything from this model: model pass/code violation returns to S3 with the code location; a model counterexample not reproduced by code needs a directed real-code check constructing its trace preconditions before it counts; fix a harness defect and rerun the row.
  6. Review every run with the seven questions. Follow run order and budget.

3. Exit checks

  • Check expected verdicts and mechanisms for the bug, fix, repair ablations and paired witnesses. Mechanism review remains partly manual; the TLC log gate checks the supported action-coverage formats mechanically.
  • Supply evidence and state covered and untested scope for every trace checklist item.
  • Mechanical: scripts/check_agreement.py checks the agreement TSV, including same-clause agreement and each mutant's passing unmutated row at the same scale. Reviewing mutant coverage, mutation_reached, evidence and mismatch resolution is partly manual; a nonempty resolution alone does not establish agreement.
  • Require agreement with the coordinator's clean reruns. Timeout, unknown, empty exploration and zero required coverage cannot pass.
  • Complete S4 even when an S7 proof is planned.

4. Rollback

Return a wrong property to S2 and an overly narrow or broad model to S3. Repair instrumentation errors and record new traces within S4. Send real code failures to S6. Carry the trace hash, event index and source anchor. Use a rollback ticket.

Concrete Examples

A PD slot model can pass its configured safety checks while rejecting a real slot-reuse trace. Diagnose the first divergence before using that model to justify a repair.

S5 — Attack model fidelity and gate strength

Read Layer 0: shared operating rules, this stage and the assigned input bundle.

Role and write scope

Red team reads reviewed artifacts and mutates only its own copies. It reports findings; the coordinator verifies them and assigns fixes.

1. Input

The complete S4 bundle and its scope statement, with hashes.

2. Procedure

  1. When delegation is available and authorized, assign independent attacks: test fidelity with missing or extra behaviors; test properties and gates with bad models, weak specs and forged output; test scale with one larger configuration. Attack the open-environment rules: a stimulus whose InputEnabled_<E> is missing or vacuous, a guard that is neither a code condition nor a registered bound, a pruned failure path, fairness on an environment action, an EnvContract_* without a documentation quote, and a named restriction inside a passing configuration.
  2. For each finding record exact command, actual output, intended output, source anchors and confidence. Mark findings as execution-supported when a run supports them, or as reproduced on real code. Items without run evidence remain leads, and so does a refutation that rests only on reading code (Gate 2). A reviewer other than the author records one review verdict per finding in the finding record.
  3. The coordinator reruns findings and merges common causes. Keep every independently confirmed failure, not just one reviewer's list.
  4. Turn a successful bypass into a mechanical check where feasible and record its ablation. Rerun the corresponding attack after gate changes.
  5. On the full adequacy track, confirm each seeded mutant and each revert from the blind set on real code before it counts, then hand scoring to a role that did not build the model (scoring).

3. Exit checks

  • Resolve each execution-supported or reproduced finding, or file a rollback ticket. The coordinator reviews these manually.
  • Update the rule registry with evidence and remaining gaps.
  • Run the corresponding attack regressions for every gate change. The source has an ASSUMES selftest; a complete generic attack suite is not bundled.
  • S5 is mandatory; a model author's success report is insufficient.
  • Apply Gate 2 to every finding S6 hands back: record the refutation attempt (outer guard, documented intent, a freedom the real system lacks) and whether it failed. Record the independent reviewer and one review verdict per finding (partly manual: the script checks the fields, the reviewer checks the claim). A finding without this record cannot be listed as verified.

4. Rollback

Return fidelity defects to S3 and a weak ideal to S2 with property-owner approval. Assign weak verdicts or gates to the coordinator for repair. Never modify the reviewed originals from the red-team role. Use a rollback ticket.

Concrete Examples

For PD KV slot ownership, attack a copied ideal action, an unchecked mapping assumption and an implementation that silently discards a posted write. These exercise mirror, ASSUMES and specification-mutation checks.

S6 — Find and repair bugs

Read Layer 0: shared operating rules, this stage and the assigned input bundle.

Role and write scope

Workers write new search and repair artifacts; approved properties and prior patches stay read-only.

1. Input

The model reviewed in S5, restricted to the scope actually trace-validated.

2. Procedure

  1. Write timestamped predictions for each bug family before running. Enable known repairs so shallow known failures do not hide deeper ones; record every switch.
  2. Parameterize hardcoded constants first. At the old values, require matching verdicts and state counts; only then increase one dimension.
  3. Classify each counterexample as model defect, known bug or new candidate. Attach code steps and a minimal trigger.
  4. Reproduce progressively: source-level reasoning, real control-code replay with stubs, controlled timing on the real system. State the actual level as Gate 3 names it (model only, CPU replay of real code, real engine), never rounded up, and any injected behavior. A named human_verifier requires real engine. Put entrypoint, launch_cmd, preconditions, injected and the hit count with its hit_unit and hit_kind in the finding record, following the recording rules; write unknown for a fact not established, never a placeholder. Build each real-code check from a clause by the four parts of a clause check, and record every refused or ineffective operation in its refusal record. Run one-GPU checks in a queue shared with another team as packed jobs (GPU check jobs): one GPU and own CPUs per run, a time cap, GPU memory back to idle, one claim across queues.
  5. Compare repair candidates in the model: the original failure disappears, other safety and liveness properties hold, and removing the repair restores the failure. Then change code and check unpatched N/N versus patched 0/N under the declared trigger. Add a reference arm: a reference implementation of the same contract where one exists, otherwise a control differing only in something a deployment can vary. Take the expected value from that run, not the author's belief; a patched arm that disagrees with the reference arm is not a validated fix.

3. Exit checks

  • Triage every candidate and record its reproduction scope.
  • Mechanical: scripts/check_refusals.py REFUSALS.tsv --docs DOC_ROOT checks that each refused or ineffective operation answers both questions of the refusal rule with the documentation quoted, and that its verdict follows from them; a refusal of a promised sequence is a failure, not a documented precondition. Whether a quote really promises or forbids the sequence remains for the reviewer. For GPU runs, scripts/gpu_pack.py run passes a job only when every run exits 0 within its cap and every GPU is idle afterwards.
  • Mechanical: scripts/grade_findings.py checks that every finding record is complete and its stored grading fields agree; evidence and reference agreement remain manual. Run it on the record file before handing back and give its command and last line as done evidence: a record it refuses is incomplete and goes back to its author, and nobody grades it by hand (who assigns).
  • For every finding, record its method class, its Gate 1 result (1a and 1b separately) and found_by (checker or reading). A finding that fails 1a or 1b is not reported as caught by formal checking. Gate 2 belongs to S5 and Gate 4 to the coordinator; S6 does not wait for them.
  • Require repair controls and the full regression to match frozen expectations.
  • A known-bug baseline matching expectations is not the same as a repaired implementation passing; require the fixed-state contract when validating a repair.

4. Rollback

Return an unrealizable counterexample to S3, or to S2 if the promise or assumption is wrong. A repair requiring a weaker promise returns to S2 for approval. Use a rollback ticket.

Concrete Examples

For PD KV slot reuse, compare repairs that preserve in-flight accounting across reuse. Check both safe release and eventual release, then remove each repair component to test its role.

S7 — Upgrade to a proof

Read Layer 0: shared operating rules, this stage and the assigned input bundle.

Role and write scope

Proof workers own assigned proof modules and auxiliary lemmas in one shared module tree. The coordinator owns theorem statements, invariant pins and accepted tool and settings manifests.

1. Input

The calibrated model selected in S1, after S4 and S5, with no unclassified mismatch between model and code. Complete invariant discovery first; pin the specification, full inductive invariant, parameter ranges and proved configuration before deductive proof starts.

2. Procedure

  1. Import the pinned model directly. The root concludes exactly its Spec ⇒ □Safety, with parameter ranges and configuration in explicit ASSUME … PROVE premises. Add no module-level ASSUME or AXIOM. Keep auxiliary definitions in the proof layer and machine-check their bridges to pinned definitions. Never rewrite the model to ease proof.
  2. Use one module tree and shared definitions from the start. Assign preservation cells for every invariant conjunct under every action, including all strengthening. Do not prove independent model copies and merge or rename them later.
  3. Prove projection lemmas IndInv ⇒ Conjunct; unfold the full invariant only at this interface. Each preservation cell receives just the old-state conjuncts it needs, including necessary type and domain facts.
  4. Expand each action once to prove effect equations and frame facts for unchanged fields. Reuse those facts in subsequent cells. Prove shared effects as family lemmas, retain every action-to-family bridge, and hide large local definitions after proving their interfaces.
  5. Split remaining goals by quantified object, updated or unchanged object, or old or new set member. Close Init, every preservation cell, stuttering and IndInv ⇒ Safety. Compose only proved results at the root; never use bounded model-checking success as a proof premise.
  6. Control facts and definitions separately: BY ONLY retains already usable definitions. Pair it with HIDE DEF and keep USE DEF inside the smallest subproof scope. A renamed large expression still expands unless hidden.
  7. Follow the proof-run protocol: keep persistent fingerprints, diagnose timeouts in the full module context with a capped step budget, and parallelize large modules at top-level theorem boundaries. Use tlapm --toolbox <start_line> <end_line> --threads 1 for each slice; start with one or two backend threads per slice. Accept slices only with the complete certificate defined in the proof gates. For lasting reductions in frontend work, split large modules and narrow DEF expansion.
  8. Keep at most one active check run per module, across all snapshots and execution hosts. That run may comprise several parallel slice processes; do not run a second whole-module or sliced copy alongside it. Overlap rounds for different modules. Refresh each downstream module unchanged once its own changed dependencies have final passing sources; do not wait for unrelated modules. Preserve exported definitions and theorem statements when repairing proof bodies. Route interface or invariant changes through review and new acceptance; invariant changes also require new pins.

For certified parallel checks, follow the slice-certificate procedure, including native inventory generation, fresh per-slice caches and independent reconciliation of obligation-location multisets.

For proof structure and tool mechanics, consult the TLAPS guide, practical hints, tactics, MongoRaftReconfig proof organization and TLAPS-Bench theorem-range slicing. These references provide no measured performance guarantee for this workflow.

3. Exit checks

  • Pass all proof acceptance gates on the assembled tree, including bridges and strengthening. Retain one complete strict result per module, from a whole-module run or an independently checked slice certificate. Bind each result to its source, full import closure and toolchain; cached leaves or a green root alone are insufficient.
  • Report the root statement and its premises verbatim. Finite-domain induction retains its bounds; an arbitrary-scale claim requires the corresponding quantification.
  • The proved object is the design model. Describe implementation evidence as validation against that model, with its alignment limits.
  • Treat final unproved, timeout, unknown, omitted, interrupted or inconclusive work as incomplete. Record runtime, memory and repeat runs without making them proof gates. No generic proof-integrity/closure checker is bundled.

4. Rollback

Send a reachable induction counterexample to S6. Return a missing premise or invariant change to S2 for review and new pins; do not inherit the old acceptance result. If strengthening fails, revisit S3. Use a rollback ticket.

Concrete Examples

For PD KV slot release, prove the design-model promise only after checking real interleavings and trace conformance. A theorem about an atomic model step does not establish atomicity in the implementation.

S8 — Package and maintain regression

Read Layer 0: shared operating rules, this stage and the assigned input bundle.

Role and write scope

One author per case skill; the coordinator owns gold and independently reviews evidence. Only the coordinator integrates changes.

1. Input

The accepted artifacts from S4, S5, S6 and S7, as required by the S1 route: models, configurations, traces, mutations, expectations and hashes.

2. Procedure

  1. Fill verify-skill: coverage, synchronized revision, assumptions, bounds, one command, expected results, triage rules and the full adequacy readout of the model at that revision.
  2. Run five checks in order: pins, drift, model, trace regression, then mutation regression. Keep the regression small and repeatable; measure and report its duration.
  3. Break protected logic in a source copy: the check must turn red. Restore it: green with the baseline. Record prediction versus actual, including failed predictions.
  4. After upstream changes, run unchanged first. Inspect all changes to covered state, including moved or new writers. Classify each as no effect, model drift, intentional promise change or possible code bug.
  5. Update models from source, not from desired verdicts. Recalibrate after alignment changes and update the sync revision only after alignment succeeds. Keep verification and product-fix changes distinct.
  6. For deductive claims, retain the frozen proof tree, tool and settings hashes, raw logs and full acceptance evidence. Publish the root statement and premises verbatim. Bind every module result, including slice certificates, to its source, full import closure and toolchain. Refresh changed closures through the root; proof-body-only changes permit unchanged downstream reruns with persistent fingerprints, while definition or invariant changes require contract review and renewed acceptance. The five-step regression cannot renew a proof claim alone.

3. Exit checks

  • Record both break and restore results. Require the independent clean regression rerun to match every expected row. Review deductive evidence under the proof gates without imposing a second same-source proof run.
  • Distinguish baseline reproduction from a required-fix check and explain stale frozen trace evidence.
  • Report maintenance as in sync, updated, bug or blocked. A blocked run does not advance the synchronized revision.
  • Identify the proved design model in deductive claims; describe the implementation as validated against it. An independent reviewer must complete the proof gates before a machine-proof claim.

4. Rollback

Return model drift to S3, then rerun S4. Send intentional promise changes to S2 and code violations to S6. Report BLOCKED if tools or alignment are unavailable; do not move the sync commit. Use a rollback ticket.

Concrete Examples

The PD example describes pins, drift, model checks, trace checks and mutations. Frozen trace checks can stay unchanged after a live source mutation; drift and source-derived model variants must detect that change.

Agents 3

vstack-main

You are the coordinator. Read Layer 0: shared operating rules and vstack-mode.

Keep exclusive write authority over spec pins, gold expectations and verdict scripts, and integrate changes yourself. The property owner approves promises; you do not replace that approval. Workers read frozen upstream artifacts.

Before assigning a stage, record its input hashes, writable paths, stage exit checklist, assurance tier and budget in the worker's task brief, generated with scripts/make_brief.py from a fields file of its own and never derived from another brief, and send the brief only when scripts/check_brief.py passes. Let the generator compute the counts a brief states. Confirm the project's resource guard recognizes the role. Assign one owner per output; use independent reviewers only where delegation is available and authorized.

Run units through the coordination loop: you write briefs, accept hand-backs and merge the unit table; you do not census, model or write specifications yourself. Send the first unit of each batch alone and accept it before fanning out the rest. Check each reply with scripts/parse_handback.py --brief, adding --changed and --sibling when workers share a tree, and return a reply whose block does not parse unread. A brief whose outputs include finding records lists python3 scripts/grade_findings.py on each record file, passing, as done evidence. When you merge findings, take every tier, grade and route from the grader's output; never assign or edit one by hand, and send a record the grader refuses back to its author. Close each batch with scripts/check_unit_table.py --plan --final, then review its hand-backs and lessons and register every new check or template field in the rule registry.

On handoff, inspect actual artifacts and independently reproduce model, trace and mutation checks from pinned inputs in a clean directory. For S7, independently recompute source and closure hashes, raw-log verdicts, static gates and any slice certificate under the proof acceptance rules. Do not require a second same-source prover run; rerun specific doubts. Read final obligation states, coverage and missing rows; unfinished or finally unproved work is not PASS.

Schedule at most one active check run per module; that run may contain parallel slices. Overlap rounds for different modules, refresh downstream results when their own changed dependencies pass, and size resources from progress and concurrent load. Follow the proof-run policy; keep resource protection separate from mathematical acceptance.

Before a finding is listed as verified, perform Gate 4: re-run its model and test from a clean copy, read both, and confirm the counts from the raw result files. Confirm the finding record, its computed danger grade and route, and one review verdict per finding from a reviewer other than its author (partly manual). Then confirm the records of the other gates: Gate 1 from S6 where its method needs it, Gate 2 from S5 and Gate 3 at real engine. Only then may a ledger row carry a named human_verifier.

A rollback ticket goes to the responsible upstream owner. Spec changes require the property owner's recorded decision, staged regression, independent review and re-pinning. Do not silently alter an expected verdict to admit a failure.

Report PASS, ISSUES or BLOCKED with scope, hashes, commands, prediction versus actual and remaining limits. Integration authority does not itself authorize publication or external writes.

vstack-redteam

Read Layer 0: shared operating rules and S5: attack model fidelity and gate strength. Restate the intended property, covered source revision and declared scope before attacking it.

Reviewed files are read-only. Change only your own copies; report findings without fixing reviewed originals. Test missing or extra model behavior against pinned code. Test properties and gates by injecting bad models, weakening copied specs or forging copied outputs. Increase a declared bound to check whether the claim extrapolates beyond evidence.

Record predictions before experiments. Each finding needs the exact command, observed and intended outputs, source anchor and confidence. Mark findings as execution-supported only after an actual model-checking or mutation run, or as reproduced after running real code. Items without run evidence remain leads. As a reviewer other than the author, record one review verdict per finding in the finding record; carry any narrowed preconditions into the write-up and computed grade.

Send the final report and rollback proposals to the coordinator. The coordinator independently reruns findings, retains every confirmed distinct failure and assigns gate repairs. Any gate change must rerun its corresponding attack. State tool, context and coverage limitations; no confirmed finding is not a proof of completeness.

vstack-worker

Read Layer 0: shared operating rules, your assigned stage skill and the pinned input bundle. Accept upstream conclusions as inputs; check their current hashes rather than starting the investigation again.

Write only the writable paths of your task brief, write each output as you go, and report every file you change under written. Do not spawn workers or integrate commits. End your reply with the brief's hand-back block: the verdict, each done-evidence command with its log and that log's exact last line, versions, files written and every rule broken. Frozen specs, configurations, gold, theorem targets and upstream outputs are read-only. Create new model or proof versions in your assigned workspace.

Timestamp predictions before calibration, mutation or a new criterion. Use the project resource guard for expensive work, keep process-group records, and stop your own recorded groups before handoff. Preserve append-only worklogs and a handoff updated at least every 15 minutes.

For proof work, follow S7. Preserve exported definitions and theorem statements when repairing proof bodies. Keep persistent fingerprints and one active check run per module, including its slices. Deliver source, full import closure and toolchain identities with raw results; refresh affected downstream modules unchanged after their own changed dependencies pass.

For an upstream defect, write a rollback ticket with an anchored counterexample or first trace divergence. Do not weaken a promise or rewrite a pin to fit a result. Strengthening IndInv is distinct from changing Safety.

Deliver each stage's input, procedure, exit checks and rollback record, with actual commands and evidence. Name unverified or incomplete obligations; no timeout or unknown becomes PASS. A bounded model result stays bounded, and a design-model theorem is not an implementation theorem.

Sponsors

Details

Marketplace
hao-ai-lab/vstack
Author
Hao AI Lab
Category
Community
License
MIT