Aperivue avatar

medsci-analysis

Reproducible statistical analysis, publication-ready figures, batch/cross-national/replication analysis, and meta-analysis synthesis.

GitHub stars
387

Install

/plugin install medsci-analysis@medsci-skills

Run it inside Claude Code after adding the marketplace with /plugin marketplace add Aperivue/medsci-skills.

Contents

Discovered from source

Skills 62

Academic AIO Skill — Medical AI Paper Visibility for AI Search Engines

Communication Rules

  • Surface the checklist in the response. Never apply AIO edits silently.
  • When a rule conflicts with journal formatting, defer to the journal and mark the item NA with the reason.
  • When introducing a rule, cite its source (TRIPOD+AI, CLAIM, STARD-AI, Wu 2025, Algaba 2024, Aggarwal 2024 GEO) by DOI or arXiv ID from External References. Never invent a citation, DOI, arXiv ID, or reporting-guideline item number; mark anything you cannot verify [VERIFY].

Section 1 — Title and Abstract Optimization

1.1 Title three-slot rule

Structure: [Task] + [Modality or anatomy] + [Model family or method class]. Include one concrete differentiator (dataset scale, new benchmark, "first …") when defensible. Avoid keyword stuffing (penalized as spam by AI overviews).

  • PASS: "Transformer-based segmentation of skull fractures on non-contrast head CT."
  • FAIL: "A novel advanced deep-learning AI machine-learning framework for medical image analysis."

1.2 Structured abstract

Use the journal-required structure (Background / Methods / Findings / Interpretation for the Lancet family; Background / Purpose / Materials and Methods / Results / Conclusion for the RSNA family; etc.). If the journal allows unstructured, still structure it internally. Each section stands alone as a semantic chunk of ≤ 3 sentences so that RAG chunk-boundary splits do not break the claim.

1.3 Opening and closing sentences

  • First sentence: the problem AND the contribution in one line. Never open with filler ("In recent years, AI has revolutionized …") — it burns the chunk LLM summarizers extract most.
  • Last sentence: explicit interpretation ("we show that …", "this implies …"). No hedging-only closes.

1.4 Taxonomy line

Include one sentence that names the field's controlled vocabulary ("diagnostic-accuracy study", "foundation-model evaluation", "LLM-as-judge", "agentic radiology workflow"). Entity linkers in AI indexes use this line.

1.5 Quantified claim

Every abstract must contain at least one numeric primary outcome with a confidence interval (for example, "AUC 0.94 [95 % CI 0.91–0.96]").

1.6 Reporting-guideline anchor

Name the guideline in the abstract or the opening sentence of Methods: "Reported following TRIPOD+AI (Collins 2024) and CLAIM 2024 (Tejani 2024)". Add STARD-AI 2025, DECIDE-AI, or TRIPOD-LLM when applicable. AIO-rule ↔ guideline-item mapping: references/reporting_guideline_mapping.md.

1.7 Keyword, MeSH, and RadLex coverage

Title, abstract, and keywords together should cover the concept's key terms, without repeating title terms in the keywords (92 % of the surveyed papers repeated key terms across title, abstract and keywords; Royal Society 2024, doi:10.1098/rspb.2024.1222). Include:

  • Core MeSH terms (verify against the NLM MeSH browser) and RadLex terms where applicable.
  • Modality synonyms ("chest radiograph (CXR)", "non-contrast CT (NCCT)").
  • Both US and UK spellings when relevant.

Section 2 — Manuscript-Level AIO

2.1 Summary box

Include the journal-specific summary box verbatim when supported; it is the fragment AI search engines most often copy or paraphrase:

  • Lancet family: "Research in context" (Evidence before this study / Added value / Implications).
  • RSNA Radiology and RYAI: 3 bullets, one claim each — labelled "Key Results" for Radiology and "Key Points" for RYAI in references/journal_summarybox_templates.yaml. Confirm the label in the current RSNA instructions for authors; the format check accepts either label for Radiology.
  • npj Digital Medicine: no summary box; Articles carry an unstructured abstract of up to 150 words.
  • Nature Medicine: editor's summary (supplied by editorial, but draft one proactively).

Journal-specific templates: references/journal_summarybox_templates.yaml. Never invent a summary-box rule: verify each template against the journal's current instructions for authors before applying it.

Deterministic format check. Validate the drafted box against its journal spec with python3 ${CLAUDE_SKILL_DIR}/scripts/check_summary_box.py --manuscript <file> --journal <stem> --strict (reads references/summary_box_specs.json: Key Points / Key Results top-level bullet count + one-claim-per-bullet, Research-in-context's three sub-blocks each opening a line and carrying text, plain-language word band). It catches the wrong-format / wrong-bullet-count box that a production technical check rejects.

Known limits. The box ends only at a heading or a bold-only label line, so a list under a plain-text label (Abbreviations:) right after the box is counted into it, and an emphasized body phrase that starts with the box label (*Key points* of prior work ...) is taken as the box; read the box by eye when either shape is present. In a Research-in-context box, a bold-only line after the last sub-block label ends the box, so a sub-heading inside Implications (**For clinicians**) cannot be told from the next section; an Implications sub-block left empty that way is reported as an advisory, not a failure.

2.2 Declarative section headings

Headings state a claim, not a generic label: "Model underperforms on rare-finding subset" beats "Subgroup analysis".

2.3 Numeric claim compression

In the Methods and in at least one Results paragraph, compress primary-outcome statistics into one sentence: "On the internal test set (n = 842), the model achieved AUC 0.94 (95 % CI 0.91–0.96), sensitivity 88.2 % (85.1–91.0), specificity 91.4 % (88.7–93.6), at an operating point of 0.37."

2.4 Reproducibility block

Include a labeled block (end of Methods or a standalone Data/Code Availability section) listing data availability and license, code availability with DOI, model weights and checkpoints, prompts and configuration files, random seeds, and compute environment. Flag code described only as "available on reasonable request": scrapers read it as not reproducible and AI tools demote the paper.

2.4a Citing an AI-assisted tool by use-class

Frame an AI-assisted tool by what it did, not by hiding it: verification/QA and analysis uses go in a Software / Code-availability statement (citable); generative drafting or humanizing goes in the journal's AI-use disclosure field, not a citation. A self-citation by the tool's author also requires a COI disclosure and should cite only the functions the work used. Use-class table and rules: ${CLAUDE_SKILL_DIR}/references/ai_tool_citation_framing.md.

2.5 Limitations enumeration

List limitations explicitly and name each one (generalizability, spectrum bias, dataset shift, single-center training, label noise). Flag "clinical grade" or "replaces radiologists" overclaims: LLM trust heuristics demote them and they invite reviewer rejection.

2.6 Standalone figure captions

Each caption re-states the claim, the dataset, and the metric, because captions survive in vector and image-retrieval indexes when the body text is lost.

Section 3 — Preprint, Channel, and Indexing Strategy

3.1 Preprint versus fast-track

  • Default: post to medRxiv (clinical), arXiv (methods, cs.CV / eess.IV), or bioRxiv on the day of journal submission. This puts the paper into Semantic Scholar within 24–72 hours and into Perplexity's web index immediately.
  • Exception: if the target journal offers a fast-track review cycle (acceptance → online within roughly 30–60 days) AND the authors prefer a single canonical version, the preprint may be skipped; compensate with aggressive post-acceptance SNS seeding and PMC deposit.
  • Never skip both preprint and fast-track — a paywall-only paper is invisible to most RAG pipelines.

3.2 Journal preprint policy

Most medical-AI venues allow preprints; a few restrict them or require disclosure. Verify the current policy on Sherpa Romeo or the journal's instructions for authors before posting.

3.3 Indexing time-lag

Read ${CLAUDE_SKILL_DIR}/references/launch_sequencing.md when planning launch timing; it lists how long each index takes to pick a paper up.

3.4 Open-access choice

Prefer gold OA with CC-BY when budget allows; otherwise green OA via preprint plus author-accepted manuscript. Closed-access papers without a preprint lose roughly 30–50 % of AI-tool citations because Elicit, Consensus, and Perplexity Academic cannot extract from paywalled PDFs.

Funder OA-policy decision tree (Plan S, NIH, UKRI, Gates, Wellcome, NRF, MoHW): references/oac_funding_checklist.yaml.

3.5 Post-acceptance channel checklist

  • Deposit AAM to PMC or Europe PMC.
  • Update ORCID and Google Scholar profile.
  • Post to Threads / X / BlueSky with DOI, one-sentence claim, and figure; long-form post on LinkedIn.
  • Submit to Papers with Code if the paper reports a benchmark.
  • Upload model or dataset to Hugging Face with a model/dataset card.

Section 12 gives the day-by-day order.

Section 4 — Review-Paper Strategy

Read ${CLAUDE_SKILL_DIR}/references/pre_draft_strategy.md when the artifact is a review or taxonomy paper, or the phase is pre-draft.

Section 5 — GitHub, CITATION.cff, Zenodo, Hugging Face

Read ${CLAUDE_SKILL_DIR}/references/repository_and_cards.md when the artifact is a README, CITATION.cff, Zenodo record, or Hugging Face model/dataset card (rules 5.1–5.6).

  • Embed JSON-LD ScholarlyArticle / SoftwareSourceCode / Dataset / Person markup in repository pages and author landing pages — templates in references/schema_markup_templates/, validated with python scripts/validate_schema.py path/to/file.jsonld.
  • Never auto-complete author lists, ORCIDs, or affiliations in CITATION.cff or Zenodo metadata; surface the empty slots to the user.

Section 6 — Authority and E-E-A-T Signals

  • Maintain an author landing page (GitHub Pages, personal domain, or institutional page) listing all papers with DOIs and open-access links.
  • Use one consistent affiliation string across papers; inconsistency fragments the author entity in knowledge graphs.
  • Keep ORCID complete and linked to Google Scholar. Re-run author disambiguation on Semantic Scholar every 6 months.
  • Cross-link related papers by the same group in Discussion sections when defensible; within-group citation raises co-retrieval in RAG.
  • Refresh repository and model cards quarterly; quarterly-updated articles outperform single-publish ones in AI-overview retention (Conductor 2026 benchmark).

Section 7 — LLM-Citation Fabrication Defense

Between 50 % and 90 % of LLM answers to medical questions are not fully supported by the sources they cite (Wu et al., Nat Commun 2025, doi:10.1038/s41467-025-58551-6). Defend the paper's identifiers:

  • Surface DOI and PMID in copy-friendly text at the top of the paper's landing page and README (for example, DOI: 10.xxxx/yyyy • PMID: 12345678).
  • Add a "How to cite" section with BibTeX, APA, Vancouver, and the plain-text line in one place.
  • Monitor incorrect citations: a Google Scholar alert for the paper's title variant, plus periodic Perplexity and ChatGPT web queries that record hallucinated bibliographic errors. Report discoverability (Perplexity / Elicit / Consensus retrieval) only from a recorded probe; never estimate it.
  • When a reviewer cites an LLM-generated reference, verify the DOI and PMID yourself before accepting it.

Section 9 — Per-Project Application and Pipeline Integration

When invoked, run in this order:

  1. Read the target artifact (title, abstract, manuscript section, README, or card) and identify its lifecycle phase: pre-draft / drafting / pre-submission / post-acceptance / post-publication.
  2. Apply Sections 1–5 and 10 relevant to that artifact, filtering each rule by its applies_to_phase field in references/checklists/AIO_GENERAL.md. Out-of-phase rules become NA rather than FAIL (e.g., do not surface §11.5 multi-disciplinary roster or §12 launch sequencing as FAIL on a pre-submission audit). Produce a PASS / PARTIAL / FAIL table, one-line reason and concrete fix per item, sorted by expected_lift (high → medium → low), in the output template of references/checklists/AIO_GENERAL.md. Render via templates/aio_audit_checklist.md.j2 when programmatic.
  3. Honour defers_to annotations to avoid duplicate audits. Items annotated with a defers_to field record only present/absent status here; item-level detail belongs to the linked skill or reference (§1.6 → /check-reporting; §3.4 / §11.3 → references/oac_funding_checklist.yaml). When the manuscript has not had a reporting-guideline audit, invoke /check-reporting first; the AIO ↔ guideline-item mapping is in references/reporting_guideline_mapping.md.
  4. Apply the Section 6 author-authority audit once per submission cycle, and Sections 11–12 as the applies_to_phase filter allows.
  5. Surface Section 7 citation-defense recommendations at post-acceptance time. For multi-repo or Hugging-Face-card team audits, run scripts/batch_metadata_audit.py.
  6. Output:
    • The phase-filtered checklist (visible).
    • A short deferred-item list with one-line status per defers_to rule.
    • At most 5 concrete edits ranked by expected_lift (high first, then medium, then low). A low-lift edit enters the Top 5 only when no high / medium item remains open.

Integration with other skills

  • Run after /self-review and /humanize, so QC-confirmed claims and the final human-readable text anchor the checklist.
  • Within /write-paper: apply Section 1 while drafting the title and abstract, Sections 2.5 and 6 (cross-linking) while drafting the Discussion, and the full checklist at QC, after the reporting-guideline check and numerical-claim audit.

Section 10 — Q&A and Entity-Extraction Optimization

10.1 Four-question Q&A block (Discussion or Appendix)

Add a labeled Q&A block, either as the closing subsection of Discussion or as a Supplementary Box:

  • What was known before this study? — two-sentence restatement of the prior state.
  • What does this study add? — two-sentence statement of the contribution.
  • How might this change clinical practice or research? — one-sentence interpretation; avoid overclaim.
  • Why does this matter? — one-sentence "so what" framing for non-specialists.

Lancet Digital Health "Research in context" already encodes the first two questions; the block extends them.

10.2 Glossary block with entity IDs

Define each domain-specific acronym inline on first use AND list them in a Glossary subsection at the end of Methods or in the Supplement, with the canonical entity ID where possible: MeSH term ID (clinical concepts), RadLex ID (radiology terms), UMLS CUI (cross-vocabulary mapping), Hugging Face model ID (named models), arXiv ID (cited methods).

10.3 Inline citation anchor text

Avoid bare reference numbers; bind each citation to its specific claim:

  • WEAK: "Prior work [12] showed efficacy."
  • STRONG: "Smith et al. (DOI: 10.xxxx/yyyy) reported a 12 % accuracy gain on the MIMIC-CXR test set [12]."

When citing one's own prior work, name the cohort or dataset explicitly to enable cross-paper retrieval.

10.4 Explicit challenge statement

Beyond the 2.5 limitations list, include a single-paragraph "Why this is hard" statement near the start of Discussion:

"Building accurate [task] for [modality/anatomy] is constrained by [data scarcity / label noise / dataset shift / regulatory uncertainty / interpretability]. Each of these has been documented [refs], and our results address [subset]."

LLM web-search systems quote challenge statements as authoritative summaries of field state (worked case: references/case_studies/kjr_mllm_2025.md).

Section 11 — First-Mover Timing and Citation-Graph Density

Rules 11.1 (topic-peak detection), 11.2 (editorial-board leverage), and 11.5 (multi-disciplinary author roster) are in ${CLAUDE_SKILL_DIR}/references/pre_draft_strategy.md; read it at pre-draft.

11.3 PMC-auto-deposit journal preference

OA journals that auto-deposit to PubMed Central reach LLM crawlers within 4–6 weeks of publication; non-PMC OA journals can take 3–6 months. When all else is equal, prefer a PMC-auto-deposit journal. Radiology/medical-AI examples (verify per submission; policies change):

  • Korean Journal of Radiology (KJR) — auto-deposit confirmed.
  • Lancet Digital Health — author-funded green OA, PMC-eligible after embargo.
  • Radiology and Radiology: AI — selected articles auto-deposit.
  • npj Digital Medicine — auto-deposit (Nature OA).
  • JAMIA — author-funded OA route.
  • JMIR — auto-deposit (PMC-indexed).

11.4 Citation-graph anchor strategy

Anchor the Discussion in 5–10 high-visibility prior works that LLM training corpora already index well, so the paper is retrieved when users query those works.

  • Identify seminal references via Semantic Scholar's "Highly Influential Citations" filter for the topic.
  • Cite them with semantic predicates (Section 10.3), not as bare lists.
  • Mix recent preprints (currency) with 2018–2022 seminal papers (graph anchoring); a 2024–2025-only citation profile has low LLM retrieval weight because of corpus cutoffs.

Section 12 — Cross-Platform Launch Sequencing

At post-acceptance / post-publication, read ${CLAUDE_SKILL_DIR}/references/launch_sequencing.md for the Day 0 → Month 1 sequence (rules 12.1–12.5).

External References

  • GEO: Generative Engine Optimization — Aggarwal et al., KDD 2024, arXiv:2311.09735.
  • LLM medical citation support — Wu et al., Nat Commun 2025, doi:10.1038/s41467-025-58551-6.
  • LLM citation bias — Algaba et al., 2024, arXiv:2405.15739.
  • ExpertQA attribution — Malaviya et al., 2024, arXiv:2309.07852.
  • TRIPOD+AI — Collins et al., BMJ 2024. EQUATOR Network.
  • CLAIM 2024 — Tejani et al., Radiology: AI 2024, doi:10.1148/ryai.240300.
  • STARD-AI — Sounderajah et al., Nat Med 2025, doi:10.1038/s41591-025-03953-8.
  • TRIPOD-LLM — Gallifant et al., Nat Med 2024, doi:10.1038/s41591-024-03425-5.
  • DECIDE-AI — Vasey et al., Nat Med 2022, doi:10.1038/s41591-022-01772-9.
  • Title, abstract, keywords guide — Royal Society Proc B 2024, doi:10.1098/rspb.2024.1222.
  • GitHub repository citation advantage — Kang et al., Inf Process Manag 2023, doi:10.1016/j.ipm.2023.103477.
  • Semantic Scholar Open Data Platform — Kinney et al., arXiv:2301.10140.

Add Journal Skill

Generate two profiles per journal from its author guidelines: a detailed write-paper profile (~100-150 lines) and a compact find-journal profile (~30 lines). Write profile content in English; the library is shared.

Key Directories

Profiles live in one of two tiers. Pick the target in Phase 0, before any extraction.

  • Public (shipped; must meet the verification bar in ${CLAUDE_SKILL_DIR}/../find-journal/POLICY.md):
    • write-paper: ${CLAUDE_SKILL_DIR}/../write-paper/references/journal_profiles/
    • find-journal: ${CLAUDE_SKILL_DIR}/../find-journal/references/journal_profiles/
  • User-local private (per-user, never pushed to git):
    • write-paper: $HOME/.claude/private-journal-profiles/write-paper/
    • find-journal: $HOME/.claude/private-journal-profiles/find-journal/

POLICY.md also documents promotion from private to public.


Phase 0: Input Collection

  • Required: full official journal name; target tier (public or private).
  • Strongly encouraged: Author Guidelines URL. If the user gives only a name, ask for this URL before proceeding; it is the primary data source.
  • Optional: field focus (used in 3.1), tier estimate (Q1/Q2/Q3), ISSN.

This skill does not judge whether a journal is legitimate. For an unfamiliar publisher, check DOAJ membership (for open-access titles) and indexing before writing a profile, and tell the user.

Public vs. Private target selection

If the user does not state the target tier, ASK before proceeding.

  • Default to private for any journal outside the user's stated area of deep expertise. Private profiles need not clear the public bar line by line, but the journal's own Author Guidelines page must still be the primary source: no inference from adjacent journals, no AI policy copied from the publisher family.
  • public only when all of these hold:
    1. The user intends to contribute the profile upstream.
    2. The user has opened the journal's homepage and author guidelines page and can attest to each field against the live source, or can paste the relevant sections.
    3. AI policy wording is transcribed from the journal's or publisher's own policy page, not inferred from a sibling journal.
    4. The user has completed or will complete the promotion checklist in ../find-journal/POLICY.md before commit.

If the target is public but the user cannot attest to (2) and (3), switch the target to private and say that promotion can happen later once the checklist is cleared.


Phase 1: Duplicate Check

Before any extraction:

  1. Glob all four directories:
    ${CLAUDE_SKILL_DIR}/../write-paper/references/journal_profiles/*.md
    ${CLAUDE_SKILL_DIR}/../find-journal/references/journal_profiles/*.md
    $HOME/.claude/private-journal-profiles/write-paper/*.md
    $HOME/.claude/private-journal-profiles/find-journal/*.md
    
  2. Check for an exact filename match on the normalized name (4.1 rules).
  3. Grep existing profiles for the journal's common abbreviation (e.g., "JCO").

If found, report the paths. If present in only one directory, offer to create only the missing counterpart; if in both, offer to update or abort. Do NOT proceed to Phase 2 without user direction.


Phase 2: Data Extraction

  • Path A (WebFetch available): fetch the Author Guidelines URL and parse it. If the page is login-gated, JavaScript-heavy, or sparse, fall back to Path B.
  • Path B: ask the user to paste the Author Guidelines content (or key sections). Never fail silently when a fetch fails.

Extraction Checklist

Mark any field that cannot be determined as [TODO: verify at journal site].

For the write-paper profile: full name; abbreviation; publisher; ISSN (print/online); frequency; Impact Factor with its JCR year; OA model; acceptance rate; peer-review type; manuscript types with word, abstract, reference and figure limits; abstract format (structured or not, headings, word limit); required sections for an Original Article; statistical requirements (p-value format, CI, effect sizes); figure specs (DPI, format, color, max count); cover letter requirements.

For the find-journal profile: a 1-2 sentence scope paragraph; 15-20 comma-separated scope keywords; accepted article types; classification (tier Q1/Q2, OA, field).

--- Gate 1: Metadata Confirmation ---

Present the extracted metadata as a structured summary. Ask:

  1. "Is this information accurate? Any corrections?"
  2. "What are the common rejection reasons for this journal?" (if not extractable)
  3. "How would you position this journal relative to similar journals in the field?"

If the user's information conflicts with the source, present the conflict and ask. Do NOT proceed to Phase 3 until the user confirms or corrects.


Phase 3: Profile Generation

3.1 Load Reference Template

Read ONE existing write-paper profile from a similar field as a format reference:

FieldTemplate to Load
RadiologyRadiology.md or European_Radiology.md
General medicineThe_BMJ.md or JAMA.md
Medical educationJMIR_Medical_Education.md
AI / digital healthnpj_Digital_Medicine.md or JMIR.md
IR / interventionalCVIR.md or JVIR.md
SurgeryAnnals_of_Internal_Medicine.md
OtherThe_BMJ.md (safest general template)

Also read ONE existing find-journal profile for the compact format.

3.2 Generate Write-Paper Profile

The detailed profile /write-paper loads when drafting for this journal. Read ${CLAUDE_SKILL_DIR}/references/write_paper_profile_template.md when writing it and fill the literal template.

Follow the canonical 11-section order exactly. /write-paper Phase 7 and /find-journal read these profiles positionally — a reordered, renamed, or omitted section silently degrades both:

  1. Journal Identity — full name, abbreviation, publisher, ISSN, frequency, impact factor, OA model, acceptance rate, peer-review type
  2. Manuscript Types and Word Limits
  3. Abstract Requirements — word cap and heading structure
  4. Required Sections (Original Article)
  5. Statistical Reporting — MANDATORY. If the guidelines specify nothing, use these defaults and mark them [TODO: verify at journal site]: exact p-values to 2-3 significant figures (p < 0.001 below that); 95% CI for primary outcomes; effect sizes in clinically meaningful units; statistical software and version identified.
  6. Figures
  7. Common Rejection Reasons
  8. Cover Letter
  9. AI Writing Disclosure Policy — permitted scope, disclosure location, AI-generated-image stance, and the policy URL
  10. Author Guidelines URL
  11. Positioning — when to submit here, when not to, and a comparison table against 2-3 similar journals (society, scope emphasis, impact factor range, distinguishing features)

Fill rules. Never guess a value. NEVER invent an Impact Factor, acceptance rate, APC, ISSN, peer-review timeline, word limit or abstract structure, because an invented value propagates straight into a drafted manuscript: mark it [TODO: verify at journal site] instead. Use an approximate range (e.g., "~15-20%") only when the source supports it. Where the journal publishes no dedicated AI policy, cite the publisher-level policy and say so explicitly.

3.3 Generate Find-Journal Profile

Follow the canonical format exactly:

# {Full Name}

## Identity
- **Abbreviation:** {abbrev}
- **Publisher:** {publisher}
- **ISSN:** {print} / {online}
- **Homepage:** {URL}
- **Author guidelines:** {URL}

## Scope
{1-2 sentence scope description.}

## Scope Keywords
{comma-separated keywords, 15-20 terms}

## Article Types Accepted
- Original Article
- Review Article
- ...

## Classification
- **Tier:** {Q1/Q2}
- **Open Access:** {Full OA / Hybrid / Subscription}
- **Field:** {field}

## Special Notes
{2-3 sentences on positioning, unique aspects, society affiliation.} AI policy: {1-line summary from write-paper profile's AI Writing Disclosure Policy section — e.g., "language editing only, dual disclosure required, AI images banned." or "follows ICMJE — disclose AI use in Methods."}

--- Gate 2: Profile Review ---

Present BOTH complete draft profiles. Ask:

  1. "Review these profiles. Any corrections needed?"
  2. "Is the Scope paragraph accurate?"
  3. "Are the Common Rejection Reasons realistic?"

Do NOT write files until the user approves both profiles.


Phase 4: Write Files and Update Counts

4.1 Determine Filename

Spaces to underscores, special characters removed: "Journal of Clinical Oncology" -> Journal_of_Clinical_Oncology.md; "JACC" -> JACC.md; "JAMA Network Open" -> JAMA_Network_Open.md.

4.2 Write Profile Files

Write the write-paper profile and the find-journal profile, each as {filename}.md, into the directory pair chosen in Phase 0 (Key Directories). For private, create the two private directories if missing. Do NOT write private profiles anywhere inside the medsci-skills git tree.

4.3 Update Profile Counts

Do not edit SKILL.md, README.md or any other file to update a profile count: counts are derived from disk at runtime, and private profiles are never counted.

4.4 Confirmation

Report the target tier and the paths written, then:

  • public: remind the user to commit the two profile files (staged by path) with a message recording which pages were opened and on what date, as POLICY.md requires.
  • private: "Files are local-only in your private profiles directory (outside this repo) — do not commit to the public repo."

Batch Mode

For several journals in one session:

  1. Collect all journal names and URLs upfront.
  2. Run Phase 1 (duplicate check) for all journals first.
  3. Run Phases 2-3 per journal, with both gates.
  4. Write all files in a single Phase 4 pass.

Statistical Analysis Skill

Generate reproducible code (Python preferred, R when necessary) for medical research analyses, run it, and produce publication-ready tables and figures from its output.

Data Privacy Check

Before reading any data file, check whether it might contain Protected Health Information (PHI):

  1. If *_deidentified.* files exist in the working directory, use those.
  2. If only raw CSV/Excel files exist, ask the user whether the data contain patient identifiers (names, national ID / RRN, contact details); if so, have them de-identify it first with /deidentify.
  3. Proceed once the user confirms the data are de-identified or contain no PHI.
  4. NEVER display raw PHI values (names, phone numbers, RRN) in your output, because console output is copied into transcripts and drafts. If you encounter them, warn the user and suggest /deidentify.

Reference Files

  • ${CLAUDE_SKILL_DIR}/references/templates/ — reusable analysis scripts; read the matching one before generating code.
  • ${CLAUDE_SKILL_DIR}/references/analysis_guides/ — methodology guides; the Phase 2 table says which to read.
  • ${CLAUDE_SKILL_DIR}/references/table-standards/ — table-standards.md (universal and AMA rules, footnote order, gtsummary pipeline), journal-profiles/ (YAML per journal: radiology, jama, nejm, lancet, european_radiology, ajr), table-types/ (one template per table type), tool-comparison.md (R/Python table tools).
  • ${CLAUDE_SKILL_DIR}/references/style/figure_style.mplstyle — figure style.

Workflow

Phase 1: Data Assessment

  1. Read the data file (CSV, Excel, TSV, or other tabular format).
  2. Report to the user: shape (rows x columns); column names and inferred types (continuous, categorical, ordinal, binary, datetime); missing values per column (count and percentage); first 5 rows; unique value counts for categorical columns.
  3. Identify the analysis unit: patient, exam, lesion, image, rater, study, etc.
  4. If a variable name or coding is uncertain, write [VERIFY: variable_name] and ask the user to confirm it against the data dictionary; never guess a column name or coding.

Phase 2: Analysis Plan

Observational designs (cohort, case-control, cross-sectional, registry, survey): before planning, look for a variable_operationalization.md from /define-variables (or an equivalent codebook-backed definition table). If none exists, warn the user and recommend running /define-variables first, because exposure/outcome/covariate definitions and cutoffs invented ad hoc from the data dictionary are a common reason reviewers reject observational work. This is a warning, not a block: proceed on explicit user confirmation and record that the artifact was not available.

Propose an analysis plan; the user decides the research question and endpoints. Mark any clinical definition, cutoff, diagnostic criterion or guideline claim you could not confirm [VERIFY].

  1. Detect the analysis type from the table in Analysis-Specific Guidelines, or accept the user's specification.

  2. List the specific tests to be performed.

  3. Identify primary and secondary endpoints.

  4. State the assumptions that will be checked (normality, homogeneity, independence).

  5. Note data cleaning needed (recoding, outlier handling, missing-data strategy).

  6. Anchor the estimand to the research question.

    • Interaction, synergy or effect modification: the primary estimand is the interaction parameter itself (a likelihood-ratio test of the interaction term, or the interaction OR/HR on one stated scale) — not a main-effect OR whose CI is then read as "no synergy". A public-health or biological synergy claim is additive-scale: report RERI, AP or S, each with a CI (R interactionR / epiR; Knol & VanderWeele reporting), not only a multiplicative product term, because a non-significant multiplicative interaction is compatible with a large additive one (and vice versa). Joint categories (high/high vs low/low) show joint association, not interaction; "stronger in A than in B" from separate stratum estimates is the difference-in-significance fallacy — report the formal interaction term.
    • Equivalence or non-inferiority: declare the margin up front (a TOST procedure, or the CI compared against a pre-stated MCID); a non-significant difference is not equivalence without a margin.
  7. Screen every categorical/binary predictor for separation before fitting anything. A predictor that perfectly predicts the outcome has no finite MLE, and the failure is silent: glm returns an odds ratio near 0 (or enormous), p ≈ 0.99, and an AUC that ends up in a table. Pathognomonic imaging signs (T2-FLAIR mismatch, the string sign, a halo sign) cause it routinely: 100% specificity means an empty cell by construction.

    python3 "${CLAUDE_SKILL_DIR}/scripts/check_separation.py" \
      --data cohort.csv --outcome idh_mutant --auto --strict
    

    COMPLETE_SEPARATION (an empty cell) and QUASI_SEPARATION (a cell below the sparsity floor) both halt the plan. With two or more predictors the script also tests them jointly (a linear combination can separate the outcome when no single predictor does); that test needs scipy, and without it the report says the joint check was not run. The remedy is a design decision, made in the plan: Firth's penalised likelihood keeps one model, while a two-stage rule — classify the sign-positive cases directly, model only the sign-negative remainder — is usually the clinically meaningful choice for a pathognomonic sign, because a sign-positive patient is already diagnosed.

Present the plan and wait for user approval before executing.

Phase 3: Execute

Generate and run a Python (preferred) or R script. Every number you report MUST come from its executed output, never estimated or hand-typed, because a number no script reproduces cannot be verified.

Script Structure

Start every script with a reproducibility header:

"""
Analysis: {description}
Date: {YYYY-MM-DD}
Random seed: 42
Python: {version}
Key packages: {package==version, ...}
"""
import numpy as np
import pandas as pd
np.random.seed(42)
Execution Rules
  1. Random seed: always np.random.seed(42) or set.seed(42).
  2. Figure style: always load the matplotlib style file:
    import matplotlib.pyplot as plt
    style_path = os.path.join(os.environ.get('CLAUDE_SKILL_DIR', '.'), 'references/style/figure_style.mplstyle')
    if os.path.exists(style_path):
        plt.style.use(style_path)
    
  3. Output files: save next to the input data, or in a user-specified output directory.
  4. Tables as CSV plus a printed markdown version (see Output Conventions); figures as PDF (vector) and PNG (300 DPI).
  5. Console output: a summary with every number that would appear in the text, formatted for direct copy-paste into the Results section.
Assumption Checking

Choose the summary and test from the design and the distribution, not a preliminary test:

  • Shape: QQ plot / histogram and skewness; for Table 1, |skewness| > 1 → median (IQR) with a rank test, else mean (SD) (table-types/table1_demographics.md). Never gate on a Shapiro-Wilk or KS P value: it flags trivial departures at large n, misses real ones at small n, and test-then-test inflates the type I error (Rochon et al. 2012).
  • Unequal variances: default to Welch's t / Welch ANOVA, not a Levene gate; Mann-Whitney tests a different hypothesis and does not fix them. State the choice and reason in Methods.
Stratified & Ordinal-Trend Reporting
  • Strata disjointness gate (before any ordinal trend test). Before a Cochran-Armitage trend test (or any analysis that treats tiers as an ordered partition), assert that the strata are mutually exclusive and exhaustive: sum(n per stratum) == unique N and sum(events per stratum) == total events. A trend test on overlapping or non-exhaustive strata is invalid. Print the per-stratum N/event table and the reconciliation (the analysis-side mirror of /self-review check_cohort_arithmetic.py PARTITION_OVERLAP).
  • Secondary stratum HR/OR: report each with (a) its reference contrast (the referent category), (b) the event count in each stratum, and (c) a sparse-stratum caveat when any stratum has few events (< 10 is unstable). "HR 1.55 in lean participants" without the referent and the events is uninterpretable.
  • Proportion CI lower-bound clamp: clamp every lower bound to max(0, lower). A zero-event Wilson/score interval can print a negative or tiny-exponent bound (e.g. 3.47e-16) that is a display artifact; report 0 (or 0.0%), and prefer an exact (Clopper-Pearson) interval for zero/near-zero cells.
Output Manifest

After all analyses complete, save _analysis_outputs.md in the output directory, in the format given in references/analysis_run_workflow.md, so /make-figures and /write-paper can find the outputs without asking.

For prespecified binary predictions on independent units, use the bundled scripts/run_analysis.py run workflow in that file: it embeds data/configuration/code/output hashes, exact counts, metric-specific denominators and the reproduction command in the same manifest (audit checks recorded versions without rewriting them; compare separates declared context and numeric equality from byte drift). It does not select thresholds or establish study validity, privacy clearance or reuse rights. Synthetic example: python3 ${CLAUDE_SKILL_DIR}/scripts/demo_analysis_run.py --out demo-project.

Phase 3.5: Generated-Code Quality Gate

Before reporting any script as final, lint every emitted .py/.R file:

python3 ${CLAUDE_SKILL_DIR}/scripts/check_generated_code.py {script.py} --strict
# or scan a whole output directory:
python3 ${CLAUDE_SKILL_DIR}/scripts/check_generated_code.py --code-dir {analysis_dir} --strict

Fix every Major before reporting the script: MISSING_SEED (randomness with no seed), HARDCODED_DATA_LITERAL (hand-typed, table-shaped data instead of read_csv()/read.csv() + subset), HARDCODED_ABS_PATH (non-portable and a PII risk), and INPLACE_SOURCE_OVERWRITE (writing to the path read as input — never modify raw data; write derived outputs to a new path). Fix the flags DEBUG_LEFTOVER and UNUSED_IMPORT when tidying. Minor, Python only: API_DEFAULT_STUDENT_T (scipy's ttest_ind without equal_var is Student's t; state Welch or the reason for Student) and API_DEFAULT_PENALIZED_OR (LogisticRegression with none of penalty, C, l1_ratio, in a file that exponentiates a coef_). R's t.test() is Welch by default.

Phase 4: Report

After execution, generate manuscript-ready text from the script's output:

  1. Results paragraph: 3-8 sentences with specific numbers, formatted as:
    • Continuous: "mean +/- SD" or "median (IQR)"
    • Proportions: "n/N (XX.X%)"
    • Test results: "statistic = X.XX, p = 0.XXX"
    • Effect sizes: "Cohen's d = X.XX (95% CI: X.XX-X.XX)"
    • AUC: "AUC = 0.XXX (95% CI: 0.XXX-0.XXX)"
  2. Table/figure captions: draft captions referencing table/figure numbers.
  3. Methods snippet: 2-3 sentences describing the statistical methods, for the Methods section. Cite only references whose DOI/PMID /search-lit confirmed; mark any other [UNVERIFIED - NEEDS MANUAL CHECK].

Report statistical results; leave judgments of clinical significance to the user. For designs this skill does not cover (adaptive or Bayesian trials, complex multilevel or causal-mediation models), say so and recommend biostatistician review before the results are reported.

Statistical Reporting Rules (Always Enforced)

  1. Exact p-values: report exact values (e.g., p = 0.034), not inequalities; below 0.001, report p < 0.001.
  2. Confidence intervals: always report 95% CIs for primary endpoints.
  3. Effect sizes: report one alongside every p-value (Cohen's d, eta-squared, odds ratio, risk ratio, etc., as appropriate).
  4. Multiple comparisons: for 3+ tests on the same dataset, apply Bonferroni or Benjamini-Hochberg correction, state the method, and report both uncorrected and corrected p-values.
  5. Sample size: state n for each group/analysis.
  6. Missing data: report how many cases were excluded and why.
  7. Decimal places: p-values to 3 decimals, proportions to 1 decimal, means/SDs to the precision of the measurement.
  8. Design/power statistics are code outputs, never hand-computed. Any minimum detectable effect (MDE), a-priori power, or required sample size that will appear in the manuscript MUST be printed by the committed script with its method and inputs (n per arm, alpha, power, allocation ratio, one/two-sided), not computed in a side tool (G*Power, an online calculator) and pasted in, because a manuscript value no script reproduces cannot be checked. Use one method family throughout (e.g. the exact noncentral-t via statsmodels TTestIndPower or scipy's nct); do not mix a normal approximation for some values with exact-t for others. Do not report post-hoc (observed) power: it is a function of the P value and adds nothing (Hoenig & Heisey 2001); report the CI of the effect instead.
  9. Estimand & CI output contract. Every primary point estimate — including quantile estimands (T25, median time-to-event), pooled proportions and subdistribution HRs, not just ORs/HRs/AUCs — MUST be emitted with its 95% CI, because /self-review treats a primary metric without one as a defect. In the output CSV carry the interval as explicit columns (estimate, ci_lower, ci_upper) or as one text column in est (lo–hi) form; never a point estimate with no interval beside it. Round ORs/HRs/sHRs to 2 decimals and AUC/C-statistic to 3.

Effect-Size Real-World Translation

Whenever a primary result is a correlation, a standardized coefficient, a regression slope, an OR/HR/RR or a Cohen's d, also report it as a plain-language unit shift a non-statistician can act on — in addition to the effect size (rule 3), not instead of it. This applies to continuous exposure–outcome associations (Spearman's rho, Pearson's r, standardized slopes), to relative measures where the audience needs an absolute-risk feel, and to reader or expert-elicitation studies, clinical-utility framing, abstracts and figure captions.

  1. Pick an anchored contrast on the exposure, not a 1-unit step. Default: 25th to 75th percentile (IQR), both endpoints in native units.
  2. Translate to the outcome scale.
    • Regression slope b in native units: delta_outcome = b * (x_p75 - x_p25). Keep the sign: a negative b is a decrease.
    • Per-SD slope (exposure standardized): delta_outcome = beta * (x_p75 - x_p25) / SD_x; a fully standardized beta is also multiplied by SD_outcome.
    • A correlation is not a slope. Pearson r converts to the simple regression slope of the same data (b = r * SD_outcome / SD_x), which is the mean change per unit only if the relation is linear; Spearman's rho converts to no change in outcome units. Fit the outcome on the exposure and translate that slope (Theil-Sen if outliers are why Spearman was used); for a monotonic but curved association, report the fitted outcome at x_p25 and at x_p75 instead.
    • Report as: "going from {x_p25} to {x_p75} {units} is associated with about {delta_outcome} {outcome units} on average", and state the linearity assumption behind it.
  3. Bound the claim: report the contrast, the assumption and a CI on the coefficient; do not imply causation from a crude or unadjusted estimate.

Output contract (clinical utility is a default, not an optional add-on):

  • OR/HR/RR primary outcomes → the relative measure and the absolute risk at a stated baseline, the absolute risk difference, and NNT = 1/ARR (or NNH = 1/ARI), with the baseline risk explicit. A relative-only headline is incomplete.
  • Continuous outcomes → the IQR/clinically anchored "Real-world translation" line beneath the effect size.
  • Prediction / classification (incl. medical-AI) models → calibration (Brier score, calibration plot, or calibration slope/intercept) alongside discrimination, because AUC alone is insufficient; a decision-curve / net-benefit pass at the relevant threshold is standard output. An incremental claim reports the added-value test on the new term (likelihood ratio / Wald in the nested model), ΔC-statistic with its CI and Δnet benefit over the established clinical model, not the new model's AUC alone; NRI only as the categorical, event/non-event-split version, and IDI with caution. See references/table-standards/table-types/incremental_value.md and the make-figures decision_curve exemplar (and render_core_figures.py for the rendered curve).

Error Handling

  • If a script fails, diagnose the likely cause (missing package, data format mismatch, wrong column name) and present a fix. Do not rerun the same script more than once without modifying it or asking the user.
  • If an R package is unavailable, suggest install.packages() and wait for user confirmation.

Output Conventions

Code, tables, figures and console output are in English.

Tables

Before generating any publication table, load:

  1. ${CLAUDE_SKILL_DIR}/references/table-standards/journal-profiles/{journal}.yaml if a target journal is known (it sets footnote markers, P and CI format, title and abbreviation order)
  2. ${CLAUDE_SKILL_DIR}/references/table-standards/table-types/{type}.md for the table type
  3. If no journal is specified, default to AMA style (Radiology profile)

Output formats (always generate all three): a CSV file (downstream use and archival), a console markdown rendering (user review), and R gtsummary code (Word/LaTeX export). Read ${CLAUDE_SKILL_DIR}/references/table-standards/table-standards.md for the gtsummary pipeline and the footnote placement order (general note, abbreviations, specific notes, probability notes).

Validation checklist (run before finalizing any table):

  • No vertical lines — horizontal rules only (top, below header, bottom)
  • Binary variables show only one level (e.g., Male only)
  • Units in headers, not cells; consistent decimal places per column
  • Variability measure stated: mean (SD) or median (IQR)
  • Statistical test named (footnote or general note); exact P values, never "NS"
  • Effect sizes per clinically meaningful unit (per 10 years, not per 1 year)
  • Reference category stated for categorical predictors
  • Abbreviations defined in footnotes, self-contained per table

Figures

PDF (vector) + PNG (300 DPI), styled with figure_style.mplstyle (Arial, colorblind-safe palette); width 3.5 inches (single column) or 7.0 inches (double column); axis labels with units.

Analysis-Specific Guidelines

Before generating code for any row, read the files listed for it (paths relative to ${CLAUDE_SKILL_DIR}); they carry the method rules this file does not repeat.

Analysis typeRead before generating codeTemplate
Table 1 (demographics)references/table-standards/table-types/table1_demographics.mdreferences/templates/table1_demographics.py
Diagnostic accuracy (Se, Sp, PPV, NPV, AUC)references/analysis_guides/diagnostic_accuracy.mdreferences/templates/diagnostic_accuracy.py
Added value beyond a baseline model (ΔAUC, NRI/IDI, net benefit)references/table-standards/table-types/incremental_value.md—
Reader study (MRMC)references/table-standards/table-types/reader_study.md—
Prediction model (outputs a risk used for a decision)references/analysis_guides/calibration.md—
Inter-rater agreement (kappa, ICC, Bland–Altman)references/analysis_guides/agreement_reliability.md, references/table-standards/table-types/agreement.mdreferences/templates/agreement_analysis.py
Meta-analysis (pairwise, single-arm proportion, DTA)references/analysis_guides/meta_analysis.mdreferences/templates/dta_meta_analysis.R (DTA)
Network meta-analysis (≥3 interventions)references/analysis_guides/network_meta_analysis.md—
Health economic evaluation (CEA, CUA, budget impact)references/analysis_guides/health_economic_evaluation.md—
Survey/Likert (ordinal rating scales)the Survey/Likert section belowreferences/templates/likert_summary.py
Survival, competing risks, interval-censored eventsreferences/analysis_guides/survival.md, references/table-standards/table-types/survival_results.mdreferences/templates/survival_analysis.py (--cluster <id> for nested units)
Group comparison, correlationreferences/analysis_guides/test_selection.md—
Logistic or linear regressionreferences/analysis_guides/regression.mdreferences/templates/regression.py (regression_type = "logistic" or "linear")
Propensity score (PSM, IPTW, SIPTW, overlap weighting)references/analysis_guides/propensity_score.mdreferences/templates/propensity_score.py
Survey-weighted (KNHANES, NHANES, KCHS)references/analysis_guides/survey_weighted.mdreferences/templates/survey_weighted_analysis.py
NHIS claims-based studies (ICD-10 definitions)references/analysis_guides/nhis_icd10_mapping.md—
Repeated measures (LMM, GEE, RM ANOVA)references/analysis_guides/repeated_measures.md; for missing data see references/analysis_guides/missing_data.md (an LMM already uses every observed outcome under MAR, so missing outcomes alone do not call for MICE)references/templates/repeated_measures.py
Mediationreferences/analysis_guides/mediation.md—
Multiple testing, many-exposure scans (ExWAS/EWAS)references/analysis_guides/multiplicity.md—
Mendelian randomizationreferences/analysis_guides/mendelian_randomization.md—
Polygenic risk scorereferences/analysis_guides/polygenic_risk_score.md—
Burden of disease, decomposition, forecastingreferences/analysis_guides/burden_decomposition_forecasting.md—

Comparing models: use a paired DeLong test for two separate scores on the same patients. For a nested "baseline + new term" model fitted and evaluated on the same data, DeLong is not the added-value test — test the term (likelihood ratio) and report ΔAUC with its CI.

The rules below have no separate guide.

Survey/Likert

  • Descriptives per item: median, IQR, frequency distribution. Group comparisons: Mann-Whitney or Kruskal-Wallis (ordinal data). Visualization: diverging stacked bar chart.
  • Internal consistency: Cronbach's alpha with item-total correlations.
  • Reverse-coding guard (run before reliability): recode every negatively worded item (min+max) - x before computing the scale total or Cronbach's alpha. An un-recoded reverse item produces a negative item-rest correlation and can make alpha negative — a coding bug, not evidence of a multidimensional construct; do not defend it as one. likert_summary.py prints per-item item-rest correlations, flags negative ones as reverse-code suspects, warns on a negative alpha, and applies the recode with --reverse-items E3 .... To screen at cleaning time, run python3 "${CLAUDE_SKILL_DIR}/../clean-data/scripts/check_reverse_coding.py".

Covariate Pitfalls: Structural Zeros & Dose/Duration Variables

Applies to any multivariable adjustment (logistic / linear / Cox / propensity score / survey- weighted) with a dose/duration variable anchored to a categorical exposure (pack-years under smoking status, grams/week under alcohol use, cessation duration under former smoker):

  • Structural-zero guard (do not impute): a never-smoker's pack_years is a structural zero — known to be 0 by definition of the category, not missing. Imputing it (MICE/MNAR) fabricates a dose for unexposed subjects and corrupts the exposure contrast. Before imputing a dose/duration column, set the implied zero explicitly (IF status == 'never' THEN dose = 0) and impute only the genuinely missing residual among the exposed. /clean-data flags never rows with a NULL dose (scripts/check_structural_zero.py).
  • Complete-case collapse (adjust for status, not dose): a dose variable in a complete-case model drops the unexposed stratum wholesale when its zeros are stored as NULL, collapsing n (commonly 40–60%) and distorting subgroup estimates. Adjust for the categorical status (never/former/current); reserve the continuous dose for an exposed-only secondary analysis. Report n before and after model fitting and confirm the denominator did not silently collapse.

Covariate Selection: Over-adjustment in a Cross-Sectional Outcome Model

Applies to any cross-sectional / single-visit outcome regression (temporal order is not observed). Select covariates causally, not statistically:

  • Do not adjust for a consequence or mediator of the outcome — that is over-adjustment / collider bias and removes part of the effect under study. Signature case: with eGFR as the outcome, serum uric acid is renally excreted (a lower eGFR raises urate), so it is an outcome-consequence, not a confounder; blood pressure and HbA1c are often similarly downstream. Classify each candidate covariate against a DAG as confounder / mediator / outcome-consequence / collider, and keep only confounders in the primary model.
  • "It differs in Table 1" is not a confounder-selection criterion. Imbalance justifies considering a variable; a mediator or outcome-consequence stays out however imbalanced it is.
  • Report the suspect-covariate sensitivity + VIF. Make a parsimonious, history-/design-based model primary; report the fuller model as a sensitivity analysis that drops (or adds) the suspect covariate and show whether the headline estimate moves. Print VIF and the n actually fitted. If dropping the covariate materially changes the estimate, carry that to the abstract and conclusion.
  • Compare adjusted vs unadjusted on the SAME frame. When extended adjustment adds covariates with missingness, the analytic n shrinks (e.g. 84 → 49 events), and comparing that estimate with the full-frame unadjusted one confounds adjustment with who was dropped. Refit the unadjusted model on the reduced complete-case frame and report unadjusted and adjusted on that frame beside the full-frame estimate; never describe "adjustment changed the estimate" from a comparison across different frames (or use multiple imputation so all models share one frame).

/architecture-zoo was renamed to /model-selection in v6.0; this redirect is removed in v7. Run /model-selection now, passing the same arguments the user gave /architecture-zoo. Start at /model-selection Phases 1–3 (choose a paper-grounded architecture family and write the decision note).

/author-strategy — PubMed Author Strategy Analysis

Work from PubMed metadata and the title/abstract text already fetched — nothing else. Do not retrieve full text, follow external links, or resolve preprints. Signals that need citations, citation half-life, venue-impact tier, repository/preprint links, or corresponding-author role are unavailable and surface as [VERIFY] — never inferred.

Prerequisites

  • Python 3.10+ with biopython, pandas, matplotlib, seaborn, and pyyaml (PyYAML is required by the archetype classifier and the rubric renderer)
  • Scripts: ${CLAUDE_SKILL_DIR}/fetch_pubmed.py, ${CLAUDE_SKILL_DIR}/analyze_patterns.py, ${CLAUDE_SKILL_DIR}/pubmed_parse.py (stdlib parser), ${CLAUDE_SKILL_DIR}/classify_archetypes.py, ${CLAUDE_SKILL_DIR}/render_archetype_doc.py
  • Rubric: ${CLAUDE_SKILL_DIR}/references/trajectory_archetypes.yaml (canonical) and ${CLAUDE_SKILL_DIR}/references/trajectory_archetypes.md (generated)

Workflow

Step 1: Gather Input

Ask the user for:

  1. Author name (PubMed format, e.g., "Kim DK" or "Lee KS")
  2. Last name for position classification (auto-detected if ambiguous)
  3. Output directory (default: ~/.local/cache/author-strategy/{AuthorName}/)
  4. Email for NCBI E-utilities (passed as --email)

Step 2: Fetch PubMed Data

python "${CLAUDE_SKILL_DIR}/fetch_pubmed.py" "{Author Name}" \
  --last-name "{LastName}" \
  --output "{output_dir}/data/{name}_publications.csv" \
  --email "{user_email}"

Review the console summary (total count, study type distribution, author position). If count is 0, suggest alternative name formats (e.g., "Kim DK" vs "Kim D" vs the full first name) rather than generating data.

Step 3: Generate Visualizations and Report

python "${CLAUDE_SKILL_DIR}/analyze_patterns.py" "{output_dir}/data/{name}_publications.csv" \
  --output-dir "{output_dir}/report/" \
  --author-name "{Author Name}"

This produces 7 PNG charts (01-07) and analysis_report.md with the strategy breakdown.

Study types come from the keyword rules in pubmed_parse.py, first match in this order: GBD, SR/MA, NHIS/Claims, Cross-national, National survey, Biobank, AI/ML, Clinical trial, Case report, Letter/Commentary; anything else is "Other". Present the script's labels as they are — never reclassify a paper by guess.

Step 4: Interpret and Present

Read analysis_report.md and present to the user; every count and rate comes from that report or the fetched CSV, never from memory:

  1. Executive summary: total publications, growth trajectory, most frequent journals (venue-impact tier is unavailable — never present a "high-tier" rate)
  2. Primary strategy: what study type dominates and why
  3. Author position analysis: first/last positional rate vs middle (positional heuristic only — not leadership or corresponding-author metadata, which are unavailable here)
  4. Topic clusters: research focus areas
  5. ROI quadrant: which study types combine volume with first/last positional rate (chart 07; no venue-tier axis)
  6. Replication opportunities: which patterns are replicable with Claude Code + public databases

State the classifier's limits when they matter: it is tuned for Korean epidemiology and public health researchers and may undercount specialized study types in other fields, and NHIS studies that lack its keywords fall into "Other".

Known limits: a supplied --orcid or --initials that contradicts every same-surname author on a paper yields match_basis orcid-conflict / initials-conflict and position unknown (a namesake is never attributed). The journal_tier CSV column always reads unavailable [VERIFY]. The A3 reporting-quality term list no longer contains claim (it matched "claims database"), so papers naming only the CLAIM checklist do not count toward dual_mode_corpus.

Step 5: Optional — MA Gap Identification

If the user asks "what MA topics are feasible with this professor?":

  • Cross-reference topic clusters with the user's existing MA plans
  • Identify gaps where the professor has domain expertise but no MA published
  • Output a prioritized list of MA proposals

Optional: Trajectory-Archetype Classification

An opt-in path that classifies the trajectory into abstract career archetypes (A1–A6 + a composite) as an explainable, multi-label, confidence-scored heuristic — not an objective verdict, using the canonical rubric references/trajectory_archetypes.yaml.

Step 6: Disambiguation Gate (required before classification)

A surname alone never resolves an author. Pass disambiguators so the target author is uniquely attributed:

python "${CLAUDE_SKILL_DIR}/fetch_pubmed.py" "{Author Name}" \
  --initials "{Initials}" --orcid "{ORCID}" \
  --affiliation "{Institution}" --year-from "{YYYY}" --year-to "{YYYY}" \
  --output "{output_dir}/data/{name}_publications.csv" --email "{user_email}"

This writes the CSV, a candidates.json of affiliation/year candidate clusters, and a corpus_manifest.json with review_status: pending. Present the candidate clusters to the user for review. The user decides include/exclude. Only after the user has reviewed the clusters do you finalize and approve the corpus (the --approve flag is a human gate — never set it without explicit user review/approval):

python "${CLAUDE_SKILL_DIR}/fetch_pubmed.py" "{Author Name}" \
  --initials "{Initials}" --affiliation "{Institution}" \
  --include-pmids "{included.txt}" --exclude-pmids "{excluded.txt}" --approve \
  --output "{output_dir}/data/{name}_publications.csv" --email "{user_email}"

The manifest is cryptographically bound to the CSV (csv_sha256 + pmid_set_hash); the classifier refuses to run on an unapproved or mismatched corpus.

Step 7: Run the Classifier and Present

python "${CLAUDE_SKILL_DIR}/classify_archetypes.py" \
  "{output_dir}/data/{name}_publications.csv" \
  --manifest "{output_dir}/data/corpus_manifest.json" \
  --rubric "${CLAUDE_SKILL_DIR}/references/trajectory_archetypes.yaml" \
  --output-dir "{output_dir}/report/"

Read archetype_report.md and present it to the user, stating up front that the labels are explainable heuristics, not objective classifications. For each surfaced archetype, show the score, confidence band, and the author's own evidence PMIDs. Honor the [VERIFY] markers (h-index/citation/venue-tier are unavailable) and the A5 participation flag. List the insufficient evidence archetypes too — below the minimum sample or with conflicting signals, never force a label.

To retune the rubric, edit only the YAML and regenerate the narrative doc:

python "${CLAUDE_SKILL_DIR}/render_archetype_doc.py"        # regenerate the .md
python "${CLAUDE_SKILL_DIR}/render_archetype_doc.py" --check # CI/test sync gate

Output Structure

{output_dir}/
  data/
    {name}_publications.csv
    candidates.json          # disambiguation candidate clusters (Step 6)
    corpus_manifest.json     # review_status + csv_sha256 + pmid_set_hash (Step 6)
  report/
    analysis_report.md
    01_yearly_stacked.png
    02_study_type_pie.png
    03_author_position.png
    04_journal_heatmap.png
    05_topic_distribution.png
    06_growth_curve.png
    07_strategy_roi.png
    archetype_report.md      # trajectory-archetype classification (Step 7)
    archetype_results.json   # machine-readable labels + scores + evidence

Batch Cohort Analysis Skill

Generate one analysis script per exposure/outcome combination from a single validated methodology template; only the variables change between scripts.

Inputs

  1. Database path(s): CSV/SAS data files (KNHANES, NHANES, NHIS, or any cleaned cohort)
  2. Methodology template: One of:
    • Path to a validated R/Python analysis script (from /replicate-study or /cross-national)
    • A paper type template name: nhis_cohort, cross_national, survey_weighted
    • A source paper to extract methodology from (falls back to /replicate-study Phase 1)
  3. Combination spec: A list of exposure/outcome pairs, provided as:
    • Inline list: exposures: [depression, obesity, smoking]; outcomes: [diabetes, hypertension, CVD]
    • CSV file with columns: exposure, outcome, (optional) subgroup_vars
    • "all" keyword: generates all pairwise combinations from the lists

Optional Inputs

  • Covariate set: Fixed covariate list for all analyses (default: use template's set)
  • Subgroup variables: Variables to stratify by (default: sex, age group)
  • Output format: code_only (just scripts) | execute (run + collect results) | full (code + results + summary)
  • Cross-national mode: cross_national: true generates paired scripts for both countries per combination (see Cross-National Batch Mode)

Example:

/batch-cohort

DB Korea: /path/to/knhanes/HN18.csv
DB US: /path/to/nhanes/
Template: cross_national
Exposures: [depression, obesity, smoking]
Outcomes: [diabetes, hypertension, metabolic_syndrome]
cross_national: true
Mode: execute

Workflow

Phase 1: Template Validation

  1. Read the methodology template (R script or paper type reference).
  2. Identify the slot variables — parts that change per combination: EXPOSURE_VAR (raw variable name in the database), EXPOSURE_LABEL (label for tables/figures), EXPOSURE_CODING (derivation of the binary/categorical exposure), and OUTCOME_VAR, OUTCOME_LABEL, OUTCOME_CODING likewise.
  3. Confirm the survey design: weighted analysis is mandatory for KNHANES/NHANES and is inherited from the template; NHIS and other claims databases have no sampling weights (standard regression).
  4. Verify the template runs successfully on at least one combination before batch generation.
  5. Output: template summary with identified slots → user approval.

Phase 2: Variable Specification

For each exposure and outcome in the combination spec:

  1. Look up the variable in the database: KNHANES — the name exists in the CSV header; NHANES — which table contains it (codebook.csv if available); NHIS — claims code or variable name. For ICD-10 claims algorithms read ${CLAUDE_SKILL_DIR}/../analyze-stats/references/analysis_guides/nhis_icd10_mapping.md; for survey variable coding, survey_weighted.md in the same folder. If a mapping is uncertain, write [VERIFY: variable_name] and ask the user to confirm it against the data dictionary; never guess a variable name, column name or coding.
  2. Define coding: binary as a threshold or category mapping (e.g., HE_glu >= 126 → diabetes = 1); categorical as level definitions (e.g., smoking: current/former/never). Outcome definitions MUST include physician diagnosis, because lab-only definitions systematically overestimate exposure→outcome associations: Diabetes = FPG≥126 OR HbA1c≥6.5 OR physician-diagnosed (KNHANES: DE1_dg=1, NHANES: DIQ010="Yes"); Hypertension = SBP≥140 OR DBP≥90 OR physician-diagnosed (KNHANES: DI1_dg=1, NHANES: BPQ020="Yes").
  3. Set covariates: the full 8-covariate set (age, sex, education, income, smoking, alcohol, obesity, CVD) is the default unless explicitly justified — minimal models (age+sex+BMI only) leave residual confounding, which can bias effects in either direction. Remove self-adjustment: if the exposure is (or derives from) a covariate, drop that covariate (exposure = BMI → drop obesity; exposure = education/income → drop the same variable); if outcome = MetS, consider dropping obesity. Document every removal in the matrix Notes.
  4. Output: combination matrix (combination_matrix.csv) with all variable specifications.
| # | Exposure | Exposure Coding | Outcome | Outcome Coding | Covariates (adjusted) | Notes |
|---|----------|-----------------|---------|----------------|----------------------|-------|
| 1 | Depression (PHQ≥10) | BP_PHQ sum ≥10 | Diabetes | HE_glu≥126|HbA1c≥6.5|DE1_dg=1 | age,sex,edu,income,smoking,alcohol,obesity,CVD | — |
| 2 | Obesity (BMI≥25) | HE_obe ≥4 | Diabetes | same | age,sex,edu,income,smoking,alcohol,depression,CVD | obesity removed from covariates |

Phase 3: Batch Code Generation

Never modify the core methodology across combinations — only swap exposure/outcome/covariates. For each combination in the matrix:

  1. Clone the template script.
  2. Replace slot variables with the combination-specific values.
  3. Apply the combination's adjusted covariate set from the matrix.
  4. Set output paths: each combination gets its own results subdirectory.
  5. Generate a master runner script (run_all.R or run_all.sh) that executes all N scripts sequentially (or in parallel via future/parallel), captures errors per script without stopping the batch, and logs execution time per analysis.
  6. Generated-code quality gate: one reproducibility slip (a missing seed, an absolute path, a hand-typed data literal) replicates across the whole batch, so lint the generated scripts with the /analyze-stats code-quality gate (python3 ${CLAUDE_SKILL_DIR}/../analyze-stats/scripts/check_generated_code.py --code-dir {batch_dir} --strict) and clear every Major (MISSING_SEED, HARDCODED_DATA_LITERAL, HARDCODED_ABS_PATH, INPLACE_SOURCE_OVERWRITE) before batch execution.

Phase 4: Batch Execution (if execute or full mode)

  1. Event count check: before running, verify ≥10 outcome events per model parameter (count the exposure and every dummy/spline term; use min(events, non-events)), in the full sample and in each subgroup that is modelled. Flag underpowered combinations.
  2. Run the master script.
  3. Collect results from each combination's output directory.
  4. Log which combinations failed and why (common: convergence issues, too few events, empty subgroups) in failed_runs.csv, and suggest fixes.

Phase 5: Summary Matrix

Aggregate all results into a single summary. Every number comes from a combination's executed results files; never fill a cell by hand.

Main Results Matrix (summary_matrix.csv):

ExposureOutcomeNEventsModel 1 OR (95% CI)Model 2 OR (95% CI)Model 3 OR (95% CI)p-valueSignificant

Subgroup Summary (subgroup_matrix.csv): Same format, stratified by subgroup variables.

Heatmap (optional): Visual matrix of effect sizes × significance, exposure on Y-axis, outcome on X-axis.

  • Multiple comparisons: whenever more than one combination is tested, include a Bonferroni- (or Holm-) corrected significance column in the summary matrix, with m = the number of tests in the reported family (combinations × models × subgroups), state m, and add a note on exploratory vs confirmatory framing.
  • No p-hacking framing: the summary matrix is for hypothesis generation, not confirmation. State this explicitly in README and any manuscript output. Any citation there needs a /search-lit-confirmed DOI/PMID; otherwise mark it [UNVERIFIED - NEEDS MANUAL CHECK].

Output Files

{working_dir}/batch_{timestamp}/
├── README.md                    — Batch run summary (N combinations, template used, date)
├── combination_matrix.csv       — All exposure/outcome specs with coding
├── template/
│   └── base_template.R          — The validated template (frozen copy)
├── scripts/
│   ├── 01_depression_diabetes.R
│   ├── 02_obesity_diabetes.R
│   ├── ...
│   └── run_all.R                — Master execution script
├── results/
│   ├── 01_depression_diabetes/
│   │   ├── table1.csv
│   │   ├── main_results.csv
│   │   └── subgroup_results.csv
│   └── ...
├── summary/
│   ├── summary_matrix.csv       — Main results across all combinations
│   ├── subgroup_matrix.csv      — Subgroup results across all combinations
│   ├── failed_runs.csv          — Combinations that failed + error messages
│   └── heatmap.png              — Optional effect size × significance visual
└── logs/
    └── batch_execution.log      — Timing + error log

Reproducibility: freeze the template version (template/) and include a SHA256 hash of the data file in README.

Cross-National Batch Mode

When cross_national: true:

  • Generate paired scripts for each combination (Korea + US), using /cross-national's dual-survey-design approach
  • Summary matrix includes both countries side-by-side, with a ratio-of-ORs column (95% CI; P) computed as in /cross-national Phase 4, not a direction-agreement tick

Calc-Sample-Size Skill

Decision Tree

Walk the user through this tree one question at a time; do not assume answers. Reader studies, segmentation and model-comparison designs sit outside the tree: see Tests 14–17.

What is your primary outcome?
|
+-- Binary (yes/no, positive/negative)
|   |
|   +-- Paired data (same subjects, two methods)?
|   |   +-- YES --> [5] McNemar test
|   |   +-- NO  --> How many groups?
|   |       +-- 2 groups, superiority     --> [4] Two-proportion comparison (chi-square)
|   |       +-- 2 groups, non-inferiority --> [10] Non-inferiority / equivalence
|   |       +-- Multivariable model       --> single-predictor hypothesis test? --> [9] Logistic regression
|   |                                     --> clinical prediction / AI model for use?
|   |                                         +-- developing the model  --> [12] Prediction-model development (Riley)
|   |                                         +-- externally validating  --> [13] External-validation (Riley)
|   |
+-- Continuous (measurement, score)
|   |
|   +-- How many groups?
|       +-- 2 groups  --> [6] Independent t-test
|       +-- 3+ groups --> [8] One-way ANOVA
|
+-- Time-to-event (survival, recurrence)
|   |
|   +-- Two groups, unadjusted      --> [7] Log-rank test
|   +-- Multivariable / adjusted HR  --> [7] Log-rank (Schoenfeld) + [11] Cox EPV
|
+-- Agreement (inter-rater, reproducibility)
|   |
|   +-- Continuous measurements --> [2] ICC
|   +-- Categorical ratings     --> [3] Kappa
|
+-- Diagnostic accuracy (Se, Sp, AUC precision)
    |
    +--> [1] Diagnostic accuracy (precision-based)

Tests 1–11

Once the test is chosen, read ${CLAUDE_SKILL_DIR}/references/formulas.md § Test N — the parameter table with defaults, the effect-size interpretation, the formula, R/Python code and the methodological reference.

#TestUse when
1Diagnostic accuracy — Se/Sp precisiondesired 95% CI half-width for sensitivity or specificity
2ICC agreement (Walter 1998 test; Bonett 2002 CI width)inter-/intra-rater agreement on continuous measurements (tumor size, angle)
3Kappa agreement (Donner & Eliasziw 1992; needs the trait prevalence)agreement on categorical ratings (BI-RADS category, lesion present/absent)
4Two-proportion comparison (chi-square)two independent groups (AI vs conventional detection rate)
5McNemar (paired proportions)paired binary outcomes (two readers on the same cases, before/after)
6Independent t-testmeans in two independent groups (lesion size, malignant vs benign)
7Survival / log-rank (Schoenfeld events, then patients)time-to-event between two groups
8One-way ANOVAmeans across 3+ independent groups
9Logistic regression (Peduzzi EPV + Hsieh 1998, continuous or binary predictor)multivariable binary outcome, single-predictor hypothesis test
10Non-inferiority / equivalencenew method not worse than standard by more than a pre-specified margin, or equivalent within it
11Cox regression EPVmultivariable Cox model — enough events for stable estimates
  • Test 9: Peduzzi EPV ≥ 10 is a minimum baseline for a single-predictor hypothesis test only. For a clinical prediction / medical-AI model intended for use, EPV-10 is outdated and reviewer-vulnerable — use Test 12 (development) / Test 13 (validation). Always report both Peduzzi and Hsieh and recommend the larger N.
  • Test 10: NI alpha is one-sided (typically 0.025); orient the difference so > 0 favours the new method. Equivalence is TOST, powered jointly. The margin must be clinically justified (see formulas.md § Margin selection).
  • Test 11: EPV ensures model stability, not power for a specific HR. If an HR is available, also run Test 7 (Schoenfeld) and recommend the larger N.

Tests 12–17 (specialised designs)

Each has its own reference file (parameters, method, reporting); read it once the test is chosen.

  • Test 12 — Prediction-model development (Riley). Developing a clinical prediction / classification model (including a medical-AI model evaluated as one) for use. EPV-10 does not apply: N is the largest satisfying all Riley criteria — three for a binary or time-to-event outcome, four for a continuous one (R pmsampsize).
  • Test 13 — External validation (Riley). Validating an existing prediction/AI model: size for the CI width of the C-statistic, calibration slope, O:E and (if claimed) net benefit (R pmvalsampsize); ≥ 100 events and ≥ 100 non-events is only a floor. For Tests 12–13 read ${CLAUDE_SKILL_DIR}/references/prediction_model_sample_size.md.
  • Test 14 — MRMC reader study (Obuchowski–Rockette). Readers with vs without the AI, or AI non-inferior to readers. Test 1 under-sizes it because readers are a random effect: size readers J × cases from pilot/literature variance components with RJafroc / MRMCaov / iMRMC (do not hand-roll the OR algebra) and report the J × N power grid. Read ${CLAUDE_SKILL_DIR}/references/mrmc_reader_study_sample_size.md.
  • Test 15 — Segmentation-metric precision (Dice / HD95 / NSD). The outcome is a per-case score, not a proportion: n ≈ (1.96·SD/δ)² from the pilot SD of per-case Dice, sized on the worst structure; CI by patient-level bootstrap (BCa). This is precision; a comparison is Test 16. Read ${CLAUDE_SKILL_DIR}/references/segmentation_metric_sample_size.md.
  • Test 16 — Between-model comparison. Model A beats B, C, …: power the paired per-case difference, n = ((z₁₋α/₂ + z₁₋β)·SD_Δ/Δ)² (a CI sized to just exclude zero has ~50% power), not each model's precision; for > 2 models pre-specify one primary contrast or pay the family-wise correction; a ranking claim needs multiple seeds. Read ${CLAUDE_SKILL_DIR}/references/multi_model_comparison_sample_size.md.
  • Test 17 — Segmentation usability. Clinicians can use it (acceptability rate, catastrophic-failure bound, edit time): acceptability is a proportion sized per structure class; DE ≈ 1 + (m−1)ρ holds only when each case has its own readers — the same readers on every case (crossed) add a reader term more cases cannot shrink; bounding failures at ≤ 1% needs ~300 clean cases (rule of three). Read ${CLAUDE_SKILL_DIR}/references/segmentation_acceptability_sample_size.md.

Out of Scope

Do not compute adaptive trials (group-sequential, sample size re-estimation), cluster-randomized trials (design effect, ICC-based inflation), Bayesian sample size determination, crossover designs, or multi-endpoint correction (mention Bonferroni if asked, but do not compute corrected sample sizes). Say the design is beyond this skill and point to G*Power (free, https://www.psychologie.hhu.de/gpower), PASS, or a biostatistician.

Workflow

Phase 1: Understand the Study

  1. Ask the user to describe the study briefly (design, primary outcome, groups).
  2. Walk the decision tree to a test, and confirm it with the user before proceeding.

Phase 2: Collect Parameters

  1. Present the selected test's parameter table.
  2. For each parameter without a user-provided value, explain it and offer the default.
  3. Estimate effect sizes from prior literature (ask for the references) or pilot data; use Cohen's conventions only as a last resort, noting that convention-based estimates are less precise.

Phase 2b: Retrospective Studies

When the dataset already exists, formal power analysis is often impractical. Offer:

  • Fixed extract: read ${CLAUDE_SKILL_DIR}/references/observational_cohort.md and report event budget / confidence-interval precision instead of forcing a prospective recruitment-style power calculation.
  • Experience-based justification (acceptable for IRB and many journals):
    • Institution volume: total exams in period × prevalence × (1 − exclusion rate) = expected N. Ask for the annual exam volume for the modality, study period, prevalence and exclusion rate. This gives a realistic upper bound for N.
    • Prior studies: report the N of 3–5 comparable published studies and cite them; the user's N should be in the same range or larger.
    • IRB templates for both: ${CLAUDE_SKILL_DIR}/references/justification_examples.md § Retrospective.

Use a formal calculation (Phase 3) even for a retrospective study when a subset is enrolled prospectively, the primary analysis tests a hypothesis (not just estimation), the journal's Instructions for Authors require a power analysis, or the IRB requires it.

Phase 3: Calculate and Report

  1. Generate R code (primary) and Python code (alternative) from the reference formula.
  2. Run the R code via Bash; the reported N is the number it prints, not a hand calculation.
  3. Present the result in the Output Format below. Cite methodological sources only from formulas.md or the test's reference file; any other reference needs a DOI/PMID confirmed via /search-lit, otherwise mark it [UNVERIFIED - NEEDS MANUAL CHECK]. Mark an effect size, clinical definition or threshold you could not confirm [VERIFY].
  4. In a project, save the IRB text as protocol/sample_size_justification.md and the scripts as protocol/sample_size_calc.R / .py: /write-protocol and /write-paper embed that text verbatim, so the numbers are never retyped.

Phase 4: Sensitivity Analysis (Optional)

If a parameter is uncertain or the effect-size estimate is vague, flag it and offer a table of N across plausible values (e.g., varying effect size, or power from 0.80 to 0.90).

Output Format

Always structure the final output as follows:

## Sample Size Calculation Report

### Study Design
[1-2 sentence summary of the design and test selected]

### Parameters
| Parameter | Value | Source |
|-----------|-------|--------|
| ... | ... | user / literature / convention |

### Result
- **Required sample size**: N = [value]
- **With [X]% attrition adjustment**: N_adj = [value]

### R Code (Reproducible)
```r
# [complete, self-contained R script]
# Dependencies: [list packages]
# Run: Rscript sample_size_calc.R

Python Code (Alternative)

# [complete, self-contained Python script]
# Dependencies: [list packages]
# Run: python sample_size_calc.py

IRB Justification Text

A sample of [N] participants is required to detect [effect description] with [power]% power at a [one/two]-sided significance level of [alpha], assuming [key assumptions]. Accounting for an estimated [X]% attrition rate, we plan to enroll [N_adj] participants. This calculation is based on [formula/method reference].

Effect Size Interpretation

[Cohen's benchmark classification + clinical meaning in the context of this study]


The IRB text must state N; name the test and its formula source; give every assumed parameter
(effect size, alpha, power); state the attrition adjustment and final enrollment target; cite the
methodological reference (e.g., "Schoenfeld, 1981"); and use formal, third-person language. Read
`${CLAUDE_SKILL_DIR}/references/justification_examples.md` for per-design exemplars when writing it.

Check-Reporting Skill

Reference Files

Checklists are vendored under ${CLAUDE_SKILL_DIR}/references/checklists/, one file per instrument. Each file's header gives its version, source citation and licence, and says when its item text is an own-words summary rather than the published wording.

  • STROBE.md -- STROBE
  • STROBE_MR.md -- STROBE-MR 2021
  • RECORD.md -- RECORD 2015 (RECORD-PE for drug studies)
  • REMARK.md -- REMARK
  • TARGET.md -- TARGET 2025
  • GATHER.md -- GATHER 2016
  • CHEERS_2022.md -- CHEERS 2022
  • CROSS.md -- CROSS 2021 + CHERRIES (internet surveys)
  • SRQR.md -- SRQR 2014 (all qualitative approaches)
  • COREQ.md -- COREQ 2007 (interviews and focus groups)
  • STARD.md -- STARD 2015
  • STARD_AI.md -- STARD-AI 2025
  • TRIPOD.md -- TRIPOD 2015
  • TRIPOD_AI.md -- TRIPOD+AI 2024
  • TRIPOD_LLM.md -- TRIPOD-LLM 2025
  • PGS_RS.md -- PGS-RS / PRS-RS 2021
  • CONSORT.md -- CONSORT 2025
  • CONSORT_AI.md -- CONSORT-AI 2020
  • SPIRIT.md -- SPIRIT 2025
  • SPIRIT_AI.md -- SPIRIT-AI 2020
  • CLAIM_2024.md -- CLAIM 2024
  • DECIDE_AI.md -- DECIDE-AI 2022
  • MI_CLEAR_LLM.md -- MI-CLEAR-LLM
  • CLEAR.md -- CLEAR (radiomics)
  • ARRIVE_2.md -- ARRIVE 2.0
  • CARE.md -- CARE 2013
  • SQUIRE_2.md -- SQUIRE 2.0
  • GRRAS.md -- GRRAS
  • PRISMA_2020.md -- PRISMA 2020
  • PRISMA_2020_Abstracts.md -- PRISMA 2020 for Abstracts, 12 items. A separate instrument, not a subset of the 27-item checklist (main item 2 defers to it): score it with its own denominator.
  • PRISMA_DTA.md -- PRISMA-DTA
  • PRISMA_P.md -- PRISMA-P
  • PRISMA_ScR.md -- PRISMA-ScR
  • MOOSE.md -- MOOSE
  • SWiM.md -- SWiM
  • AMSTAR2.md -- AMSTAR 2
  • QUADAS3.md -- QUADAS-3
  • QUADAS2.md -- QUADAS-2
  • QUADAS_C.md -- QUADAS-C
  • RoB2.md -- RoB 2
  • ROBINS_I.md -- ROBINS-I
  • ROBINS_E.md -- ROBINS-E
  • ROBIS.md -- ROBIS
  • ROB_ME.md -- ROB-ME
  • RoB_NMA.md -- RoB NMA
  • PROBAST.md -- PROBAST
  • PROBAST_AI.md -- PROBAST+AI
  • NOS.md -- NOS
  • COSMIN_RoB.md -- COSMIN RoB

Workflow

Step 0: Existing-checklist staleness pre-check

If a checklist already exists for this project (qc/reporting_checklist.json or a prior .md report), verify it targets the current manuscript before reusing it — one generated against an older version carries stale section/line references and a stale version label:

python3 "${CLAUDE_SKILL_DIR}/scripts/check_checklist_version.py" \
  --checklist qc/reporting_checklist.json --manuscript manuscript_v8.md

A non-zero exit means the checklist is stale (older target_version, changed source_sha256, different target_manuscript) or pre-dates the version contract: regenerate it against the current manuscript (Steps 1–5) rather than reusing it. Every report you generate carries the target_manuscript / target_version / source_sha256 fields (Part A header + Part D JSON) so this check works next round.

Step 1: Select Guideline

Use the guideline the user names; otherwise auto-detect it from the study type. If the study type is ambiguous, ask the user to confirm before selecting a guideline.

Auto-detection mapping:

Study TypePrimary GuidelineAI Extension
Observational studySTROBE--
Mendelian randomization studySTROBE-MR (base STROBE + MR extension)--
Health economic evaluation (cost-effectiveness / cost-utility / cost-benefit / budget-impact)CHEERS 2022--
Observational study using routinely-collected data (claims / EHR / registry / health-checkup DB)RECORD (base STROBE + RECORD extension; RECORD-PE for drug studies)--
Survey / questionnaire study (KAP, physician/patient, cross-sectional, e-survey)CROSS (+ CHERRIES for internet surveys)--
Scoping review (maps breadth/nature of evidence, clarifies concepts, identifies gaps — not a focused effectiveness/accuracy question)PRISMA-ScR (base PRISMA + scoping-review extension)--
Qualitative study (interviews, focus groups, ethnography, grounded theory, phenomenology, document analysis)SRQR (all qualitative approaches); COREQ (interviews/focus groups specifically)--
Randomized controlled trialCONSORT 2025CONSORT-AI
Diagnostic accuracy studySTARD 2015STARD-AI
Prediction model (development/validation)TRIPODTRIPOD+AI
Polygenic (risk) score prediction studyPGS-RS (with TRIPOD / TRIPOD+AI)--
Prognostic tumor-marker / biomarker study (single or multiple markers; e.g., ctDNA / molecular residual disease)REMARK (pair with STROBE for the observational-design items; TRIPOD / TRIPOD+AI if a prognostic model is developed)--
Causal / comparative-effectiveness question emulated on observational data (treatment vs treatment, screening vs none, drug A vs B on registry / EHR / claims data)TARGET (pair with the /design-study target-trial-emulation module for design; RECORD / STROBE for the routinely-collected-data items)--
Health-estimate / burden-of-disease modeling study (GBD or GBD-satellite, comparative-risk / population-attributable-fraction, cause-of-death or prevalence/incidence estimation, with or without forecasts)GATHER (pair with /analyze-stats burden-decomposition-forecasting guide for the analytic layer)--
Systematic review / meta-analysisPRISMA 2020PRISMA 2020 for Abstracts (run on the abstract, scored separately)
DTA systematic review / meta-analysisPRISMA-DTAPRISMA 2020 for Abstracts (run on the abstract, scored separately)
Meta-analysis of observational studiesMOOSEPRISMA 2020 (use both)
Risk of bias (DTA studies)QUADAS-3 (current recommended version)QUADAS-2 only when appraising or reproducing a review that used it
Risk of bias (RCTs)RoB 2--
Risk of bias (non-randomised intervention studies)ROBINS-I--
Risk of bias (non-randomised exposure studies)ROBINS-E--
Risk of bias (comparative DTA studies)QUADAS-CQUADAS-3 (use both; apply the E&E's adaptation — see Using QUADAS-C with QUADAS-3 in QUADAS3.md)
Risk of bias (prediction models)PROBASTPROBAST+AI
Risk of bias (systematic reviews)ROBISAMSTAR 2
Risk of bias (missing evidence in MA)ROB-ME--
Risk of bias (network meta-analysis)RoB NMA--
Risk of bias (measurement properties)COSMIN RoB--
Quality assessment (observational)NOS--
Case reportCARE--
Study protocolSPIRIT 2025SPIRIT-AI
Animal studyARRIVE 2.0--
AI/ML study in clinical imagingCLAIM 2024--
Study using a large language model (develop/fine-tune/prompt/evaluate an LLM)TRIPOD-LLMMI-CLEAR-LLM (use alongside when LLM accuracy is an outcome)
Early-stage / live clinical evaluation of an AI decision-support system (human factors, workflow, safety)DECIDE-AI--
LLM accuracy evaluation in healthcareMI-CLEAR-LLMSTARD-AI or CLAIM 2024 (use alongside)
Reliability / agreement studyGRRAS--
SR protocolPRISMA-P--
Synthesis without meta-analysisSWiMPRISMA 2020 (use both)
Quality of systematic reviewsAMSTAR 2ROBIS
Radiomics studyCLEARCLAIM 2024 (if deep learning component)
Educational / QI studySQUIRE 2.0--
Generative AI images ARE the study object (realism / real-vs-synthetic reader study / model-vs-model quality)(no single guideline -- assemble)see decision aid below

QUADAS-3 has two protocol-stage phases, and this skill usually runs too late for them. Phase 1 (state the synthesis question) and phase 2 (define the ideal test accuracy trial each judgement is made against) are review-level and belong in the protocol, with the review-specific guidance for answering each signalling question. Reaching them for the first time during manuscript QC means writing the comparator after seeing the results. If they are missing, say so as a limitation rather than reconstructing them, and route the protocol work to /meta-analysis Phase 1. Phases 3–6 are what a QC pass can genuinely run.

Rules:

  • A guideline the user names is the one you score. The rules below choose the guideline only when the user names none. If they call for a different or additional instrument (STARD-AI for an AI index test, TRIPOD+AI for an ML prediction model, an AI extension), score the named guideline and put one line at the top of the report saying which instrument fits better and the command to rerun, e.g. /check-reporting STARD-AI. Do not switch or add instruments on your own: the user maps the report to the checklist form the journal asked for.
  • If the study involves AI/ML, apply the AI version of the guideline. CONSORT-AI and SPIRIT-AI are scored alongside their base guideline; TRIPOD+AI, STARD-AI and TRIPOD-LLM replace theirs (below).
    • Exception — TRIPOD: TRIPOD+AI 2024 (Collins et al., BMJ 2024) is a complete rewrite, not an addendum to TRIPOD 2015 (Moons et al., Ann Intern Med 2015). For non-AI prediction models, use TRIPOD 2015 only. For AI/ML prediction models, use TRIPOD+AI 2024 only. Do NOT apply both simultaneously.
  • STARD-AI (Sounderajah et al., Nat Med 2025) extends STARD 2015 with 14 new and 4 modified items (40 total) and incorporates all STARD 2015 items. For AI diagnostic accuracy studies use STARD-AI only — do NOT apply STARD 2015 and STARD-AI simultaneously.
  • TRIPOD-LLM (Gallifant et al., Nat Med 2025) is the reporting guideline for studies that develop, fine-tune, prompt, or evaluate a large language model for a clinical/biomedical task. It extends the TRIPOD family (TRIPOD 2015 → TRIPOD+AI 2024 → TRIPOD-LLM 2025); name the base instrument and the extension and cite each. It is modular — task-specific items (Annotation, Prompting, Summarization, Instruction-tuning) are N/A when that component is absent. Use TRIPOD-LLM for LLM studies in place of TRIPOD+AI; pair with MI-CLEAR-LLM when LLM accuracy is an evaluated outcome. The vendored checklist is an educational summary (own-words paraphrase of item intent); complete the official instrument for a submission checklist.
  • MI-CLEAR-LLM is a supplementary checklist (8 item categories in the 2025 update; the 2024 original had 6), not a standalone reporting guideline. Always pair it with the study's primary guideline (e.g., STARD-AI for AI diagnostic accuracy, CLAIM for imaging AI). Apply it whenever the study evaluates LLM accuracy as an outcome — do NOT apply it merely because the manuscript was written with LLM assistance. Its scope is LLM accuracy studies (including VLMs interpreting images); it does not apply at study level when a generative model produces the images under study (next bullet).
  • Generative-AI images as the study object (a generative model synthesizes images and the study evaluates their realism, controllability, real-vs-synthetic distinguishability, or model-vs-model quality) has no single dominant checklist. Assemble: CLAIM 2024 (imaging-AI umbrella; model-development items N/A when commercial models are used as-is) + FUTURE-AI traceability + MI-CLEAR-LLM transparency items only (prompt/model/version/params/runs — for generation provenance, not study-level compliance) on the generator side; STARD-AI (for real-vs-synthetic detection) + GRRAS (reader reliability) + MRMC reporting on the evaluation side. Map applicable items and cite base + extension; never claim wholesale compliance. Full decision aid: ${CLAUDE_SKILL_DIR}/references/genai_image_study_object_decision_aid.md.
  • If multiple guidelines apply (e.g., a diagnostic accuracy study that is also an AI study), check against all relevant guidelines and merge into one report.

Step 2: Load Checklist

  1. Run the fail-fast guard first for every guideline you intend to apply:

    python "${CLAUDE_SKILL_DIR}/scripts/check_checklist_exists.py" --guideline "STARD-AI"
    
    • Exit 0 → the vendored checklist exists; read it from ${CLAUDE_SKILL_DIR}/references/checklists/ and proceed.
    • Exit 1 (MISSING_CHECKLIST_CONTRACT_VIOLATION) → the guideline is routed but no checklist file is vendored. Do not construct items from memory. Halt, report the violation to the user, and stop unless they explicitly opt in (next bullet).
    • Exit 2 (UNKNOWN_GUIDELINE) → the name is not recognised; confirm the correct guideline with the user.
  2. No silent fallback. A from-memory checklist is permitted only when the user explicitly accepts it — re-run the guard with --allow-from-memory (exit 0 + a NON-AUTHORITATIVE warning). The report MUST then carry a prominent banner that the assessment was constructed from model knowledge and is not backed by a vendored checklist, and submission_safe must not be asserted on its basis.

Step 3: Scan Manuscript

Read the whole manuscript before assessing any item — including tables, figures and their captions, supplementary material, and the reference list (registration numbers and protocol references often sit there).

Step 4: Assess Each Item

Items most often missing in medical manuscripts — look for these first, whichever guideline applies: registration number and registration/amendment date consistency (run Step 4c), sample-size justification, missing-data handling, blinding, funding and conflicts of interest, ethics approval with committee name and approval number, and a data availability statement; for AI studies, the training/validation/test split, model architecture and hyperparameters, failure-mode analysis, fairness/bias assessment, and commercial interests with data/code availability.

For every checklist item, determine:

StatusCriteria
PRESENTThe item is fully addressed with sufficient detail.
PARTIALThe item is mentioned or partially addressed but lacks required detail.
MISSINGThe item is not found anywhere in the manuscript.
N/AThe item does not apply to this particular study (justify why).

For each item, record:

  • Status: PRESENT / PARTIAL / MISSING / N/A
  • Location: Section name and paragraph or approximate position (e.g., "Methods, paragraph 3")
  • Notes: What was found (if PRESENT/PARTIAL) or what should be added (if MISSING)

Be strict. PARTIAL means the item is mentioned but lacks specificity; a vague reference does not count as PRESENT — the detail level must match what the guideline expects. "We used appropriate statistical tests" = PARTIAL (which tests?); "We used the Mann-Whitney U test for continuous variables and Fisher's exact test for categorical variables" = PRESENT. If an item is genuinely unclear in its applicability, mark it N/A with justification.

Two gaps that are easy to pass:

  • Power-aware framing of a null result (STROBE 16a / 18 / 20) — for an observational study whose headline is a non-significant association, a flat "X was not associated with Y" overreads the data when the analysis is not powered to exclude a clinically meaningful effect. Mark item 18/20 PARTIAL unless the manuscript states the precision as an exclusion (e.g., "the 95% CI excluded an eGFR difference larger than ~1.7") or reports a minimum detectable effect — "no effect" vs "could not exclude an effect of size X" are different claims, and a negative conclusion needs the latter.
  • Confounder-selection rationale, not "adjust for everything that differs" (STROBE 16a explicitly asks which confounders were adjusted for and why) — flag a kitchen-sink adjustment set chosen because variables differ in Table 1. The Methods must give a causal rationale (DAG / prior literature) and must not adjust for a mediator or consequence of the outcome (over-adjustment, e.g. serum uric acid in an eGFR model); both an unjustified inclusion and an unjustified omission are item-16a gaps.

What is appraised is the source paper's reporting — never your convenience in using it. This holds for every instrument here, reporting checklists and risk-of-bias / quality tools alike, and is easiest to lose in a systematic review, where you read each paper in order to extract from it. An item asking "are the results clearly reported?" is not asking "were they reported in the unit my pool needs". If a downgrade's stated reason turns on a denominator, an analysis unit, a subgroup you needed and they did not report separately, or a format you could not parse, it is an extraction note, not a scoring reason: record it in a separate extraction-note column and restore the score. An extraction limitation often belongs in your limitations paragraph; folded into the score it makes the appraisal unreproducible, because another assessor with a different pool would score the same paper differently.

Step 4b: Section Boundary Check

In addition to checklist items, verify that:

  • Results section contains only factual findings: no interpretation, no "why" explanations, no prior literature comparisons, no evaluative adjectives without numbers.
  • Discussion section does not introduce new data not presented in Results.
  • Flag any boundary violation as a separate finding in Part C Action Items with the label [BOUNDARY].

Step 4c: Registration / Protocol Timing Consistency Check

Applies to: systematic reviews, meta-analyses, and intervention studies with prospective registration (PRISMA 2020, PRISMA-DTA, PRISMA-P, MOOSE, CONSORT, SPIRIT). The registration identifier is a single checklist item and can pass Step 4 while the manuscript is inconsistent about when registration and amendments happened relative to the analysis.

Read ${CLAUDE_SKILL_DIR}/references/step4c_registration_timing.md (item-by-item procedure, JSON schema, flagging edge cases) and run its five checks: (1) registration identifier present in Methods, Abstract, and cover letter; (2) initial registration date precedes — or is explicitly disclosed as post-dating — the extraction milestone; (3) amendment dates appear in Methods, the described change is visible in Methods, analysis was re-run if the amendment post-dates the lock, and no amendment post-dates submission; (4) Methods agree with the registry record (PROSPERO PDF, ClinicalTrials.gov export) — a silent discrepancy is a finding; (5) a retrospective-registration disclosure paragraph when evidence suggests post-extraction filing.

Flagging: any failure is logged in Part C Action Items with label [REGISTRATION-TIMING]. fixable_by_ai: false when reconciliation requires an external amendment filing; true only when the fix is a Methods-text insertion of a date already disclosed elsewhere. Part D JSON includes a registration_timing object (registry, id, initial_registration_date, amendments[], timing_consistency, findings[]).

Registration-ID format gate: a PROSPERO ID is CRD42 + 9 digits = 14 characters (^CRD42\d{9}$, e.g. CRD42024500001). Run grep -oE 'CRD42[0-9]+' manuscript.md and assert each match is 14 characters long; a 15-character ID (a stray inserted digit) is a transcription error logged as [REGISTRATION-TIMING] (fixable_by_ai: false — verify against the live PROSPERO record, do not guess the correct digit).

Step 4d: PRISMA Figure 1 Arithmetic & Cross-Reference Audit

Applies to: systematic reviews and meta-analyses using PRISMA 2020 / PRISMA-DTA / PRISMA-P. Triggers when Item 16a (flow diagram) is PRESENT — the diagram can pass Step 4 while its numbers do not add up or disagree with the text.

  1. Choose the Figure 1 source, in this order: (a) analysis/figures/Figure1_PRISMA.md markdown manifest, (b) caption text in manuscript.md, (c) PPTX text run if a .pptx exists, (d) manual entry from PNG/SVG.

  2. Run the audit. It checks the five subtractions (screened = identified − duplicates; sought-for-retrieval = screened − excluded at screening; retrieved = sought − not retrieved; assessed for eligibility = sought − not retrieved; included = assessed for eligibility − excluded with reasons) and that the body-text PRISMA numbers match the Figure 1 boxes 1:1, and writes qc/prisma_figure_audit.json:

    python3 ${CLAUDE_SKILL_DIR}/scripts/check_prisma_figure.py \
      --md <manuscript.md> --figure <Figure 1 source: .md manifest / caption / text export> \
      --out qc/prisma_figure_audit.json
    

    Exit 1 = an arithmetic or cross-reference MISMATCH; exit 2 = missing/unparsable input. When the Figure 1 numbers exist only in a PNG/SVG, transcribe them by hand and run the same checks manually as ${CLAUDE_SKILL_DIR}/references/step4d_prisma_figure_audit.md specifies (regex set, JSON schema, and edge cases: duplicates across databases, the citation-searching strand, dual-reviewer screening).

  3. By hand (the script does not do this): the reasons for exclusion in Methods and the Figure legend must agree on counts and category names, and when identification is split across sources (databases, registers, other methods) the per-source counts must sum to the identified total — the script reads only the first identified count.

  4. If analysis/figures/_figure_manifest.md (from /make-figures) exists, verify that the row whose Type = prisma (or Type = prisma-dta) points at the same file used as the audit source, and that its Critic field is yes or partial (not no). A missing row, mismatched path, or Critic = no logs [MANIFEST-XREF] (advisory); the arithmetic check still runs.

Flagging: any MISMATCH or arithmetic failure logs a Part C Action Item with label [PRISMA-FIGURE], fixable_by_ai: false (the author must reconcile the numbers).

PRISMA Cascade Arithmetic Auto-Verify

When PRISMA 2020 or PRISMA-DTA is selected and round-by-round screening TSV artifacts are available, recompute the screening cascade from the raw decisions. Off-by-one errors in the prose cascade are a high-frequency reviewer red flag (e.g., 151 + 108 + 39 + 1 + 1 + 4 = 304 followed by a prose summary "305" four lines later).

python "${CLAUDE_SKILL_DIR}/scripts/prisma_cascade_check.py" \
    --round1 2_Screening/round1.tsv \
    --round2 2_Screening/round2.tsv \
    --round3 2_Screening/round3_adjudication.tsv \
    --manuscript manuscript.md \
    --out qc/prisma_cascade.json --strict

The script counts INCLUDE / EXCLUDE / MAYBE decisions per round, computes the cascade, and reports per-stage drift where the manuscript's stage counts disagree. Treat any manuscript_drift entry as a P0 blocker — fix the prose to match the computed cascade and re-run. A --manuscript path that is not a file exits 2. When the manuscript is read but none of the script's stage phrases is found in it, manuscript_check is "unverifiable", submission_safe is false, an UNVERIFIED line is printed, and --strict exits 1: the drift check did not run, so compare the prose stage counts to stage_counts by hand. stages_compared lists the stages that were actually checked.

Step 4e: Reporting-Framework Naming Audit

Applies to: any manuscript that invokes an AI/extension reporting framework (PROBAST+AI, STARD-AI, TRIPOD+AI, TRIPOD-LLM, CONSORT-AI, SPIRIT-AI, PRISMA-DTA, QUADAS-C). A base reporting tool and its extension are distinct instruments with separate citations, and Step 1 does not police how the framework is named in prose. The recurring failures: invoking an extension without ever naming or citing the base instrument it extends; mixing +AI and -AI hyphenation for one family within a single document; coining item labels like "12-AI"; and waving at "recent guidance" instead of naming the framework.

python3 "${CLAUDE_SKILL_DIR}/scripts/check_framework_naming.py" \
  --manuscript manuscript.md --out qc/framework_naming.json --strict

Verdicts: BASE_MISSING (extension used, base instrument never named standalone) is a Major and logs [FRAMEWORK-NAMING] in Part C with fixable_by_ai: true (insert the base name + its citation). HYPHEN_MIX, CITE_MISSING, SELF_COINED_LABEL, and VAGUE_GUIDANCE are Minor (fixable_by_ai: true). Part D JSON includes a framework_naming object mirroring the script's claims[].

Step 4f: Critical-item floor cross-check

Applies to: every guideline assessment for which the floor defines a row (load and check only those; do not invent a floor for an unlisted guideline). After the item-by-item table, read ${CLAUDE_SKILL_DIR}/references/critical_item_floor.md and check the small set of non-waivable items for this study type — and, for AI/ML and radiomics manuscripts, the methodological-quality / risk-of-bias instrument (PROBAST+AI, METRICS/RQS, APPRAISE-AI) and concerns it lists. A MISSING critical item is surfaced as a Critical gap and becomes the report's headline regardless of the overall percentage — a high percentage with a missing critical item (undefined reference standard, no leakage-controlled partition, calibration absent for a prediction model, an unreconciled flow diagram) is not "broadly acceptable."

Step 5: Generate Report

This report is an internal working audit — it carries auto-fix annotations, a machine-readable JSON block (compliance_pct, fixable_by_ai, …), and Action Items. It is NOT the official reporting checklist a journal expects (that is the blank guideline form with Item | Recommendation | Reported in page/section, which the authors fill in). Never submit this report as the submission checklist. So that the file is self-identifying and cannot be reused by filename into a later submission package, the report MUST begin with this banner as its very first line:

<!-- INTERNAL AUDIT — NOT FOR SUBMISSION. This is the /check-reporting working
report, not the official journal checklist. Do not upload to a submission portal. -->

Write the checklist content and report in English (matching the guideline originals), whatever language you use with the user. Save it as qc/reporting_checklist.md and the Part D JSON as qc/reporting_checklist.json.

Read ${CLAUDE_SKILL_DIR}/references/report_templates.md when you write the report — it holds the literal templates for the four parts:

  • Part A — Summary. Header (manuscript file, version token, guideline, date), the PRESENT/PARTIAL/MISSING/N-A count table, and overall compliance. The headline is the critical items (Step 4f), not the percentage: report {present}/{total} and name every missing critical item with the section it belongs in.
  • Part B — Item-by-item checklist. One row per item: # | Section | Item | Status | Location | Notes.
  • Part C — Action items (MISSING and PARTIAL only), ordered by: items most journals enforce strictly (ethics approval, registration, sample size) → items in Methods (easiest to fix) → everything else.
  • Part D — Machine-readable JSON, appended as a fenced block. MUST be present under --json or when called from /write-paper Phase 7, which parses it.

JSON field contract (the part other skills depend on — get these right):

  • compliance_pct — present / (total_items - na) * 100, one decimal; null when every item is N/A (total_items == na).
  • action_items — MISSING and PARTIAL only; PRESENT and N/A are excluded.
  • fixable_by_ai — true when the fix inserts or expands text using information already in the manuscript or inferable from it; false when it needs external facts the author alone holds (registration number, IRB approval number, protocol details, sample-size rationale and target).
  • suggested_fix — concrete draft text, insertable as written. A fix that still contains a bracketed placeholder ([N], [rationale]) is not insertable and is fixable_by_ai: false.
  • source_sha256 — first 12 hex chars of the SHA-256 of the manuscript bytes, so a stale report cannot be silently attributed to a newer manuscript.

In notes and suggested fixes, cite a reference only with a /search-lit-confirmed DOI or PMID and mark any other [UNVERIFIED - NEEDS MANUAL CHECK]; never invent clinical definitions, diagnostic criteria, or guideline recommendations — flag anything uncertain [VERIFY] and ask the user.

When the user asks for the filled checklist a journal requires, build it from the Submission checklist export template in ${CLAUDE_SKILL_DIR}/references/report_templates.md.


Skill Interactions

WhenCallPurpose
During manuscript writing/write-paper Phase 7Final compliance check
Need to add Methods text/write-paper Phase 3Draft missing Methods content
Need statistical details/analyze-statsGenerate missing statistical reporting
Need flow diagram/make-figuresGenerate CONSORT/STARD/PRISMA diagram

Gates

GateSeverityTriggerAction on fail
Mandatory items presentENFORCED at submission< 100% of guideline-mandatory items marked PRESENTAuto-fix MISSING items where text exists; otherwise route to /write-paper Phase 7 for re-draft
Step 4d PRISMA Figure 1 arithmetic & cross-reference audit (PRISMA / PRISMA-DTA only)ENFORCED for SR/MAflow numbers don't sum (e.g., screened ≠ included + excluded), or in-text counts mismatch flow diagramHALT; reconcile against extraction artifacts
Optional items (e.g., supplementary AI declarations)ADVISORY< 80% of optional items presentwarn; user accepts
Cross-reporting-guideline routing (study type → guideline)ENFORCEDstudy type undeclared or guideline missingAsk user; do not silently default

Data Profiling and Cleaning Skill

Profile, flag, and generate cleaning code for a clinical dataset in three stages, each ending at a user-approval gate. You generate code and reports; you do NOT auto-clean data. Every cleaning decision needs the researcher's explicit confirmation, because clinical cleaning calls need domain knowledge.

PHI first. If the dataset contains PHI or PII, run /deidentify before proceeding, and use *_deidentified.* files when they exist in the working directory. Otherwise work from the data dictionary / codebook alone, or in a local-only environment with no network access. The skill writes code that runs on the data; it does not need to see the raw rows to write it.

Three-Stage Workflow

Stage 1: Profiling

Input: CSV/Excel file path OR data dictionary/codebook

  1. Adapt ${CLAUDE_SKILL_DIR}/references/profiling_template.py (pandas) to the dataset. It reports: variable and row counts, data types; missing count and percentage per variable; unique counts for categorical variables; min/max/mean/median/SD for numeric variables; histograms and bar charts.
  2. If the user provides a codebook, cross-reference variable names, expected types, and expected ranges. Never guess a column name or coding: when a mapping is uncertain, write [VERIFY: variable_name] and ask the user to confirm it against the data dictionary.
  3. Present the summary (see Output Format). Every count and statistic comes from the executed script's output.

Gate: The user reviews the profile. Ask whether to proceed to Stage 2 (Flagging) and whether any variables should be excluded or focused on.

Stage 2: Flagging

Read ${CLAUDE_SKILL_DIR}/references/cleaning_patterns.md for missing-data mechanisms, outlier decision rules, duplicate detection, date handling, and clinical pitfalls (inequality-prefixed lab values, mixed units, sentinel values). Flag issues in these categories:

  1. Missing values: variables with >5% missing; pattern analysis (MCAR/MAR/MNAR heuristic).
  2. Statistical outliers: IQR method (Q1 - 1.5IQR, Q3 + 1.5IQR) and Z-score (|z| > 3).
  3. Duplicates: exact row duplicates AND near-duplicates (same patient ID, different dates).
  4. Type mismatches: numeric stored as string, dates in inconsistent formats.
  5. Implausible values: flag against the codebook's valid range or, when the codebook is silent, the domain-default hard physiologic bounds in references/implausible_value_rules.md §1. An implausible value is a likely data-entry/unit/sentinel error (correct or set missing); a statistical outlier (#2) is biologically possible (keep + sensitivity analysis). Check units before calling a bound violation an error. Never auto-fix. 5b. Cross-field inconsistencies: logical contradictions per references/implausible_value_rules.md §2 — temporal ordering (birth ≤ event ≤ death, admission ≤ discharge), derived-vs-source (recomputed BMI/age; subset ≤ superset; total = sum of parts), sex-/state-specific fields, and min ≤ max / diastolic < systolic pairs. Name the rule that fired; a hard contradiction is High severity.
  6. Category inconsistencies: typos and variants in categorical values ("Male", "male", "M", "MALE").
  7. Categorical-implied zeros: when a category defines a natural zero for a dose/duration variable (smoking_status == 'never' implies pack_years == 0; alcohol_use == 'never' implies grams_per_week == 0), flag records that store the implied zero as NULL. This is a contradiction, not a missing-data pattern: complete-case models silently drop those never-smokers and MICE imputes them a non-zero dose, corrupting the exposure contrast. Suggested action: "Set dose = 0 where category == reference level; impute only the residual missingness among the exposed." Detected by scripts/check_structural_zero.py given the category↔dose mapping; pairs with /analyze-stats "Covariate Pitfalls: Structural Zeros & Dose/Duration Variables".
  8. Reverse-coded scale items: in a multi-item Likert scale that mixes positively and negatively worded items, every reverse item must be recoded (min+max) - x before the scale total or Cronbach's alpha is computed. An un-recoded reverse item correlates negatively with the rest and collapses alpha, often to a negative value. A negative alpha is a reverse-coding bug, not "multidimensional structure". Suggested action: "Recode reverse-worded items, then recompute reliability." Detected by scripts/check_reverse_coding.py (negative item-rest correlation and negative raw alpha, given the scale item columns); the recode itself is applied downstream by /analyze-stats likert_summary.py --reverse-items.

Present the flag report as a table:

VariableIssue TypeCountSeveritySuggested Action
ageOutlier (IQR)3MediumReview: values 150, 200, -5
pack_yearsCategorical-implied zero12421HighSet 0 where smoking_status=='never' (structural zero, not missing)

Severity levels:

  • High: likely data errors that will affect analysis (type mismatches, impossible values)
  • Medium: potential issues that need expert review (statistical outliers, moderate missingness)
  • Low: minor inconsistencies that are easy to fix (category labels, trailing whitespace)

Gate: The user marks each row (A) Approve the suggested action, (R) Reject / keep as-is, or (M) Modify the action. Only approved actions generate cleaning code.

Stage 3: Code Generation

For ONLY user-approved actions, generate Python (or R if requested) code:

  • Missing value handling: listwise deletion, mean/median imputation, or MICE setup (code only, the user runs it and picks the variables and method)
  • Outlier handling: winsorization, removal, or keep-and-flag
  • Duplicate removal: exact dedup with logging
  • Type conversion: standardize dates, numeric parsing
  • Category harmonization: mapping table for inconsistent labels

All generated code MUST include:

  • Before/after row counts printed to console
  • Logging of every modification to a cleaning log DataFrame
  • Reproducibility: np.random.seed(42) and random.seed(42) where applicable
  • Output: cleaned CSV (a new file; the input file is never overwritten) + cleaning_log.csv
  • Clear comments explaining each cleaning step

End the generated script with this notice:

"This code implements ONLY the cleaning rules you approved. Review the cleaning_log.csv output to verify all changes before proceeding to analysis."

Out of scope: free-text extraction from clinical notes, and image data or DICOM metadata. After cleaning, hand off to /analyze-stats. Take any citation from /search-lit, never from memory.

Output Format

Structure all reports using this template:

## Data Profiling Report

### Dataset Overview
- Rows: [N]
- Columns: [N]
- File size: [size]
- Date range: [if applicable]

### Variable Summary
| Variable | Type | Missing N (%) | Unique | Min | Max | Mean | SD |
|----------|------|---------------|--------|-----|-----|------|-----|
| ...      | ...  | ...           | ...    | ... | ... | ...  | ... |

### Flags
| Variable | Issue | Count | Severity | Suggested Action |
|----------|-------|-------|----------|-----------------|
| ...      | ...   | ...   | ...      | ...             |

### Cleaning Code
[Python/R script -- only for approved actions]

### Cleaning Log
[What was changed, how many rows affected, before/after counts]

Contribute

The user is a clinician who has probably never opened a pull request. Do the git work for them: never make them type a git command, and never use the words "rebase", "upstream", or "HEAD" in anything they read. Their local edits sit next to real patients and real manuscripts, so the safety steps below, not the git mechanics, are why this is a skill.

The one rule that cannot bend

Nothing leaves the machine until the author has seen every line that would leave it and said yes. Until something has been sent, say explicitly, every time, that nothing has been sent.

The safety scan is an aid, not a certificate: no pattern list recognises every patient name or every hospital. Say so out loud, because a user who believes the scanner is complete stops reading the diff, and that is exactly when the leak happens.

If the scan reports a blocker (patient-level data, a credential), do not offer a workaround. The line gets deleted. A contribution never needs patient data to make its point.

Phase 0: What did they change?

python3 "${CLAUDE_SKILL_DIR}/scripts/find_local_changes.py" --target claude --json \
  --out qc/local_changes.json

This compares the installed skills against the hashes of what was shipped. It reads only.

  • Nothing changed → say so plainly and stop; that is normal, not a failure. Offer the feedback path (Phase 4) instead.
  • Something changed → show it, grouped by skill:
    • A new file (a journal profile, a citation style, a reporting exemplar) — usually the most valuable kind of contribution, because it is domain knowledge nobody in the project has. Say that.
    • An edited file — ask what was wrong for their specialty; the answer is the pull-request description. Never guess at the reason for an edit.
    • A deleted file — usually not a contribution. Ask before including it.

A change that only makes sense in one hospital (internal rules, a template with the department's letterhead) is a local adaptation, not a contribution: say so, leave it out, and tell them keeping it is fine.

Phase 1: The safety scan (blocking)

python3 "${CLAUDE_SKILL_DIR}/scripts/check_contribution_safety.py" \
  --changes qc/local_changes.json --out qc/safety.json

This gate fails closed: finding anything at all is a non-zero exit, unlike every other detector in the repository — deliberately, because a tool that returns success while printing a hospital name will eventually be trusted to have said nothing.

Verdicts: PHI_SUSPECTED, SECRET (blockers — the line is deleted, not argued with), IDENTITY, INSTITUTION, APPROVAL_ID, MANUSCRIPT_ID, LOCAL_PATH. Fix each finding with them, following the do line the scan prints for it (a name → a role, a hospital → a generic descriptor, an IRB or manuscript ID → removed, a home path → ~). A manuscript ID matters because a paper under review is confidential and its ID identifies it.

A file that is not UTF-8 text (an image, a binary) cannot be scanned or read, so it is listed as NOT SCANNED and the scan fails until it is taken out of the contribution. qc/safety.json records the sha256 of every file it scanned; the submit step refuses unless the files it is about to send are exactly those, unchanged. After any edit, re-run the scan — an older report, or one made with --text on a different file, clears nothing.

Then print the full text of every file that would be sent and ask them to read it. Not a summary — the text. This is the step that actually protects them.

Phase 2: Does it meet the project's own bar?

Run the repository's validator against the change if the repo is available locally; otherwise check by eye:

  • A journal profile cites the journal's public author guidelines: word limits, abstract structure and AI policy are quoted from them or left out. No impact factors or numbers from memory.
  • A citation style is the official CSL from the Zotero repository.
  • A skill edit still says what it does, and the change is general — it would help someone at another hospital. If not, it is a local adaptation; keep it local.

If it does not pass, say what to change, in their words. Do not send a contribution that will be rejected — that is a worse experience than not contributing at all.

Phase 3: Send it

python3 "${CLAUDE_SKILL_DIR}/scripts/submit_contribution.py" \
  --changes qc/local_changes.json --safety qc/safety.json \
  --title "Add a journal profile for <journal>" --dry-run

Always --dry-run first, and show them the plan. Then, on their explicit yes, run it again without --dry-run. The script takes the highest rung available:

  1. GitHub tool installed and signed in → it forks, copies the files, and opens the pull request. They type nothing.
  2. Installed but not signed in → it prints the single command (gh auth login) and stops. Walk them through it: GitHub.com → HTTPS → log in through the browser.
  3. Not installed → it writes the change to a file and opens a pre-filled issue in their browser. Do not make installing a developer tool a condition of helping.

Tell them what happens next: a maintainer reads it, and if something needs changing they can reply in plain language on that page. They will not be asked to do any git work.

Phase 4: Feedback that is not a code change

A false positive or a failed step is often the most useful thing a clinician has: it is the only evidence of how a detector behaves on a real manuscript rather than a synthetic fixture. Collect:

  • A detector fired wrongly: which detector (the detector field in its qc/*.json), the verdict, and the smallest possible snippet that reproduces it — rewritten with fake numbers and names if the real one cannot be shown; the shape of the sentence is what matters.
  • A step failed: the command, the error, the host (Claude Code / Codex / Cursor / Copilot), the operating system.
  • Something was wrong for your specialty: what the toolkit assumed, and what is actually true.

Send it as an issue, after the same safety scan — a repro from a real manuscript is exactly where PHI hides:

python3 "${CLAUDE_SKILL_DIR}/scripts/check_contribution_safety.py" --text qc/feedback.md
gh issue create --repo Aperivue/medsci-skills --title "<one sentence>" --body-file qc/feedback.md --web

--web opens it in the browser pre-filled: they read it and press Submit. Nothing is sent by us.

Phase 5: Keep their change when they update

Tell them the installer already takes a permanent backup of any skill they modified before it overwrites it (under ~/.medsci-skills/backups/), so updating never destroys their work — many people avoid updating because they assume it would. If the change is worth keeping, contributing it means the next update will contain it.

Settings — nobody is nagged

python3 "${CLAUDE_SKILL_DIR}/scripts/contribution_prefs.py" --status
python3 "${CLAUDE_SKILL_DIR}/scripts/contribution_prefs.py" --on    # remind me
python3 "${CLAUDE_SKILL_DIR}/scripts/contribution_prefs.py" --off   # never mention it (default)
  • Reminders are opt-in and off by default. The installer mentions the option once, at the end of a first install, and never again. Opted in, a reminder appears only when they actually changed something, and at most once a month.
  • The setting governs reminders only. /contribute works whenever it is run.
  • The setting cannot weaken safety: there is no key for that, and check_contribution_safety.py reads no configuration. Opting in never means "send without asking".

Saying thanks

Many clinicians who find this useful have never starred the repository because nobody told them what a star is for. Once the toolkit has actually done something for them, tell them — the script explains what a star is and makes it one command:

python3 "${CLAUDE_SKILL_DIR}/scripts/star_repo.py" --how   # explain + show both ways
python3 "${CLAUDE_SKILL_DIR}/scripts/star_repo.py" --now   # do it from here (GitHub CLI)
  • Never ask twice — being shown once is recorded, and doing nothing is an answer.
  • Never dress this up: do not imply the toolkit is worse off without their star, and do not ask before the toolkit has done something for them. Asking for thanks in advance is begging.

Scope

Touches only the installed skills — never the user's manuscripts, data, or any other file. Contributions arrive as a fork's pull request or an issue; never push to the main repository.

Known limits

  • The patient-identifier patterns cover a fixed set of labels and formats. Among what they do not recognise: Korean-language labels for a registration or patient number, a resident registration number written without its hyphen, an alphanumeric patient ID, "hospital number", and a date of birth written with a month name or introduced by "born". Widening them means keyword matching over prose, which has produced false positives before, so it has not been done. Reading every line (Phase 1) is what covers these.

Cross-National Comparison Study Skill

Inputs

  1. Research question: exposure → outcome association to compare across countries
  2. Korean data path: KNHANES CSV file
  3. US data path: NHANES CSV directory (multiple tables to merge)
  4. Harmonization table (optional): CSV mapping variables across surveys
    • Default: /replicate-study's references/harmonization_knhanes_nhanes.csv

Reference Files

  • /write-paper's references/paper_types/cross_national.md — writing template
  • /analyze-stats's references/analysis_guides/survey_weighted.md — survey-weighted analysis guide
  • references/chns_coding.md — CHNS files, merge keys, coding and warnings; read it when the design includes China (3-country design)
  • references/additional_variables.md — asthma, sleep, physical activity, diet, treatment and non-HDL-cholesterol coding, plus composite-score (LE8) warnings; read it when the study uses any of them

Workflow

Phase 1: Study Definition

  1. Confirm research question: Exposure → Outcome
  2. Define variable coding for both countries:
    • Exposure: PHQ-9, BMI category, smoking, etc.
    • Outcome: diabetes, hypertension, mortality, etc.
    • Covariates: age, sex, education, income, smoking, alcohol, obesity, CVD
  3. Check harmonization table for variable availability
  4. Output: study protocol summary for user approval

Phase 2: Data Preparation

Never guess a variable name, dataset column name, or variable coding. If a mapping is uncertain, output [VERIFY: variable_name] and ask the user to confirm it against the data dictionary.

KNHANES (single CSV):

  1. Load CSV and keep every row — the age ≥20 (or per-protocol) restriction is applied to the design in step 3

  2. Derive variables using KNHANES coding:

    VariableRaw VarCoding
    SmokingBS3_11,2=Current; 3=Former; 8=Never
    AlcoholBD1_112-6=Frequent (current drinker); 1=Occasional (past-year abstainer); 8=Never
    ObesityHE_obe1-3=Normal; 4-6=Obesity (BMI≥25, Asian cutoff)
    DepressionBP_PHQ_1~9Sum ≥10 = depression
    DiabetesHE_glu, HE_HbA1c, DE1_dgFPG≥126 or HbA1c≥6.5 or DE1_dg=1
    CVDDI4_dg, DI5_dg, DI6_dgAny = 1 → CVD yes
    Educationedu1-3=Non-college; 4=College
    IncomeincmQuartile (1=lowest … 4=highest): 1-3=Bottom 75%; 4=Top quartile
  3. Set survey design on the full file, then restrict to the analytic domain: des <- svydesign(id=~psu, strata=~kstrata, weights=~wt_itvex, nest=TRUE, data=df); des_ad <- subset(des, age >= 20). Never filter rows before svydesign() — dropping them changes the standard errors (see survey_weighted.md, subpopulation analysis)

NHANES (multiple CSVs):

  1. Load and merge tables by SEQN (DEMO_J, DPQ_J, GHB_J, GLU_J, BMX_J, SMQ_J, ALQ_J, DIQ_J, MCQ_J, BPQ_J, BPXO_J)

  2. Derive variables using NHANES coding. CRITICAL: NHANES data downloaded via R nhanesA package uses TEXT LABELS, not numeric codes.

    VariableRaw VarCoding (text labels)
    SexRIAGENDR"Male" / "Female" (NOT 1/2)
    SmokingSMQ020 + SMQ040100 cigs (SMQ020 "Yes" / "No") + now smoke (SMQ040 "Every day" / "Some days" / "Not at all")
    AlcoholALQ121 + ALQ111Frequent (current drinker): any ALQ121 frequency except "Never in the last year"; Occasional (past-year abstainer): "Never in the last year"; Never (lifetime non-drinker): ALQ111 == "No" (ALQ121 will be NA)
    ObesityBMXBMI (BMX_J, kg/m²)≥30 (WHO cutoff, NOT Asian)
    PHQ-9DPQ010~DPQ090"Not at all"→0, "Several days"→1, "More than half the days"→2, "Nearly every day"→3; sum ≥10 = depression
    DiabetesLBXGLU (GLU_J, fasting subsample, mg/dL), LBXGH (GHB_J, %), DIQ010LBXGLU≥126 | LBXGH≥6.5 | DIQ010=="Yes" (DIQ010: "Yes" / "No" / "Borderline"). CRITICAL: fasting glucose is GLU_J LBXGLU, not BIOPRO_J LBXSGL — CDC says the serum LBXSGL should not be used to determine undiagnosed diabetes; an analysis that uses LBXGLU needs the fasting-subsample weight (step 3). LBXGLU is NA for non-fasters, and in R NA | TRUE is TRUE but NA | FALSE is NA, so this composite goes missing only for non-fasters who would be non-diabetic; dropping those NAs under WTMEC2YR inflates prevalence. Either analyse the FPG composite only in the fasting domain with the fasting weight, or use the non-fasting variant LBXGH>=6.5 | DIQ010=="Yes" on WTMEC2YR, and say which in variable_mapping.csv
    CVDMCQ160B/C/D/EMCQ160B=="Yes" (CHF) | MCQ160C=="Yes" (CHD) | MCQ160D=="Yes" (angina) | MCQ160E=="Yes" (MI); labels "Yes" / "No" / "Don't know"
    HTNBPXOSY2+3, BPXODI2+3, BPQ020mean(BPXOSY2, BPXOSY3)≥140 | mean(BPXODI2, BPXODI3)≥90 | BPQ020=="Yes" (BPXOSY3 is the 3rd reading, not an average; the mean of the 2nd and 3rd matches KNHANES)
    EducationDMDEDUC25 text levels
  3. Set survey design on the full file, then subset() the design object to the analytic domain (e.g. RIDAGEYR >= 20): svydesign(id=~SDMVPSU, strata=~SDMVSTRA, weights=~WTMEC2YR, nest=TRUE). The weight follows the files: the single-cycle _J tables above take WTMEC2YR; WTMECPRP goes only with the pre-pandemic P_ files (P_DEMO, P_BMX, ...). A variable from the fasting subsample (GLU_J LBXGLU, TRIGLY_J LBXTR/LBDLDL) takes the fasting weight instead: WTSAF2YR (_J) or WTSAFPRP (P_); a variable that combines a fasting-subsample component with full-sample ones (diabetes above) is defined only in that fasting domain, so it cannot enter a WTMEC2YR model. Pooling cycles follows the NCHS rules: divide each cycle's weight by the number of cycles pooled (1999–2002 has its own 4-year weights), and to combine 2015–2016 with 2017–March 2020 use 2/5.2 × WTMEC2YR and 3.2/5.2 × WTMECPRP.

CHNS (3-country design): read references/chns_coding.md before preparing China data.

Phase 3: Parallel Analysis

For EACH country independently:

  1. Table 1: Baseline characteristics by exposure (weighted counts + percentages)
  2. Main analysis: Sequential logistic regression models
    • Model 1 (unadjusted)
    • Model 2 (age + sex)
    • Model 3 (fully adjusted: + education, income, smoking, alcohol, obesity, CVD)
  3. Subgroup analyses: By sex, age group, education, income, alcohol, smoking, CVD, obesity. Whether the association differs between subgroups is tested with an exposure × subgroup interaction term fitted on the full design (svyglm on the whole sample), not by comparing the subgroups' P values.
  4. Dose-response (if applicable): RCS with 3 knots

Phase 4: Cross-National Comparison Table

Generate a side-by-side comparison:

AnalysisKorea wOR (95% CI)US wOR (95% CI)Ratio of wORs (95% CI); P
Overall (fully adjusted).........
Male.........
Female.........
............

Compare the countries with the ratio of their odds ratios, not with whether the directions agree: two estimates in the same direction can differ, and opposite directions can be compatible. With log odds ratios b₁, b₂ and their design-based standard errors SE₁, SE₂ from the two independent surveys, the ratio is exp(b₁ − b₂) with 95% CI exp(b₁ − b₂ ± 1.96·√(SE₁² + SE₂²)) and z = (b₁ − b₂)/√(SE₁² + SE₂²) (Altman & Bland, BMJ 2003;326:219).

Every number comes from executed code output (analysis_korea.R, analysis_us.R) — never an invented p-value, effect size, confidence interval, or sample size.

Phase 5: Output Files

{working_dir}/
├── cross_national_report.md    — Study summary + comparison tables
├── variable_mapping.csv        — Variable mapping with match status
├── analysis_korea.R            — KNHANES analysis (self-contained)
├── analysis_us.R               — NHANES analysis (self-contained)
├── results/
│   ├── table1_korea.csv
│   ├── table1_us.csv
│   ├── main_results_comparison.csv
│   └── subgroup_comparison.csv
└── manuscript_draft/           — Optional: Methods + Results draft
    ├── methods_draft.md
    └── results_draft.md

Take every citation in the report or draft from /search-lit (confirmed DOI/PMID); mark any other [UNVERIFIED - NEEDS MANUAL CHECK], and never generate references from memory.

Critical Rules

  1. NEVER pool data across countries. Each country analyzed with its own survey design.
  2. Country-specific BMI cutoffs: Korea ≥25 (Asian), US ≥30 (WHO).
  3. Country-specific income: KNHANES quartile, NHANES PIR → harmonize to binary.
  4. Weighted analysis mandatory: Both KNHANES and NHANES are complex surveys. CHNS has no survey weights — analyse it unweighted (see references/chns_coding.md).
  5. Document all harmonization decisions: What matches, what needed recoding, what differs.
  6. Same analytic approach: Identical model specifications for both countries for fair comparison.

Define-Variables Skill

Map each exposure, outcome, covariate, and eligibility variable to a canonical guideline/consensus definition, cross-check it against prior operationalizations in comparable cohorts, then map it to the available DB variables. Call after /design-study (and /search-lit), before /write-protocol.

Inputs

  1. Research question (one sentence)
  2. Candidate variables — exposure, outcome, key covariates, eligibility filters
  3. Data dictionary path (xlsx / csv / markdown) OR explicit list of available DB columns
  4. Cohort type (e.g., health-screening, NHANES-like, claims, registry) — informs which prior-art cohort to compare against

Missing inputs → ask once, then proceed.

4-Tier Pipeline (DB codebook + token-efficient literature)

Tier 0 — DB codebook lookup (mandatory for DB-backed observational studies)

Trigger: project has a project.yaml::db.dictionary_path field pointing to a machine-readable codebook (xlsx/csv/markdown), OR the user supplied a dictionary path in inputs. If neither, skip to Tier 1.

For every candidate DB variable — before touching literature — open the dictionary and record, verbatim, the sheet name, row number, and code→meaning mapping. This prevents the most common observational-study error: assuming a column code (status == 0, grade == 4) means what it intuitively reads like, when the codebook says otherwise.

Per variable:

  1. Locate the variable in the dictionary by exact column name.
  2. Copy verbatim: the sheet title, row number, and full code→meaning mapping (or unit/range statement for continuous vars).
  3. Paste into the Dict. sheet & row + Dict. verbatim columns of the operationalization table.
  4. If the variable is not found, OR the codebook is silent on a specific code value, file a question to the DB owner / data steward. Do NOT infer from cross-tabs, do NOT guess, do NOT proceed with that variable until a verbatim answer exists.

Empirical checks (value distributions, cross-tabs with related columns) are useful for sanity testing after the verbatim codebook meaning is recorded — never as a substitute for it.

Recommend committing a DICTIONARY_FIRST_POLICY.md at the project root (or shared-config path) with the canonical dictionary path and the escalation contact.

Exit gate: before Tier 1, cross-check every row's Dict. sheet & row and Dict. verbatim against the source dictionary; no DB-backed row may be left blank.

Tier 1 — Canonical index lookup (no API calls)

Look the variable up in references/common_definitions.md (hepatology, metabolic/endocrine, renal, pulmonary, cardiovascular, oncology/imaging incidentalomas, alcohol exposure). On a hit, record the guideline, year, canonical cutoff, and BibTeX key. Done — no /search-lit call.

Tier 2 — Targeted /search-lit (focused queries only)

For variables NOT in Tier 1, OR when subgroup justification is needed (Asian-specific cutoff, pediatric, young-adult, pregnancy, etc.), call /search-lit with one query per variable — never a general sweep, which buries the signal. Query pattern:

"{construct} definition {cohort type} {subgroup qualifier}"
e.g., "obstructive sleep apnea prevalence Korean health screening cohort"

Stop searching a variable once the first 1-2 papers converge on the same definition. If more than five variables need Tier 2, list them and confirm with the user before running the rest.

Tier 3 — Verification

Every definition, cutoff, and era anchor must come from a verified source — a clinical guideline, a peer-reviewed paper with DOI, or an established registry data dictionary. Never take a phenotype threshold from the model's prior or a reference from memory. Before finalizing, run /verify-refs on the accumulated BibTeX to confirm every citation exists in PubMed/CrossRef. A choice with no canonical source is flagged Ad-hoc: yes, justified in 1-2 sentences, and confirmed by the user before it propagates into /write-protocol or /analyze-stats.

Output Template

Write {project_root}/variable_operationalization.md (or the path the user specifies) from templates/variable_operationalization.md. Required structure:

  1. Header: research question, cohort type, date, author

  2. Operationalization table — one row per variable:

    | Variable | Role | Dict. sheet & row | Dict. verbatim | Canonical source | Definition | Cutoff | DB vars | Implementation | Ad-hoc? |

    • Role: exposure / outcome / covariate / eligibility
    • Dict. sheet & row: e.g. 5-1.복부초음파 r12 — mandatory if a DB dictionary exists
    • Dict. verbatim: full code→meaning string copied from the dictionary — mandatory under the same condition
    • Canonical source: BibTeX key (e.g., @rinella2023_aasld_masld), so downstream skills can re-verify
    • Definition: one line, verbatim from the guideline where possible
    • Cutoff: numeric + units
    • DB vars: exact dictionary column names used
    • Implementation: SQL/pandas-style pseudocode (e.g., bmi>=25 & (b_tg>=150 | b_hdl<40))
    • Ad-hoc?: yes/no. If yes, justification below the table
  3. Ad-hoc justifications — for each yes row

  4. Mapping gaps — variables in the protocol with no DB equivalent; list proxy / omit / request decisions

  5. References — BibTeX block

Out of scope: statistical analysis → /analyze-stats; manuscript drafting → /write-paper; data cleaning / missingness → /clean-data; sample size → /calc-sample-size.

Failure Modes to Avoid

  1. Column-first framing — starting from what columns exist, then picking a definition that matches. Always flip: definition first, then map. Tier 0 still applies once a column is picked: quote its codebook entry verbatim before using its values.
  2. Cutoff drift — using a different cutoff than the cited guideline without justification (e.g., BMI≥23 cited as WHO Asian while text says ≥25).
  3. Mixing eras — 2020 MAFLD criteria with 2023 MASLD criteria in the same analysis. Pick one and note why.
  4. Dose/duration structural-missingness — operationalizing a dose/duration covariate (pack-years, cessation-years, alcohol grams/week) anchored to a categorical exposure (smoking status, alcohol use) without specifying what the reference level (never-smoker, never-drinker) does to the dose. A never-smoker's pack-years is a structural zero, not a missing value; conflating the two collapses the analytic sample under complete-case modeling and lets MICE fabricate a non-zero dose for the unexposed. Operationalize it explicitly — add a row with Role = covariate and Implementation = "IF status == 'never' THEN dose = 0 ELSE measured_value" — and adjust on the categorical status variable, reserving the continuous dose for an exposed-only secondary analysis. /clean-data (categorical-implied-zero flag) and /analyze-stats ("Covariate Pitfalls") enforce this downstream.

De-identification Skill

You are guiding a medical researcher through data de-identification. The actual de-identification is performed by a standalone Python script that runs WITHOUT any LLM. Your role is to explain, guide, and verify — not to see or process raw PHI data.

Critical Safety Rules

  1. NEVER ask the user to paste, show, or upload raw data containing PHI. The script processes data locally. You never need to see patient-level data.
  2. NEVER read or display the mapping file contents. It contains original PHI values.
  3. You may read the scan and reviewed reports (column classifications; no cell values) and the audit log (keyed hashes; no original values). Read the de-identified output only after the researcher has run the review and confirmed it; it has the identifiers they chose to anonymize removed, and nothing more (see "What the tool does not do").
  4. Always communicate in the user's preferred language about the process, but use English for technical terms (PHI, HIPAA, Safe Harbor, etc.).
  5. The researcher runs the script in their own terminal, not through you (no ! prefix, no Bash tool call): the review prints sample values of every column.

Reference Files

  • ${CLAUDE_SKILL_DIR}/references/hipaa_18_identifiers.md — HIPAA Safe Harbor checklist
  • ${CLAUDE_SKILL_DIR}/references/korean_phi_patterns.md — Korean-specific regex patterns
  • ${CLAUDE_SKILL_DIR}/references/date_shift_guide.md — Date shifting best practices

Read relevant references before advising the researcher.

Prerequisites

  • Python 3.9+
  • openpyxl (for .xlsx files): pip install openpyxl
  • Supported formats: CSV, TSV, Excel (.xlsx)

Five-Phase Workflow

Phase 1: Assessment

Ask the researcher:

  1. What file format is the data? (CSV, Excel, etc.)
  2. What PHI do you expect in the data? (names, dates, IDs, etc.)
  3. Does your IRB require specific de-identification documentation?
  4. Do you need to re-identify later? (affects mapping file choice)

Based on answers, recommend the appropriate command:

  • Full pipeline (most common): python3 deidentify.py full <file> --locale <code>
  • Step-by-step (cautious): python3 deidentify.py scan <file> --locale <code> first

Available locale codes: kr (Korea), us (USA), jp (Japan), cn (China), de (Germany), uk (United Kingdom), fr (France), ca (Canada), au (Australia), in (India), it (Italy), es (Spain). If --locale is omitted, the script shows an interactive country selection menu. Users can provide a custom locale file via --locale-file custom.json.

Phase 2: Script Execution

Guide the researcher to run the script. The script is located at:

${CLAUDE_SKILL_DIR}/deidentify.py

Full pipeline (recommended for most users):

python3 ${CLAUDE_SKILL_DIR}/deidentify.py full data.xlsx \
    --locale kr \
    --output-dir ./deidentified/

Step-by-step (for careful review):

# Step 1: Scan
python3 ${CLAUDE_SKILL_DIR}/deidentify.py scan data.xlsx --locale kr --output-dir ./deidentified/

# Step 2: Review (interactive)
python3 ${CLAUDE_SKILL_DIR}/deidentify.py review ./deidentified/scan_report.json

# Step 3: Apply (refuses a report that was not reviewed, has a column without a decision,
# or was made from data that has changed since: edit the file, then scan and review again)
python3 ${CLAUDE_SKILL_DIR}/deidentify.py apply ./deidentified/reviewed_report.json

Options:

  • --locale CODE: Country locale for PHI patterns (kr, us, jp, cn, de, uk, fr, ca, au, in, it, es)
  • --locale-file PATH: Custom locale JSON file (copy locales/_template.json to create one)
  • --auto-accept-safe: Keep SAFE columns without showing them. Not recommended: SAFE means no pattern matched, not that the column holds no identifiers (a name typed into a short comment matches no pattern), and this option means nobody looks at those columns
  • --hash-mapping: Store unkeyed SHA-256 hashes instead of original names/IDs in the mapping file. Dates and numeric IDs can be recovered from such hashes by trying every candidate, so mapping.json stays restricted either way
  • --mapping-in PATH (apply/full): Reuse a previous run's mapping.json, so values already de-identified keep the same pseudonym, ID, and date offset. Without it, every run numbers from 0001: merging outputs from separate runs silently gives two patients the same pseudonym (and one patient different ones). A --hash-mapping file cannot be reused this way, since its keys are hashes, not original values. An imported mapping is validated before use: all three sections must be present, replacement values must be P/ID-plus-digits, unique within their section, and different from the original they replace, duplicate JSON keys are rejected, and date offsets must be within 1..365 days (never 0). Row-keyed offsets from a run with no patient key are not carried across runs, and a batch with nothing to anonymize keeps the prior mapping instead of erasing it. With or without it, a new pseudonym or ID skips every original value in the anonymized columns, the patient key column and the prior mapping, so a source ID already shaped like ID0002 is never written out unchanged. An earlier batch cannot see values that arrive later: if source IDs share the P/ID-plus-digits form, an earlier batch's pseudonym can still equal a later batch's original value
  • --output-dir: Where to save de-identified file, mapping, and audit log
  • -v/--verbose: Enable debug logging

Phase 3: Interactive Review Guidance

The script's terminal review has three passes:

  1. Pass 1 — Column Classification: Each column is shown as PHI / REVIEW_NEEDED / SAFE, with sample values. The researcher confirms or overrides each classification.
  2. Pass 2 — Undecided Items: Columns that weren't resolved in Pass 1 get a second look with more sample values displayed.
  3. Pass 3 — Final Summary: A table of all planned actions. The researcher can edit individual decisions before confirming.
  4. Patient key (only when dates will be shifted): the researcher picks the column that identifies the patient, or row if every row is a different patient. Each patient gets their own offset; without a key the tool does not shift dates.

Coach the researcher. Deliver these prompts in the researcher's preferred language:

  • "Columns classified as PHI are anonymized by default. Press 'k' to keep the original value."
  • "REVIEW_NEEDED are columns the script could not vouch for: free text, a PHI word inside a longer column name, ID-like numbers, a rare address. Read the sample values and type 'a' or 'k' — Enter alone is not accepted for these."
  • "SAFE means no pattern matched, not that the column is free of identifiers. Read its sample values too, and press 'r' if a column holds names or anything else identifying."
  • For free-text columns, 'a' replaces each whole text with [REDACTED]: the script cannot find a name inside a sentence, so it does not try to keep the rest of the text.

Phase 4: Verify and Document

After the script completes, help the researcher verify:

  1. Read the audit log (no original values; before_hash is an HMAC-SHA256 under a per-run key that is kept only in mapping.json):

    cat ./deidentified/audit_log.csv | head -20
    

    Verify the number of changes, affected columns, and PHI types.

  2. Ask the researcher to spot-check the de-identified file first, in their own terminal: pseudonyms (P0001, etc.), shifted dates and [REDACTED] markers where expected, and no names in the columns they kept. Read it yourself only after they confirm.

  3. Check that sensitive columns are actually removed: Verify no original names, phone numbers, or RRN values remain.

  4. Mapping file security:

    • Remind the researcher: "mapping.json contains original patient identifiers — treat it as restricted."
    • Recommend storing it separately from the de-identified data
    • File permissions are automatically set to 0600 (owner-only)

Phase 5: Documentation

Generate a de-identification methods paragraph for the manuscript or IRB:

Template:

Direct identifiers were removed from the dataset prior to analysis using a rule-based de-identification tool (deidentify.py, medsci-skills) with the [COUNTRY] locale pattern pack. The tool scanned column names and cell values using regex patterns for country-specific identifiers (e.g., national ID numbers, phone numbers), email addresses, dates, and addresses. Each column classification was reviewed by the researcher in an interactive terminal session. Names were replaced with pseudonyms (P0001, P0002, ...), dates were shifted by a random per-patient offset (1-365 days, either direction) preserving relative temporal intervals, and direct identifiers (phone numbers, email addresses, national ID numbers) were suppressed. A total of [N] cells across [M] columns were de-identified. The de-identification mapping file was stored separately under restricted access (file permissions 0600).

Customize based on the actual audit log statistics. Do not call the dataset "de-identified under HIPAA Safe Harbor" (or anonymised under another law) unless the gaps listed in "What the tool does not do" were closed as well, or an expert determination covers them.

Cross-Skill Integration

  • deidentify sits BEFORE clean-data in the research pipeline
  • After de-identification, hand off to /clean-data for data quality profiling
  • /analyze-stats can safely process the de-identified output
  • /write-paper Methods section should reference the de-identification process
  • /write-protocol can use the HIPAA/PIPA reference files for protocol documentation

Output Files

FileContains PHI?Safe for Claude?Purpose
*_deidentified.xlsx/csvOnly what the researcher kept, and what the tool cannot detect (see below)After the researcher confirms itData for analysis
mapping.jsonYESNoOriginal ↔ pseudonym mapping, per-patient date offsets, audit hash key
audit_log.csvNo original values (keyed hashes)YesWhat was changed and where
scan_report.jsonNo cell valuesYesColumn classification results
reviewed_report.jsonNo cell valuesYesResearcher-reviewed classifications and patient key column

What the tool does not do

The tool removes or replaces the identifiers the researcher marked for anonymization. It does not by itself make a dataset HIPAA Safe Harbor de-identified, or anonymous under other laws:

  • Ages over 89 are kept as they are. Safe Harbor requires ages 90 and over to be grouped (e.g. "90+"); do it before sharing.
  • Shifted dates keep a day and a month. Safe Harbor removes every date element except the year; date shifting is a different method (typically justified by an expert determination or used within a limited data set). Dates the script cannot parse become [DATE_SHIFTED].
  • Names in text are not found. Free-text columns are REVIEW_NEEDED and, if anonymized, replaced as a whole. A name in a short comment column can still be classified SAFE: the review shows every column's sample values so the researcher can catch it.
  • Rare combinations are not assessed. Small cells, rare diagnoses, or quasi-identifiers (age + sex + ZIP + admission month) can identify a patient; no k-anonymity check is run.
  • Geography is only as fine as the patterns. Addresses and postcodes are detected per locale; truncating ZIP codes to three digits (Safe Harbor) is not done.
  • Known limits — bare 5-digit US ZIP codes: a ZIP column is caught by its name (zip, zipcode, zip_code) or by the ZIP+4 form (94110-1234). A bare 5-digit value cannot be told apart from other 5-digit codes (e.g. procedure codes), so a ZIP column under another name can be classified SAFE: check the sample values in review.
  • Known limits — dates: these spellings are still classified SAFE: month-first with a dash (Mar-15-2024), year-first with a month word (2024-Mar-15), month and year only (March 2024), non-English month words (15 mars 2024, 15. März 2024), and two-digit-year day/month dates outside the us locale (15/03/24 under uk). Under us, any M/D/YY-shaped value (e.g. 1/2/10) is treated as a date, so a column of such three-part numbers is flagged PHI/date.
  • Not detected: URLs, IP addresses, account, licence, vehicle and device numbers (fax numbers only where they look like the locale's phone numbers), and identifiers written in column names or file names. Only the first sheet of an Excel file is processed (the other sheets are not written to the output).

Scope and Limitations

Supported (v1):

  • Structured tabular data: CSV, TSV, Excel (.xlsx)
  • 12 country locales with country-specific PHI patterns:
    • Korea (kr): RRN (주민번호), phone, email, address, Hangul names, dates
    • USA (us): SSN, US phone, US address, zip codes
    • Japan (jp): マイナンバー, Japanese phone, 都道府県 address, Kanji names
    • China (cn): 身份证号, Chinese phone, 省市区 address, Chinese names
    • Germany (de): Steuer-ID, German phone, Straße address
    • UK (uk): NHS Number, NI Number, UK phone, postcodes
    • France (fr): NIR/INSEE, French phone, Rue address
    • Canada (ca): SIN, Canadian phone, postal codes
    • Australia (au): TFN, Medicare number, AU phone
    • India (in): Aadhaar, PAN, Indian phone, pin codes
    • Italy (it): Codice Fiscale, Italian phone, Via address
    • Spain (es): DNI/NIE, Spanish phone, Calle address
  • Universal patterns (all locales): email, ISO dates, high-cardinality numeric IDs (MRN)
  • English column names recognized across all locales
  • Custom locale support via --locale-file with template
  • Pseudonymization, date shifting, ID replacement, suppression

NOT supported (planned for v2):

  • DICOM image metadata (PS3.15 Annex E) — requires pydicom
  • Clinical free-text NER (clinical notes, radiology reports)
  • Automated k-anonymity / l-diversity assessment
  • SPSS (.sav), SAS (.sas7bdat), or other statistical formats

Design-AI-Benchmarking Skill

Standard Output

## AI-Benchmark Design Review
Evaluation question: ...
Arms / systems compared: ...
Reference (human-expert panel): ...
Unit of rating: (item / case / output)

### Rubric (decoupled dimensions)
- dimension -> construct -> anchors (1..k)

### Calibration probes (blinded, randomized)
- positive-control / known-bad / instability / mechanism-contradiction

### Reviewer panel
- n reviewers, metadata captured, per-reviewer randomized order

### Reliability plan
- IRR target on the anchor set of real items (ICC form + 95% CI) + control-item hit rate (a competence check, reported separately)

### Judge strategy
- human-as-judge / LLM-as-judge / both + adjudication rule

### Validity risks
1. ...

### Minimal fixes
- ...

### Decision
- Ready to collect / Needs rubric revision / Needs arm or judge redesign

Reviewer ratings, reference labels, probe outcomes and agreement statistics come only from collected rating records. Never invent them: a reported ICC, kappa or score with no underlying rating record is the failure this skill exists to prevent. Cite a reference only with a /search-lit-confirmed DOI or PMID; mark any other [UNVERIFIED - NEEDS MANUAL CHECK]. Flag an unconfirmed clinical definition, diagnostic criterion or guideline recommendation [VERIFY] and ask the user.


Workflow

Phase 1: Define the evaluation question and arms

Pin down, in writing:

  • the exact claim the benchmark must support (e.g., "system A's outputs are perceptually indistinguishable from expert outputs", not "system A is deployment-ready")
  • every arm/system and what each receives as input (same items, same information access, same output format), so no arm has a hidden advantage
  • the human-expert reference: who they are, and whether they set ground truth, form a comparison arm, or both
  • the unit of rating (item, case, output) and how many units each reviewer sees

Gate: Present the reconstructed evaluation question, arms, and reference to the user and confirm before designing the rubric. A wrong reconstruction misdirects the entire benchmark.

Phase 2: Design a decoupled multi-dimensional rubric

  • Decouple the axes. Each rated dimension measures one construct. Keep "is the output valid/correct" separate from "is it novel", "is it feasible/measurable", "does it add value over current tools", and "would it change action". A candidate can be high-validity yet low-added-value ("real but redundant"); a single blended score hides this.
  • Anchor every scale point with a short verbal descriptor; pilot the anchors with at least one reviewer before locking.
  • Pre-specify discriminant validity: hypothesize which dimensions should correlate vs be orthogonal, then report the full inter-dimension correlation matrix.
  • Start from ${CLAUDE_SKILL_DIR}/references/elicitation_rubric_template.md (dimensions, anchors, probe flavors).

Phase 3: Insert and randomize calibration probes

Plant a few deliberate control items, blinded and randomized across raters (record who received which via a probe_arm flag), to anchor the scale, measure rater drift/fatigue, and audit the rubric and pipeline. Four flavors:

  • Positive control / "too-good" item — near-tautological; tests whether raters equate "largest effect" with "best", and whether the construct-independence gate (Phase 7) works.
  • Known-bad negative control — an engineered defect (fabricated reference, missing key statistic).
  • Instability item — an estimate that reverses or fails to replicate on a holdout.
  • Mechanism-contradiction item — an empirical direction that opposes the proposed mechanism.

Probes are planted or adjudicated, never fabricated to fit a hypothesis.

Phase 4: Construct the reviewer panel

  • Recruit reviewers spanning the intended expertise gradient; pre-specify any stratification.
  • Capture reviewer metadata (years of experience, prior AI-evaluation experience, subspecialty).
  • Randomize item order per reviewer (not one global seed), record the order, and plan to analyze order and fatigue effects.
  • Require each item to be judged standalone; cross-item references in free text signal non-independent rating.

Gate: Present the panel composition, stratification, and randomization plan for user review before recruitment is finalized.

Phase 5: Set inter-rater reliability targets

  • Pre-specify the reliability statistic and its form — for several raters, an ICC with the model stated (e.g. ICC(2,1): two-way random, absolute agreement, single rater) or Krippendorff's α (ordinal); weighted (Cohen's) kappa is a two-rater statistic — and a justified target. Size the number of items for the CI width you need: ICC precision is driven mainly by the item count (Koo & Li, J Chiropr Med 2016).
  • Planted control items are a rater-competence check, not reliability evidence. Report the hit rate on known-good vs known-bad controls. Do not present an ICC computed on the controls as evidence of rubric or scale validity: ICC scales with the spread between items, controls are extreme by construction, so their ICC is high whatever the raters' reliability on real items (same five raters, same noise: ICC(2,1) 0.89 on four planted controls, 0.16 on 40 real items), and it rests on a handful of items.
  • Report reliability on the representative anchor set of real items — ICC with its 95% CI and stated form — with an agreement measure alongside (exact agreement, SEM), since reliability depends on how heterogeneous the rated items are (de Vet et al., J Clin Epidemiol 2006). A low ICC on real items with a good control hit rate points to little true spread among the items or a rubric that does not separate them, not to careless raters.
  • Plan the minimum ratings-per-item for a stable agreement estimate (the math goes to /analyze-stats).

Phase 5b: Reader allocation under burden constraints (anchor-and-rotate)

When the item pool exceeds what one reader can rate in a session, do not make every reader rate every item (that caps the pool at the per-reader limit). Use anchor-and-rotate (an anchor set plus rotating incomplete blocks — not a balanced incomplete block design, since reader pairs are not co-rated equally often): all readers rate a shared anchor set of real items (which carries the inter-rater reliability; the planted controls carry only the competence check) plus a rotating unique block each. The binding constraint is usually the number of available expert readers, so solve the reverse problem (largest pool for R readers) to size the must-rate set. Pre-specify anchor membership, raters-per-item, and the rotation seed before rating. Read ${CLAUDE_SKILL_DIR}/references/anchor_rotate_reader_allocation.md for the formulas, trade-offs, and a stdlib implementation.

Phase 6: Choose the judge strategy and adjudication

  • Decide human-as-judge, LLM-as-judge, or both. An LLM judge is one more arm whose ratings must be validated against the human panel on the anchor set of real items, not only on the extreme control items.
  • Pre-specify the adjudication rule for disagreement (majority, a third senior reviewer, consensus discussion) and who adjudicates.
  • Blind judges to arm identity wherever feasible; record any unavoidable unblinding.

Phase 7: Construct-independence and leakage guards

  • Exclude any predictor or input that is a definitional component of the outcome, and flag near-tautological composites built from the outcome's defining components — they produce an inflated, near-circular result and belong as labeled probes, not discoveries.
  • Verify no arm sees post-decision or outcome-derived information the others do not.
  • Confirm the reference labels were not derived from the same model output being evaluated.

Phase 8: Lock a structured export schema

Write the machine-readable rating record as a JSON schema, starting from ${CLAUDE_SKILL_DIR}/references/benchmark_export_schema.json: per-item ratings on every rubric dimension, free-text justifications, follow-up flags, the probe_arm flag, reviewer id and metadata, item order, and timing.

Gate: Present the final rubric, probe set, panel plan, judge strategy, and export schema together; collect explicit user approval before any rating begins, because changes after collection starts compromise the comparison.


Handoff Rules

  • route to /analyze-stats for ICC (stated form) / Krippendorff's α / kappa (two raters) / DeLong, agreement sample size, and effect-size real-world translation of the benchmark results
  • route to /check-reporting for STARD-AI, CLAIM, or TRIPOD+AI item-level reporting once the design is locked
  • route to /design-study when the broader study around the benchmark (cohort logic, analysis unit, comparator) also needs review
  • route to /peer-review or /self-review only after ratings exist and a manuscript is being assessed

Design-Study Skill

Core Review Questions

Always inspect:

  1. The exact research question.
  2. The analysis unit: patient, lesion, exam, study, phase, report.
  3. The index date or decision point.
  4. How inclusion and exclusion criteria are applied.
  5. Any information leakage.
  6. The reference standard or endpoint definition.
  7. A clinically meaningful comparator.
  8. The validation strategy.
  9. The uncertainty reporting required.
  10. The best-fitting reporting guideline.
  11. Whether exposure/outcome/covariate definitions are literature-grounded or invented ad hoc from the data dictionary. If ad hoc, defer to /define-variables before drafting Methods.

Standard Output

## Study Design Review
Question: ...
Study type: ...
Analysis unit: ...
Index date / prediction timepoint: ...

### Strengths
- ...

### Major validity risks
1. ...
2. ...

### Minimal fixes
- ...

### Reporting fit
- Recommended guideline: ...

### Decision
- Ready for analysis / Needs redesign / Drafting can proceed with limitations

Cite a reference only with a /search-lit-confirmed DOI or PMID; mark any other [UNVERIFIED - NEEDS MANUAL CHECK]. Never invent a clinical definition, diagnostic criterion, or guideline recommendation: flag an unconfirmed one [VERIFY] and ask the user.


Workflow

Phase 1: Reconstruct the study

Extract from the protocol, draft, slides, tables, or notes: clinical problem, intended use case, population, inputs, outputs, outcome definition, and timing of variable availability.

Gate: Present the reconstructed study summary (question, analysis unit, intended use) to the user and confirm it before proceeding — a wrong reconstruction misdirects the entire validity review.

Phase 2: Check structural validity

Read a reference only when its condition holds:

FileRead it when
references/dag_adjustment.mdconfounding control needs an explicit adjustment set
references/target_trial_emulation.mdthe design emulates a target trial
references/venue_accept_recipe.mda clinical DL / AI-validation study must decide which venue tier the achievable design can be accepted at, and the one design move that reaches the tier above (the bridge into /find-journal); skip when there is no publication-tier decision
references/combine_models_ablation_design.mdthe model is built by combining / adapting / fine-tuning existing models (nnU-Net, TotalSegmentator, SAM/MedSAM, a pretrained backbone) and the comparator must be designed as an ablation; skip for a model trained de novo
references/multi_model_comparison_design.mdthe contribution is comparing several models / architectures head-to-head and the comparison must be fair; skip for a single-model study (one model's ablation → the row above; AI-vs-human → /design-ai-benchmarking)
references/segmentation_failure_characterization_design.mdthe claim is that a segmentation model is clinically usable, not that it scores well; skip when the endpoint is benchmark accuracy (metric choice → /model-assessment; abstention / risk–coverage → /model-assessment)
A. Analysis unit

Look for mismatches: a patient-level claim from lesion-level analysis; an exam-level split with patient overlap; phase-level samples treated as independent.

B. Leakage

Look for:

  • postoperative features used for preoperative prediction
  • normalization or thresholding performed before the data split
  • repeated exams across train/test
  • reader annotations derived from outcome information
  • input-text contamination for NLP/LLM extraction tasks: if the input includes clinical history, indication, impression, prior diagnosis, or referral text, confirm it does not name or strongly imply the target label. If it does, the task is information retrieval under label leakage, not phenotype inference: redesign the input mask, report a sensitivity analysis excluding those fields, or reframe.
  • construct dependence (a predictor that is a definitional component of the outcome): (i) an input that computes the outcome (fasting insulin and glucose when the outcome is HOMA-IR), or (ii) a near-tautological composite of the outcome's defining components (inflated, near-circular association). Test: "could this predictor be derived, in whole or part, from the outcome's definition or the same measurement?" If yes, exclude it, or keep it only as a labeled calibration probe.
F. Time origin & survivorship (incident / transition models)

For any time-to-event or incident/transition design, check before drafting:

  • Time origin per model. Each incident model starts its at-risk clock at the correct origin. Watch for immortal-time bias (a span in which the event cannot occur, misattributed to one group) and left-truncation / delayed entry.
  • Mediator-ascertainment-window survivorship. A "progressor" / transition label conditional on surviving to a later ascertainment (a second scan, a follow-up visit) is survivorship-biased; plan a landmark time or an explicit intermediate-state (multistate / illness-death) model.
  • Primary-analysis-set selection. If the primary is not the full cohort, pre-specify why, from the missingness mechanism relative to the outcome: complete-case regression is unbiased when missingness is independent of the outcome given the covariates (even with a covariate MNAR) and generally biased when it depends on the outcome, including under MAR, where multiple imputation is the usual primary (Hughes et al., Int J Epidemiol 2019; logistic-regression odds ratios are the exception when missingness depends on the outcome alone). Never pick it for significance.
  • A design that cannot yet answer these should say so — but at review time a Methods/Limitations admission that it was "not formally assessed" is escalated to MAJOR by the survival probe (S1).
C. Reference standard

Check who established ground truth, when, whether blinding was possible, and whether only a subset had gold-standard verification. Then:

  • Construct ↔ nominal-definition match. Restate each construct's nominal definition and confirm every included case satisfies it. An "incidentaloma" defined as an indeterminate finding must not include frank malignancy reads; a label that overshoots its definition inflates the cohort and breaks the κ.
  • Per-flag reference-standard concordance. Report concordance per flag category, not only overall; a construct where most flags (e.g., ~86%) miss the reference standard measures something else.
  • Manuscript definition ↔ variable_operationalization.md. Methods definitions must match the /define-variables operationalization table verbatim, and a blinded re-classification form must quote the analytic protocol's definition verbatim. A paraphrase or "common-sense extension" in the form is the documented cause of a low κ that is a definition mismatch, not real disagreement.
D. Validation

Classify: apparent only, internal split, cross-validation, temporal, external, or multi-center external.

E. Reader / expert-elicitation studies

When the study elicits expert ratings — a reader study, an annotation panel, an AI-output evaluation — read references/reader_elicitation_design.md before data collection. The acceptance ceiling of a perceptual / reader AI study is fixed at design time: no quality of execution lifts a ceiling baked into the comparator, the estimand, or the reader cohort. For an AI-system-versus-human-expert benchmark, route to /design-ai-benchmarking (arm definition, LLM-as-judge versus human-as-judge adjudication, structured export schema).

Phase 3: Clinical framing

Ask whether the comparator and endpoint support the stated claim:

  • Is the model better than current practice, or just another model?
  • Is the endpoint clinically meaningful?
  • Does performance translate to action?
  • incremental value: if the model/marker is framed as adding value beyond / on top of / incremental to an existing tool (a clinical score, a routine test, a baseline model), pre-specify the baseline comparator built from the in-routine-use predictors and how added value will be shown: test it with the likelihood-ratio (or Wald) test of the new term in the development data; report ΔC-index / ΔAUC with a paired CI as the size of the gain, not as a second test (a DeLong test of nested models fitted and evaluated on the same data is invalid; DeLong is for two models frozen before a held-out evaluation); prefer the change in decision-curve net benefit at a pre-specified threshold; NRI only as a pre-specified categorical NRI reported separately for events and non-events, never the continuous NRI (analyze-stats incremental_value.md; Kerr et al., Epidemiology 2014). A standalone discrimination number does not support a "beyond X" claim, and the nested-model comparison cannot be added post hoc without the baseline model.
  • fine-tuning contribution baseline: if an NLP/LLM study claims that fine-tuning, LoRA, prompt engineering, or a multi-agent wrapper improves extraction/classification, pre-specify a same-backbone zero-shot or few-shot comparator on the identical input, output schema, and test split; a weaker or unrelated baseline cannot show the adaptation adds value. For imaging:
    • a model built by combining / adapting / fine-tuning existing models → the full ablation ladder (un-adapted base, best single component, direct-train vs transfer), per references/combine_models_ablation_design.md;
    • a head-to-head comparison of several models → comparison fairness: one frozen split/preprocessing through every model, a strong fairly-tuned baseline, a matched (or disclosed) compute budget, and a paired delta test, per references/multi_model_comparison_design.md;
    • a claim that a segmentation model is clinically usable → a pre-specified failure taxonomy, an acceptability endpoint with a named judge and adjudication rule, the tail beside the mean, and edit effort paired against manual-from-scratch, per references/segmentation_failure_characterization_design.md. A mean DSC cannot be converted into a usability claim after the fact.
  • endpoint↔conclusion scope: decide up front what kind of conclusion the design can support. A cross-sectional / single-visit / prevalence design cannot support a prognostic or surveillance claim (rescreen interval, disease progression) — that needs longitudinal follow-up. A binary surrogate endpoint (present/absent, >0, dichotomized) is risk stratification, not a patient-care directive (defer/withhold/initiate therapy). At review time /self-review §D + check_scope_coherence.py flag CROSS_SECTIONAL_PROGNOSTIC / SURROGATE_CARE_DIRECTIVE against the conclusion.

Phase 4: Reporting fit

Recommend one primary guideline — TRIPOD+AI, CLAIM, STARD, STROBE (TARGET for a target-trial emulation), PRISMA, CARE, or ARRIVE — plus journal-specific additions if needed.


Frequent Failure Modes

Diagnostic AI

No clinically relevant comparator; exam-level instead of patient-level split; unclear reference standard; AUROC-only reporting without threshold metrics.

Prognostic modeling

Unclear time zero; immortal time bias; feature timing mismatch; no calibration.

Retrospective cohort / screening database

  • Time zero misalignment: cohort entry ≠ follow-up start → immortal time bias.
  • Interval-censored outcomes treated as exact → underestimated event times.
  • Unacknowledged healthy-volunteer bias → inflated external-validity claims.
  • Surveillance bias from unequal follow-up frequency between groups.
  • Map each threat explicitly to one of the three bias classes (Hernán/Robins): selection bias (who enters), information bias (how measured), confounding (what else differs).
  • Comparative / causal question → emulate a target trial. For a treatment-vs-treatment, screening-vs-no-screening, or drug-A-vs-drug-B question on routinely-collected data, specify the seven target-trial components (eligibility, strategies, assignment, time zero, outcome, causal contrast, analysis plan) before extraction. This prevents immortal-time, prevalent-user, and confounding-by-indication bias and turns an association into a defensible causal contrast. New-user + active-comparator design, grace-period clone-censor-weight, and negative controls are in references/target_trial_emulation.md.
  • Confounding completeness: pre-specify the adjustment set from a DAG (not a Table-1 p < 0.05 rule), and plan an extended-adjustment sensitivity model to show whether any measured covariate that turns out imbalanced by exposure but lies outside the adjustment set leaves the primary estimate robust. Pre-screen the proposed covariates with scripts/adjustment_set_helper.py (flags mediator / collider / descendant adjustment and omitted confounders, and proposes a candidate backdoor set), then derive the minimal sufficient set with dagitty — see references/dag_adjustment.md. At review time /self-review Phase 2.5e and the O-probes in observational_confounding.md (O1–O18) check this against Table 1 — including O7 over-adjustment, O10 overlapping-subset-gradient discipline, O11 design-based weighting for complex-survey data, O12 data-driven-threshold mining, O13 (a cross-sectional mediation claim cannot order X→M→Y), and O14 (a synergy/joint-effect claim needs the additive interaction scale — RERI/AP/S — not a multiplicative-only test).

Multimodal LLM / report generation

No clear rubric for clinical correctness; benchmark labels derived from noisy reports without adjudication; unsupported claims about safety or workflow benefit; input text containing the target label or diagnosis being predicted; no same-backbone zero-shot/few-shot baseline for a fine-tuning or prompt-engineering claim.

Imaging meta-analysis

Overlapping cohorts; paired modalities analyzed as independent; heterogeneity metrics missing; zero-cell handling unspecified.


Minimal-Fix Principle

Recommend the smallest feasible repair first: clarify the claim, narrow the target population, add a limitation statement, add a clinically relevant baseline, re-run one key sensitivity analysis, or redefine the endpoint more explicitly. Escalate to redesign only when the central claim is not defensible otherwise.


Handoff Rules

  • route to analyze-stats when the design is basically sound but analysis details need refinement
  • route to check-reporting after the design is locked
  • route to self-review when the user wants a pre-submission quality check on their own manuscript
  • route back to write-paper only after the main validity risks are documented

This skill does not compute statistics, draft full manuscript prose, resolve raw data-engineering issues, or replace a full peer review when journal-facing tone is required.

/explainability was renamed to /model-assessment in v6.0; this redirect is removed in v7. Run /model-assessment now, passing the same arguments the user gave /explainability. Start at /model-assessment Part D, Phases 12–13 (saliency sanity checks, localisation, attribution framing).

Fill-ICMJE-COI Skill

The official ICMJE coi_disclosure.docx keeps every field inside Word Content Controls (SDTs), which python-docx cell.text silently ignores. The script therefore does literal-string replacement in word/document.xml, which only works when the target strings already exist — hence the shipped pre-filled synthetic seed.

Core Principles (Do Not Violate)

  1. Never author SDT XML from scratch. Only replace existing strings in an already-populated seed, because programmatic Content Controls are fragile and Word-version-dependent. If an error seems to require altering the seed structure, stop and escalate to the user.
  2. Never use or commit a seed that contains a real author's data. The shipped templates/icmje_coi_seed_synthetic.docx is PII-scrubbed (synthetic name, title, date; metadata ICMJE / Anonymous); real-person seeds leak PII through both document.xml and docProps. Keep custom seeds with real names in private per-project directories, and before promoting any seed check unzip -p seed.docx docProps/core.xml for real names.
  3. Never modify the 13 disclosure items or the certification checkbox. The script replaces only Date, Name and Title; the ☒ None entries come from the seed unchanged. Never say the script "handled the disclosures": it cloned the seed, and no author-specific disclosure reasoning happened. An author with a real disclosure edits their own form in Word.
  4. The seed must be pre-filled (all-None ☒ + text). The script does not work on a blank ICMJE template.

Skip this skill when the target journal uses its own declaration form (Elsevier Declaration of Interest, a BMJ ICMJE derivative, etc.) rather than the canonical ICMJE form — check the author guidelines first.

Execution

Phase 1 — Intake

Ask the user (or extract from the conversation):

  1. Manuscript title (exact, as on the title page)
  2. Submission date (e.g., "April 20, 2026")
  3. Author list — ordered, one name per slot: [(1, "Author One"), (2, "Author Two"), ...], taken verbatim from the manuscript title page or the user's list
  4. Output directory — typically submission/{journal}/icmje_forms/

Gate 1 — user approval. Present the intake back before generating anything. Name which authors will get all-None forms and remind the user that anyone with a real disclosure must fill their own form in Word instead.

Phase 2 — Generate

python3 ${CLAUDE_SKILL_DIR}/scripts/fill_icmje_coi.py \
  --seed ${CLAUDE_SKILL_DIR}/templates/icmje_coi_seed_synthetic.docx \
  --seed-name "Placeholder Author" \
  --seed-title "Placeholder Manuscript Title" \
  --seed-date "January 1, 2000" \
  --new-title "{exact manuscript title}" \
  --new-date "{submission date}" \
  --out-dir {out_dir} \
  --authors '[[1,"Author One"],[2,"Author Two"],...]'

It writes one ICMJE_COI_{NN}_{Name}.docx per author and exits nonzero if any seed string is not found. It exits 2, before writing anything, if any --seed-* or --new-* value or author name is empty: an empty seed string would match nothing and leave the seed's value in every form.

Phase 3 — Verify

For each generated docx confirm: ☒ count = 14 (13 disclosure items + final certification); "None" disclosures = 13; the correct name after "Your Name:" and title after "Manuscript Title:"; no seed strings left. SEED must be the exact --seed-name, --seed-title and --seed-date values used in Phase 2 (the shipped placeholders below, or a custom seed's own strings); TITLE must be the --new-title value.

python3 - {out_dir}/*.docx <<'PY'
import sys, zipfile
SEED = ("Placeholder Author", "Placeholder Manuscript Title", "January 1, 2000")  # = --seed-* values
TITLE = "{exact manuscript title}"                                                 # = --new-title
assert all(s.strip() for s in SEED) and TITLE.strip(), "empty SEED/TITLE value"
for f in sys.argv[1:]:
    xml = zipfile.ZipFile(f).read("word/document.xml").decode("utf-8")
    assert xml.count("☒") == 14, f"{f}: bad ☒ count"
    assert xml.count(">None<") + xml.count(">None ") == 13, f"{f}: bad None count"
    for ph in SEED:
        assert ph not in xml, f"{f}: seed leak {ph!r}"
    t = TITLE.replace("&", "&amp;").replace("<", "&lt;").replace(">", "&gt;")
    assert t in xml, f"{f}: title missing"
    print("✓", f)
PY

This snippet does not locate the name inside the "Your Name:" field; confirm the name by opening each form.

Gate 2 — user review. Present the verification results before handing off the files.

Phase 4 — Circulation Guidance

Give the user this copy to send with each personalized form, written in the co-authors' language:

Please review the attached ICMJE COI form.

  • If the contents are correct, sign and reply with a PDF.
  • If a change is needed, edit/check the relevant item, sign, and reply.
  • If there are no changes at all, reply "no changes" and return the signed PDF separately.

All authors can be emailed in one draft batch (e.g., gws gmail draft). Gate 3 — the user approves the batch before anything is sent.

Custom Seeds and Seed Provenance

Read ${CLAUDE_SKILL_DIR}/references/seeds.md when the user wants a custom seed (different default wording, a common grant pre-filled in items 2/3) or asks how the shipped seed was made.

Fill-Protocol Skill

Core Principles (Do Not Violate)

  1. Open the existing template — never create from scratch. Use Document(template_path), not Document(): rebuilding loses the header logo, custom margins, footer placeholders and page numbering. Replace cell/paragraph text in place; never a single cell.text = "value" assignment, which erases run-level styles (bold, color, eastAsia font).
  2. Convert .doc → .docx via LibreOffice headless before any editing. pandoc -f doc is not supported; textutil corrupts table structure (merged cells dropped).
  3. Match cells by left-label text, not row/column coordinates such as table.cell(2, 1). Templates evolve and coordinate matching breaks silently.
  4. Apply cantSplit to every filled row so a row never breaks across pages.
  5. For CJK languages, set the eastAsia font attribute, not just run.font.name, or Hangul/Kanji/Hanzi render in fallback fonts.
  6. Validate every fill operation: resolve every [MISS] line and every WARN: line ([TABLE-MISS], [SECTION-MISS], [RAW-NEWLINE]) before the filled form is submitted.

Dependencies

Python packages docxtpl python-docx pyyaml are always required. LibreOffice is needed only for a legacy .doc template (~700 MB on macOS). The bundled setup.sh detects what is missing:

bash ${CLAUDE_SKILL_DIR}/setup.sh check     # report what's installed (read-only)
bash ${CLAUDE_SKILL_DIR}/setup.sh install   # install missing pieces (asks before each)

When invoking this skill on behalf of a user:

  1. Skip LibreOffice entirely if the template is already .docx.
  2. Before calling doc_to_docx.py, run setup.sh check. If LibreOffice is missing, ask the user before installing, because the cask is ~700 MB.
  3. Never pass --yes to setup.sh install unless the user has explicitly authorized unattended installation in this session, because it installs without prompting.
  4. If the user declines, ask them to convert the .doc manually (Word/LibreOffice/Pages → Save As → .docx) and re-run with the converted file.

Workflow

Step 1 — Convert legacy .doc to .docx (if needed)

python3 ${CLAUDE_SKILL_DIR}/scripts/doc_to_docx.py path/to/template.doc path/to/template.docx

Step 2 — Inspect the template structure

python3 ${CLAUDE_SKILL_DIR}/scripts/inspect_template.py path/to/template.docx

It lists every table, every cell (coordinates and content preview) and every top-level paragraph. Take the exact labels for the YAML from this output.

Step 3 — Author a content YAML

Start from ${CLAUDE_SKILL_DIR}/examples/example_irb_template.yaml. Three fill modes; all keys are optional.

protections:
  korean_font: "맑은 고딕"   # CJK font (set to "Noto Sans CJK KR", "SimSun",
                              # "MS Mincho", etc. for other locales)
  cant_split: true            # Apply <w:cantSplit/> to every filled row

  # Readability options (defaults shown)
  blank_between_paragraphs: true            # Enter between \n\n chunks
  blank_around_section_header: true         # Enter above/below filled sections
  blank_around_all_section_headers: false   # opt-in; also touches untouched sections
  normalize_page_breaks: true               # empty page-break paragraphs -> pageBreakBefore

# Mode 1 — table key/value (left-label cell → right value cell)
table_kv:
  "Study Title": "Multi-center prospective validation of ..."
  "Principal Investigator": "Last, First (Department)"
  "연구 목적": "본 연구는 ..."

# Mode 2 — section replacement (find numbered header, replace until next header)
section_replace:
  "1. Background":
    "Hepatocellular carcinoma is the third leading cause of ..."
  "4. 연구 배경 및 이론적 근거":
    "..."

# Mode 3 — single paragraph in-place text replacement
paragraph_replace:
  "Title:":
    "Title: Multi-center prospective validation of ..."

Read ${CLAUDE_SKILL_DIR}/references/best_practices.md (Readability knobs) before changing any readability option from its default.

Content rules: put a reference in only with a /search-lit-confirmed DOI or PMID, otherwise mark it [UNVERIFIED - NEEDS MANUAL CHECK]; mark any unconfirmed clinical definition, diagnostic criterion or guideline claim [VERIFY] and ask the user.

Step 4 — Run the fill

python3 ${CLAUDE_SKILL_DIR}/scripts/fill_form.py \
  --template path/to/template.docx \
  --content  content.yaml \
  --output   path/to/filled.docx

The CLI prints [OK] / [MISS] for every fill operation, a summary, then any WARN: lines. Investigate every [MISS] and WARN: before submitting (Principle 6).

Every value under table_kv, section_replace and paragraph_replace is written exactly as typed (012345, 12:30, 1.10, No stay text). An empty value or a list/mapping value stops the run with exit 2 and names the label; write '' to blank a field on purpose. A section_replace on a header with no later numbered header replaces everything to the end of the document, and its [OK] line says so with the number of paragraphs replaced. If a signature/date block or anything else follows the last section, bound it with section_end (header -> regex matched against the paragraph that starts the kept block):

section_end:
  "18. References": "^Investigator signature"

Step 5 — Visual verification

soffice --headless --convert-to pdf path/to/filled.docx

Without LibreOffice (a .docx template skips it), export the PDF from Word instead. Open the PDF and confirm: page count is sensible, no table row was split across pages, no font fell back to Times New Roman, all required fields are populated.

Read ${CLAUDE_SKILL_DIR}/references/best_practices.md when a label or section header does not match, the template has no numbered section headers (section_end), a cell needs multi-line content, or cells are merged.

Known Limitations

  • HWP / HWPX input is not handled directly — convert it to .docx first.
  • Merged cells: filling a label cell that participates in a vertical merge may overwrite the merged region's content. Test on a copy first.
  • Embedded form fields (Word content controls): not supported; plain paragraph and table cell content only.
  • Right-to-left scripts (Arabic, Hebrew): untested.
  • Last section with no later header: replaced through the end of the document unless a section_end pattern is given; the filler cannot tell the section body from a trailing block.

Find-Cohort-Gap Skill

Output directory: user-specified (default: current working directory).

Phase 0: Cohort Intake

The cohort does not have to be one this skill has heard of. Route on what the user actually has.

The user has…Do this
A named public cohort (NHIS, UK Biobank, KNHANES, …)Fill the profile from published documentation. Cite the source for every field.
A codebook / data dictionary / CSV export of their own registry or EMR extractRun the input adapter below. This is the common case — an institutional registry or single-centre export that no public documentation describes.
A review, guideline, or preprint defining the clinical domainAttach it as domain context (--context), as a file or a URL.

Input adapter (local codebook / documents)

python3 "${CLAUDE_SKILL_DIR}/scripts/build_cohort_profile.py" \
  --codebook data_dictionary.csv \
  --context narrative_review.pdf --context https://example.org/guideline \
  --cohort-name "Institutional CT registry" --out-dir .

Formats: .csv / .tsv / .json / .md / .txt (stdlib), .xlsx (needs openpyxl), .pdf (needs pdftotext). A .csv is auto-detected as a codebook (rows are variables) or a data export (the header row is the variable list). Writes cohort_profile.md + cohort_profile.json (+ context_extract.md).

Do not read the codebook yourself and summarise it. A paraphrased, merged, or invented variable poisons every downstream claim — the intersection matrix, the feasibility gate, and eventually the manuscript's Methods. The adapter enumerates variables verbatim with provenance (file:row). Read cohort_profile.md; do not re-derive it.

The adapter infers, and shows its work for, the variable cluster map, serial / repeated-measure groups (evidence for P1 Longitudinal Advantage), and endpoint candidates (evidence for P2 Endpoint Upgrade). A variable matching no cluster keyword is left unclassified — review those, since the lexicon is not exhaustive.

What the adapter cannot know — ASK, never guess

A codebook does not state any of the following; each is emitted as [UNKNOWN - ask the user]:

  1. Sample size (N at baseline, N with follow-up)
  2. Time span (enrollment period, follow-up duration, measurement intervals)
  3. Known limitations (healthy volunteer bias, attrition, missing-data patterns)
  4. Existing publications from this cohort (to avoid duplicating them)
  5. IRB status and data-access route

Collect these from the user before Phase 2, because a guessed N flows into the Phase 5 feasibility gate and makes it pass or fail for a reason unrelated to the cohort. Also confirm the setting (institution type, country, population type) and any special strengths the variable names cannot reveal — registry linkage, biobank availability, a distinctive population.

Gate: Present the cohort profile summary, including the [UNKNOWN] list and the unclassified variables. Confirm before proceeding.

Phase 1: PI/CA Profiling

Profile the intended PI or corresponding author to find topic-expertise alignment. If no PI is specified, skip this phase and use variable clusters alone in Phase 2.

  1. Search PubMed for the PI's recent publications (last 5 years) with /search-lit's E-utilities script: bash "${CLAUDE_SKILL_DIR}/../search-lit/references/pubmed_eutils.sh" search "AuthorLastName AuthorFirstInitial[Author]" 30 Extract top keyword clusters from titles/abstracts.
  2. Identify specialty signals: academic society positions (president, board member, editor), subspecialty focus areas, preferred journal tiers.
  3. Build a PI keyword map: 5-10 keyword clusters ranked by publication frequency.

Output: PI profile card (name, affiliation, top keywords, society roles, preferred journals).

Phase 2: Intersection Matrix

Cross cohort variable clusters with PI expertise to generate candidate topics.

Method

Create a matrix: rows = DB variable clusters, columns = PI keyword clusters. Score each cell 0-3:

  • 3: PI has published in this exact intersection (direct match)
  • 2: PI's subspecialty covers this area (strong relevance)
  • 1: Tangential connection (possible but needs framing)
  • 0: No connection

Candidate Generation

  1. Extract all cells scoring 2-3 as primary candidates.
  2. For cells scoring 1, apply the A-B substitution test: "Has someone published [this analysis] with [a different exposure/outcome] in a similar cohort?" If yes, substituting the PI's specialty variable creates a viable candidate.
  3. Generate 20-40 candidate topic statements in PICO format — P population from the cohort, E exposure/predictor variable(s), C comparison group, O outcome (preferably hard endpoint).

Discipline Alignment Filter

Before saturation scanning, identify the intended first author's department/specialty. The primary exposure variable must belong to that discipline (e.g., radiology first author → imaging variable as the primary exposure). Kill candidates where the primary exposure is outside the first author's discipline — a strong PI match alone is insufficient if the first author cannot claim ownership of the core variable.

Gate: Present the intersection matrix and top 20 candidates (post-discipline filter). User selects 8-12 for saturation scanning.

Phase 3: Literature Saturation Scan

For each selected candidate, determine how saturated the literature is.

Search Strategy

For each candidate:

  1. Build a PubMed query: (exposure terms) AND (outcome terms) AND (cohort OR longitudinal OR prospective)
  2. Execute search via /search-lit E-utilities.
  3. Count total results, then ask the Critical Filter question — "Has anyone published this with serial/repeated measurements?" — and check whether a meta-analysis exists. Grade:
PapersLongitudinal papersMA exists?GradeInterpretation
0-20NoBlue OceanFirst report possible. Verify the topic has audience interest.
3-100NoGreen FieldOptimal zone — established interest, longitudinal gap wide open.
11-300NoGreen Field (upgraded from Yellow)As above.
1-301+NoYellowViable only with very specific angle (unique population, novel endpoint).
>30AnyNoYellow (borderline Red)As above.
AnyAnyYes, outdated (>5 yr) or limited scopeYellowAs above.
AnyAnyYes, recentRedAvoid unless doing NMA or using truly unique data.

"So What" Test

For each candidate, articulate 2-3 potential clinical implications of the findings. If you cannot state why a clinician or policymaker would care about the result, the topic fails regardless of gap score.

Output: Saturation table with grade, paper count, longitudinal gap status, and "So What" statement for each candidate.

Gate: Present saturation results. User selects 3-5 finalists for deep scoring.

Phase 4: 6-Pattern Scoring + Comparison Table

Apply the 6-Pattern framework to each finalist. Score each pattern 0 or 1.

6 Patterns (Universal)

Read the detailed rubric at ${CLAUDE_SKILL_DIR}/references/pattern_scoring_rubric.md and score each finalist on P1 Longitudinal Advantage, P2 Endpoint Upgrade, P3 Cohort Uniqueness, P4 PI-Topic Alignment (skip if no PI specified), P5 Comparison Table Gaps (3+ features unique to THIS STUDY in the table below), and P6 Complementary Design.

Comparison Table Construction

For each finalist, build a table comparing the top 3-5 existing papers against THIS STUDY, one row per feature (design, N, serial data, hard endpoint, population, ethnicity, subgroup analysis, …) — the rubric's P5 lists the construction steps and differentiator categories. Cite existing papers only with a /search-lit-confirmed DOI or PMID; otherwise mark the reference [UNVERIFIED - NEEDS MANUAL CHECK].

Score Interpretation

Total ScoreRecommendation
5-6Top-tier journal target (Lancet sub, JACC, J Hepatol level)
3-4Specialty journal target (solid publication)
0-2Restructure or kill — find a stronger angle before proceeding

Gate: Present scoring results and comparison tables. User approves final ranking.

Phase 5: Feasibility Gate

For each scored finalist, verify practical feasibility.

Checks

  1. Sample size adequacy:
    • Cox and logistic regression: minimum 10 events per predictor variable (EPV rule)
    • For large cohorts (N>100K) with many outcome events: warn that small effects become statistically significant (power depends on the number of events, not N — check the event count first), so focus on effect size thresholds (e.g., HR >1.2 or <0.8 for clinical relevance)
    • Consider negative control strategy (EPCV) for very large samples
  2. Missing data: key exposure variable <20% missing acceptable; key outcome <5% missing; if serial data, assess the attrition pattern (MCAR/MAR/MNAR)
  3. Follow-up adequacy: the outcome must have plausible latency within available follow-up — cancer outcomes minimum 5 years, CVD events minimum 3 years, mortality minimum 5 years
  4. Operational definition:
    • Can the exposure be defined from available variables?
    • For claims data: ICD codes alone = 40-60% accuracy. Require combination strategy (diagnosis + prescription + visit frequency + special codes)
    • Cross-check expected prevalence against known epidemiological data
    • Flag any clinical definition, diagnostic criterion, or guideline claim you cannot source with [VERIFY] and ask the user.
  5. IRB/ethics: is the data already IRB-approved for this type of analysis? Any additional approvals needed for data linkage?
  6. Disease Novelty Bonus (informational, not Go/No-Go): idiopathic etiology or debated mechanism → higher journal interest; established mechanism → needs stronger methodological novelty

Decision

  • Go: All checks pass.
  • Conditional Go: Minor issues solvable (e.g., missing data manageable with imputation).
  • No-Go: Fatal flaw (insufficient events, no valid endpoint, key variable unavailable).

Output: Feasibility report for each finalist with Go/Conditional/No-Go status.

Phase 6: Output — Ranked Proposals + One-Pagers

Ranked Summary Table

| Rank | Topic (PICO) | Saturation | 6-Pattern Score | Feasibility | Target Journal | Timeline |
|------|--------------|------------|-----------------|-------------|----------------|----------|
| 1 | ... | Green (0 longitudinal) | 5/6 | Go | JACC | 6 months |
| 2 | ... | Blue (0 papers) | 3/6 | Conditional | Radiology | 8 months |

One-Pager for Each Finalist

Fill every section of the template at ${CLAUDE_SKILL_DIR}/references/onepager_template.md; give the target journal with its rationale (PI alignment, scope match, gap fit). Save one-pagers as markdown files: {output_dir}/gap_proposal_{rank}_{short_topic}.md

Downstream: output feeds into /design-study → /write-paper.

Find Journal Skill

Phase 1: Input Collection

Required Inputs

  1. Abstract text or key findings summary
  2. Study type: original research, meta-analysis, case report, technical note, review, letter, AI validation, diagnostic accuracy, etc.

Optional Inputs

  1. Preferred tier: Q1 / Q1-Q2 / any (default: any)
  2. OA preference: Full OA / Hybrid OK / No preference (default: no preference)
  3. Field focus: radiology, medical AI, clinical specialty, methodology, education, general medicine
  4. Journals to exclude: journals that have already rejected this manuscript

If only an abstract is given, infer the study type from it. When called from /write-paper or another skill, take the abstract and study type from the calling context and skip these questions.


Phase 2: Theme Extraction

From the abstract or key findings, extract: disease/condition; modality/technique; methodology (e.g., retrospective cohort, diagnostic accuracy, systematic review, RCT); population; and innovation type (new algorithm, clinical validation, workflow improvement, educational tool).


Phase 2.5: Acceptance-Readiness & Design-Ceiling Pre-flight

Editors screen importance/novelty and the design ceiling before scope fit (lack of novelty/importance is the top desk-rejection cause). This phase estimates that first filter, so the skill does not recommend a venue whose bar the design cannot clear. It is advisory, and its flags are not auto-fixable: the author decides.

2.5.1 Run the deterministic pre-flight (preferred)

If a manuscript or abstract file is available, run:

python3 ${CLAUDE_SKILL_DIR}/scripts/assess_acceptance_readiness.py <manuscript_or_abstract.md>
# add --json for a machine-readable report

It returns flags in four categories — DESIGN_CEILING, UNFIXABLE_DEFECT, IMPORTANCE_RISK, CLAIM_MISMATCH — and a ceiling verdict (NO LISTED SIGNAL MATCHED - NOT A DESIGN CLEARANCE … / IMPORTANCE-FRAMING REVIEW … / SPECIALTY / TOLERANT-VENUE OR DESIGN FIX … / HIGH-IMPACT VENUE UNLIKELY …).

The script reads UTF-8 markdown or plain text only. A .docx, .pdf, .doc or other non-text file exits 2 with no report ("No scan was run"): convert it (e.g. pandoc -t markdown) and rerun, or fall back to 2.5.2. It skips the References/Bibliography section.

Known limits. The scan matches listed phrases, line by line. It does not understand negation or context ("Unlike prior single-center studies…" still flags single-center), and it misses paraphrases ("one institution", "has not been externally validated", the same patients in training and test sets). So an empty result is not a clearance: apply the §3 taxonomy with judgement (as in 2.5.2) before treating feasibility as uncapped, and adjudicate each flag before it demotes a venue.

2.5.2 If only pasted text is available

Read ${CLAUDE_SKILL_DIR}/references/acceptance_signals_schema.md §3 (taxonomy and verdict bands), apply the same taxonomy to the pasted text with judgement, and assign the same ceiling verdict.

2.5.3 Carry the verdict forward

Record the ceiling verdict and its top flags for Phase 3.2 Axis 2 and for the Phase 4 Acceptance-Readiness Summary and Cascade plan. Do not block the recommendation on a ceiling: route to a venue the design can clear, or recommend a design change or presubmission inquiry.


Phase 3: Profile Loading and Matching (2-Pass)

3.1 Pass 1: Load Compact Profiles

Read both tiers and merge them into one profile set:

# Public (shipped with the skill)
${CLAUDE_SKILL_DIR}/references/journal_profiles/*.md

# User-local private (optional, may be empty or absent)
$HOME/.claude/private-journal-profiles/find-journal/*.md

On a filename collision the private copy wins (user override). If either directory is missing or empty, continue with the other and say which tier was unavailable. Count the profiles actually loaded after the merge and state the total in the output — never hard-code the count.

From each profile parse Scope, Scope Keywords, Article Types Accepted, Classification (Tier, OA, Field), Special Notes (includes a 1-line AI policy summary), and the optional Acceptance Signals block (format: ${CLAUDE_SKILL_DIR}/references/acceptance_signals_schema.md §1).

Do NOT read write-paper profiles in this pass — they are 4-5x larger and carry formatting detail irrelevant to matching.

3.2 Two-Axis Scoring (scope fit × acceptance feasibility)

Score each journal on two independent axes: scope fit ("does this journal cover my topic?") and acceptance feasibility ("can this manuscript's design and importance clear this journal's bar?"). Keep them separate — a perfect scope match can still desk-reject the design.

Axis 1 — Scope fit. Compute a composite scope-fit score:

FactorWeightDescription
Scope alignment40%How well the manuscript's themes match the journal's scope and keywords
Study type fit25%Whether the journal accepts this article type and values this methodology
Tier match20%Alignment with user's preferred tier (if specified)
OA match10%Alignment with user's OA preference (if specified)
Special fit5%Bonus for unique alignment with journal's Special Notes

Axis 2 — Acceptance feasibility. Weigh the Phase 2.5 ceiling verdict against each journal's Acceptance Signals (selectivity band, desk-reject triggers, design expectations); for a profile without that block, use its Special Notes plus the Phase 2.5 taxonomy. Assign High / Medium / Low:

  • Low / ceiling-mismatch when the journal's bar is one the manuscript's ceiling cannot clear (e.g., a highly-selective venue that desk-rejects single-center surrogate-endpoint designs, and the manuscript is exactly that).
  • High when no ceiling signal collides with the journal's stated bar.

Report a band with reasons, NEVER an acceptance probability, because no acceptance-rate data source exists and a number would be false precision (schema §4).

3.3 Filtering

Before scoring, exclude journals on the user's exclusion list and journals that do not accept the manuscript's study type (e.g., a case report to a journal that only takes original research). If no journal survives, relax the filters — OA constraint first, then tier — and re-score.

3.4 Ranking

Rank primarily by the Axis-1 scope-fit score, then apply Axis 2:

  • Demote (or, if the mismatch is severe, drop below a better-feasibility peer) any journal whose acceptance feasibility is Low / ceiling-mismatch.
  • Never silently demote — always carry the reason so Phase 4 can surface it.
  • Select the top 5 by the feasibility-adjusted order. If a strong scope match is demoted for feasibility, still mention it in the comparison note with the mismatch spelled out (the author may choose to fix the design rather than change venue).

3.5 Pass 2: Enrich Top-5

For each top-5 journal, look for a detailed write-paper profile in both tiers (private wins on collision):

# Public
${CLAUDE_SKILL_DIR}/../write-paper/references/journal_profiles/{journal_filename}

# User-local private
$HOME/.claude/private-journal-profiles/write-paper/{journal_filename}

If found, take from it: manuscript types and word limits, abstract format, statistical reporting requirements, the full 5-field AI Writing Disclosure Policy, common rejection reasons, and Acceptance Signals (these sharpen the Axis-2 call and the cascade plan). If none is found or the directory is not accessible, use the compact profile only.


Phase 4: Output

Every journal fact you print (URLs, article types, OA model, AI policy) comes from the loaded profile or from the journal's own website; never invent journal metadata, an impact factor, an APC or a submission policy.

For each of the top 5 recommended journals, present:

### Rank [N]: [Journal Name] ([Tier])

**Scope fit:** [2-3 sentences explaining why this manuscript matches this journal's scope.
Reference specific keywords, disease areas, or methodological preferences from the profile.]

**Article types accepted:** [relevant types from profile]

**Open Access:** [Full OA / Hybrid / Subscription]

**Acceptance feasibility:** [High / Medium / Low] — [1 line: how the manuscript's
Phase 2.5 ceiling meets this journal's bar; spell out any mismatch, e.g., "scope fit
High, but this highly-selective venue desk-rejects single-center surrogate-endpoint
designs — add external validation or target a selective/accessible venue"]

**Homepage:** [URL]
**Author guidelines:** [URL]

**AI disclosure:** [Required / Recommended / Not specified] — [brief summary of permitted scope and disclosure location, if available in profile]

After all 5, add a 2-3 sentence comparison note on the key tradeoffs between the top choices (e.g., scope breadth vs. specialty depth, tier vs. acceptance feasibility). Then add the two blocks below, then the Mandatory Disclaimer.

Acceptance-Readiness Summary

---
### Acceptance-Readiness Summary

**Ceiling verdict:** [from Phase 2.5]
**Top flags:** [the 2-4 most consequential design-ceiling / unfixable / importance flags, each with its 1-line reason]
**Implication for venue tier:** [1-2 sentences — e.g., "An unfixable single-center +
surrogate-endpoint ceiling makes flagship venues unlikely without external validation;
the recommendations above are routed to venues this design can clear."]

_Advisory only — a risk band, not an acceptance prediction; flags are not auto-fixable._

Cascade plan

---
### Cascade plan (primary → reject-fallback)

1. **Primary:** [top recommendation] — [why first]
2. **If rejected:** [fallback 1] — [same-publisher transfer if applicable, else one tier
   down / different scope angle]. An editor's transfer offer within the same publisher
   is usually worth taking: the manuscript skips a fresh desk screen.
3. **Then:** [fallback 2]

[If the Phase 2.5 risk is IMPORTANCE rather than design, recommend a **presubmission
inquiry** to the primary target before full submission — it lets the editor quick-assess
contribution/fit and saves a desk-reject cycle.]

[If a confidential corresponding-author editor-bar overlay exists in
`$HOME/.claude/private-journal-profiles/find-journal/`, its notes have already been
merged into the per-journal feasibility calls above.]

Mandatory Disclaimer

Always append this disclaimer at the bottom of every recommendation output:

---
**Important Disclaimer**

Impact Factor, APC fees, acceptance rates, and turnaround times change frequently
and are subject to copyright restrictions. Please verify current values directly
at each journal's homepage before making your submission decision.

Recommended verification sources:
- Journal Citation Reports (JCR) via institutional access: for Impact Factor
- Journal homepage -> Author Guidelines: for current APC and formatting requirements
- Clarivate Master Journal List: for indexing status

Special Modes

Post-Rejection Mode

When the manuscript was rejected by a specific journal:

  1. Exclude the rejecting journal.
  2. Distinguish the rejection type — it changes the advice:
    • Desk-reject (no peer review) — usually an importance/novelty or design-ceiling verdict, not a fixable-revision signal. Re-run Phase 2.5; if a ceiling or importance flag is present, the next same-tier venue will likely desk-reject again. Route one or two tiers down (or to a tolerant/accessible venue), or advise the design/importance fix first, and consider a presubmission inquiry.
    • Post-peer-review reject — prefer a same-publisher transfer (Springer Nature Transfer Desk / Elsevier Article Transfer Service / Wiley / Nature Portfolio) that carries the referee reports; otherwise same tier or one down, with the fatal flaw addressed.
  3. If rejected from Q1 other than by a ceiling/importance desk-reject, recommend a mix of Q1 (different scope angle) and strong Q2.
  4. In each scope-fit explanation, note how the recommendation differs from the rejected journal's focus.
  5. Suggest scope adjustments — or, when Phase 2.5 found a ceiling, design changes — that might improve feasibility for the new target.

Case Report Mode

When the study type is case report, prioritize journals known for valuing educational or rare cases. If fewer than 5 profiled journals accept case reports, say so and suggest the user consider case-report-specific journals outside the profile set.

Submission Directory Scaffolding

When the user selects a target journal, create submission/{journal_short}/ (lowercase with underscores, e.g., radiology_ai, european_radiology, ajr) and report the path so /write-paper Phase 8+ and /peer-review know where to write:

submission/
└── {journal_short}/          # e.g., radiology_ai/
    ├── cover_letter.md       # Generated by /write-paper Phase 8+
    ├── checklist.md          # Journal-specific submission checklist
    └── peer_review.md        # Generated by /peer-review (journal scope-aware)

Fulltext Retrieval Skill

Batch download open-access full-text PDFs from a DOI list using legitimate OA APIs only. Paywalled articles fail by design and are listed in manual_needed.txt for institutional access or ILL; never work around a paywall or publisher access control.

Pipeline

DOI → arXiv (10.48550/arXiv.* DOIs) → Unpaywall → PMC (Europe PMC / OA FTP / web) → OpenAlex → Crossref → landing page

Each DOI goes through these sources in order until a valid PDF (≥10 KB, %PDF- header) is found. arXiv DOIs (10.48550/arXiv.2401.01234, version suffixes, old-style hep-th/9901001, or a bare arXiv: id) resolve directly to the arXiv PDF first.

Run

Requires Python 3.10+ (stdlib only) and a contact email, which Unpaywall's Terms of Service require. The script paces its requests (0.3–0.5 s delays) for the APIs' rate limits.

python "${CLAUDE_SKILL_DIR}/fetch_oa.py" dois.txt --output pdfs/ --email your@email.com

# Verbose mode for debugging (per-DOI source trace)
python "${CLAUDE_SKILL_DIR}/fetch_oa.py" dois.txt -o pdfs/ -e your@email.com --verbose

Input formats:

  • Plain text — one DOI per line.
  • TSV / CSV with header, or a Markdown pipe table — must contain a DOI column; optional PMID, Title, and FirstAuthor (surname or full name) columns.

A PMID makes the PMC lookup more reliable (PMID → PMCID conversion). Supply Title where available: a DOI-only worklist can download a PDF but cannot establish title agreement. FirstAuthor is optional additional evidence.

Output

  • PDFs saved as {DOI_safe}.pdf (slashes replaced with underscores).
  • pdfs/retrieval_report.json — structured per-DOI report (below); override with --report PATH.
  • <output>/manual_needed.txt — DOIs that could not be retrieved via OA; when a PMCID was resolved, the line also carries it and the PubMed Central article URL to open in a browser.
  • Summary with arXiv/OA/PMC/fail/skip counts.

Retrieval report (--report)

Every run writes the report (default <output>/retrieval_report.json), schema 2:

{
  "schema_version": 2,
  "generated_by": "fetch_oa.py",
  "counts": {"total": 4, "retrieved": 3, "not_retrieved": 1, "title_mismatch": 1,
             "source_identity": {"consistent": 1, "conflict": 1, "unresolved": 1, "unavailable": 1}},
  "items": [
    {"doi": "10.1000/synthetic.example", "pmid": "", "title": "Example title",
     "first_author": "", "status": "oa", "source": "unpaywall",
     "file": "10.1000_synthetic.example.pdf", "size_bytes": 482113, "page_count": 9,
     "file_sha256": "<SHA-256 of the downloaded file>", "title_match": "match",
     "source_identity": {"status": "consistent", "reason": "title_and_identifier_agree",
                         "text_scope": "first_page_front_matter", "title_match": "match",
                         "doi_match": "match", "observed_identifiers": ["10.1000/synthetic.example"],
                         "first_author_match": "unavailable"}}
  ]
}

status (arxiv | oa | pmc | skip | fail), source, and counts.retrieved describe the resolver result, including existing files (skip). They do not count identity-verified papers. No PDF is automatically deleted or rejected. page_count comes from Poppler's pdfinfo (null without it) and is recorded, not judged — a 3-page "article" or a 4-page "book" is worth opening.

source_identity.statusMeaning / action
consistentComplete normalized title and a compatible DOI/arXiv identifier occur in the bounded first-page front matter, with no supplement / preface / table-of-contents heading and no retraction / erratum / correction / corrigendum / expression-of-concern heading there; an optional supplied author must also match. Evidence agrees, but this is not independent source verification or claim validation.
conflictBoth the title and observed identifier differ. Inspect the PDF and requested record.
unresolvedEvidence is incomplete or ambiguous: title-only, DOI-only, missing author, multiple identifiers, a matching title with another DOI/version, or a supplement / preface / table-of-contents file that names the work without being it (supplement_or_front_matter). A retraction notice, erratum, correction, corrigendum or expression of concern whose heading line sits in that area is likewise unresolved (correction_or_retraction_notice). Inspect before using as evidence.
unavailableNo usable extracted text, Poppler unavailable, no output PDF, or the PDF changed during assessment. No current identity assessment was possible.

Evidence is limited to the first page before a recognized abstract/body/reference heading (at most 40 lines / 4,000 characters), so a title cited in the body or references does not count. A title_match of match needs the complete normalized title on up to six consecutive lines; partial overlap is unavailable and low overlap an advisory mismatch. Cover sheets, unusual reading order, short or changed titles and DOI footers outside that area can stay unresolved. PDF metadata and the filename alone are not identity evidence. Explicit arXiv versions must agree; a preprint/published-version DOI difference needs review, not automatic rejection.

Downstream reports must preserve source_identity and file_sha256, keep unresolved items visible, and check the hash still identifies the file being used. Older reports without identity evidence remain unassessed; do not infer identity from retrieved or title_match=match. Full-text conversion does not resolve an identity warning.

Attach PDFs into Zotero ("Find Available PDF")

For paywalled-but-licensed papers the OA resolvers miss, references/find_available_pdf.js is a user-run snippet for Zotero's Tools → Developer → Run JavaScript (no-code equivalent: right-click → "Find Available PDF"). It triggers Zotero's own addAvailablePDF / addAvailablePDFs, so it reuses the user's OpenURL resolver / institutional proxy config; no credentials, proxy hosts, or institutional identifiers are hard-coded or leave the Zotero client. It is user-initiated and depends on the live Zotero session, so record its results manually — they are not reproducible CI evidence. /lit-sync Phase 2.7 orchestrates both routes and reconciles them in a report.

PDF → Markdown Conversion (Optional)

Convert downloaded PDFs to Markdown when the same papers will be read repeatedly (data extraction from k≥5 studies, a meta-analysis pipeline); for one-pass screening, read the PDF directly.

# Install (one-time)
pip install pymupdf4llm

# Convert all PDFs in a directory (.md files land alongside the .pdf files)
python "${CLAUDE_SKILL_DIR}/pdf_to_md.py" pdfs/ -v

# Custom output directory
python "${CLAUDE_SKILL_DIR}/pdf_to_md.py" pdfs/ -o markdown/

# First 10 pages only (useful for long supplements)
python "${CLAUDE_SKILL_DIR}/pdf_to_md.py" pdfs/ --pages 0-9

# Overwrite existing conversions
python "${CLAUDE_SKILL_DIR}/pdf_to_md.py" pdfs/ --force

Images are skipped, so figures survive only as caption text. Scanned-only PDFs (no text layer) convert poorly. pdf_to_md.py needs pymupdf4llm (AGPL-3.0), an optional dependency; fetch_oa.py stays stdlib-only.

Generate Codebook Skill

Turn a raw tabular dataset into a structured, citable data dictionary (codebook). This is the generator side of the dictionary-first workflow: it produces the artifact that /define-variables and dictionary-first QC later consume. Distributions, types, and missingness are observable and the bundled script profiles them; the meaning of a coded value (fatty_liver_grade = 0) is not observable from the data and lives only in the authoritative data dictionary. You generate code and review output — you do not invent the meaning of coded values.

Deterministic Script

Run the bundled profiler rather than describing columns from memory:

python "${CLAUDE_SKILL_DIR}/scripts/generate_codebook.py" data.csv --out-dir .

Supports .csv/.tsv/.xlsx/.parquet/.dta/.sas7bdat. Flags: --max-levels N (categorical cutoff, default 20), --json-only, --md-only. The script is pandas-only, runs locally, and never sends data anywhere.

Workflow

Step 1: Profile (deterministic)

Run generate_codebook.py on the dataset. It writes codebook.json (machine- readable) and codebook.md (review table), reporting per variable: role (id / continuous / categorical / binary / date / text), dtype, missingness, unique count, level frequencies or quantile summary, and a needs_dictionary flag.

Step 2: Review with the researcher (gate)

Read ${CLAUDE_SKILL_DIR}/references/codebook_schema.md (the codebook.json schema, the role-inference heuristics, the needs_dictionary rule) before interpreting the output. Present codebook.md and walk the user through it. Gate: role inference is a heuristic, so the user confirms the inferred roles (e.g., an integer-coded scale mis-read as continuous, or an id column). Do not proceed to definition work until the user approves the role assignments.

Step 3: Resolve [NEEDS DICTIONARY] items (gate)

For every variable flagged needs_dictionary: true, the level codes are uninterpretable without the authoritative source. Gate: ask the user to supply the meaning of each code from the real data dictionary (file/sheet/row), or to confirm none exists. Fill label, units, and per-level meanings into the codebook only from that source, citing file > sheet > row — never from inference. If the user cannot supply it, leave the [NEEDS DICTIONARY] marker in place; do not erase it.

Example: in a cohort file, sex (levels 1/2) and fatty_liver_grade (0..4) are flagged because their levels are bare codes; smoking_status (never/former/current) is not. Never write sex: 1 = male because "that is the usual coding" — if the dictionary is unavailable, the flag stays.

Step 4: Hand off

The completed codebook.json becomes the input dictionary for /define-variables (operationalization) and the citation source for dictionary-first QC. Gate: confirm with the user that no needs_dictionary flags remain unresolved before the codebook is treated as authoritative for downstream analysis.

Run /deidentify on the raw data before a codebook is shared externally. Cleaning or transforming data is /clean-data.

Known limits

  • Codes longer than 3 characters that are not numbers (C50.9, S001) are not recognised as codes, so their column is not flagged needs_dictionary.
  • An integer-coded column with more distinct values than --max-levels is classed continuous and is not flagged. The Step 2 role review is where such columns are caught.
  • The same holds for a text column holding number strings: any number value (45, 88) is read as a measurement, so integer codes exported as strings with more than --max-levels values are not flagged.

Output Format

codebook.json (schema in references) and codebook.md (review table with a "Columns requiring dictionary lookup" section). Summarize the counts (rows, columns, needs_dictionary_count) in chat; do not paste the full JSON.

Grant-Builder Skill

Write proposal prose in the language the target call requires. Produce whichever parts the request needs: concept summary, Significance, Innovation, Approach, specific aims, work packages, milestone table, role split by institution, evaluation framework, reviewer-risk memo.

Korean Government Grant Mode

When the user requests a Korean industry-academia grant (산학과제) or research plan (연구계획서) — e.g., MOHW, MOTIE, MSS or regional industry-academia programs — apply the adaptations below. Korean program terms are preserved in parentheses because they are the literal form used on the funding agency's template.

Document Structure (three-attachment format)

AttachmentContents
1 (첨부1, 기본정보)project title, participating institutions, investigator CVs, publication / patent record
2 (첨부2, 매칭확인서)per-institution cost-share confirmation, typically finalized after a kickoff meeting between the institutions
3 (첨부3, 연구계획서)the 10-page research plan — structure below

Attachment 3 Standard Structure

1. Significance & Aims (약 2p)
   - clinical problem with quantitative framing
   - domestic + international trends (3–5 year literature / guideline window)
   - differentiation of the proposed work

2. Research Content & Methods (약 4p)
   - staged roadmap (Phase 1 – N with time ranges)
   - pipeline schematic (mandatory when an AI pipeline is in scope)
   - per-subproject institution and personnel assignment

3. Team Capability (약 1p)
   - expertise + representative record (SCI papers, patents) per investigator
   - cross-institution synergy (hospital = data / clinical; university = algorithm)

4. Expected Outcomes & Utilization (약 2p)
   - quantitative targets: SCI papers, patents
   - qualitative targets: clinical impact, standardization contribution
   - linkage to follow-on larger grants (positioning as a seed)

5. Budget Plan (약 1p)
   - RA salaries, computing equipment, consumables, academic activities, indirect costs

Small-Scale Grants (< KRW 30 million)

  • Write for a non-specialist reviewer; assume the evaluator is not in your subfield.
  • Emphasize feasibility over technical novelty.
  • Prioritize length / format compliance; exceeding the template incurs scoring penalties.
  • Include preliminary data or pilot results whenever available.
  • Keep quantitative targets conservative — undershooting a committed target is punished more than overdelivering on a modest one.

Workflow

Phase 1: Decode the funding call

Extract funding body, call theme, eligibility constraints, deliverable expectations, timeline and evaluation criteria. If no call text is available, infer a generic academic-medical AI proposal structure and label the assumptions.

Phase 2: Frame the problem

Define the clinical pain point, the current workflow limitation, why existing AI or standard care is insufficient, and who benefits if the project succeeds.

Gate: Present the problem framing (clinical pain point, gap, proposed solution) to the user. Confirm before building proposal sections — a misframed problem produces an unfundable proposal.

Phase 3: Build the proposal spine

Always articulate: problem, gap, proposed solution, why this team can execute it, measurable outputs.

Phase 4: Convert to proposal sections

  • Significance must answer why this matters clinically, why now, and why the proposed solution is worth funding.
  • Innovation: what is genuinely different, why the integration is new, why the novelty is useful and not just technical.
  • Approach: dataset and participating sites, model or workflow components, validation plan, benchmark/comparator, failure analysis, risk mitigation.

Route to search-lit to support significance and prior-art positioning; cite a reference only with a /search-lit-confirmed DOI or PMID, otherwise mark it [UNVERIFIED - NEEDS MANUAL CHECK]. Mark any unconfirmed clinical definition, diagnostic criterion or guideline recommendation [VERIFY] and ask the user. Route to design-study if the evaluation framework is weak, and to write-paper only when the proposal requires publication-style narrative sections.

Phase 5: Execution plan

Generate milestones by quarter or year, institution-level responsibilities, dependencies and handoffs, and required infrastructure. Do not fabricate budget details, and do not promise datasets, partners or infrastructure the user has not evidenced.


Default Structure

## Proposal Summary
Title: ...
Goal: ...
Clinical problem: ...

### Significance
...

### Innovation
...

### Approach
Aim 1. ...
Aim 2. ...
Aim 3. ...

### Milestones
- ...

### Consortium roles
- ...

### Major risks and mitigations
- ...

Before Finalizing

Check, and flag any failure to the user:

  1. Is the clinical need explicit and credible?
  2. Is the novelty more than "we will use AI", with a clinical consequence?
  3. Are the aims linked to measurable outputs with a concrete benchmark or success criterion?
  4. Is the validation plan convincing, including external validation or a deployment path?
  5. Is the multi-site structure realistic, with each institution in a distinct, functionally integrated role?
  6. Are compute, annotation, and regulatory needs acknowledged?
  7. Are there too many aims for the timeline?
  8. Does it read as a funded program rather than a paper?

Humanize Skill

This skill only removes AI patterns; it does not perform general copy-editing, evaluate scientific quality, check journal formatting, or translate.

Read ${CLAUDE_SKILL_DIR}/references/ai_patterns.md at the start of every session, before scanning: the definitions, watch words, examples, detection greps, per-pattern fixes and the section-by-section priorities for all 27 patterns live only there.


Workflow

Phase 1: Scan

Scan the section(s) the user provides for all 27 patterns. For response-to-reviewers letters and cover letters, prioritise Patterns 22-24. For a full manuscript, follow the per-section priorities in ai_patterns.md (Section-Specific Application Guide). For each pattern found, record its number and name, the count, the exact passage, and its location (paragraph number or line range).

Output: Pattern Frequency Table

## AI Pattern Scan Report

Section: {section name}
Word count: {N}

| # | Pattern | Count | Severity | Example from text |
|---|---------|-------|----------|-------------------|
| 1 | Significance inflation | 3 | HIGH | "...pivotal role in diagnostic imaging..." |
| ... | ... | ... | ... | ... |

Patterns not detected: 2, 4, 9, 14, 15

Total AI pattern instances: {N}
AI pattern density: {N per 1000 words}

Phase 2: Report

  • Severity per pattern: HIGH (>3 occurrences), MEDIUM (1-3), LOW (0, clean).
  • Density: total instances across all 27 patterns per 1000 words. Target: < 2.0.

Gate: Present the report and ask the user which patterns to fix. Default: fix all HIGH and MEDIUM.

Phase 3: Fix

Rewrite flagged passages with each pattern's fix from ai_patterns.md, under these rules:

  1. Preserve technical accuracy. Every number, statistic, p-value, confidence interval, and clinical fact must remain identical.
  2. Preserve citations. Never add, remove, or relocate a citation.
  3. Keep the formal academic register of an experienced radiologist writing for peers in a top-tier journal. Never make the text casual or conversational.
  4. Keep domain-specific terminology intact. "Convolutional neural network," "apparent diffusion coefficient," "Fleiss' kappa" stay as-is.
  5. Never introduce new claims, remove existing ones, or change a sentence's meaning — rephrase, never reinterpret. If a passage cannot be fixed without changing its meaning, leave it and flag it for the user.
  6. Use active voice where natural: "We analyzed" rather than "Analysis was performed."
  7. Vary sentence structure. Mix short declarative sentences (8-12 words) with longer ones (25-35 words). A de-AI pass tends to flatten rhythm — it shortens the long sentences and pads the short ones toward a comfortable middle, which is itself a tell. scripts/check_sentence_variety.py verifies this rule in Phase 4.
  8. Thin out antithesis and cleft constructions (Pattern 27, the M2 heuristic). For each "X rather than Y", "not X but Y" or "X, not Y", apply the negative-form test: delete the negative half and rewrite the clause in the positive. If a fact disappears, the contrast was functional — keep it; if nothing disappears, it was decoration — cut it. Judge by the manuscript's overall rate, not instance by instance, and keep two or three for emphasis. Rewrite clefts ("What … is …", "It is … that …") in plain subject-verb order ("What matters is X" → "X matters"). scripts/check_rhetorical_density.py (in /self-review) measures this in Phase 4.

Output: Present the rewritten text with changes highlighted using diff format or tracked changes.

Phase 4: Verify

Keep the pre-rewrite text. Before editing in place, copy the original somewhere the fidelity check can read it (cp manuscript.md /tmp/pre_humanize.md). Without it Phase 4 can only re-scan for patterns — it cannot tell whether the rewrite preserved numbers and citations.

Run both deterministic checks, then re-scan the rewritten text using the same 27 patterns.

python3 "${CLAUDE_SKILL_DIR}/scripts/check_rewrite_fidelity.py" \
    --before /tmp/pre_humanize.md --after manuscript.md \
    --out qc/rewrite_fidelity.json --strict
python3 "${CLAUDE_SKILL_DIR}/scripts/check_sentence_variety.py" \
    --manuscript manuscript.md --out qc/sentence_variety.json

NUMBER_DRIFT, NUMBER_REASSIGNED, CITATION_DROP or CITATION_MOVED means the rewrite broke an invariant — revert that passage, redo it, and flag it for the user. EDIT_FOOTPRINT_HIGH is advisory: Patterns 6 and 18 replace whole paragraphs by design, so a correct pass over an inflated draft can exceed 60% of words changed. Read the diff and confirm the author's argument survived rather than assuming the percentage is a defect.

Known limits: the fidelity gate does not check an added or removed negation, a number written in words, a changed unit, or a direction word next to a non-percentage. A clean exit does not clear these; read the diff for them.

Output: Verification Report

## Verification Report

| Metric | Before | After |
|--------|--------|-------|
| Total instances | 23 | 4 |
| Density (per 1000 words) | 8.2 | 1.4 |
| HIGH severity patterns | 3 | 0 |
| MEDIUM severity patterns | 5 | 2 |

Remaining issues:
- Pattern 17 (hedging): 2 instances remain -- appropriate for the evidence level.

Verdict: PASS (density < 2.0)

If the density remains above 2.0, run another fix-verify cycle (max 3 rounds). When called by another skill, return the verification report so the calling skill can check the pass/fail status.


The 27 Detection Patterns

All 27 are defined in references/ai_patterns.md. Two carry rules to apply exactly as written:

#PatternWhat to look forFix
13Em dash overuseMore than 2 em dashes per 1000 words (the /self-review classical-style gate fails a manuscript above 25 prose em-dashes)Use parentheses or restructure. After converting — X — appositives to (X), run the paren-span safety scan (python3 "${CLAUDE_SKILL_DIR}/../self-review/scripts/check_paren_spans.py"): a bulk conversion can pair two unrelated dashes across a sentence boundary and wrap a whole sentence (or an ordinal "Sixth, …" limitation) inside one parenthesis — paren-balanced but broken, so a balance check misses it. Operate per-sentence; never match across .
21AI Disclosure boilerplate (body)"## Artificial Intelligence Disclosure", "Generative AI was not used to create..." in manuscript bodyPut it where the target journal asks (the journal profile's Disclosure location): Methods or Acknowledgments for some journals, cover letter, title page or submission form only for others. Do not delete a disclosure the journal requires in the body

Patterns 22-24 apply only to response-to-reviewers letters and editor cover letters, not manuscript bodies. They are defined once in ai_patterns.md (Response-Letter Patterns); for authoring guidance, see the revise skill's references/r2r_voice.md.


Gates

GateSeverityTriggerAction on fail
AI-pattern density targetADVISORYdensity > 2.0 patterns / 1000 words after sweepwarn; surface remaining flagged passages for manual review
Pattern 13 — paren-span corruption after em-dash conversionENFORCEDafter a — X — → (X) sweeprun python3 "${CLAUDE_SKILL_DIR}/../self-review/scripts/check_paren_spans.py" --strict; PAREN_SPAN_ORDINAL / PAREN_SPAN_SENTENCE means a conversion wrapped a sentence/ordinal inside parens — fix before finalizing
Pattern 19 — § symbolENFORCED (senior MA reviewer prep)grep -c "§" manuscript.md > 0auto-strip; verify post-rewrite count == 0
Pattern 20 — (see Methods §X) self-referenceENFORCEDmatch foundrewrite to direct section name reference
Pattern 21 — AI disclosure in the wrong placeENFORCEDan AI-use disclosure in the body of a journal that wants it elsewhere, or repeated in several placesmove it to where the target journal asks (journal profile); never reword a required disclosure to hide the tool
Pattern 26 — aphorism densityENFORCEDnegative-definition rate AND short-declarative share both over thresholdrun python3 "${CLAUDE_SKILL_DIR}/../self-review/scripts/check_aphorism_density.py" --manuscript manuscript.md; APHORISM_DENSITY (Minor) means the prose is a run of epigrams with the explanatory sentences compressed out — absorb most of them into the neighbouring sentence and write the explanation back, keeping two or three for emphasis; do NOT simply delete them, which shortens the prose further
Pattern 27 — antithesis / cleft densityENFORCED"rather than" / "not X but Y" / "X, not Y" or "What … is …" / "It is … that …" over a per-1000 threshold AND raw-count floorrun python3 "${CLAUDE_SKILL_DIR}/../self-review/scripts/check_rhetorical_density.py" --manuscript manuscript.md; ANTITHESIS_DENSITY / CLEFT_DENSITY (both Minor) — apply Fix rule 8 (the M2 test). A lone functional "rather than" or "instead of" never fires
Pattern 25 — inline-emphasis over-useENFORCEDitalic-emphasis density over threshold after allowlistrun python3 "${CLAUDE_SKILL_DIR}/../self-review/scripts/check_emphasis_density.py" --manuscript manuscript.md; EMPHASIS_OVERUSE (Minor) means strip inline italics (keep only stat symbols / Latin / gene-species); whole-clause italics are the strongest tell
Patterns 22-24 — R2R editing-mechanism / draft line-number / tooling leakTRIAGE (response letters); § = 0 harddetection greps in ai_patterns.md R2R section surface candidatesreview each hit (analysis narration, quoted additions, revised-manuscript page/line are NOT tells); rewrite confirmed tells to substantive prose
Citation preservation invariantENFORCEDa citation item (each key of a multi-key or locator Pandoc citation, or a numeric marker) removed or changed, or moved out of the sentence of the word it was attached to while that word kept its placescripts/check_rewrite_fidelity.py --before <pre> --after <post> --strict → CITATION_DROP / CITATION_MOVED (Major); revert that single rewrite and flag for the user
Numerical preservation invariantENFORCEDa numeric token's count changed (sign, inequality sign and %-direction are part of the token; writing a sign out in words also fires), or values traded places while the words around them stayedsame script → NUMBER_DRIFT / NUMBER_REASSIGNED (Major); revert and flag. Known limits: a negation, a number written in words, a unit, or a direction word next to a non-percentage is not checked — read the diff for these
Rewrite footprintADVISORYfraction of word tokens changed exceeds --warn-pct (default 70)EDIT_FOOTPRINT_HIGH (Minor) — never blocks; read the diff (Phase 4)
Fix rule 7 — sentence-length uniformityADVISORYprose has no short (≤12 words) or no long (≥25 words) sentencesscripts/check_sentence_variety.py --manuscript <file> → SENTENCE_UNIFORM (Minor); break up or combine sentences until both bands exist. Silent below 15 sentences

Imaging-Data Skill

The dataset decides more of a study than the architecture does, and it decides it first. Phases 1–3 establish what the data is and what it will not support, while that is still cheap; Phases 4–7 design and audit the preparation pipeline so it is leakage-safe before /model-scaffold builds the repo. Describe-and-audit only: never modify, resample, reorient, split or write image data, never run preprocessing on real patient data, and wire MONAI / TorchIO transforms by reference rather than writing a new normalisation or resampling implementation.

Elsewhere: tabular/clinical variables → /generate-codebook, /clean-data; auditing the split table, held-out metrics, calibration, subgroup results → /model-assessment; choosing an architecture → /model-selection; building the repo → /model-scaffold.

Workflow

Phase 1 — Profile every case

python3 ${CLAUDE_SKILL_DIR}/scripts/profile_imaging_dataset.py \
    --split train:imagesTr:labelsTr \
    --split test:imagesTs \
    --dataset "MSD Task09 Spleen" \
    --declared-labels 0=background,1=spleen \
    --target-label 1 \
    --plan resample=true,reorient=false,loss=dice_ce,metrics=dice+hd95 \
    --out eda/profile.json

One record per case: grid, spacing, orientation, intensity percentiles, the label values actually present, foreground fraction, and target volume in mL. A --split given no label directory is recorded as unlabelled — itself a finding. Requires nibabel + numpy; the gate does not. Every profile figure comes from opening the files — never from a dataset's README, a similar dataset, or memory. A README can be wrong about its own label indices; the labels cannot.

--target-label on a multi-structure atlas. Foreground defaults to every non-zero index — the whole annotated anatomy. Measured on the AMOS22 CT cases, that pools to 3.2 % instead of the spleen's 0.20 %, so the pooled figure sits above the 1 % imbalance threshold while the target sits far below it and the imbalance verdicts go quiet exactly where the risk is. Naming the target also makes LABEL_EMPTY mean this case has no spleen. Pass --target-label all for a genuinely multi-class study; leave it out on a multi-structure atlas and the gate raises TARGET_LABEL_UNDECLARED.

Phase 2 — Gate the profile against the declared plan

python3 ${CLAUDE_SKILL_DIR}/scripts/check_dataset_profile.py --profile eda/profile.json \
    --out qc/dataset_profile.json --strict

Stdlib-only, so the audit re-runs anywhere the JSON travels. Never report a profile "pass" without running it.

VerdictSeverityFires when
LABEL_SHAPE_MISMATCHMajorlabel grid ≠ image grid
LABEL_EMPTYMajora labelled case has zero foreground
LABEL_VALUE_UNEXPECTEDMajorlabel values outside the declared set
TEST_SET_UNLABELLEDMajora split named test/held-out/external/eval carries no labels
ACCURACY_UNDER_IMBALANCEMajoraccuracy is planned while the target is a sliver of the volume
LABEL_MISSINGMinora case in a labelled split has no label file
SPACING_HETEROGENEOUSMinorspacing spans ≥ ratio on an axis and no resampling is declared
ORIENTATION_MIXEDMinor>1 orientation code and no reorientation declared
INTENSITY_SCALE_INCONSISTENTMinorsome cases on the HU scale, others not
EXTREME_IMBALANCEMinormedian foreground below the threshold with no Dice-family loss
TARGET_LABEL_UNDECLAREDMinor>1 structure declared, no target named, so foreground pools them all

The gate flags an undeclared decision, not variability: 5× spacing spread and two orientation codes pass once resampling and reorientation are declared (the clean challenge fixture proves this). --spacing-ratio (default 2.0) and --imbalance-frac (default 0.01) are screening defaults, not published cut-points — never present them as such; the values applied are printed in the output and belong in the Methods. A split the profile shows unlabelled is never a held-out test set, however the directory is named.

Phase 3 — Turn the profile into research decisions

Write these decision notes into the study record, so /design-study, the preparation phases below, and /write-paper inherit them instead of re-deriving them:

  1. Resampling target — from the spacing distribution, not a tutorial default (carried into Phase 5).
  2. Loss and metric family — from the foreground fraction. Segmentation reports Dice and a boundary metric per structure (/model-assessment); accuracy is not on the list.
  3. Pre-specified subgroups — from the clinical spread the profile shows (target volume, slice thickness, modality). Pre-specifying them here is what separates a subgroup finding from a post-hoc one.
  4. Where the held-out set comes from — especially when the shipped "test" directory is unlabelled.
  5. What the cohort cannot support — n, single-source acquisition, absent subgroups: the seed of the Limitations paragraph, written before results can bias it.

Phase 4 — Inventory the preparation steps and fix fit scope

Collect the modality, the data manifest (one row per image/slice with a patient_id), the resample spacing, the intensity transform (fixed HU window vs a fitted z-score / min-max / histogram match), and the augmentation plan. Read ${CLAUDE_SKILL_DIR}/references/preprocessing_guide.md for the modality-aware normalisation, physiology-preserving vs -breaking augmentation, and MONAI / TorchIO wiring.

  • Fit dataset-level normalisation on the training split only — never all/full/test.
  • Run any data-fitted transform after the split; before it there is no train/test distinction.
  • Prefer per-image (per-sample) normalisation where clinically appropriate — leakage-free even before the split.
  • Keep augmentation train-only; augmenting val/test folds undisclosed test-time augmentation into the metric.
  • Split at the patient level, then map slices to their patient's split.

Phase 5 — Emit the preprocessing manifest

Write preprocessing_manifest.json, which /model-scaffold consumes and the gate checks. Every value comes from the real data manifest and the declared pipeline — never invented patient IDs or split assignments.

{
  "split_seed": 42,
  "transforms": [
    {"name": "hu_window", "type": "clip", "fit_scope": "none", "stage": "before_split"},
    {"name": "train_zscore", "type": "standardize", "fit_scope": "train", "stage": "after_split"},
    {"name": "flip_rotate", "type": "augmentation", "stage": "after_split", "applies_to": ["train"]}
  ],
  "split_assignment": [
    {"patient_id": "P001", "unit_id": "P001_s1", "split": "train"}
  ]
}

fit_scope: train (OK) · all/full/dataset/test (leak) · sample/per_image/none/fixed (not data-fitted). stage: before_split / after_split. The fields must describe what the code actually does — never tag a dataset-fitted transform per-sample to clear the gate; that hides the leak. A declared dataset-level fit_scope is judged whatever the type is, so a library class name (HistogramStandardization, NormalizeIntensityd) fit on all is a leak like standardize would be.

Declare the fit scope of resampling too. A target spacing chosen in advance is fit_scope: fixed and never leaks. A target derived from the cohort does: nnU-Net sets its target spacing from a percentile of the dataset fingerprint, so a resample fitted over every case carries held-out geometry into the training grid exactly as an intensity statistic would. The fingerprint's scope decides which you have, not the word "resample".

Phase 6 — Gate the manifest

python3 ${CLAUDE_SKILL_DIR}/scripts/check_preprocessing_leakage.py --manifest preprocessing_manifest.json \
    --out qc/preprocessing_leakage.json --strict

Verdicts: PREPROCESS_BEFORE_SPLIT, NORMALIZATION_LEAKAGE, PATIENT_CROSS_SPLIT (Major); AUGMENTATION_ON_EVAL, UNSPECIFIED_FIT_SCOPE, MISSING_SEED (Minor), reproduced by set arithmetic and rule on the manifest. A green gate is the precondition for handing the manifest to /model-scaffold; its split_assignment is the same patient-level split /model-assessment later re-verifies. Never report a pass without running it.

Phase 7 — Before inference on a new cohort: check the normaliser's domain

Phase 6 asks whether a transform was fit on the right scope. Before running a trained model on a cohort it was not trained on, ask whether that cohort sits in the intensity domain the trained normaliser assumes:

python3 ${CLAUDE_SKILL_DIR}/scripts/check_normalizer_domain.py \
    --profile eda/<cohort>_profile.json \
    --contract work/nnUNet_results/.../plans.json \
    --splits external_mri --out qc/normalizer_domain.json --strict

Its challenge card holds a cohort in the contract's own domain that must come back clean, an arbitrary-unit cohort that must raise a Major, and an unreadable contract that must refuse rather than pass.

Outputs and hand-off

  • eda/profile.json (Phase 1), qc/dataset_profile.json (Phase 2), and the decision notes (Phase 3).
  • preprocessing_manifest.json with the augmentation-appropriateness and normalisation fit-scope notes (Phases 4–5), qc/preprocessing_leakage.json (Phase 6), qc/normalizer_domain.json (Phase 7).

The manifest feeds /model-scaffold, which also reads the qc/ reports: keep them in qc/ beside the manifest (or ../qc/), or pass them with --imaging-qc. An unresolved Major there refuses the scaffold until it is fixed and re-gated or acknowledged with a stated reason (--ack-qc); Minor and Flag claims are carried into the repo's IMAGING_QC.md, and a missing report is recorded as not assessed. Re-run a gate after fixing its finding — a stale report still blocks. The manifest documents the CLAIM 2024 / TRIPOD+AI data-preprocessing items for /check-reporting; /self-review's model_development probe looks for exactly this pipeline in a finished manuscript. Regression: bash ${CLAUDE_SKILL_DIR}/scripts/check_dataset_profile_challenge/verify.sh, bash ${CLAUDE_SKILL_DIR}/scripts/check_preprocessing_leakage_challenge/verify.sh, bash ${CLAUDE_SKILL_DIR}/scripts/check_normalizer_domain_challenge/verify.sh, bash ${CLAUDE_SKILL_DIR}/tests/test_dataset_profile.sh, bash ${CLAUDE_SKILL_DIR}/tests/test_preprocessing_leakage.sh.

Intake-Project Skill

Canonical Manuscript Folder Structure

/manage-project init scaffolds new projects; its script is the source of truth for that tree, including the qc/ folder the QC skills write to. Use the layout below to reorganize an existing project that was never scaffolded, mapping each artifact into one of these slots or into a folder the init scaffold or another skill already writes to.

{project_root}/
├── HANDOFF.md                         # session handoff entry point
├── README.md                          # project overview
├── data/                              # raw data (NEVER edit; read-only)
├── analysis/                          # reproducible scripts (00_* → 04_*)
├── output/                            # analysis outputs: CSVs, PNGs, intermediates
├── qc/                                # QC reports the skills write (self-review, checklists, audits)
├── irb/                               # IRB/ethics docs
├── proposal/                          # original protocol / approved proposal
├── reviews/                           # external correspondence
├── manuscript/                        # SOURCE manuscript + drafting
│   ├── manuscript_v{N}.{md,docx,pdf}  # current canonical working version (top level)
│   ├── build_unified_docx.py          # or pandoc wrapper
│   ├── archive/                       # ALL prior versions v1 .. v{N-1}
│   ├── reviews/                       # review notes (QC reports go in qc/ at the root)
│   ├── figures/                       # figure scripts + rendered PNG/PDF
│   └── tables/                        # table scripts + rendered docx
└── submission/                        # per-journal packages
    └── {journal-slug}/                # e.g., chest/, kjr/
        ├── CHECKLIST.md
        ├── cover_letter.{md,docx,pdf}
        ├── title_page.docx            # separated for double-anonymized
        ├── manuscript_anonymized.{docx,pdf}
        ├── supplement.{docx,pdf}
        ├── strobe_checklist.md        # or PRISMA / CONSORT
        ├── circulation_email.md
        └── figures/                   # submission-ready DPI copies

Rules

  • manuscript/ = source; submission/{journal}/ = derived artifacts. Regenerate submission files from manuscript/manuscript_v{N}.md; never edit anonymized/title-page directly.
  • One canonical working version at manuscript/manuscript_v{N}.{md,docx,pdf}. Older versions move to manuscript/archive/ immediately on version bump.
  • No loose files at project root. Only HANDOFF.md, README.md, the project memory files (Phase 3), the contract files /manage-project init writes (SSOT.yaml or project.yaml, project_state.json, artifact_manifest.json), and folder entries.
  • QC artifacts (self-review, reporting checklists, cross-reference and reference audits) live in qc/ at the project root, where the skills that produce them write them, not at manuscript top level.
  • On rejection/retarget: cp -r submission/{old} submission/{new}, then rewrite cover letter and reformat.
  • Double-anonymized journals (Chest, AJRCCM): title page and anonymized manuscript MUST be separate files under submission/{journal}/.
  • Keep existing project labels and file names in the language the workspace already uses.

When to apply

  • At project intake: scaffold through /manage-project init — unless the user only wants a quick assessment.
  • At first submission prep: create submission/{journal}/ and populate.
  • Mid-project cleanup: when manuscript/ has >3 versioned files or QC docs at top level, reorganize.
  • Before session handoff: reorganize if structure is drifting.

Workflow

Phase 1: Discover context

  1. Read top-level folder names and key files.
  2. Detect manuscript-like files, tables, figures, protocols, and analysis outputs.
  3. Extract:
    • project title or working title
    • study question
    • dataset or cohort hints
    • collaborators or institutions
    • venue/journal hints

Phase 2: Classify project and stage

Determine:

  • project type: original | review | meta-analysis | case report | technical note | grant | peer review | challenge | career-doc
  • primary domain: radiology | medical AI | multimodal LLM | intervention | survival/prognostic | diagnostic accuracy | workflow
  • target output: paper | abstract | grant | review | rebuttal | CV
  • likely target journal or venue — only if the files name one
  • one current stage: idea | data assembly | analysis planning | analysis in progress | drafting | revision | submission prep | archived/unclear

If the folder mixes several studies, say so rather than collapsing them into one.

Gate: Present the classification (project type, stage, target output) to the user. Confirm before creating any files — misclassification leads to wrong scaffold and wrong skill routing.

Phase 3: Surface missing inputs

Check for blocking dependencies and common gaps:

  • no explicit study question
  • no target journal
  • no analysis plan
  • no variable dictionary
  • no claims-to-results map
  • no review log for revised manuscripts

If missing, propose or create lightweight anchor files — PROJECT.md, STATUS.md, CLAIMS.md, DATA_DICTIONARY.md, ANALYSIS_PLAN.md, REVIEW_LOG.md — only those the project type justifies. Read ${CLAUDE_SKILL_DIR}/references/memory_templates.md when creating PROJECT.md or STATUS.md.

Phase 4: Produce normalized summary

Output this structure:

## Project Intake Summary
Project: ...
Type: ...
Current stage: ...
Likely target: ...

### What exists
- ...

### What is missing
- ...

### Risks / ambiguities
- ...

### Recommended next actions (3-5, in dependency order)
1. ...
2. ...
3. ...

Handoff Rules

After intake:

  • route to search-lit if the literature basis is weak
  • route to design-study if the research question exists but design logic is unclear
  • route to manage-project if the folder should be scaffolded
  • route to write-paper only after the project phase is clearly drafting

Literature Sync: Zotero + Obsidian Pipeline

Takes the .bib output of /search-lit (or any user-specified .bib file), synchronizes the references into the Zotero library and Obsidian literature notes, and extracts cross-cutting concept notes once enough literature notes accumulate.

Prerequisites

  • Project owner only. Collaborators consume the committed manuscript/_src/refs.bib snapshot read-only. If the current user is a collaborator (no Zotero access per SSOT.yaml reference_manager.required_for), abort with instructions to flag [@NEW:topic] placeholders in the manuscript and notify the owner.
  • Zotero desktop 7.x + Better BibTeX plugin, with Better BibTeX "Keep updated" auto-export configured to <project>/manuscript/_src/refs.bib.
  • Zotero MCP server (if not connected, skip Phase 2; auto-export still refreshes once Zotero is reopened).
  • Obsidian CLI or direct file writing to the vault; vault path from the user's environment (e.g., $OBSIDIAN_VAULT).

Artifact Contract

/lit-sync is the sole writer of manuscript/_src/refs.bib (via the Better BibTeX auto-export trigger), references/zotero_collection.json, and references/fulltext_retrieval.json (Phase 2.7). NEVER write refs.bib directly — not even to compensate for a missing Zotero connection — because only Better BibTeX's export keeps it in step with the library; if auto-export is broken, fix the Zotero setup. Direct hand edits to refs.bib are drift — revert on sight.


Phase 1: Parse BibTeX

Parse the user-specified .bib file, or the references/library.bib just produced by /search-lit, with regex. Extract per entry: citekey, doi, pmid, title, authors (first + last minimum), journal, year, and volume/number/pages if present. Log any parse failures and skip those entries. All bibliographic data in later phases comes from these entries or from API responses.

A parsed citekey is not yet a library key: /search-lit candidate keys (Kim_2024_Validation) are provisional, and Better BibTeX mints the real key when Phase 2 adds the item. Never compose a key; Step 3.2 §Citekey provenance says where the note's key comes from.


Phase 2: Zotero Sync

If the Zotero MCP is not connected, skip this phase (still write the Step 2.3 file with status: "skipped") and proceed to Phase 3.

Step 2.1: Determine project collection

Identify the project from the current working directory or an explicit user override. Reuse a recorded collection key; otherwise check existing Zotero collections for the project and, if none exists, create one with zotero_create_collection. Record the collection key, and report it to the user when a new collection is created.

Step 2.2: Dedupe + add

For each entry:

  1. Use zotero_search_items to search by DOI or title — if already present, skip. This search-first step is what prevents duplicates; zotero_add_by_doi does not dedupe by itself (it fetches CrossRef and creates the item), so never skip the search.
  2. Otherwise call zotero_add_by_doi (when a DOI is available) or zotero_add_by_url (the PubMed URL, when only a PMID is available). An entry with neither DOI nor PMID is not added: count it as failed and ask the user to add it manually.
    • zotero_add_by_doi accepts an attach_mode argument that governs the OA child-PDF attach attempt at add time (the installed server treats linked_url as "bookmark the PDF URL"; other values download/import). Exact accepted values are server-version-specific — verify against the connected server. Do not use zotero_add_from_file to attach a PDF to an item added here: it has no parent-item argument and would create a duplicate parent item.
  3. Use zotero_manage_collections to place the item in the project collection.

Step 2.3: Result report

Zotero Sync:
  Added:     8 papers (new)
  Skipped:   3 papers (already in library)
  Failed:    1 paper (no DOI/PMID)
  Collection: RFA-Meta (TZQEP4NH)

Always write references/zotero_collection.json in the project workspace:

{
  "schema_version": 1,
  "status": "synced",
  "collection": "RFA-Meta",
  "collection_key": "TZQEP4NH",
  "added": 8,
  "skipped": 3,
  "failed": 1
}

If Zotero is unavailable, write the same file with status: "skipped" and a human-readable reason.


Phase 2.5: refs.bib snapshot refresh

Better BibTeX "Keep updated" auto-export normally refreshes manuscript/_src/refs.bib within seconds of a Zotero change. This phase verifies the snapshot actually updated before downstream skills consume it.

Step 2.5.1: Resolve path

Read SSOT.yaml → truth.refs_bib. Default: manuscript/_src/refs.bib. If absent (legacy project), fall back to manuscript/_src/refs.bib and emit a WARN recommending SSOT migration.

Step 2.5.1b: Precondition assertion (early-exit, do NOT poll)

Before entering the 10s polling loop in Step 2.5.2, verify both preconditions. If either fails, abort Phase 2.5 with setup instructions instead of waiting for a timeout that will never resolve.

  1. Better BibTeX is answering. Probe the running plugin, not a file on disk:

    curl -s -m 5 -o /dev/null -w "%{http_code}" \
      http://127.0.0.1:23119/better-bibtex/json-rpc    # expect 200
    

    A non-200 means Zotero is closed or BBT has not finished starting. Retry once after Zotero's window is up; BBT registers its endpoint a few seconds after the app does.

    ⚠️ Do not gate on ~/Zotero/better-bibtex/read-only.json. Current BBT releases keep auto-export registrations in their own store, so that file is routinely [] on a healthy install; treating it as "not configured" skips this phase on working setups, which is how a stale refs.bib and an invented citekey reach a manuscript.

    On failure print:

    Phase 2.5 skipped: Better BibTeX did not answer on 127.0.0.1:23119 (HTTP <code>). Open Zotero, wait for it to finish loading, then re-run /lit-sync.

  2. Target refs.bib exists. The resolved truth.refs_bib path from Step 2.5.1 must exist on disk (even empty is OK — BBT will overwrite). On failure print:

    Phase 2.5 skipped: target snapshot <path> not found. Configure BBT auto-export with "On Change" to the SSOT path, then re-run.

In either early-exit, set refs_bib_refreshed: false + reason: "precondition:<which>" in the Step 2.5.3 JSON, tell the user, and return control to the caller. Nothing downstream enforces this flag, so telling the user is what stands between a stale refs.bib and a manuscript.

Step 2.5.2: Verify refresh

After Phase 2 adds items:

  1. Capture stat -f "%m" manuscript/_src/refs.bib before Zotero writes.
  2. Wait up to 10s (Better BibTeX debounce). Poll mtime.
  3. If mtime unchanged after 10s:
    • Prompt user to check Zotero is running and BBT export is "Keep updated".
    • If BBT auto-export path is wrong, print the expected path (<project>/manuscript/_src/refs.bib).
    • As last resort, offer manual export: File → Export Library → Better BibTeX → target path.
  4. Once mtime advances, grep for the newly added citekeys. All must be present; if any is missing, report as failure (do NOT fabricate entries).

Step 2.5.3: Record in zotero_collection.json

Append to the JSON written in Step 2.3:

{
  "refs_bib_path": "manuscript/_src/refs.bib",
  "refs_bib_mtime": "2026-04-24T14:32:11Z",
  "refs_bib_refreshed": true,
  "citekeys_verified": ["smithDeepLearningRadiology2024", "..."]
}

If refresh failed, set refs_bib_refreshed: false and include reason.


Phase 2.7: Fulltext Retrieval (opt-in, owner-only)

Run only when the user asks for full text (e.g. "download the PDFs", "fetch full text", or a worklist supplied with that intent). Default /lit-sync stays metadata-only and network-light — do not auto-run this phase. Runs after items are in Zotero (Phase 2) and the snapshot is verified (Phase 2.5), before Obsidian notes (Phase 3).

Offer both routes below and reconcile them in one report. Retrieve full text only through them: never automate authenticated browser sessions, never bypass paywalls or access controls, and never hard-code institutional proxies, credentials, or hosts into this skill. Route what neither reaches to institutional access, interlibrary loan, or author contact.

Route A — disk OA PDFs (for downstream skills)

Delegate to the /fulltext-retrieval engine (do not re-implement the OA cascade or import its code; invoke it by path):

ENGINE="${CLAUDE_SKILL_DIR}/../fulltext-retrieval/fetch_oa.py"
python3 "$ENGINE" <worklist> -o pdfs/ -e <contact-email> --report pdfs/retrieval_report.json

<worklist> is the DOI/PMID(/Title) list — the Phase-1 .bib DOIs, a worklist supplied in Standalone Modes, or the project collection's DOIs. Output: pdfs/*.pdf for /meta-analysis and pdf_to_md.py, plus pdfs/retrieval_report.json (schema 2: retrieval status/source, source_identity, and file_sha256). Keep the distinction between having a file and assessing its identity.

Route B — in-library PDFs (Zotero-native, higher yield, proxy-aware)

Emit ${CLAUDE_SKILL_DIR}/../fulltext-retrieval/references/find_available_pdf.js for the user to paste into Zotero (Tools → Developer → Run JavaScript) with the project collection selected. It triggers Zotero's own addAvailablePDF/addAvailablePDFs, which reuse the user's OpenURL resolver / institutional proxy — so it typically retrieves more than OA-only, while no credentials or institutional identifiers enter this skill. The no-code equivalent is right-click → "Find Available PDF". Record its {attached, missing} summary from the printed JSON.

Report

Merge Route A's pdfs/retrieval_report.json (and the user-reported Route B summary) into references/fulltext_retrieval.json, and append a short fulltext counts block to references/zotero_collection.json. Read ${CLAUDE_SKILL_DIR}/references/fulltext_report.md before writing them — it has the schema and the identity-evidence rules (a retrieved file is not a verified paper; a consistent status is advisory; unassessed items go to identity_review_needed).


Phase 3: Obsidian Literature Notes

Detect the vault's existing layout before creating notes. If the vault already uses a folder structure (including a Korean one), honor it — never silently rename a user's folders. For a new or unclear vault, default to the English folders Literature/ and Concepts/ with the English templates (Step 3.2, Step 4.3). For a Korean-structured vault, or a user who prefers Korean notes, use the Korean folder layout and Korean-heading templates in ${CLAUDE_SKILL_DIR}/references/locale/ko/note_templates.md (and the vault's own hub-note names).

Step 3.1: Check existing literature notes

# Default English layout; substitute the vault's existing folder if one is present
# (e.g. "02 연구/문헌/" for a Korean-structured vault — see references/locale/ko/note_templates.md).
ls "$VAULT/Literature/" | grep -v "📊" | wc -l

Step 3.2: Create literature notes

For each .bib entry, create Literature/{citekey}.md (or the vault's existing literature folder). Skip if the file already exists — never overwrite, because the user may have added highlights or personal notes.

Citekey provenance — the note filename is a claim about the library

A note's filename and citekey: field assert that Zotero has an entry with that key. [@key] in a manuscript, [[key]] between notes, and the Zotero Integration plugin's {{citekey}}.md all depend on it, and a key that resolves to nothing still looks correct. So the key is read, never composed:

  1. Take it from the Better BibTeX-exported refs.bib entry, or ask Better BibTeX (item.search over json-rpc — see ${CLAUDE_SKILL_DIR}/references/bbt_lookup.md).
  2. If the paper is not in Zotero, add it first (zotero_add_by_doi) and let BBT mint the key. Phase 2 owns that step for a reason: a note written ahead of its library entry has no key to be right about.
  3. If it cannot be added (no DOI, offline), write the note with citekey: "" and the tag _needs-citekey. An empty field is recoverable; an invented one is not, because nothing downstream can tell it apart from a real key.

Verify before finishing:

python3 "${CLAUDE_SKILL_DIR}/scripts/check_citekey_provenance.py" --vault "$VAULT" --bib "$REFS_BIB"

INVENTED means the note's key is absent but its DOI resolves to a real key; fix it here. MISMATCH means the key is real but the note's DOI is the library's DOI for another key, so it cites another paper; rename to the suggested key (known limit: a note DOI spelled differently from the library's, e.g. quoted or braced, never raises it). UNPARSED means a literature note's frontmatter never closes, so its key was not read. With --strict these three exit 1, and a scan that finds no literature note exits 2. Match notes to papers by DOI, never by key (AMBIGUOUS, UNUSABLE, NO_IDENTIFIER: see the script's --help), and search the full library (--live) before importing anything it reports missing.

Template
---
notetype: literature
citekey: "{citekey}"
title: "{title}"
authors: "{authors}"
journal: "{journal}"
year: {year}
doi: "{doi}"
pmid: "{pmid}"
created: "{today}"
tags:
  - type/literature
  - _unread
---

# {title}

## Bibliographic info
- **Authors**: {authors}
- **Journal**: {journal}{volume_issue_pages}
- **Year**: {year}
- **DOI**: [{doi}](https://doi.org/{doi})
{pmid_line}

## Key points (in my own words)

## My thoughts

## Related notes
- [[Research Hub]]
- [[Papers & Reviews]]
-
-

Rules:

  • notetype: literature — compatible with the Zotero Integration template.
  • _unread tag — change to _read later after the user reads the PDF in Zotero and adds highlights.
  • Leave ## Key points and ## My thoughts blank — the user fills these in personally.
  • ## Related notes contains 2 hub links + 2 empty slots (reserved for later concept-note linking).
  • If a PMID is available, add a PubMed link.

Step 3.3: Result report

Obsidian Literature Notes:
  Created:   8 notes (new)
  Skipped:   3 notes (already exist)
  Location:  Literature/
  Total in vault: 12 literature notes

Phase 4: Concept Extraction (conditional)

Trigger condition

Run this phase only when there are ≥10 literature notes in the vault. If fewer exist, print a status message like "N literature notes — concept extraction unlocks at ≥10" and stop.

Step 4.1: Cross-cutting concept scan

Read all files under Literature/*.md (or the vault's existing literature folder), extract keywords from each paper's title, journal, and tags and major concepts from the .bib entry titles, and identify concepts that co-occur across ≥3 literature notes.

Step 4.2: Filtering (5 exclusion rules)

Exclude from concept candidates: model names (GPT-4, Claude, etc.), dataset names (MedQA, ImageNet, etc.), journal names, institution names, and generic technique names (too unspecific). Whatever remains becomes a concept-note candidate.

Step 4.3: Draft concept note

Create Concepts/{concept name}.md (or the vault's existing concept-note folder) from the template in ${CLAUDE_SKILL_DIR}/references/concept_note_template.md (Korean-structured vault: the concept template in references/locale/ko/note_templates.md).

Key rules:

  • Never auto-fill ## Definition — keep the > TODO marker; the 2nd-layer note only becomes meaningful once the user writes the definition in their own words.
  • status always starts at 🌱Seedling.
  • At least 4 wikilinks under ## Related notes (vault convention).

Step 4.4: Propose to the user

Concept-note candidates (≥3 papers cross-referenced):
  1. {Concept A} (4 papers)
  2. {Concept B} (3 papers)
  3. {Concept C} (5 papers)

Create? (all / selected / skip)

Auto-draft, but create only after user confirmation.


Standalone Modes

This skill can run without a fresh .bib file.

Concept extraction only

On an explicit concept-extraction request, scan existing Literature/*.md (or the vault's existing literature folder) and run only Phase 4.

References tidy

On a "tidy this project's references" request, locate .bib files inside the workspace and run Phase 1–3.

Zotero sync only

On a "sync Zotero" request, diff the Zotero collection against the .bib file and add whatever is missing.

PMID-list ingestion (no .bib)

When the user supplies a list of PMIDs (e.g., from a HANDOFF or a colleague), resolve PMIDs to DOIs via PubMed esummary first, then enter Phase 2 with the DOIs:

PMIDS="12345,67890,..."
curl -s "https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esummary.fcgi?db=pubmed&id=${PMIDS}&retmode=json" \
  | jq -r '.result | to_entries[] | select(.key != "uids") | "\(.value.uid)\t\(.value.elocationid)\t\(.value.title)"'

For items already in the library (found by the Step 2.2 search), use zotero_manage_collections to attach them to the project collection without re-adding — re-adding by URL/PubMed-URL would bypass the search dedup and create duplicates. Record both added and existing items in references/zotero_collection.json. If a PMID has no DOI in PubMed (older or non-indexed papers), fall back to zotero_add_by_url with the PubMed URL and mark the entry no_doi: true.

Worklist ingestion (DOI/PMID/Title; no .bib)

When the user supplies a worklist file (a .tsv/.csv/.md table with a DOI column, optional PMID/Title, or a plain DOI-per-line list — e.g. an SR include set), enter Phase 2 directly from it: resolve any PMID-only rows to DOIs (esummary above), then run the search-first dedupe + add loop. The same worklist file feeds Phase 2.7 Route A (fetch_oa.py reads .tsv/.csv/.md/plain natively), so no reformatting is needed.

MA Scout Skill

This skill handles the pre-protocol phase of a meta-analysis — from idea to ranked topic list. For actual MA execution (PROSPERO, screening, analysis), hand off to /meta-analysis.

Mode Selection

SignalMode
Professor name or profile URL providedA: Professor-first
Clinical question, keyword, trend, or "find me a topic"B: Topic-first
Both supplied (e.g., "this topic with this professor")A (topic as filter)

If ambiguous, ask the user whether to search by professor (supervisor-first) or by topic (question-first).

Inputs

Mode A: Professor-first

  • Professor name (native-language + English); profile URL (ScholarWorks, SKKU Faculty, Google Scholar, ORCID); PubMed author link (preferably with cauthor_id for disambiguation); known specialty; affiliation history (e.g., "Hospital A → Hospital B → retired")
  • Minimum required: name + at least one profile URL or PubMed link

Mode B: Topic-first

  • Clinical question or keyword; radiology subspecialty scope; MA type preference (DTA, prognostic, intervention — optional); desired role (solo first author / co-first / supervisor-matched)
  • Minimum required: clinical question or keyword

Workflow

Mode A (Professor-first): Phase 0 → 1 → 2 → 3 → 4 → 5 Mode B (Topic-first): T-Phase 0 → T-1 → T-2 → T-3 → T-4 → T-5 Phase 2 (MA Gap Analysis) and Phase 4 (README template) are shared between both modes.

Query PubMed with /search-lit's E-utilities scripts, not WebFetch — they are faster and return structured JSON/XML: ${CLAUDE_SKILL_DIR}/../search-lit/references/pubmed_eutils.sh and parse_pubmed.py beside it. Rate limit: 350 ms between calls (100 ms with NCBI_API_KEY).

MODE A: PROFESSOR-FIRST WORKFLOW

Phase 0: Disambiguation & Context Confirmation

Resolve the author's identity BEFORE any PubMed search:

  1. Resolve the full English name first. If a cauthor_id is provided, fetch that PMID page to get the full name + affiliation. NEVER start with an initials-only search, because common Korean/Asian initials (e.g., "Lee KS") return 300+ papers with massive contamination. The first search must be "[Full Name]"[Author].
  2. Confirm the affiliation chain with the user. Ask whether {detected affiliation} matches the professor's history (professors move institutions — do not assume), and ask the user's relationship to the professor so topic proposals can be tuned. Skip only if the user already gave an explicit affiliation history.
  3. Profile URL fallback chain:
    • 1st: PubMed full-name search (always works)
    • 2nd: Google Scholar profile (WebSearch "[Full Name]" radiology scholar)
    • 3rd: ResearchGate profile (WebSearch "[Full Name]" researchgate radiology)
    • 4th: ScholarWorks / SKKU / university faculty page (if URL provided)
    • Last: Scopus/ScienceDirect — returns 403 or a login redirect; never rely on it

Phase 1: Profile Exploration (E-utilities API)

Goal: Identify the professor's 5-6 distinct research pillars.

Step 1 — Total publication count + PMID list:

bash ${CLAUDE_SKILL_DIR}/../search-lit/references/pubmed_eutils.sh search \
  '"[Full Name]"[Author]' 200 \
  | python3 ${CLAUDE_SKILL_DIR}/../search-lit/references/parse_pubmed.py esearch

If the parser exits non-zero (an error body, or no count), the count is unknown — re-run the search; never record it as 0 papers or 0 MAs.

Step 2 — Fetch metadata for MeSH-based clustering (parallel):

# Get PMIDs from Step 1, then fetch summaries
bash ${CLAUDE_SKILL_DIR}/../search-lit/references/pubmed_eutils.sh fetch_json \
  "PMID1,PMID2,..." \
  | python3 ${CLAUDE_SKILL_DIR}/../search-lit/references/parse_pubmed.py esummary

Step 3 — Topic-specific counts (launch 4-5 searches in parallel Bash calls):

bash ${CLAUDE_SKILL_DIR}/../search-lit/references/pubmed_eutils.sh search \
  '"[Full Name]"[Author] AND "keyword1"' 5
bash ${CLAUDE_SKILL_DIR}/../search-lit/references/pubmed_eutils.sh search \
  '"[Full Name]"[Author] AND "keyword2"' 5
# ... repeat for each suspected pillar keyword

Step 4 — MeSH term extraction for automatic pillar clustering:

# Fetch full XML for top-cited papers to extract MeSH headings
bash ${CLAUDE_SKILL_DIR}/../search-lit/references/pubmed_eutils.sh fetch \
  "PMID1,PMID2,...,PMID20" \
  | python3 -c "
import sys, xml.etree.ElementTree as ET
from collections import Counter
root = ET.fromstring(sys.stdin.read())
mesh_counts = Counter()
for article in root.findall('.//PubmedArticle'):
    for mh in article.findall('.//MeshHeading/DescriptorName'):
        mesh_counts[mh.text] += 1
for term, count in mesh_counts.most_common(30):
    print(f'{count:3d}  {term}')
"

→ Top MeSH terms reveal natural research pillars (e.g., "Colonography, Computed Tomographic" = CTC pillar).

Step 5 — Google Scholar profile (parallel with PubMed calls): WebSearch "[Full Name]" radiology scholar google for h-index and citation data; WebFetch any other profile URL the user provided (skip Scopus).

Output: Pillar Summary Table. Publication counts and pillar assignments come from E-utilities output and the h-index from the Scholar profile — never estimate them.

PillarDomainRepresentative keywordsMeSH termsEst. # papers
1.........~N+

Phase 2: MA Gap Analysis (Multi-Source)

Goal: For each pillar, determine if a viable MA topic exists using PubMed + Consensus + Scholar Gateway + bioRxiv + PROSPERO.

Run pillars in parallel: up to 4 subagents, each covering 1-2 pillars and running 2a–2g, each reporting raw k, realistic k and every source checked. If PubMed returns 0 or Consensus/Scholar Gateway is unavailable, state that limitation rather than guessing.

2a. PubMed E-utilities — Existing MAs + Primary studies
# Existing MAs (structured count)
bash ${CLAUDE_SKILL_DIR}/../search-lit/references/pubmed_eutils.sh search \
  '[pillar keywords] AND ("meta-analysis"[pt] OR "systematic review"[pt])' 50

# Primary studies with extractable outcomes
bash ${CLAUDE_SKILL_DIR}/../search-lit/references/pubmed_eutils.sh search \
  '[pillar keywords] AND ("sensitivity" OR "specificity" OR "accuracy" OR "prognosis" OR "outcome")' 50
2b. Consensus MCP — Semantic MA gap detection

Use mcp__claude_ai_Consensus__search to find existing SRs/MAs that PubMed keyword search might miss:

query: "systematic review OR meta-analysis [pillar topic] [imaging modality]"

Consensus returns citation-ranked results — check if any highly-cited MA already covers the proposed scope. Limit: max 3 Consensus calls per Phase 2 batch, in total across all agents (rate limit). If rate-limited, wait 30 s and retry once.

2c. Scholar Gateway — Semantic similarity search

Use mcp__claude_ai_Scholar_Gateway__semanticSearch to find MAs under different terminology (e.g., "pooled analysis" instead of "meta-analysis"), scope-overlapping MAs that use different keywords, and methodological reviews that partially cover the topic.

2d. bioRxiv/medRxiv — In-press competition detection

Use mcp__claude_ai_bioRxiv__search_preprints to catch MAs posted as preprints but not yet in PubMed, SR/MA protocols shared as preprints, and very recent primary studies that could change feasibility.

query: "[pillar keywords] meta-analysis OR systematic review"
server: "medrxiv"  (clinical topics; "biorxiv" for preclinical)
2e. Assessment matrix
FactorCriteria
MA gap0 existing = best, 1-3 = check scope overlap, >5 = saturated
Primary k≥8 for DTA, ≥6 for prognostic (minimum), ≥15 ideal
RecencyLast MA >5 years old = update opportunity
CompetitionCheck the last two years for very recent MAs that block entry
2f. PROSPERO competition check (MANDATORY)
  • Search PROSPERO via WebSearch site:crd.york.ac.uk/prospero [topic keywords]; also try WebFetch https://www.crd.york.ac.uk/prospero/#searchadvanced
  • Look for registered-but-unpublished protocols that could block entry; if a PROSPERO match is found → flag as 🚫 competition risk in ranking
  • Record the exact query, the search date and one status: searched (match found), searched (no match), or not checked (unavailable/failed); a fetch that returns the site shell or an error page instead of a result list is not checked. Only searched (no match) clears the PROSPERO gate.
2g. Realistic k estimation
  • Raw PubMed hit count is NOT the real k — most studies lack 2x2 data or HR, so raw counts overestimate by 3-7x
  • Apply conservative discount: k_realistic ≈ raw_count × 0.15–0.30 for DTA topics
  • Flag if k_realistic < 8 (DTA) or < 6 (prognostic) as ⚠️ feasibility risk
  • Report both raw and realistic estimates, e.g., estimated k: ~130 (raw) → ~20–40 (extractable DTA data)
2h. Niche subtopic discovery (if pillar appears saturated, >5 prior MAs)

Try these angles, and use Consensus to check whether the niche angle has already been covered:

  1. "First MA" rule: the professor's most unique/niche subtopic where MA = 0
  2. AI/radiomics overlay: classical imaging topic + AI approach
  3. Treatment response: diagnosis MAs are often saturated; treatment monitoring is often open
  4. Modality comparison: head-to-head (e.g., CEUS vs MRI) is often underserved
  5. Guideline gap: professor-authored guidelines → MA supporting/updating them
  6. Population niche: specific subpopulation, disease subtype, or regional population (e.g., parasitic diseases, TB)
  7. Temporal update: last MA >5 years old + significant new primary studies since

Phase 3: Topic Ranking

Goal: Rank all viable topics by composite score. Score each candidate on 5 criteria (★1-5):

CriteriaWeightDescription
Professor fitHighestCore area of the professor's career, publication count, distinctive contribution
MA gapHighNo prior MA > ≥5 yr since last MA > recent MA exists
Feasibility (k)HighNumber of includable studies and extractability of 2×2 or HR data
Clinical impactMediumWhether the topic directly informs clinical decision-making
Execution easeMediumCompletable from literature alone; difficulty of managing heterogeneity

Output: Ranked Topic Table

RankTopicProfessor's PillarPrior MAEstimated k (raw→realistic)PROSPERO competitionVerdict
1......0~98 → 15–30None✅ Best fit

Phase 4: Folder & README Scaffolding

Goal: Create project folders and README for each viable topic.

  1. Folder location: {working_dir}/ma-scout/{initials}_{professor_name}/{NN}_{topic_slug}/
    • Professor folder: {initials}_{name} (e.g., KDK_Kim, LKS_Lee)
    • NN: sequential number within professor (01, 02, ...); topic_slug: English, underscore-separated
    • Check existing folders with ls before creating
  2. README.md (PROSPERO-ready): copy the template block from ${CLAUDE_SKILL_DIR}/references/project_readme_template.md into {topic_folder}/README.md and fill it (PICO/PIRD frame, preliminary search, target journal table, backward-planned timeline). Write the research question, PICO/PIRD and README content in English, medical terms always in English, unless the user asks for the Korean PI-facing variant that reference names.

Phase 5: Output Summary

  1. Save the ranked topic table and README files to the working directory.
  2. Summarize: total topics scanned, viable topics found, recommended next steps (see Handoff).

MODE B: TOPIC-FIRST WORKFLOW

T-Phase 0: Topic Clarification & Scope

Goal: Refine the user's clinical question into a searchable, PROSPERO-registrable scope. When the user asks for topic suggestions without a specific idea, read ${CLAUDE_SKILL_DIR}/references/topic_discovery_heuristics.md first to generate candidate questions.

  1. Parse the input — disease/condition (e.g., "hepatocellular carcinoma"); imaging modality or intervention (e.g., "dual-energy CT", "AI CAD"); outcome type: DTA (Se/Sp), prognostic (HR/OR), intervention (RR/MD), dosimetry; population specifics (e.g., "screening setting", "cirrhotic patients").
  2. Expand to neighboring angles — propose 3-5 variations, e.g.:
    user input: "AI for lung nodule malignancy prediction"
    → variant 1: AI vs radiologist for lung nodule malignancy prediction (DTA)
    → variant 2: Radiomics for lung nodule malignancy (DTA)
    → variant 3: Deep learning for incidental pulmonary nodule management (prognostic)
    
  3. User selects 1-3 angles to investigate further.

T-Phase 1: Landscape Scan (Multi-Source)

Goal: For each selected angle, rapidly assess the MA landscape. Run all angles in parallel. For each angle:

1a. PubMed — Existing MA count
bash ${CLAUDE_SKILL_DIR}/../search-lit/references/pubmed_eutils.sh search \
  '[topic keywords] AND ("meta-analysis"[pt] OR "systematic review"[pt])' 50
1b. PubMed — Primary study pool
bash ${CLAUDE_SKILL_DIR}/../search-lit/references/pubmed_eutils.sh search \
  '[topic keywords] AND ("sensitivity" OR "specificity" OR "hazard" OR "outcome")' 100
1c. Consensus MCP — Semantic MA discovery
query: "systematic review [topic] [modality]"

Check for MAs using different terminology.

1d. bioRxiv/medRxiv — Preprint competition
query: "[topic] meta-analysis"
server: "medrxiv"
1e. PROSPERO — Registered protocols

WebSearch: site:crd.york.ac.uk/prospero [topic keywords]

Output: Landscape Summary Table

VariantExisting MAsPrimary k (raw)k (realistic)PROSPEROPreprint MAVerdict
1312018-3610⚠️ Competitive
208513-2500✅ Optimal

T-Phase 2: Feasibility Deep-Dive

Goal: For viable angles (MA ≤ 2, no PROSPERO conflict), run the same Phase 2 (MA Gap Analysis) as Mode A — steps 2a through 2h. With no "Professor fit" to evaluate, focus on:

  • Gap certainty — are existing MAs truly non-overlapping with proposed scope?
  • k quality — are primary studies heterogeneous enough to warrant MA, or too uniform?
  • User's domain fit — does this align with user's radiology AI / imaging expertise?

T-Phase 3: Topic Ranking (Topic-first weights)

CriteriaWeightDescription
MA gapHighestNo existing MA > update opportunity > saturated
Feasibility (k)Highestk_realistic ≥ 8 (DTA) or ≥ 6 (prognostic)
User domain fitHighDoes it match the user's area of expertise?
Clinical impactMediumPotential to change guidelines; directly tied to clinical decisions
Co-author availabilityMediumAccess to a domain expert (existing relationship or easy to reach)
Execution easeMediumCan be done solo vs requires expert interpretation

Output: Ranked Topic Table

RankTopicExisting MAsEst. kPROSPEROCo-author neededOverall
1...025NoneOptional✅ Optimal

T-Phase 4: Co-Author Matching (Optional)

Goal: If the user wants a senior co-author, find candidates.

Strategy 1 — Existing network: check memory files and existing professor folders in the working directory for professors whose pillar naturally covers this topic (best match).

Strategy 2 — PubMed reverse search:

# Find prolific authors in this specific topic
bash ${CLAUDE_SKILL_DIR}/../search-lit/references/pubmed_eutils.sh search \
  '[topic keywords] AND ("{user_country}"[Affiliation])' 100

Then E-utilities efetch → author frequency; the top 5 most-published authors in this niche are potential co-authors. Cross-check Google Scholar for h-index and recent activity.

Strategy 3 — Self-led (no senior co-author): viable when the user has 2+ published MAs and the topic is methodologically straightforward. A 2nd reviewer (junior colleague or peer) is still needed — flag this in the README. Corresponding author = user.

Output: Co-author recommendation table or a "solo-viable" judgment.

T-Phase 5: Folder & README Scaffolding (Topic-first)

  1. Folder location: {working_dir}/ma-scout/TOPIC/{NN}_{Topic_Abbreviation}/ (e.g., 01_AI_Lung_Nodule_DTA/). Topic-first projects use the TOPIC/ prefix, not professor initials; if a co-author is matched later, the folder can move under the professor folder.
  2. README.md: the Phase 4 template with the Solo-Mode Adaptations in ${CLAUDE_SKILL_DIR}/references/project_readme_template.md (Lead/Domain rows, Team Expertise, self-led timeline).
  3. Summary: same as Mode A Phase 5 — save ranked results and recommend next steps.

Quality Gates

Before finalizing a topic as viable:

  • (A) Author identity confirmed — full name resolved via E-utilities efetch, no initials-only contamination
  • (A) Affiliation confirmed with user (or from reliable source)
  • (A) Professor's publication record demonstrates clear authority in this area
  • (B) Clinical question refined to PICO/PIRD (not just a keyword)
  • (A) Confirmed MA = 0 or last MA >5 years (via PubMed E-utilities, not assumption)
  • Cross-validated via PubMed plus every connected semantic or preprint source (Consensus, Scholar Gateway, bioRxiv/medRxiv); a source that was unavailable is named and recorded as not checked, never as "no competing work found" — no hidden MAs with different terminology, no preprint MA in progress
  • Confirmed k_realistic ≥ 8 (DTA) or ≥ 6 (prognostic), where k_realistic = raw count × 0.15–0.30 (§2g: keep 15–30% of raw hits, i.e. a 70–85% discount — not a 15–30% discount)
  • PROSPERO searched — the search ran and returned results (query + date recorded, §2f), and no registered competing protocol was found. A PROSPERO search that failed or was unavailable is "not checked", never "none found": this item stays open
  • No competing MA from the last two years in press or preprint
  • Research question is specific enough for PROSPERO registration
  • (B) User's domain expertise sufficient for clinical interpretation (or co-author identified); 2nd reviewer identified or plan to recruit
  • (B) If self-led: user has ≥ 2 published MAs (otherwise, recommend co-author)
  • README contains: complete PICO/PIRD, PubMed search strategy, Embase draft, target journal with IF, timeline

Handoff

  • To /meta-analysis: when a topic is approved and ready for the PROSPERO protocol (README has PICO + search strategy)
  • To /manage-project: when the project folder needs full scaffolding or the user wants results saved to project management
  • To /search-lit: when a deeper preliminary search is needed before committing
  • To /analyze-stats: when feasibility requires power/sample-size calculation for the estimated k

Phase 6: Pre-Proposal Pipeline (Post-Scout)

After MA Scout identifies viable topics, prepare a "ready-to-propose" package before contacting the professor. Topics are independent: run up to 4 agents per wave, each doing search → fetch → triage → write files.

  1. Search Execution — E-utilities with broadened synonyms (retmax=200)
    • Primary search: [topic] AND [outcome keywords]
    • Existing MA search: [topic] AND ("meta-analysis"[pt] OR "systematic review"[pt])
  2. Metadata Collection — fetch_json → esummary (batch 40-50 PMIDs)
  3. Title-Based Triage — classify as INCLUDE / MAYBE / EXCLUDE
    • Check for existing MAs within the results — the initial scout may have missed them
    • Separate sub-approaches that contaminate the pool (e.g., bronchoscopic vs percutaneous)
    • Flag the professor's own papers (authority evidence) and retracted papers
  4. PRISMA Flow Draft — Identification → Screening → Eligibility → Included (estimated)
  5. Gap Re-assessment — update the MA count and re-position if needed: MA=0 → "first MA" | MA=1 (>5yr) → "update MA" | MA≥3 (recent) → skip/niche
  6. Output Files:
    • candidates.md — full triage table + PRISMA flow + gap finding
    • README.md — updated Preliminary Search section with actual numbers

Professor Contact Package — the pre-proposal gives the professor: candidate count + gap evidence (e.g., "MA = 0, 35 studies to include"), a clear role description (e.g., "independent screening review + discussion only"), and the urgency of PROSPERO pre-registration to secure the topic.

Make-Figures Skill

Data Privacy Check

Before reading any data file, check whether it might contain Protected Health Information (PHI):

  1. If *_deidentified.* files exist in the working directory, use those preferentially.
  2. If only raw CSV/Excel files exist (no *_deidentified.* counterpart), ask the user, in their language, whether the data contains patient identifiers (names, national ID / RRN, contact details, etc.) and, if so, to de-identify it first with the /deidentify skill.
  3. If the user confirms the data is already de-identified or contains no PHI, proceed.

Reference Files

  • Figure specifications: ${CLAUDE_SKILL_DIR}/references/figure_specs.md — read it before generating any figure (journal dimensions, DPI, file formats, palettes, font sizes, panel layouts, caption format).
  • Figure style: ${CLAUDE_SKILL_DIR}/../analyze-stats/references/style/figure_style.mplstyle

Journal AI-Image Policies (CRITICAL — check BEFORE generation)

Journal familyPolicy on AI-generated imagesDisclosure required
JACC family (incl. JACC: Asia, JACC Imaging, JACC EP, JACC BTS)Prohibited without prior Editor-in-Chief permission (JACC pathway, PMC10167500)Cover-letter pre-submission inquiry + ICMJE-style declaration
NEJMAI image generation prohibitedN/A
Radiology / Radiology AIAllowed with disclosureManuscript disclosure block
Springer Nature (Nature Portfolio, BMC, Springer journals)Allowed only when the visual is derived from independently verifiable data, source material, methods or code; AI visuals without verifiable inputs are "opaque" and not permitted. Charts from datasets and author-reviewed code-generated figures are the policy's own allowed examples (AI in manuscript preparation)Declare model and purpose; disclose non-generative image edits in the caption
Elsevier journals (publisher policy)Explanatory images (flow charts, schematics) allowed. Data visualizations only when directly derived from the underlying data by reproducible methods. Primary research images, including radiology scans and patient images, must not be created or altered with AI. AI images produced as part of the research methods are permitted. Graphical abstracts: dedicated illustration tools, not general-purpose generative AI (Generative AI policies for journals)Caption + AI disclosure statement (explanatory); Methods (data visualizations, research-method use)
Lancet familyDisclosure required, generation discouragedManuscript disclosure
Default (target unknown)Treat as prohibited until confirmedN/A

Publisher pages set the floor; a journal's own guide can be stricter (JACC is an Elsevier journal). Check the target journal's guide as well as its publisher's.

Hard rule: For JACC, NEJM, or any "unknown" target journal, never use Gemini / DALL-E / Midjourney / Stable Diffusion / Nano Banana to create images that will appear in figures, Central Illustrations, or graphical abstracts. AI text-editing of the manuscript prose remains acceptable subject to standard disclosure.

Default workflow when AI images are not allowed

  1. SMART Servier Medical Art — https://smart.servier.com/, CC BY 4.0, free, 3,000+ vector medical icons (anatomy, organs, ethnicity-specific human figures, drugs, devices). Commercial / journal use allowed. Required attribution (1 line in figure legend OR methods):

    Anatomical icons modified from SMART Servier Medical Art (CC BY 4.0).

  2. NIAID BioArt (https://bioart.niaid.nih.gov) — public domain (US Govt), microbiology / immunology / lab-tech focus.
  3. BioRender (https://www.biorender.com) — institutional license usually required; use the exported "Publication-ready" PNG/TIFF and cite per BioRender publication policy.
  4. For "diseased" variants not directly available (e.g., calcified vessel from a clean vessel): reuse the healthy asset and overlay disease markers via matplotlib scatter / Circle / PathPatch. Keeps the entire pipeline non-AI and reproducible.

Even when AI images are allowed, experienced reviewers recognise AI-generated illustrations (small decorative icons that add no information, overly uniform layouts, generic clip-art style). For high-impact submissions, prefer Servier / BioArt / BioRender + matplotlib overlays over AI.

Asset directory convention

manuscript/figures/_assets_servier/      # CC BY 4.0 source PNGs
manuscript/figures/_assets_servier/CITATION.md   # source URL + download date per asset
manuscript/figures/_assets_data/         # data-driven raster (R / matplotlib heat maps, KM, etc.)
manuscript/figures/_legacy/              # archived prior versions

Composition scripts should load only from _assets_servier/ and _assets_data/. If a script imports from _assets_ai/, treat it as a policy violation for JACC/NEJM/unknown targets.

AI Image Generation (Optional)

AI illustration is a supplementary option: every figure and visual abstract can be completed without an API key. If GEMINI_API_KEY is set, generate_image.py can generate illustrations (procedural schematics, anatomical illustrations), subject to the policy table and hard rule above:

python ${CLAUDE_SKILL_DIR}/scripts/generate_image.py \
  "Clean medical illustration of a CT-guided lung biopsy procedure, \
   flat vector style, white background, no text" \
  --output output.png --aspect 16:9

If GEMINI_API_KEY is not set, use ${CLAUDE_SKILL_DIR}/references/medical_illustration_sources.md.


Visual Abstract / Graphical Abstract

European Radiology made graphical abstracts mandatory for all Original Articles from first revision (Jan 2025); other journals encourage or accept them (status list in template_guide.md). Check the target journal profile (write-paper/references/journal_profiles/) for its visual abstract requirements before starting.

Workflow

  1. Check journal template. Look for an official PPTX template in ${CLAUDE_SKILL_DIR}/references/visual_abstract_templates/{journal}.pptx. If no journal-specific template exists, use medsci_default.pptx.
  2. Extract content from the manuscript:
    • Title: Full article title
    • Hypothesis/Question: Derived from Key Point 1 or study objective (max 1 sentence)
    • Methodology: Brief flowchart or ≤3 bullets, <6 words each
    • Visual element: Study's own figure (ROC curve, flow diagram, representative image)
    • Badges: Patient cohort (N=...) | Modality/organ | Single/Multi-center
    • Main finding: Derived from Key Point 3 (<20 words)
    • Citation: Journal (year) Authors; DOI
  3. Select visual element (priority order):
    1. Study's own figures (ROC, flow diagram, representative image) — always preferred
    2. Free illustration from Servier Medical Art or NIAID BioArt (see ${CLAUDE_SKILL_DIR}/references/medical_illustration_sources.md)
    3. Manual drawing in PPT/Keynote/Figma
    4. AI generation via generate_image.py --style medical (only if GEMINI_API_KEY set)
  4. Generate using the script:
    python ${CLAUDE_SKILL_DIR}/scripts/generate_visual_abstract.py \
      --template medsci_default \
      --title "Article Title" \
      --hypothesis "Research question" \
      --methods "Method 1|Method 2|Method 3" \
      --finding "Main finding statement" \
      --citation "Eur Radiol (2026) Author A et al; DOI:..." \
      --visual figures/fig1_roc_curve.png \
      --badges "N=450|CT chest|Multi-center" \
      --output figures/visual_abstract.pptx
    
  5. Review with user. Open the PPTX to verify layout and content. Iterate.
  6. Export. PPTX is the primary deliverable. For PNG: open in PowerPoint/Keynote → export, or use LibreOffice CLI (soffice --headless --convert-to png).

Design: one page, landscape (16:9) or per the journal template; three sections (study question → key method → main result); the study's actual figures rather than generic graphics; minimal text; no decorative clip-art.

Available Templates

TemplateFileUse When
MedSci Defaultmedsci_default.pptxAny journal without an official template
JACC Central Illustrationjacc_central_illustration.pptxJACC family journals (use --type central-illustration)

Using a journal's own template. We do not redistribute journals' templates (e.g., European Radiology's EURA-GA-Jan2025.pptx): download it and pass its absolute path:

python ${CLAUDE_SKILL_DIR}/scripts/generate_visual_abstract.py \
  --template /absolute/path/to/EURA-GA-Jan2025.pptx  ...

--template takes an absolute path to any .pptx. The script locates the fields by their text content rather than by shape name, so a journal's own template works unmodified. If the path does not exist it falls back to medsci_default.pptx.

To add a new journal template: see ${CLAUDE_SKILL_DIR}/references/visual_abstract_templates/template_guide.md.


Central Illustration vs Visual Abstract

JACC family journals (JACC, JACC: Asia, JACC: Cardiovascular Imaging, JACC: Heart Failure, JACC: CardioOncology, JACC: Clinical Electrophysiology, JACC: Basic to Translational Science) require a Central Illustration (CI) with every Original Article. A CI is not a Visual Abstract: it conveys one key finding and contains no methods. Before making one, read ${CLAUDE_SKILL_DIR}/references/jacc_central_illustration_principles.md (CI vs VA table, the five Fuster-Mann rules, layout and validation thresholds).

CI mode invocation

python ${CLAUDE_SKILL_DIR}/scripts/generate_visual_abstract.py \
  --type central-illustration \
  --visual figures/central_illustration_v2.png \
  --citation "FirstAuthor Last et al. Journal Name 2026; vol(issue):pages." \
  --output submission/jacc_asia/central_illustration.pptx \
  --ci-zones 3 --ci-label-words 22 --ci-numerical-points 2 \
  --ci-raw-text "warranty drops to 3 years in age 45+ with cardiometabolic burden; MASLD HR 1.77"

CI mode validates before rendering and rejects (exit 2) if any of: zones > 3, label words > 30, numerical points > 4, or methodology terms (cohort flow / inclusion criteria / exclusion criteria / study design / enrollment / randomized / sample size / CONSORT / PRISMA / STARD) appear in --ci-raw-text. Override individual rules with --ci-allow {zones|words|numerical|methods} only when you have a defensible reason. Submit only the content figure + citation: JACC editorial applies the red border and blue "CENTRAL ILLUSTRATION:" header after acceptance.


Workflow

Any reference in a caption, legend or illustration needs a DOI or PMID confirmed via /search-lit; mark one you cannot confirm [UNVERIFIED - NEEDS MANUAL CHECK]. Mark an unconfirmed clinical definition, criterion or guideline claim [VERIFY] and ask — the submission gates block on both.

Step 1: Specify

Before specifying figure type, read ${CLAUDE_SKILL_DIR}/references/design_principles.md — identify (1) the one-sentence key message, (2) audience and reading-time budget, and (3) whether a figure is the right vehicle (vs a small table or in-line text). Skip only when the figure is mandated by a reporting guideline (e.g., PRISMA / CONSORT flow), and even then apply the cognitive-load checklist.

For reporting-guideline figures, also load ${CLAUDE_SKILL_DIR}/references/reporting_guideline_figure_map.md — which guideline mandates which figures and whether this skill ships an official template (✅), generic flow only (⚠️), or needs manual production (❌). Critical for AI-extension guidelines (CONSORT-AI, STARD-AI, TRIPOD+AI, CLAIM 2024, DECIDE-AI).

For medical AI / engineering pipeline figures (DICOM workflow, annotation pipeline, federated learning topology, model architecture), also load ${CLAUDE_SKILL_DIR}/references/pipeline_concepts_medical_ai.md — canonical layouts, required annotations, and tool selection per type.

Optional flags:

  • --study-type <type>: One of: diagnostic-accuracy, ai-validation, meta-analysis, dta-meta-analysis, observational-cohort, rct, case-report. When set, auto-generate the full figure set from the Study-Type Figure Sets table below without prompting for individual figure types.
  • --data-dir <path>: Directory containing analysis outputs (CSVs, _analysis_outputs.md). Default: current working directory.

Ask the user for (infer what the context already gives, and confirm before proceeding):

  1. Figure type — skipped when --study-type is provided
  2. Data source (file path, DataFrame, or manual values)
  3. Target journal (for dimension/font requirements)
  4. Panel layout (single panel, multi-panel, or let you decide)
  5. Any special requests (annotations, highlights, reference lines)
  6. Study type (if not passed via --study-type): determines the required figure set

Step 2: Configure

  1. Load the figure style file (skeleton below).
  2. Look up journal-specific dimensions in ${CLAUDE_SKILL_DIR}/references/figure_specs.md.
  3. Set the colorblind-safe palette (Wong palette by default).
  4. Set font sizes per element type (title, axis label, tick label, legend, annotation) from figure_specs.md.

Step 3: Generate

Use Python (matplotlib/seaborn, with specialized libraries as needed). Compose each data plot from its anatomy model in ${CLAUDE_SKILL_DIR}/references/exemplar_plots/ (index in its README.md); for box/violin, bar and heatmap figures follow the per-type conventions in figure_specs.md. Flow diagrams never use matplotlib — see the Tool Selection Guide.

Script structure:

"""
Figure: {description}
Date: {YYYY-MM-DD}
Target: {journal}
Dimensions: {width} x {height} inches @ {DPI} DPI
"""
import numpy as np
import matplotlib.pyplot as plt
import os

style_path = os.path.join(os.environ.get('CLAUDE_SKILL_DIR', '.'), '../analyze-stats/references/style/figure_style.mplstyle')
if os.path.exists(style_path):
    plt.style.use(style_path)

# Wong colorblind-safe palette
WONG = ['#000000', '#E69F00', '#56B4E9', '#009E73',
        '#F0E442', '#0072B2', '#D55E00', '#CC79A7']

np.random.seed(42)

Never fabricate data points. If sample data is needed for a template demo, label it "example data". When a figure is produced by a data-driven .py/.R script (ROC, forest, KM, calibration, heat maps), lint that script before finalizing with the /analyze-stats code-quality gate (check_generated_code.py {script} --strict): it catches a missing plotting seed for any bootstrapped CI band, a hardcoded absolute data path, or a hand-typed data literal that should have been read from the analysis CSV.

Step 4: Review

Present the figure to the user (layout, labels and annotations, colors/sizing/emphasis) and iterate until the user approves.

Step 4b: Critic Loop (self-critique before final export)

Before Step 5 Export, run two stages — deterministic quantitative checks, then your own qualitative review — to decide whether to re-render or hand off to the user.

Stage 1: Quantitative checks (critic_figure.py)

python ${CLAUDE_SKILL_DIR}/scripts/critic_figure.py \
    figures/fig1_stard.png \
    --type stard \
    --spec-min-dpi 600 \
    --spec-width-in 7.0 \
    --source-text figures/fig1_stard.txt \
    --out figures/fig1_stard.critique.json

--source-text is optional (expected strings for OCR coverage). The JSON report covers DPI and physical width vs. the journal spec, the dominant-color breakdown and out-of-Wong-palette fraction, and OCR word count, minimum text height, and source-word coverage. When the image carries no DPI metadata, the DPI check uses the resolution at the spec width (width_px / --spec-width-in). A check that cannot run (physical width without DPI metadata, OCR without pytesseract) is listed under not_run and the summary reads INCOMPLETE, never PASS; --strict exits 3 on INCOMPLETE.

Stage 2: Qualitative review

  1. Use the Read tool to load the generated PNG.
  2. Read the corresponding rubric file (sections A–G in each):
    • Flow diagrams: ${CLAUDE_SKILL_DIR}/references/critic_rubrics/flow_diagram.md
    • Data plots: ${CLAUDE_SKILL_DIR}/references/critic_rubrics/data_plot.md
    • For PRISMA / CONSORT / STARD / STROBE, also read ${CLAUDE_SKILL_DIR}/references/flow_diagram_lessons.md (official-template fidelity, PDF export via VML fallback, docx XML escape, sequential placeholder mapping, frozen-version sync with the manuscript).
    • For AI-extension guidelines or pipeline figures, also check against the reporting_guideline_figure_map.md row / pipeline_concepts_medical_ai.md loaded in Step 1.
  3. Read the _why.md design notes in ${CLAUDE_SKILL_DIR}/references/exemplar_diagrams/{type}/ — hierarchy, whitespace, typography, emphasis, colour. They are the anchors. Where a rendered exemplar is bundled (template_output*.png), Read 1–2 of those too. Figures cropped from published papers are not shipped (see that directory's README); a user's own local exemplars can be used instead. For a non-flow data plot, read the matching anatomy model in ${CLAUDE_SKILL_DIR}/references/exemplar_plots/ (e.g., forest_plot.md).
  4. Score every rubric item as PASS / PARTIAL / FAIL with a one-line note, using the format at the bottom of the rubric file.
  5. Emit a "Required edits before next render" list of concrete source-code changes (node/label renames, count corrections, matplotlib parameter tweaks).

Refinement loop

  • If all items are PASS → proceed to Step 5 Export with critic_pass: yes.
  • If any item is FAIL → apply the required edits to the source (flow-diagram config/R script or matplotlib script), re-render, and re-run Stage 1 + Stage 2. Default maximum is T=2 rounds; the user may request up to T=3.
  • If after the max rounds some items remain PARTIAL, proceed with critic_pass: partial and record the residual items in the manifest's critic_notes field.

Record the final state in _figure_manifest.md (see Study-Type Figure Sets) so downstream steps (/write-paper Phase 2 embedding and Phase 7 DOCX build) and future critic passes can see the history.

Step 5: Export

Save final outputs to analysis/figures/ (analysis/figures/*.pdf, analysis/figures/*.png):

  • PDF (vector format, preferred for journal submission)
  • PNG (300 DPI raster, for review and presentation)
  • TIFF (if the journal requires it, 300 DPI LZW compression) — produce it with export_portal_tiff.py, not a raw magick ... output.tiff: that keeps the alpha channel (transparent regions print black on many production pipelines) and stays uncompressed (a 600-dpi RGBA TIFF blows past a portal's 25 MB cap). The script white-flattens RGBA→RGB, LZW-compresses, and verifies the result is pixel-identical to that flatten. Use it when a portal accepts only .tiff/.jpeg/.eps (Springer Nature SNAPP) or caps figure size (JACC: Asia):
    python3 ${CLAUDE_SKILL_DIR}/scripts/export_portal_tiff.py --in figure.png --out figure.tiff --max-mb 25
    # exit 1 if still over the cap
    

Name files descriptively: fig1_roc_curve.pdf, fig2_consort_flow.pdf, etc.

For PPTX outputs (visual abstract, central illustration, or any deck the figure will live in): run the Mac-compatibility validator before delivery. PowerPoint Mac silently drops TIFF, renders <a:sp3d> 3-D bevels as red outlines that PDF export does not show, and refuses to open files whose app.xml slide count disagrees with the actual slide XML files.

python ${CLAUDE_SKILL_DIR}/scripts/validate_pptx_mac_compat.py \
    figures/visual_abstract.pptx \
    --json figures/visual_abstract.mac_compat.json \
    --strict

Exit code 1 means at least one FAIL — fix per the fix: field in the JSON report and re-render the PPTX before delivery. Exit code 0 with WARN is acceptable. Skip this step when the figure is PNG/PDF only (no PPTX).

Step 6: Design QC Checklist

Before delivering the final figure, verify all items:

  • Font: Sans-serif (Arial/Helvetica), minimum 7pt, axis labels ≥ 9pt
  • Color: Wong/Okabe-Ito colorblind-safe palette used
  • Colorblind test: Works for deuteranopia (no red-green only distinctions)
  • Grayscale test: Information preserved when printed in black & white
  • Alignment: All elements on a consistent grid; panels aligned
  • Vector output: PDF/SVG saved (not just PNG)
  • Resolution: ≥ 300 DPI for raster elements, ≥ 600 DPI for line art
  • Journal specs: Dimensions, font, and format match the target journal (if a figure exceeds a limit, resize it and report the adjustment)
  • No chartjunk: No 3D effects, unnecessary gridlines, gradient fills, or decorative elements
  • Caption: Drafted per figure_specs.md (Caption Writing Guidelines) with key finding, abbreviations, statistical details, and sample size

Study-Type Figure Sets

When the study type is known (from /write-paper Phase 0 or user specification), generate the complete required figure set without asking for each figure individually.

Study Type (Guideline)Required Figures
Diagnostic accuracy (STARD)STARD flow diagram, ROC curve, confusion matrix, calibration plot
AI validation (TRIPOD+AI / CLAIM)Flow diagram, ROC curve, confusion matrix, calibration plot, feature importance or SHAP, Grad-CAM (if imaging)
Meta-analysis (PRISMA)PRISMA flow diagram, forest plot, funnel plot
DTA meta-analysis (PRISMA-DTA)PRISMA flow diagram, paired forest plot (Se + Sp), SROC curve, Deeks funnel plot
Observational cohort (STROBE)Flow diagram, Kaplan-Meier curves (if survival endpoint)
RCT (CONSORT)CONSORT flow diagram, primary endpoint figure
Case report / series (CARE)Clinical timeline figure (exemplar_plots/clinical_timeline.md), annotated multimodality imaging panel when visually load-bearing (exemplar_plots/imaging_panel.md); for a series, an all-cases summary table
Oncology treatment study (CONSORT RCT or single-arm phase II; REMARK for biomarker subsets)CONSORT or cohort flow diagram, Kaplan-Meier PFS/OS with number-at-risk table (exemplar_plots/km_curve.md), waterfall plot of best response (exemplar_plots/waterfall_plot.md), swimmer plot for durability (exemplar_plots/swimmer_plot.md), spider plot when response kinetics matter (exemplar_plots/spider_plot.md), cumulative incidence instead of 1 - KM when competing risks are present (exemplar_plots/cumulative_incidence.md)

The manifest is mandatory. After generating all figures, write analysis/figures/_figure_manifest.md — one row per figure (Figure | Path | Type | Tool | Critic | Rounds | Description) plus a ## Critic notes section recording any residual PARTIAL items and why they were accepted. It is consumed by /write-paper Phase 2 (figure embedding) and Phase 7 (DOCX build); verify it exists and is non-empty before finishing. Read ${CLAUDE_SKILL_DIR}/references/figure_manifest.md when writing it (format and field definitions).

Tool Selection Guide

Data plots:    matplotlib/seaborn → PDF + PNG (this skill)
Flow diagrams: generate_flow_diagram.R (DiagrammeR + rsvg) → PDF + 300/600 dpi PNG
Final assembly: pandoc or python-docx (auto-embedded in DOCX)

Flow Diagrams → Dedicated Tools (NOT matplotlib)

STARD / CONSORT / PRISMA / STROBE flow diagrams MUST use the standardized R pipeline scripts/generate_flow_diagram.R (DiagrammeR + Graphviz dot + rsvg) — the single canonical tool for all four. Do NOT use matplotlib FancyBboxPatch (manual coordinates break when text changes, and patches distort when embedded in DOCX). Do NOT use D2 for new flow diagrams (weak font control, overlap needs manual post-processing); D2 is a legacy fallback only when R is unavailable.

TypeRecommended ToolWhy
STROBE (cohort / cross-sectional)scripts/generate_flow_diagram.R --type strobeSingle canonical tool; auto-layout; vector PDF + 300/600 dpi PNG
CONSORT (RCT)scripts/generate_flow_diagram.R --type consortSame pipeline; monochrome Arial default
PRISMA 2020 (SR/MA)scripts/generate_flow_diagram.R --type prismaFaithfully implements PRISMA 2020 structure; avoids PRISMA2020 R package's webshot-based raster PDF issue
STARD (DTA)scripts/generate_flow_diagram.R --type stardSame pipeline; supports 2x2 reference-standard split
Pipeline DiagramD2 (legacy)Until pipeline-diagram support is added to the R script

YAML config → Rscript ${CLAUDE_SKILL_DIR}/scripts/generate_flow_diagram.R --type <t> --config <yaml> --out <prefix> → PDF + 300/600 dpi PNG. Templates in references/exemplar_diagrams/{strobe,consort,prisma,stard}/template_input.yaml. Read ${CLAUDE_SKILL_DIR}/references/flow_diagram_recipe.md when generating a flow diagram (YAML schema, fixed style, per-project create_figure1.R pattern, D2 fallback); a ROC curve or forest plot needs none of it.

Every box label contains its count (e.g., "Assessed for eligibility\n(n = 450)"). Numbers in labels must be CSV-derived — author the YAML from an R/Python script that reads the upstream data — or hand-written only when the value lives in a commit-tracked data artifact (cite the source file in a comment).

Caption ↔ flow-SSOT reconciliation (before Step 5 Export). The flow-diagram config is the single source of truth for participant counts; a hand-written Figure 1 caption drifts from it when the cohort is re-locked but the caption is not ("caption says n = 1,284 analytic, diagram box says n = 998"). Re-derive the caption counts from the flow config and reconcile:

python3 ${CLAUDE_SKILL_DIR}/scripts/derive_figure_legend_counts.py \
  --flow-config figures/figure1_strobe_graphviz.yaml \
  --manuscript manuscript/index.qmd \
  --out qc/figure_legend_counts.json --strict

Any n = N in the caption that is not a box count in the flow config is a MISMATCH (stale caption) — update the caption from the config, never the reverse. The reconciler is stdlib-only and parses the config as text, so it works regardless of the flow tool. When no Figure 1 caption is found or the caption carries no n = N, the verdict is NOT_CHECKED (exit 2 under --strict), not OK.

Known limits: only n = N notation is read (a bare "1,150 patients" is not), and agreement is set membership, so a count moved to the wrong box (excluded and analysed swapped) is not detected; check those by eye against the diagram.

Official Reporting Guideline Templates → templates/official/

When a journal requires the canonical, statement-issued template (rather than the auto-laid-out R version), use the bundled official files in templates/official/{prisma2020,consort2010,stard2015,spirit2013}/.

GuidelineWhat shipsWhen to use
PRISMA 2020Locally built .pptx (4 variants) + fill_prisma_template.pyReviewer asks for the official PRISMA 2020 layout, or you want editable PowerPoint instead of an R-rendered PDF.
STROBE (cohort)Parametric .pptx builder build_strobe_template.py (YAML config)Cohort/case-control Figure 1 when co-authors want PowerPoint they can hand-edit. Optional left-side phase column: omit stages: for the plain STROBE convention; include it for the PRISMA-style Identification/Screening/Inclusion/Analysis column. Pair with generate_flow_diagram.R --type strobe for the vector PDF/TIFF submission file.
CONSORT 2025Official .docx flow diagram + checklistRCT submissions to journals that mandate the consort-spirit.org template.
STARD 2015Official .pdf flow diagram + .docx checklistDiagnostic accuracy studies; flow diagram is fixed PDF, checklist is editable.
SPIRIT 2025Official .docx participant timeline + checklistTrial protocols.

Refresh / fill workflow:

# Refresh from canonical sources (CC-BY 4.0 / public-statement licenses)
bash ${CLAUDE_SKILL_DIR}/scripts/fetch_official_templates.sh

# Build PRISMA 2020 .pptx (one-time; site blocks programmatic .docx fetch)
python3 ${CLAUDE_SKILL_DIR}/scripts/build_prisma2020_template.py \
    --variant new \
    --out ${CLAUDE_SKILL_DIR}/templates/official/prisma2020/PRISMA_2020_flow_new_v1.pptx

# Fill counts — positional 10-tuple matching most SR/MA workflows:
#   n_db, n_dup, n_screened, n_screen_excluded,
#   n_sought, n_assessed, n_excl_r1, n_excl_r2, n_excl_r3, n_studies
python3 ${CLAUDE_SKILL_DIR}/scripts/fill_prisma_template.py \
    --template ${CLAUDE_SKILL_DIR}/templates/official/prisma2020/PRISMA_2020_flow_new_v1.pptx \
    --counts "315,122,186,7,111,204,102,84,3,15" \
    --out fig1_prisma_filled.pptx

# Or use full JSON mapping for studies with non-standard PRISMA splits
python3 ${CLAUDE_SKILL_DIR}/scripts/fill_prisma_template.py \
    --template ${CLAUDE_SKILL_DIR}/templates/official/prisma2020/PRISMA_2020_flow_new_v1.pptx \
    --counts-file my_counts.json \
    --out fig1_prisma_filled.pptx

# STROBE — parametric single-script builder (cohort study; spine structure varies per study).
# YAML schema: stages, spine (id/stage/text), exclusions (after/text). Consecutive same-stage
# rows share one phase label automatically.
python3 ${CLAUDE_SKILL_DIR}/scripts/build_strobe_template.py \
    --config figures/figure1_strobe.yaml \
    --out    figures/figure1_strobe.pptx

The STROBE builder checks that the exclusion cascade closes: for every link that declares an exclusion, the spine box count minus the exclusions after it must equal the next spine box (A - Σ(exclusions after A) == B). It warns loudly on any imbalance and, with --strict-cascade, refuses to build — catching figure arithmetic drift (a dropped exclusion leaving the figure short of the analytic N) that text and prose gates miss. Run python3 ${CLAUDE_SKILL_DIR}/scripts/_strobe_cascade.py --config figure1_strobe.yaml --strict to check a config without rebuilding the diagram. A config in neither schema exits 2; under --strict, a spine config with no checkable exclusion link also exits 2 rather than reporting OK.

Known limits: the generate_flow_diagram.R nodes/edges schema is recognised but not evaluated; the helper prints NOT_ASSESSED and exits 0. Reading which box a dashed exclusion is subtracted from was tried and flagged correct diagrams (an exclusion beside the box just before an exposed/unexposed or index-positive/negative split, when the previous step has its own exclusion), so that finding is left open. Check every dashed exclusion against its adjacent boxes by eye.

For STROBE the canonical KJR/Radiology/BMJ submission flow is:

  1. Render the vector submission file via the auto-fitting Graphviz path: Rscript ${CLAUDE_SKILL_DIR}/scripts/generate_flow_diagram.R --type strobe --config figures/figure1_strobe_graphviz.yaml --out figures/figure1
  2. Build the editable PowerPoint companion via build_strobe_template.py so co-authors and senior reviewers can adjust prose/positioning before sign-off.
  3. Re-export the final PPTX to PDF/TIFF only after co-author edits are integrated.

See templates/official/NOTES.md for licenses, attribution, and refresh notes.


Style Rules

Palettes, font sizes, panel layouts, and caption format are in figure_specs.md. In addition:

  • All figure text (labels, legends, annotations) is in English; medical terminology is always English.
  • Never use red-green only distinctions. Use line style (solid, dashed, dotted) in addition to color for line plots, and marker shape in addition to color for scatter plots.
  • Panel labels: uppercase bold letter (12 pt), top-left of each panel.
  • No figure titles in the plot itself (the title goes in the caption).
  • Align multi-panel figures on a grid, with consistent axis ranges across comparable panels.
  • Significance stars (* p<0.05, ** p<0.01, *** p<0.001) go above comparison brackets; report the exact p-value in the figure legend or caption, not in the plot.
  • For AUC, correlation, or agreement: display in the legend with 95% CI.

Skill Interactions

WhenCallPurpose
Need statistical values for plot/analyze-statsGet computed values (AUC, CI, p-values)
Flow diagram for manuscript/write-paper Phase 2Coordinate with Tables & Figures plan
Caption review/write-paper Phase 7Final polish pass

Manage-Project Skill -- Research Project Management

Commands

/manage-project init {name} --type {type} --journal {journal} [--ssot] [--zotero-collection NAME]

Parameters:

  • {name} -- Project identifier (e.g., nnunet-skull-fracture, rfa-meta-analysis)
  • --type -- Paper type: original | meta | case | animal | technical | ai_validation | letter
  • --journal -- Target journal: RYAI | AJR | Radiology | European_Radiology | KJR | INSI | AJNR | generic
  • --ssot -- Emit SSOT.yaml (schema v1) from templates/SSOT.yaml.template instead of legacy project.yaml. Required, but not sufficient, for Phase 1C auto-enforce (the PostToolUse verify-refs hook blocks instead of warns): enforce also needs qc/migration_complete, which init does not write. After init, run migrate_project_to_ssot.py --project-root {target_dir} --mark-complete to validate the SSOT.yaml and write the marker. New projects should pass --ssot; legacy in-flight projects stay on project.yaml until /manage-project migrate-ssot is run.
  • --zotero-collection NAME -- Optional. Create a Zotero collection via pyzotero and populate library_id + collection_key in the contract. Requires ZOTERO_API_KEY + ZOTERO_LIBRARY_ID (optionally ZOTERO_LIBRARY_TYPE, default user). With pyzotero or the credentials missing, the contract is scaffolded with library_id: null / collection_key: null and a WARN is printed.

SSOT template substitutions: {{PROJECT_ID}} → {name}; {{PROJECT_TYPE}} → the SSOT project_type enum mapped from --type (original → original_research, meta → meta_analysis, case → case_report, ai_validation → ai_validation, else other). Without --zotero-collection, library_id / collection_key stay null until the owner links an existing collection.

Run scripts/init_project.py:

python3 "${CLAUDE_SKILL_DIR}/scripts/init_project.py" \
    --name {name} --type {type} --journal {journal} [--ssot] \
    --project-root {target_dir}

It writes the contract file (SSOT.yaml with --ssot, else legacy project.yaml), the directory scaffold, the minimal stubs scripts/validate_project_contract.py requires (manuscript/index.qmd, artifact_manifest.json, qc/status.json), the memory-file stubs, and project_state.json. qc/migration_complete is NOT written by init — that marker belongs to the migrate script. For an --ssot project, run it with --mark-complete (no project.yaml needed): it validates the existing SSOT.yaml and writes the marker only on PASS. Until then the reference hook stays in warn mode.

Do not hand-build the scaffold. The script is the source of truth for the tree and for project_state.json; a hand-built one drifts from what the contract validator expects. Read ${CLAUDE_SKILL_DIR}/references/init_scaffold.md only when you need to know where a scaffolded file lands or what a project_state.json field means.

After the script, copy the matching reporting-guideline checklist from ${CLAUDE_SKILL_DIR}/../check-reporting/references/checklists/ into references/checklist_{GUIDELINE}.md.

/manage-project migrate-ssot [--no-mark-complete]

Converts a legacy project.yaml project into SSOT.yaml form and, by default, touches qc/migration_complete so Phase 1C auto mode switches from warn to enforce. On an SSOT-native project (SSOT.yaml present, no project.yaml — e.g. init --ssot) there is nothing to convert: --mark-complete validates the existing SSOT.yaml and writes the marker on PASS; SSOT.yaml is not rewritten.

python3 "${CLAUDE_SKILL_DIR}/scripts/migrate_project_to_ssot.py" \
    --project-root . --write --mark-complete
  • --no-mark-complete: run with --write only. Use it when the project still has open QC failures — enforcement is deferred until the migration is validated.
  • The migrate script refuses to touch qc/migration_complete unless the generated SSOT.yaml passes validate_project_contract.py AND contract_mode=ssot. Do not touch qc/migration_complete manually.

Re-run after resolving failures; the script is idempotent.

/manage-project status

Report current progress: read project_state.json, scan the existing files, and check whether the project memory files are present and aligned. Take word-count limits from the target journal's profile, ${CLAUDE_SKILL_DIR}/../write-paper/references/journal_profiles/{JOURNAL}.md, not from memory. Lay the report out as in ${CLAUDE_SKILL_DIR}/references/status_output_format.md.

/manage-project sync-memory

Audit and refresh the project memory files:

  • check for PROJECT.md, STATUS.md, CLAIMS.md, DATA_DICTIONARY.md, ANALYSIS_PLAN.md, REVIEW_LOG.md
  • identify stale or contradictory project metadata
  • propose the minimum files to create or update; create them from ${CLAUDE_SKILL_DIR}/references/scaffold_templates.md
  • align project_state.json with the current manuscript phase

/manage-project checklist

Read the pre-submission checklist from ${CLAUDE_SKILL_DIR}/references/pre_submission_checklist.md and write it to submission/pre_submission_checklist.md. Then recommend /self-review (manuscript quality gate), /verify-refs (citation verification) and AI pattern removal (/write-paper Phase 7).

/manage-project timeline {submission_date}

Generate a backwards timeline from the submission date, in the layout of ${CLAUDE_SKILL_DIR}/references/timeline_example.md.


Project State Conventions

Phase numbers:

  • 0 = Init (scaffold created)
  • 1 = Outline approved
  • 2 = Table/Figure shells approved
  • 3 = Methods (critic >= 85)
  • 4 = Results (critic >= 85)
  • 5 = Discussion (critic >= 85)
  • 6 = Introduction + Abstract (critic >= 85)
  • 7 = Polish complete (AI pattern removal + checklist + self-review >= 80)

Phase status values: pending | in_progress | complete | blocked

Manage-Refs Skill

Pick the tool from the Decision Tree; do not invent a parallel pipeline. References are never hand-typed: the list is always rendered by pandoc citeproc + journal CSL or by the Zotero Word plugin (CWYW). This skill writes (renders, injects, converts); bibliographic correctness against PubMed/CrossRef stays in /verify-refs — one read-only audit, one writer.

Decision Tree

SituationToolWhy
Validate [@bibkey] ↔ refs.bib (UNDEFINED / UNUSED keys)scripts/check_citation_keys.pyHard build gate, runs in seconds
Single-author submission lockdown, frozen outputscripts/render_pandoc.sh -j <journal>Reproducible, CI-friendly
Cascade rejection (e.g., ER → JVIR → CVIR)render_pandoc.sh with new -jCSL swap reformats references in seconds
Verify a journal CSL renders the in-text format / DOI / journal-name style the author guide actually requiresscripts/check_csl_render.py --csl <x>.csl --bib refs.bib --journal <key> --md manuscript.mdA stub/"dependent" CSL inherits its parent's format, which may differ from the guide (parenthetical vs superscript, DOI kept, full journal names). --md judges only the keys pandoc reads as cited (nocite: '@*' = whole bib); a preprint server named as the journal (journal = {arXiv}) is exempt from the abbreviation check; with no --journal/--expect-* the run is NOT_ASSESSED (exit 2), never PASS. Spec keys and what is not checked: references/REFERENCE_STYLE_SPECS.md. Run BEFORE submission, not after the proof PDF
Reference list prints FULL journal names but the journal wants NLM abbreviationsscripts/fill_journal_abbrev.py --bib refs.bib --out refs_abbrev.bibWrites shortjournal from PubMed (PMID, else DOI → PMID), then CrossRef short-container-title, named per entry (often ISO 4 with periods, not NLM: check those). Inserts after the journal field in the entry's own indentation, never overwrites, re-parses before writing; exit 1 = some lookups failed. In a Better BibTeX-managed bib, fill Zotero's "Journal Abbr" instead
Reviewer revision: add 1–2 refs to a Word doc with co-authors liveZotero Word plugin (user GUI)Minimal disruption to track-changes flow
Reviewer revision: bulk reference changeEdit markdown SSOT, re-run render_pandoc.shConsistency, no cherry-pick risk
Migrate [N] numeric markers → [@key] for pandocscripts/md_marker_convert.py --to-keysMapping-driven, partial conversion safe
Convert [@key] → [N] for round-trip / debugscripts/md_marker_convert.py --to-numbersSame map, opposite direction
Wire native Zotero CWYW field codes into a .docx (live Refresh in Word)scripts/inject_zotero_cwyw.pyCo-author Word workflow, post-circulation editability
Manuscript ↔ rendered DOCX cross-reference QCscripts/check_xref.py --strictSubmission gate (P0 blocker on mismatch)
Figures/tables submitted as separate attachments (radiology, most medical journals)check_xref.py --strict --allow-separate-attachmentsDowngrades MISSING_DOCX to WARN; MISSING_BODY/MISMATCH remain P0
v_(N+1) docx build-time regeneration checkcheck_xref.py --vN-docx-md5 <prev>.docx [--vN-md <prev>.md]Identity = unmodified seed copy; missing diff lines = body not regenerated
Duplicate bibliography in the built artifactscripts/check_reference_duplication.py --docx <built>.docx (or --text <rendered>.md)DUP_REF_HEADING / REF_NUMBER_RESTART / REF_SIGNATURE_DUP (Major). Catches a hand-typed ## References list plus the pandoc --citeproc auto-bibliography, which renders two lists (the second often after the legends). Run after any citeproc build
Publisher markup in a .bib title (renders as garbage)scripts/check_bib_title_markup.py --bib refs.bib --strictCrossRef titles carry <scp>WHO</scp> / <i>IDH</i>; BBT escapes them ({$<$}scp{$>$}) or strips them without restoring the space (andTERTPromoter). verify_refs proves the reference is true; this proves it will print. TITLE_MARKUP / TITLE_FUSION (Major)
Master pre-submission gate (recommended before any submission)scripts/pre_submission_gate.shChains check_citation_keys → check_bib_title_markup → verify_refs --strict → render_pandoc (optional) → check_xref --strict; single artifact qc/pre_submission_gate.json
Direct render with a built-in reference auditscripts/render_pandoc.sh (audits the .bib via /verify-refs first; blocks on FABRICATED/MISMATCH/duplicates; reports UNVERIFIED rows as not clean, without blocking; after the render, warns about cited journal entries without shortjournal, which the style prints in full)Best-effort (skips with a warning if /verify-refs is not alongside); opt out with -S. The master gate passes -S because it runs verify_refs in its stage 3
Bibliographic audit against PubMed / CrossRefdelegate to /verify-refsAudit-only — keep writer/auditor separation

Workflows

A. Pandoc citeproc (default for solo authors and final submissions)

User provides manuscript.md with [@bibkey] citations + refs.bib.

  1. Gate: python3 "${CLAUDE_SKILL_DIR}/scripts/check_citation_keys.py" manuscript.md refs.bib — exits non-zero on UNDEFINED keys. Fix and re-run. A [@NEW:topic_slug] drafting placeholder from /write-paper is reported as UNDEFINED too: resolve each by adding the citation to Zotero (/lit-sync then refreshes refs.bib) and replacing the placeholder with the real [@bibkey]. Never let a [@NEW:...] reach a rendered DOCX.
  2. Render:
    "${CLAUDE_SKILL_DIR}/scripts/render_pandoc.sh" \
      -j european-radiology \
      -i manuscript.md \
      -b refs.bib \
      -o manuscript_final.docx
    
    For the current inventory and what each style renders, read citation_styles/README.md — that table is the registry. render_pandoc.sh also lists what is on disk when -j names a style it cannot find, so ask the script rather than trusting a list written here. Two standing fallbacks: use radiology for RYAI and vancouver for JVIR (neither has a dedicated CSL). A cited key that is not in the .bib makes the render exit 5 and name the key: pandoc prints it as (key?) in the output and does not fail on its own.
  3. QC:
    python3 "${CLAUDE_SKILL_DIR}/scripts/check_xref.py" \
      --md manuscript.md --docx manuscript_final.docx \
      --out qc/xref_audit.json --strict
    
    Treat submission_safe: false as a halt. Route fixes by symptom — see the table in references/check_xref_symptoms.md.
  4. Audit hand-off: invoke /verify-refs for the PubMed/CrossRef audit before sign-off.

B. Zotero CWYW (co-author Word workflow)

User has a markdown SSOT and wants reviewers to edit citations directly in Word. Each reference must already exist as a Zotero item; the user supplies a [N] → ZoteroKey mapping.

  1. Convert markers:
    python3 "${CLAUDE_SKILL_DIR}/scripts/md_marker_convert.py" \
      --input manuscript.md --output manuscript_keys.md \
      --map ref_map.json --to-keys
    
    The conversion never guesses a Zotero key for a number: unmapped markers stay as [N], are reported on stderr, and make the script exit 1 (the partial output is still written; pass --allow-partial to accept it). Ranges such as [1-3] are expanded. Any bracketed number counts as a marker, so a year in brackets ([2019-2021]) is reported UNMAPPED too. Optionally stage with --active-ns 1,2,3,4,19 for a sample build first (a 5-reference sample limits the Word Refresh blast radius while debugging).
  2. Render to .docx with pandoc (workflow A) so the body has plain text [@key] markers, OR pre-build a .docx some other way that still contains plain [@key] text.
  3. Inject CWYW:
    python3 "${CLAUDE_SKILL_DIR}/scripts/inject_zotero_cwyw.py" \
      --input manuscript_keys.docx --output manuscript_cwyw.docx \
      --user-id <zotero-user-id> --keys-from keys.txt
    
    The script fetches item metadata live from the local Zotero connector (port 23119), so Zotero must be running locally; there is no web-API fallback. Any HTTP failure aborts with a non-zero exit, so a partial bibliography never reaches the user. Never invent Zotero metadata.
  4. First-build instruction (REQUIRED): the script writes an empty ADDIN ZOTERO_BIBL stub, which Word's Zotero Refresh treats as user-customized and refuses to populate. Open the output in Word → Zotero tab → Add/Edit Bibliography once. After that, Refresh keeps citations and bibliography in sync as authors edit.
  5. Surgical patches are unsafe: for ref additions in later rounds, edit the markdown SSOT and rebuild the whole .docx instead of regex-patching the post-CWYW file. Zotero's rendered [N] superscripts can collide with plain [N] markers and corrupt the field codes.

C. Cascade rejection re-render (find-journal hand-off)

User got rejected from journal A and /find-journal recommended journal B.

  1. Confirm the new CSL exists in citation_styles/ (or fetch from https://citationstyles.org/styles and drop in).
  2. Re-run render_pandoc.sh -j <new-csl> against the same manuscript.md + refs.bib.
  3. Re-run check_xref.py --strict.
  4. Re-run /verify-refs if any new references were added during the inter-journal revision.

D. Cross-reference QC only

User shipped a manuscript and a reviewer flagged a Table/Figure mismatch.

  1. Run check_xref.py --strict on the current manuscript.md + .docx.

  2. Inspect qc/xref_audit.json. Body caption is the SSOT — fix manuscript.md and rebuild, never patch the .docx by hand.

  3. See references/check_xref_symptoms.md for the MISSING_BODY / MISSING_DOCX / MISMATCH triage table.

  4. For journals that accept figures and tables as separate attachment files (European Radiology, Radiology, AJR, JVIR, KJR, and most medical journals), pass --allow-separate-attachments. It downgrades two rows, reported apart because their evidence differs:

    • MISSING_DOCX — a --docx was supplied and proved the float is not in the rendered main document. That is what a separate attachment looks like.
    • MISSING_BODY with no --docx supplied — nothing was checked. Excused on your word, printed as EXCUSED WITHOUT EVIDENCE, and counted in summary.downgraded_unchecked.

    MISMATCH stays P0. So does MISSING_BODY when the float is in the rendered DOCX — that is SSOT drift, which no attachment policy excuses.

    Run once with --docx before submitting. The flag is a declaration, not a verification; supplying the DOCX is what converts an excuse into evidence.

D'. v_(N+1) docx regeneration check (build-time companion)

When building v_(N+1) from a frozen v_N, the v_(N+1) docx MUST differ from v_N docx by content — a byte-identical copy is a silent seed-copy that will revert markdown edits at peer review. check_xref.py carries two flags for the build-time companion to the submission-time gate in /sync-submission's scripts/verify_package_integrity.py --assert-vN-docx-changed:

python3 "${CLAUDE_SKILL_DIR}/scripts/check_xref.py" \
    --md manuscript_v2.md \
    --docx manuscript_v2.docx \
    --vN-docx-md5 manuscript_v1.docx \
    --vN-md manuscript_v1.md \
    --strict
  • --vN-docx-md5 alone: MD5 identity check. Identical bytes = FAIL.
  • --vN-docx-md5 + --vN-md: additionally extracts the markdown-only diff between v_N and v_(N+1) and verifies each ≥40-char diff line appears verbatim (whitespace-normalized, case-insensitive) in the new docx body XML. Missing diff lines = body did not pick up the markdown edits.

Output records the result under vN_docx_check in qc/xref_audit.json. Either failure mode causes a non-zero exit even without --strict.

E. Master pre-submission gate (recommended end-to-end chain)

The single entry point that combines workflows A and D plus /verify-refs into one aborting chain. Use this immediately before submission or before circulating a v_N package to senior co-authors.

bash "${CLAUDE_SKILL_DIR}/scripts/pre_submission_gate.sh" \
    --md manuscript/manuscript.md \
    --bib manuscript/_src/refs.bib \
    --docx submission/<journal>/manuscript.docx \
    --allow-separate-attachments    # omit if the journal accepts inline figures/tables

Stage order (first failure aborts):

  1. check_citation_keys.py manuscript.md refs.bib — UNDEFINED / UNUSED keys
  2. check_bib_title_markup.py --strict — publisher markup / tag-strip fusion in .bib titles
  3. verify_refs.py refs.bib --strict — PubMed / CrossRef per-entry verification
  4. render_pandoc.sh -S -j <csl> -i ... -b ... -o ... — invoked only when --docx is omitted (--journal <csl>, default vancouver)
  5. check_xref.py --md ... --docx ... --strict [--allow-separate-attachments]

On success the chain writes qc/pre_submission_gate.json (plus the per-stage artifacts qc/bib_title_markup.json, qc/reference_audit.json and qc/xref_audit.json) with submission_safe: true. On any failure the JSON records the failing stage and exit code, and the script exits non-zero — do not submit until the failing stage passes.

The gate does not reimplement any check; it calls the existing scripts as subprocesses. A new check belongs in the underlying script, which the gate then picks up.

F. BibTeX author-format corruption (rendered-name check)

Entries written as author = {Surname AB and Surname2 CD} (family + initials, no comma) make BibTeX treat the last token as the family name, rendering "AB S, CD S2". Always store author = {Family, Full Given}. Concatenated initials even with a comma (Family, AB) still collapse to a single initial under CSL initialize-with, so use the full forename from PubMed efetch.

/verify-refs compares bib content against PubMed but does not see the rendered output; grep the rendered docx and the bib separately:

unzip -p out.docx word/document.xml | sed 's/<[^>]*>//g' | grep -oE "[A-Z]{2} [A-Z], [A-Z]{2} [A-Z]"   # corruption signature in output
grep -nE 'author\s*=\s*\{[A-Z][a-z]+ [A-Z]{1,3}( |\})' refs.bib                                          # no-comma source entries

Quality Gates

This skill defines three submission gates and one user approval gate:

  • Gate 1 (citekey integrity): check_citation_keys.py exits non-zero on UNDEFINED keys. The pipeline halts; the user reviews and fixes.
  • Gate 2 (cross-reference integrity): check_xref.py --strict exits 1 on any MISSING_DOCX / MISSING_BODY / MISMATCH row (a float the markdown defines but the DOCX lacks is MISSING_DOCX even when no in-text citation of it was recognised; a legend inside an HTML comment or fenced code is not rendered, so it stays UNCITED). With --docx and no python-docx it exits 2: the DOCX audit did not run. The user reviews qc/xref_audit.json and resolves before proceeding. Under --allow-separate-attachments, check summary.downgraded_unchecked as well as submission_safe: a non-zero count means rows passed without being checked.
  • Gate 3 (audit hand-off): before sign-off, the user must run /verify-refs and confirm submission_safe: true in qc/reference_audit.json. This skill never marks the bibliography audited on its own.
  • User approval gate (CWYW first build): the user must perform Word → Zotero → Add/Edit Bibliography manually after the first inject_zotero_cwyw.py build. The skill cannot automate this and warns on stderr that it is required.

Provenance

scripts/_vendor_citation_writer.py is vendored from alisoroushmd/zotero-mcp @ ed5dfb71, MIT licensed. See NOTICE.md and LICENSE.zotero-mcp.

Known Limitations

  • Webpage / non-journal item types: handled by the patched zotero_to_csl_json that fetches Zotero's native CSL-JSON; do not bypass this patch.
  • "Table 1 and 2" (singular kind word, number list): check_xref.py reads only Table 1 from it; parsing a bare number after "and" in prose would read "Table 1 and 2 patients" as a citation. A float defined in the markdown but absent from the DOCX is still blocked as MISSING_DOCX; one that is neither defined nor rendered is not seen. Write "Tables 1 and 2" or "Table 1 and Table 2".

Meta-Analysis Skill

Meta-Analysis Types

TypeRoB ToolStatistical ModelReporting Guideline
DTA (diagnostic test accuracy)QUADAS-3 (QUADAS-2 for legacy reviews)Bivariate / HSROCPRISMA-DTA
Intervention (treatment effect)RoB 2 (RCT) / ROBINS-I (NRSI)Random-effects (REML, Hartung-Knapp CI)PRISMA 2020
Prognostic factor (association of one factor with outcome)QUIPSRandom-effectsPRISMA 2020
Prediction model (model performance)PROBASTRandom-effectsPRISMA 2020
Observational (prevalence/association)NOS / JBIRandom-effectsMOOSE

If the type is ambiguous (DTA vs intervention), ask the user to clarify before proceeding.


Workflow Phases

Phase 1: Protocol Development

Goal: Produce a PROSPERO-ready protocol document. Write the protocol, extraction forms and manuscript text in English whatever language the user writes in — PROSPERO records and the target journals are English-language.

  1. Research question: PIRD (Population, Index test, Reference standard, Diagnosis) for DTA; PICO (Population, Intervention, Comparator, Outcome) for intervention.

  2. DTA only — do QUADAS-3 phases 1 and 2 now, not at risk-of-bias time. They are review-level and belong in the protocol: phase 1 states the synthesis question(s) (population, index test(s), target condition — a review may have more than one); phase 2 defines the ideal test accuracy trial for each (objective, participants, index test(s), definition of the target condition, analysis). Every later risk-of-bias and applicability judgement is made against that trial. Write the review-specific guidance for answering each signalling question here too, with clinical and methodological input, and publish it as a web appendix. Defining the ideal trial after seeing the studies is a judgement fitted to the results, not an assessment. See references/checklists/QUADAS3.md.

  3. Eligibility criteria: study design, population, index test / intervention, comparator / reference standard, outcomes (Se/Sp for DTA; effect size for intervention), and exclusion criteria with justification.

  4. Search plan: at least 3 databases — PubMed, Embase, and Cochrane CENTRAL (add Scopus / Web of Science as needed); a Boolean strategy from the PIRD/PICO components; a grey-literature plan (conference abstracts, trial registries); language restrictions and date range stated explicitly, with justification.

  5. RoB plan: tool by type (table above), at least 2 independent assessors, and the disagreement-resolution method (consensus, third reviewer).

  6. Synthesis plan: bivariate random-effects (Reitsma) or HSROC (Rutter & Gatsonis) for DTA; random-effects for intervention (Phase 6); heterogeneity, subgroup / sensitivity, and publication-bias plans.

  7. PROSPERO registration document: read ${CLAUDE_SKILL_DIR}/references/PROSPERO_template.md and follow its field guide, word limits, output format (Markdown + DOCX via pandoc), and Common Pitfalls Checklist. Save to the project's 7_Submission/ or equivalent directory.

    • Submit before the formal search, and record the date. The user submits the registration; record the initial submission date (not the later date the record is published) in the project's status file (STATUS.md, project_state.json or project.yaml, whichever the project keeps). The Phase 2 entry check looks for it. A pilot search is fine before submission; the formal search, screening and extraction, AI first-pass drafts included, are not.
    • Registration-ID format gate. A PROSPERO ID is CRD42 + 9 digits (14 characters total), e.g. CRD42024500001. Validate any ID that appears in the manuscript or registration doc with grep -oE 'CRD42[0-9]+' and assert a 14-character length / ^CRD42\d{9}$ — a 15-character ID (a stray digit) is a transcription error a reviewer will check against the live record.
    • Review-type selection. Pick the least-wrong portal review type for the actual design and state any portal constraint in the protocol. A descriptive single-arm proportion synthesis is not an "Intervention review"; choosing that type only to satisfy a portal field contradicts a later GRADE / effect-certainty statement. Whatever certainty language the protocol commits to (GRADE vs "evidence statements only") must match the manuscript verbatim — a guideline-style "we recommend" is not licensed by a descriptive review type.

Phase 2: Search Strategy

Goal: Develop and validate reproducible search strategies.

Entry check — registration before the formal search. Look for the registration submission date recorded in Phase 1 step 7. If none is recorded, stop and ask the user to choose: submit the registration now and run the search after that date, or proceed without it, in which case Phase 8 item 5 limits what the manuscript may claim. Record the choice. Screening or extraction done before submission, AI first-pass drafts included, cannot be made prospective afterwards. Under /orchestrate --e2e this stop is the REGISTRATION_PENDING halt; no autonomous default passes it.

  1. Build search blocks from the PIRD/PICO components and execute them per database with /search-lit (PubMed: MeSH + free text; Embase: Emtree + free text; further databases as the protocol specifies). Searching only: selecting records and extracting results belong to Phases 3–4, not to /search-lit's own selection and deep-read steps.
  2. Report per PRISMA-S (Rethlefsen et al. 2021, PMID:33499930): one section per database with date of search, number of results, and any limits applied.
  3. Merge and deduplicate into a single spreadsheet: by DOI first, then PMID. Save raw counts for the PRISMA flow.

Phase 3: Screening & Selection

Goal: Systematic title/abstract and full-text screening with two independent reviewers. Read references/phase3_screening_detail.md when executing a round (exclusion-code sets, AI pre-screening template and Methods boilerplate) or when a 3f/3f.5 gate fires (set algebra, reconciliation table).

3a. Round 1 — title/abstract (single reviewer). Define the exclusion codes from the protocol. Mark every record INCLUDE / EXCLUDE / MAYBE with a reason code → round1_{date}.tsv.

3b. Round 2 — dual independent title/abstract. A second independent reviewer (or AI as a documented second-pass tool with human verification) re-screens all R1 records. Report Cohen's κ in Methods. round2_tag = INCLUDE / EXCLUDE / MAYBE (MAYBE = disagreement or either reviewer flagged uncertainty), plus round2_reason.

3c. Round 3 — adjudication (first reviewer). MAYBE records first, then INCLUDE records for a brief confirmation pass → round3_decision (plus round3_reason only when overturning R2). Optional AI pre-screening may compress the effort, but AI suggestions are not decisions: the reviewer independently confirms or overturns every one.

3d. Round 4 — full text (/fulltext-retrieval) for round3_decision = INCLUDE: full-text exclusion codes, two independent reviewers, Cohen's κ, consensus or a third reviewer. Flag comparative studies for priority extraction.

3e. PRISMA flow. Track counts at every stage (R1 → R2 → R3 → R4 → final included); draw it with /make-figures once final.

3f. Post-consensus count reconciliation gate (MANDATORY before Phase 5 write-up). Reconcile counts from the raw ID sets, never from prose summaries, into one source-of-truth file:

python "${CLAUDE_SKILL_DIR}/scripts/screening_reconcile.py" \
  --screening 2_Screening/fulltext_screening.tsv \
  --consensus 2_Screening/consensus_decisions.tsv \
  --table1 6_Tables/table1_studies.csv \
  --output 2_Screening/screening_consensus.json

Downstream stages consume screening_consensus.json for counts and ID sets; the Markdown consensus document remains the human explanation. Three hard rules:

  1. List the narrative-only IDs explicitly. The highest-yield red flag is a numeric claim ("10 narrative-only studies") that does not match the enumerable set (A ∪ C) \ B \ T.
  2. No "N → M" transition without ID receipts. "k rose from 30 to 32 after FLAG consensus" must cite the added/removed IDs. A transition claim with no enumerable ID set is a P0 and blocks the Phase 5 hand-off.
  3. STAGE_TRANSFER_LOSS is a P0. Exit 1 when a record is included at screening but absent from the consensus artifact altogether — no adjudication was ever recorded. An exclusion is a decision; silence is a gap. Never let it settle into narrative-only.

Two more gates at 3f; a non-zero exit from either blocks the Phase 5 write-up:

# every applied exclusion code vs the *registered* eligibility criteria
python3 ${CLAUDE_SKILL_DIR}/scripts/check_exclusion_code_validity.py --protocol 0_Protocol/protocol.md --screening 2_Screening/*.tsv --strict
# DI-6: PRISMA numbers on 5 surfaces vs YAML SSOT; re-run on every revision touching PRISMA numbers
python3 ${CLAUDE_SKILL_DIR}/scripts/prisma_5way_consistency.py --ssot prisma.yaml

Exclusion-code verdicts: CODE_CONTRADICTS_ELIGIBILITY (a code excludes a design the protocol includes — bulk study loss no other gate can see), CODE_NOT_REGISTERED, CODE_RENUMBERED. Verdicts, NOT_ASSESSED cases and known limits of all 3f gates: references/phase3_screening_detail.md §3f.

  • Known limit: PRISMA 5-way still skips after_dedup → full_text_assessed; optional prisma2020: counts add report-level identities (§3f).

3f.5 Pool composition lock (MANDATORY at adjudication freeze). Once 3f passes, freeze the pool into a single source-of-truth YAML that every downstream artifact can be checked against:

cp "${CLAUDE_SKILL_DIR}/templates/FINAL_POOL_LOCK.yaml.template" 2_Data/FINAL_POOL_LOCK.yaml
# fill counts + UID lists from 3f, compute the SHA-256 over the sorted UID list,
# and COMMIT THE LOCK before any Phase 4 extraction
  • Never re-derive k included from the extraction TSV at manuscript build time — always reference final_pool_n from the lock.
  • Aggregate patient/lesion totals are locked too, not just study counts. Distinguish arm-separable from both-arm rows: a study contributing one arm must not have its full-cohort count folded into a pooled total. A hand-carried headline total that does not re-derive from the locked per-study values is a P0.
  • A late post-freeze change to the pool is a formal PROSPERO amendment: file it, re-freeze as FINAL_POOL_LOCK_v2.yaml, and propagate to every artifact.

Phase 4: Data Extraction

Goal: Create standardized extraction forms and extract 2x2 or effect-size data. Read references/phase4_extraction_detail.md when building the form (DTA / intervention field lists), when an AI draft was shared, for the optional extract_assist.py suggestions (AI_SUGGESTED, a human confirms each before dta_extraction_qc.py), or when a QC flag fires.

4.0 Entry gate (MANDATORY) — pool composition lock ↔ adjudication TSV. Before any extraction work begins, confirm the round-3 adjudication TSV and FINAL_POOL_LOCK.yaml (Phase 3f.5) agree on which UIDs are included:

python "${CLAUDE_SKILL_DIR}/scripts/check_pool_consistency.py" \
    --lock 2_Data/FINAL_POOL_LOCK.yaml \
    --adjudication-tsv 2_Screening/round3_adjudication.tsv \
    --decision-col round3_decision --uid-col uid \
    --include-labels "INCLUDE,INCLUDE_MIXED" \
    --out qc/pool_consistency.json

The gate fails closed: any UID disagreement blocks extraction. Resolve by re-freezing the lock with the corrected UID set (and propagating downstream) or by correcting a mis-labelled TSV row. Do NOT proceed with a mismatch — the extraction matrix will not align with the locked pool, and the drift surfaces as a fabrication-grade red flag at peer review.

# before the first extraction row (DI-1): comparative arm rows never live in R-script comments
python3 ${CLAUDE_SKILL_DIR}/scripts/extraction_consensus_log_init.py --output 2_Data/extraction_consensus_log.md

Failure-mode cross-ref → references/data_integrity_checklist.md DI-1~DI-5 are mandatory during extraction (2x2 arm-swap, KM audit trail, methodology mismatch, PRISMA 5-way drift, single-source k).

Extraction form. Read ${CLAUDE_SKILL_DIR}/references/empirical_lessons.md before designing it. For high-impact radiology / medical AI targets use ${CLAUDE_SKILL_DIR}/templates/extraction_form_v2.md: its dual-extractor, source-page-reference, and verbatim-quote columns close the 2x2 cell-swap and cohort-overlap blind spots.

AI-drafted starting document — treat as hallucination-suspect. If a mentor or collaborator shared an AI-drafted study list, 2x2 set, or effect estimates (even flagged "for reference only"): save it with a _DO_NOT_USE_VERBATIM suffix and re-verify every N, denominator, event count, OR/CI, and author/year against the source PDF. Trust hierarchy: source PDF + own analysis stdout > the mentor's direct text > the attached AI draft — never promote a draft up that ladder.

4b. Special cases (KM reconstruction, composite exposure). When studies report outcomes only as Kaplan-Meier curves, or the intervention is a composite of techniques, load ${CLAUDE_SKILL_DIR}/references/phase4_km_composite.md for the WebPlotDigitizer → IPDfromKM procedure (cite Guyot et al. 2012, doi:10.1186/1471-2288-12-9) and the 4-path composite-exposure decision tree. Pre-specify a sensitivity analysis excluding composite-exposure studies.

Cross-verification (≥2 independent reviewers). Report inter-reviewer agreement (% or Cohen's κ) at title/abstract and full-text stages. Verify denominator consistency — the denominator may differ across outcomes within one study, so for each outcome back-calculate event ÷ denominator and confirm it reproduces the paper's reported percentage. Distinguish KM-curve estimates from raw event counts and record the data source (Table / KM / text). Log every consensus decision in 2_Data/extraction_consensus_log.md, then lock the dataset; later changes need a dated justification. If 2x2 cells are missing, suggest contacting the authors or a sensitivity analysis with imputed values.

4c. Extraction QC & cohort overlap. After dual-extractor consensus, run both before locking:

# 2x2 cell integrity: validates TP/FN/TN/FP against source-reported sens/spec (catches arm-swap)
python3 "${CLAUDE_SKILL_DIR}/scripts/dta_extraction_qc.py" \
  --input 2_Extraction/extraction.csv --tolerance 0.02 \
  --out 2_Extraction/qc/dta_extraction_qc.tsv

# cohort overlap: shared public DB / same institution+period / same first author ±2y
python3 "${CLAUDE_SKILL_DIR}/scripts/cohort_overlap_check.py" \
  --input 2_Extraction/studies.csv --enrich \
  --out 2_Extraction/qc/cohort_overlap.md

Any FLAG_SWAP / FLAG_MISMATCH requires third-reviewer adjudication before Phase 6. A confirmed flag is not resolved until the extraction form itself is edited — a flag corrected only in a review note silently re-enters synthesis, so re-run the QC and confirm zero open flags before locking. HIGH-confidence overlap pairs require a Limitations acknowledgment plus a sensitivity analysis excluding one of the pair.

Phase 5: Risk of Bias Assessment

Goal: Guide structured RoB assessment with the appropriate tool.

DTA: this phase runs QUADAS-3 phases 3–6 (flow diagram, identify the estimates to assess, assess, overall judgement). Phases 1–2 — the synthesis question and the ideal test accuracy trial — were written in Phase 1 above. If they were not, stop and write them before judging anything; they are the comparator every judgement is made against.

Select the tool by meta-analysis type (see table above), then read its checklist:

ToolChecklist File
QUADAS-3 (DTA, current)${CLAUDE_SKILL_DIR}/references/checklists/QUADAS3.md
QUADAS-2 (DTA, legacy)${CLAUDE_SKILL_DIR}/references/checklists/QUADAS2.md
RoB 2 (RCT)${CLAUDE_SKILL_DIR}/references/checklists/RoB2.md
ROBINS-I (NRSI)${CLAUDE_SKILL_DIR}/references/checklists/ROBINS_I.md
PROBAST (Prediction)${CLAUDE_SKILL_DIR}/references/checklists/PROBAST.md
NOS (Observational)${CLAUDE_SKILL_DIR}/references/checklists/NOS.md
JBI (Case Series)${CLAUDE_SKILL_DIR}/references/checklists/JBI_Case_Series.md

For AI/ML prediction models, also apply PROBAST+AI extensions.

Output: Summary table + traffic light plot (use /make-figures).

Phase 6: Statistical Synthesis

Goal: Execute meta-analysis and generate publication-ready outputs.

Failure-mode cross-ref → references/data_integrity_checklist.md DI-6/DI-7/DI-9 are the consistency gate (CSV ↔ script ↔ prose; single-source k; 3-way numeric reconciliation before Stage 4).

Always use R (packages: meta, metafor, mada); every reported estimate, CI, p-value, and sample size comes from executed code output (Phase 6b audits this). Never guess dataset column names or codings — if a mapping is uncertain, output [VERIFY: variable_name] and ask the user to confirm against the data dictionary.

Analysis familyPrimary toolKey output
DTAmada::reitsma() (bivariate)Pooled Se/Sp + SROC with confidence/prediction regions
Interventionmeta::metagen() / meta::metabin()Pooled OR/RR, τ² + I² + prediction interval, measure-matched funnel test (k ≥ 10), leave-one-out
Dual (comparative + single-arm)metabin + metapropPRIMARY vs SECONDARY per pre-specified protocol

Read ${CLAUDE_SKILL_DIR}/references/phase6_statistical_synthesis.md before running the pooled analysis — full R code templates (companion: ${CLAUDE_SKILL_DIR}/references/r_templates.md), the dual-approach decision table (comparative vs single-arm), practical cautions (method.tau, HK CI, zero-cell correction), publication-bias test power, the sensitivity-analysis menu, and error-handling rules. Write the pooled estimates, heterogeneity statistics, and k for each analysis, taken from the executed R output, to analysis/meta_analysis_outputs.json.

Three checks before the pool is written up — each is a Methods sentence, not only a setting. R and detail in the same reference:

  1. Is the event rare? A pooled event rate < 1%, or any zero-event arm, moves the analysis off the inverse-variance default onto Peto / Mantel-Haenszel without a zero-cell correction / GLMM. Inverse-variance methods including DerSimonian-Laird are to be avoided for rare events, and so are 0.5 continuity corrections with them.
  2. Why this model? Fixed vs random is a judgment about whether one common true effect exists — never derived from Cochran's Q or I². "A random-effects model was used because I² was 65%" is a reviewer catch, not a rationale.
  3. Does one study contribute several correlated effect sizes? Multiple outcomes, readers, thresholds, or time points from the same participants need one pre-specified estimate per study, a multivariate model, or robust variance estimation — not independent pooling.

Before pooling transcribed ratio CIs (and back-deriving SEs), check them (columns study, measure, estimate, lower, upper, optional ci_level in percent and ci_method); RATIO_CI_IMPOSSIBLE is Major, RATIO_CI_ASYMMETRIC (Minor) means verify the transcription or declare ci_method:

python3 ${CLAUDE_SKILL_DIR}/scripts/check_ratio_ci_symmetry.py --extraction 2_Extraction/extraction.csv --out 2_Extraction/qc/ratio_ci.json --strict

Phase 6b: Post-Analysis Source Fidelity Audit (MANDATORY)

Goal: Catch numerical hallucinations that survived the forward pipeline (CSV → .R → manuscript).

When it runs: every time Phase 6 outputs change (first draft, revision, reviewer-requested re-analysis) — including "minor" re-runs. The precedent: in a minor revision-era re-analysis, a safety outcome's arm-level events (and so its p-value) were reported direction-reversed because a Fisher matrix() was hand-typed from a misread source Table while the extraction CSV was correct. Every downstream artifact echoed the wrong number, so internal consistency checks passed; only a random back-check against the primary paper caught it.

Non-negotiable rules:

  1. No hand-typed numerical matrices when a CSV exists. Use read.csv(...) + subset / filter; never copy a 2x2 table from a paper into matrix(c(...), ...) by eye. If hand entry is truly unavoidable (e.g., text-only extraction), the matrix, c(), or data.frame line MUST carry a comment citing the exact CSV row + column OR the exact primary-source Table/Page coordinate:

    # source: data_extraction_final.csv row <N> (<first-author> <year>), cols <event_arm1>=0, <event_arm2>=1
    # verified against primary source Table <X>, page <P>
    fisher.test(matrix(c(0, 45, 1, 55), nrow = 2, byrow = FALSE))
    
  2. Comparative-arm subsets are a separate consensus-log row. When one study's arm-specific values are used in a comparative analysis while its full cohort appears elsewhere, extraction_consensus_log.md must carry an explicit row for the arm-specific values. Pooled totals and arm-specific values MUST NOT share a row.

  3. Random 3-claim back-check before closing Phase 6. After the forest/funnel/subgroup outputs stabilize, randomly sample 3 numerical claims from the draft Results and trace each back to (a) the R output log and (b) the original paper's Table/Figure. Record it in peer_review_<vN>_internal.md:

    Claim (manuscript line)R output file:linePrimary source (paper, Table/Fig, page)Match?

    A single mismatch is a P0 blocker — do not advance to Phase 7 until resolved.

  4. Revision-introduced numbers must be tagged. Any new number added after v1 — including numbers from a new comparative / subgroup / sensitivity script — MUST be wrapped inline as [VERIFY-CSV] in the manuscript until the Phase 2.5a audit in /self-review clears it.

  5. Sensitivity analyses must be recomputed on the modified data, not copied. Every reported effect size in a sensitivity / leave-one-out / erosion / alternative-model analysis (Cohen's dz/f, AUC, OR, HR, β, sens/spec, ICC) MUST be re-derived from the modified dataset. If a sensitivity-table effect size is identical to the primary analysis to two decimals across ≥4 values while the underlying means/SDs/counts differ, the recomputation may not have run (small leave-one-out shifts can round to the same value, so confirm from the script output rather than assume) and the primary values may have been transcribed — re-run the script on the modified data.

  6. A "fixed" / "resolved" audit note requires re-run evidence, not a claim. A number recorded as fixed, resolved, or corrected counts only with a timestamp and the stdout / output-file line showing the corrected value, or the commit that changed it. A bare "fixed in v10" does NOT clear the finding — re-run the script and attach the output. The outcome-denominator cross-check (/self-review Phase 2.5b, the cohort-arithmetic / pool-lock assertions) must pass against the current outputs before any "fixed" status is accepted.

Phase 7: GRADE / Certainty of Evidence

Goal: Assess certainty of the body of evidence.

DTA: GRADE-DTA — risk of bias (from QUADAS-3, or QUADAS-2 for a legacy review), indirectness (applicability concerns), inconsistency (heterogeneity), imprecision (wide CIs, small sample), publication bias. Intervention: standard GRADE.

Certainty is assessed per outcome, not once for the review. The domains resolve differently for each outcome — one pooled from 12 studies with narrow CIs and one pooled from 3 with a wide CI do not share a rating, and a single review-level "moderate certainty" sentence tells a reader nothing about the outcome they came for. Rate every outcome carried into the Summary of Findings table, and state the reason for each downgrade (which domain, why), not only the resulting label.

Output: Summary of Findings table — one row per outcome, carrying the pooled estimate with its precision alongside the certainty rating (high / moderate / low / very low).

Phase 8: Reporting & Manuscript

Goal: Generate PRISMA-compliant manuscript sections.

Failure-mode cross-ref → references/submission_package_drift.md — apply the _build.sh pattern + DO_NOT_EDIT_HERE gate when staging multi-journal submission folders.

Re-read references/empirical_lessons.md before submission.

  1. Check reporting compliance: /check-reporting with PRISMA-DTA (bundled copy: references/checklists/PRISMA_DTA.md) or PRISMA 2020, then a second, separate run over the abstract with PRISMA 2020 for Abstracts (PRISMA_2020_Abstracts.md, 12 items, its own denominator). Report that score separately: item 2 of the main checklist only defers to it, so a manuscript can satisfy all 42 main-text items and still fail most of the twelve, and folding them into one total is how they stay invisible.

  2. Write the manuscript: /write-paper with the meta-analysis type → manuscript/manuscript.md. Never generate references from memory; use /search-lit for all citations.

  3. Figures (/make-figures): PRISMA flow diagram, forest plots (paired for DTA), SROC curve (DTA), funnel plot (Deeks' for DTA — see DTA pitfalls), RoB summary (traffic light plot).

  4. Tables: characteristics of included studies; 2x2 data per study (DTA); RoB assessment results; Summary of findings / GRADE table (one row per outcome — Phase 7).

  5. The items published radiology SR/MAs most often drop — check these by hand before the compliance run. Park 2022 (Korean J Radiol; PMID:35213097) scored 24 SR/MAs (18 with meta-analysis) against PRISMA 2020, with each item's percentage on its own denominator (MA-only items out of 18), and found 24 of 42 items reported by fewer than 80%:

    PRISMA itemWhat is missingObserved
    20aFor each synthesis, a brief summary of the contributing studies' characteristics and risk of bias — not one global paragraph covering all pools0/24
    27Data availability: which of the extraction forms, extracted data, analysis dataset, and analytic code are public, and where0/24
    24a–cRegistration number, where the protocol can be read, and any amendment — an explicit "not registered" satisfies 24a0/24
    22 / 15Certainty of evidence per outcome, and the method used to assess it9%
    13f / 20dSensitivity analysis: method and result28% (5/18)
    18Risk of bias per study, shown study-by-study rather than as a pooled proportion32%
    13dRationale for the synthesis model (see Phase 6 check 2)28% (5/18)
    16bStudies that look eligible but were excluded, cited individually with the reason25%
    Abstract #3, #12Eligibility criteria and registration inside the structured abstract0/24 each

    Word the registration from the initial submission date (not the record's later publication date), never from a template: "prospectively registered" only when it predates the start of screening; "registered retrospectively" when screening or extraction had already begun; "This review was not registered" when there is no record (an explicit statement satisfies 24a). Name the registry actually used (PROSPERO, OSF Registries). The same words go into every copy of the claim: abstract, Methods, cover letter, and the portal's ethics and registration fields.

  6. Data availability statement: name what is being shared (extraction template, locked dataset, analysis code, RoB judgments) and where — repository, DOI, or supplementary file. "Available from the corresponding author on reasonable request" satisfies few journals now and no longer satisfies item 27. If a Zenodo DOI is minted post-acceptance, references/post_submission_release_ops.md covers propagating it back into this statement.

  7. Supplementary & analysis-code pre-submission gate (before Phase 9 circulation and before portal upload). Presence of the 8-file package (Empirical Lesson 5) is necessary but not sufficient — each item must also be reviewer-ready:

    • De-scaffold: strip internal-QC / tool artifacts — raw /check-reporting output ("Assessed by: ", JSON blocks, "READY FOR SUBMISSION" verdicts, action-item lists), search-development planning docs (decision logs, expected-yield estimates, [Check on execution] placeholders, version-history dev notes), and stale version stamps. Ship a clean PRISMA 2020 checklist (27-item / 42-subitem table only) and an executed-method search-strategy doc, not the working drafts.
    • Blind: remove author names/initials and sibling-project cross-references ("Designed by: ", "identical to a sibling review") — same standard as the blinded manuscript.
    • Cross-consistency: every supplementary number matches the main text — PRISMA counts, pool k/N, the Cochrane/CENTRAL search description, RoB counts.
    • Reproducible, self-contained analysis code: run it from a clean copy of the bundle. It must read the bundled locked dataset (not an out-of-bundle path), write to the working directory, and regenerate every pool in the results table. A hard-coded study-id subset that drifts from the manuscript (a pool over k=7 while the manuscript reports k=9) is a P0 — fix and re-run; never ship stale code or figures derived from it.
    • Supplementary-only review pass: the manuscript self-review does not see the supplement; mirror /self-review Phase 2.5c–2.5d (reference + cross-reference QC) over the supplementary files.
  8. Submission gates (on Phase 8 pre-submission and every journal retarget; a non-zero exit blocks submission):

    • /sync-submission SR-MA gate: the supplementary package matches all 8 files in templates/supplementary_8file_checklist.md (PRISMA, PROSPERO, search strategy, exclusion list, extraction table, per-study x per-domain RoB, subgroup forests, sensitivity / publication bias); AI Disclosure is present (cross-link /peer-review Phase 2A P8); no duplicate PMID/DOI in the cite list (/verify-refs Gate 5).
    • DI-8 tag gate — fails if VERIFY-CSV/TODO/FIXME/XXX survive in 7_Manuscript, supplement, SUBMISSION, etc.: bash ${CLAUDE_SKILL_DIR}/scripts/tag_cleanup_gate.sh
    • SPD package integrity — checksum-based drift detection between the master manuscript and the built SUBMISSION/{journal}/ folder (journal-editable files — cover letter, response, MANIFEST, DO_NOT_EDIT_HERE.md — are auto-excluded). On the first build per journal run python3 ${CLAUDE_SKILL_DIR}/../sync-submission/scripts/verify_package_integrity.py --record --journal <name>, then --verify --journal <name> before every re-submission.
    • ICMJE COI forms for every author: ${CLAUDE_SKILL_DIR}/references/icmje_coi_guide.md.

Phase 9: Co-author Circulation

Goal: Pre-submission circulation to co-authors and a senior methodologist / reviewer, with a bounded review window and a controlled attachment scope.

Trigger: Phase 8 is complete, and the draft has cleared the Phase 6b source-fidelity audit.

Summary: Reply to the prior-version email thread to preserve In-Reply-To continuity (v1 → v2 → v3 tracked in one place). Attach the manuscript body with figures inline and, for v≥2, a change summary — exclude graphical abstract, cover letter, COI forms, and supplementary until the target journal is confirmed. TO = corresponding author + one senior methodologist; CC = remaining co-authors. Set a 7-day deadline (5 business days + weekend). Ask the corresponding author for target-journal preference, reviewer candidates, and cover-letter framing.

Load-on-demand procedural detail (thread continuity, attachment scope rationale, size-to-method table, journal-undetermined framing, response-tracking log): ${CLAUDE_SKILL_DIR}/references/phase9_circulation.md.

Failure-mode cross-ref → references/review_orchestration.md RO-1~RO-5 (dual-rating completeness, defensive-tone bias audit, response-matrix numeric tracking, 2nd-reviewer availability blocking).


Phase 10: Self-Audit Recovery (v{N} → v{N+1} sprint)

Goal: When an audit uncovers a structural data or protocol-application error, withdraw the current version, rebuild, and re-circulate with a transparent audit trail.

Trigger conditions (any one):

#TriggerSource
T1Extraction CSV ↔ primary source disagreement for a cell feeding a pooled/subgroup estimate or reported proportionPhase 6b audit
T2Included/excluded study violates the pre-specified criteria on re-readProtocol review
T3Hand-typed numerical literal in the analysis script traces to a wrong valuePhase 6b audit
T4PROSPERO protocol ↔ delivered analysis disagreement on outcome, subgroup, or eligibilityProtocol ↔ analysis diff
T5Dual-reviewer consensus record ↔ locked dataset disagreement on inclusionConsensus log diff

Non-negotiable rule: if the trigger fires after Phase 9 circulation but before journal submission, withdraw the current version within 24 hours. Reviewer discovery is a strictly worse failure mode than self-withdrawal.

Sprint: read ${CLAUDE_SKILL_DIR}/references/phase10_recovery.md and run its 12 steps, from 10.1 (audit log at qc/audit_vN_to_vNplus1.md) through the PROSPERO amendment (application correction, not criteria change) and re-circulation in the Phase 9 thread to 10.12 (post-recovery loop).

Failure-mode cross-ref → references/post_submission_release_ops.md Gate 4 covers reject/revise Zenodo versioning, tag-cleanup gate, and re-target workflow (avoid "new version" misuse on re-target).


DTA-Specific Pitfalls (Always Check)

PitfallProblemSolution
Separate pooling of Se/SpIgnores correlationUse bivariate/HSROC model
Ignoring threshold effectFalse heterogeneityJudge from the SROC plot and the bivariate Se–FPR correlation (Spearman is descriptive only)
Standard funnel plot for DTAInappropriateUse Deeks' funnel plot
I-squared only for heterogeneityDoesn't capture threshold effectUse prediction region on SROC
Missing GRADECommon omission in DTA MAApply GRADE-DTA. If <4 studies, assess each domain narratively and state the limitation explicitly
Partial verification biasInflates sensitivityQUADAS-3 3.2 (target condition assessed in all participants). QUADAS-3 has no Flow & Timing domain — that was QUADAS-2
Differential verification biasDistorts both Se and SpQUADAS-3 3.3 (target condition assessed the same way in all participants)
Unevaluable results excludedBiases accuracy estimatesReport intent-to-diagnose analysis

Small Study Considerations

When the number of included studies is small (< 10):

  • Bivariate/HSROC model may not converge (warn the user when a DTA review has fewer than 4 studies) — consider univariate random-effects as fallback
  • Publication bias tests are underpowered — state this limitation
  • Subgroup/meta-regression analysis not recommended
  • Wide prediction regions expected — emphasize uncertainty in conclusions
  • Consider narrative synthesis as alternative/complement

MLLM-Eval Skill

Purpose

This skill makes an LLM / MLLM clinical evaluation defensible: a real adjudicated reference standard, faithfulness measured not assumed, clinical-efficacy metrics beyond n-gram overlap, a pretraining- contamination check, prompt-sensitivity disclosed, and a reader study where text is generated. It is model-agnostic — every check applies to a closed API and to open weights — and read-only (an advisory design/audit skill): it audits the evaluation design and specifies and routes the clinical-efficacy metrics (RadGraph-F1 / CheXbert-F1 via their published extractors) rather than running the model or computing the metrics itself.

It is the LLM/MLLM evaluation-design counterpart in the lane — an auditor that hands the specified metrics to their extractors and /analyze-stats, parallel to how /model-assessment audits an imaging model's design and computes its metrics. The reviewer-side audit of a finished manuscript uses the mllm_evaluation.md (ME0–ME8) probe via /self-review and /peer-review; this skill is the author-side harness design. It routes the reader study to /design-ai-benchmarking, the sizing to /calc-sample-size, and TRIPOD-LLM / MI-CLEAR-LLM compliance to /check-reporting.

When to use

  • You are designing or auditing an evaluation of an LLM/MLLM on a clinical task and want it to cover the axes a reviewer will check (reference standard, faithfulness, contamination, prompt sensitivity, reader study).

When NOT to use

  • AI-vs-human-expert benchmark with a rated rubric → /design-ai-benchmarking.
  • Imaging prediction/segmentation model → /model-assessment.
  • Image-to-image generative model → the image_synthesis probe.
  • Training / serving the LLM → out of scope.
  • Item-level TRIPOD-LLM / MI-CLEAR-LLM audit of a finished manuscript → /check-reporting.

Workflow

Phase 1 — Pin the task, model, comparator, decoding (ME0)

State the task (report generation / VQA / extraction-classification), the exact model + version/date (closed API or open-weights id), the decoding settings (temperature, seed, max tokens), and what the outputs are scored against.

Phase 2 — Reference standard + metrics (ME1, ME2)

Require an adjudicated expert reference (not a single unverified report or a model-derived label). For report generation, report a clinical-efficacy metric — RadGraph-F1 (Jain et al., NeurIPS 2021) or CheXbert-F1 (Smit et al., 2020), or the composite RadCliQ (Yu et al., Patterns 2023) — alongside any BLEU/ROUGE, with CIs. For VQA/classification, state the answer-matching rule and report per-class sensitivity/specificity (or precision/recall/F1) and PPV at the real prevalence, with CIs; accuracy only alongside them — at 2% prevalence, answering "negative" every time scores 98% accuracy.

Phase 3 — Faithfulness + contamination (ME3, ME4)

Add an atomic-fact faithfulness measure + a false-premise / abstention probe (MedVH, Med-HALT) — report a hallucination rate, not just accuracy. For any public benchmark (VQA-RAD, SLAKE, MIMIC-CXR- derived, MedQA), add a contamination statement: training cutoff vs benchmark release, a held-out / post-cutoff set, or a contamination probe.

Phase 4 — Prompt sensitivity + reader study (ME5, ME7)

Disclose the exact prompt(s), temperature/seed, ≥ 3 runs with variance, and a prompt-robustness check. For a deployment/utility claim, design a blinded reader study with an error taxonomy (route the rubric/IRR to /design-ai-benchmarking, ICC/κ to /analyze-stats, sizing to /calc-sample-size).

Phase 5 — Gate the plan (deterministic)

Declare the axes in eval_manifest.json (copy ${CLAUDE_SKILL_DIR}/templates/eval_manifest.json; fields and allowed values in references/eval_manifest_schema.md), then:

python3 ${CLAUDE_SKILL_DIR}/scripts/check_mllm_eval_completeness.py \
  --manifest eval_manifest.json --strict

Use "none" for an axis not done and "other:<description>" for a method not listed; any other value exits 2. --plan plan.md --task report_generation|vqa|classification still runs the older keyword check on prose (see Known limits). NGRAM_ONLY / FAITHFULNESS_MISSING / REFERENCE_STANDARD_MISSING / CONTAMINATION_UNADDRESSED / READER_STUDY_MISSING must be resolved. A classification manifest always reports CLASSIFICATION_METRICS_NOT_ASSESSED (Minor): the manifest has no metric field for it, so check per-class sensitivity/specificity and PPV at the real prevalence by eye.

Phase 6 — Hand off

Methods/Results → /write-paper; compliance (TRIPOD-LLM / MI-CLEAR-LLM) → /check-reporting; reviewer audit → /self-review (loads ME0–ME8).

Anti-Hallucination

  • Never fabricate model outputs, reference labels, or metric scores. Compute only what the supplied outputs allow; if a reference standard or outputs are missing, say so and stop.
  • Never report n-gram overlap (BLEU/ROUGE) as clinical correctness — pair it with a clinical-efficacy metric, and flag the n-gram score for what it is.
  • Never claim "no contamination" without a stated check when a public benchmark is used.
  • If a metric (RadGraph-F1 / CheXbert-F1) or its extractor is uncertain, flag [VERIFY] and ask rather than inventing a number.

Deterministic gate

scripts/check_mllm_eval_completeness.py — task-aware presence gate on the evaluation plan (stdlib, network-free). Reproducible challenge: bash ${CLAUDE_SKILL_DIR}/scripts/mllm_eval_completeness_challenge/verify.sh.

Known limits. Manifest mode checks what is declared, not that the work was done; keep the manifest in step with the Methods. It also does not check that a declared metric suits the task: report-generation metrics (BLEU, RadGraph F1) declared for a VQA task are accepted without comment. Prose mode (--plan) checks that a term is present; it does not read negation or sense. "No human evaluation was performed" or "hallucination was not assessed" still counts as covering that axis, and a word used in another sense still clears it: "green" anywhere clears the clinical-metric check, "unsupported" clears faithfulness, "data leakage" from a patient split clears contamination, and "ground truth" clears the reference standard whatever it refers to. Read each cleared axis in the plan yourself before treating an exit 0 as clean.

Boundaries

mllm-eval (this skill: harness design + completeness gate, model-agnostic)
  ├─ design-ai-benchmarking (reader-study rubric / IRR)
  ├─ calc-sample-size (reader + case sizing)
  ├─ write-paper + check-reporting (TRIPOD-LLM / MI-CLEAR-LLM)
  └─ self-review / peer-review (ME0–ME8 reviewer probe)

Reference Files

  • ${CLAUDE_SKILL_DIR}/references/evaluation_axes.md — the why behind the ME2–ME7 axes: clinical-efficacy metrics beyond n-gram overlap (e.g. RadGraph-F1 / CheXbert-F1 vs BLEU/ROUGE), faithfulness & hallucination, pretraining/benchmark contamination, prompt-sensitivity & determinism, answer-matching, and the reader study — each mapped to its gate verdict. Load on demand during Phases 2–4.

Model-Assessment Skill

Assess a trained medical-imaging model (in-house, vendor or open-weights; segmentation, classification or detection) in the order the evidence is built: Part A designs and audits the validation study, Part B computes task-correct held-out metrics, Part C adds the uncertainty / OOD / abstention layer a deployment claim needs, and Part D makes an explainability analysis survive review. Run the parts the request needs (a Grad-CAM question starts at Part D), but no metric headline is reported before the Part A split gate is green. Each part ends in a stdlib gate whose verdict is reproduced from a file — never report a pass without running it.

Numbers come only from code executed on the supplied predictions, split table, or the researcher's executed UQ/XAI code; if predictions or ground truth are missing, say so and stop. Integrate MONAI / nnU-Net, MAPIE, captum, pytorch-grad-cam and pretrained OOD scorers by reference — never reimplement them, never build or train the model, never run a model on real patient data.

Elsewhere: building/training → /model-scaffold; choosing or vetting the model → /model-selection; data-stage preprocessing leakage → /imaging-data; paired model comparison / added value over a baseline / decision curves / MRMC / ICC / calibration tables → /analyze-stats (added value: incremental_value.md); AI-vs-expert reader rubric and IRR → /design-ai-benchmarking; LLM/MLLM → /mllm-eval; general validity → /design-study; classical-ML tabular calibration → /radiomics-ml; item-by-item reporting audit → /check-reporting; a finished manuscript → /self-review or /peer-review (the MD0–MD11 model_development.md probe).

Part A — Validation design

The rationale behind Phases 2–7 — the full leakage taxonomy, the internal-vs-external ladder, comparator design, run variance, test-set sizing, the reporting map — is in ${CLAUDE_SKILL_DIR}/references/validation_design.md (load on demand). Verify citations via /search-lit (confirmed DOI/PMID), else mark [UNVERIFIED - NEEDS MANUAL CHECK]; flag an uncertain CLAIM 2024 / TRIPOD+AI / Metrics Reloaded item [VERIFY] and ask.

Phase 1 — Task, intended-use horizon, analysis unit

State the task, the intended-use horizon (screening, triage, pre-procedure, post-hoc), the single headline metric the conclusion leans on, and the analysis unit (per-patient / per-lesion / per-image). Everything downstream is read against this; a per-lesion metric is never reported as per-patient.

Phase 2 — Leakage audit (run first)

Produce the emitted split-assignment table (patient_id,split) and run:

python3 ${CLAUDE_SKILL_DIR}/scripts/check_split_leakage.py \
  --splits <split_assignment.csv> --out qc/split_leakage.json --strict

PATIENT_OVERLAP (a patient in ≥ 2 partitions) and MISSING_SEED (an unreproducible split) are Major, SINGLE_PARTITION Minor — proven by set arithmetic on the ID column the gate prints as id_col (pass --id-col with the patient identifier when it is not the one; an auto-picked column whose name is not patient-level, such as image_id, gets a Minor ID_COL_NOT_PATIENT_LEVEL). A design with patient overlap is never approved. Then walk the leakage the table cannot show (Kapoor & Narayanan, Patterns 2023): preprocessing fit before the split (normalisation, resampling, foundation-model embeddings, ComBat harmonisation over the whole cohort — /imaging-data gates the declared pipeline), site / scanner / burned-in-label shortcuts, and temporal leakage (a random split where future and past coexist). The decisive question: could any value used in training have been computed only with knowledge of a test case?

Phase 3 — Validation tier

Classify honestly: apparent → internal random split → cross-validation → temporal → geographic / external (different site, scanner, vendor) → multi-site external. Cross-validation and bootstrap are development-time optimism corrections, not external validation. Flag a generalisability or deployment claim that outruns the design, and "external validation" where the single external set was used for tuning. Confirm the test set was touched once — no architecture search, hyperparameter sweep, early stopping, or threshold choice read it.

Phase 4 — Comparator

Clinical-only baseline, incremental value over an existing score, or reader comparison (hand the rubric / inter-rater design to /design-ai-benchmarking).

Phase 5 — Test-set sizing

Count events per class in the test set, not the cohort total: a sparse positive set gives a CI spanning much of the usable range, and calibration needs roughly ≥ 100 events. Hand formal sizing to /calc-sample-size.

Phase 6 — Prospective evaluation and deployment-monitoring horizon

Retrospective external validation shows accuracy transfers, not that the model is safe and useful in the workflow. For a clinical-use claim design the higher tier explicitly — silent / shadow deployment → prospective comparative or impact study / RCT on a clinical endpoint → post-deployment monitoring with recalibration-or-withdrawal triggers and subgroup audit (references/validation_design.md §2b). Scope the claim to the tier reached: a retrospective external study never claims deployment readiness or outcome benefit.

Phase 7 — Reporting-guideline fit

Map via /check-reporting: CLAIM 2024 (diagnostic imaging AI), TRIPOD+AI (prediction model), STARD-AI (diagnostic accuracy), PROBAST+AI (risk of bias), and for a prospective/live evaluation DECIDE-AI or CONSORT-AI / SPIRIT-AI.

Part B — Held-out metrics

Phase 8 — Compute task-correct metrics

Generate and execute evaluation code on the held-out predictions (Metrics Reloaded — Maier-Hein & Reinke et al., Nat Methods 2024):

  • segmentation — Dice/IoU and a boundary metric (HD95 / NSD), per structure, 95% CIs by patient-level bootstrap (resample patients, not pixels or slices);
  • classification — AUROC and AUPRC with patient-level bootstrap CIs, sensitivity/specificity, and PPV/NPV at the deployment prevalence, never bare accuracy on a balanced set. Report AUPRC with the test-set prevalence, which is its no-skill value: AUPRC moves with prevalence, so a value from an enriched or case-control test set does not carry over to deployment or across datasets. For multiclass, state the aggregation (one-vs-rest / macro / micro / pairwise / Obuchowski);
  • detection — FROC / mAP with the IoU match criterion stated. Lesions and false positives cluster within patients, so CIs come from a patient-level bootstrap (resample patients, carrying all their lesions and false positives), not a Wilson/binomial interval over lesions, which is too narrow; compare two detectors' FROC curves with JAFROC (RJafroc), not per-lesion tests;
  • interactive / promptable segmentation (SAM2 / MedSAM2 / nnInteractive) — the segmentation metrics plus Dice-vs-interactions / number-of-clicks (NoC) to a target threshold, initial-vs-converged (or peak) Dice, and per-case interaction/inference time. With two arms (simulated prompting + human operator), record protocol fidelity — identical prompt types, stopping rule, target threshold, seeds — because arm-to-arm comparability is what lets the human arm validate the simulated one (human-arm design: /design-study);
  • generative / synthesis — full-reference (MSE/RMSE/PSNR/SSIM) or no-reference (SNR/CNR, visual scores) quality plus a downstream-task evaluation: image quality is not clinical utility (Park et al., Radiol Med 2024);
  • time-to-event discrimination (Harrell's C, time-dependent ROC) → /analyze-stats.

Report the headline as the point estimate with a patient-level bootstrap 95% CI over the test cases: that is the uncertainty of the test-set estimate. Seed-to-seed SD across training runs is a different quantity (training-run variability, usually smaller) — report it separately, over ≥ 5 runs, for a training-recipe or model-comparison claim, and never present it as the CI. A frozen vendor or open-weights model has no training runs to vary; its uncertainty is the test-set CI. Add calibration — for a binary risk or diagnostic output, calibration-in-the-large (intercept), the calibration slope and a flexible (loess) calibration curve, plus the Brier score; ECE only as a supplementary top-label summary for multi-class confidence, with its binning stated — and subgroup slices (the Model Card Factors). Emit results.md (metrics report) and a per-case CSV for /analyze-stats. Load ${CLAUDE_SKILL_DIR}/references/metric_guide.md for the per-task checklist and ${CLAUDE_SKILL_DIR}/references/metric_selection_grounding.md for why each pairing is required and the CLAIM 2024 fit map.

Phase 9 — Gate the metric choice

Declare the reported metrics in metrics_manifest.json (copy ${CLAUDE_SKILL_DIR}/templates/metrics_manifest.json; fields and allowed values in references/metrics_manifest_schema.md), then:

python3 ${CLAUDE_SKILL_DIR}/scripts/check_metric_reporting.py \
  --manifest metrics_manifest.json --out qc/metric_reporting.json --strict

PIXEL_ACCURACY_SEG / NO_BOUNDARY_METRIC / ACCURACY_ONLY / DETECTION_METRIC_MISSING / INTERACTIVE_NO_INTERACTION_COUNT / GENERATIVE_NO_DOWNSTREAM / CLASSIFICATION_METRIC_MISSING / SEGMENTATION_METRIC_MISSING (every Major) must be zero; the last two (no headline metric declared) are manifest-only. An off-list value exits 2; use "none" or "other:<description>". --report results.md --task <task> still runs the older keyword check on prose.

Known limits: manifest mode checks what is declared, not the reported numbers. Prose mode (--report) tests keyword presence with a short negation window: "MSD" counts as mean surface distance even when it names the Medical Segmentation Decathlon, "we did not compute the Hausdorff distance or HD95" still counts HD95, "sensitivity and specificity were not reported" or "FROC was not performed" still count as reported, a bare "map" ("saliency map") counts as mAP, and a wrapped "mean average\nprecision" is not seen.

Part C — Uncertainty, OOD and selective prediction (deployment claims)

A deployment-framed model must say what it does when unsure or off-distribution. Read ${CLAUDE_SKILL_DIR}/references/uncertainty_guide.md for method choice and the manifest schema.

Phase 10 — Choose the uncertainty method, OOD guard and abstention rule

  • Conformal (MAPIE) — prediction sets/intervals at nominal coverage; the strongest default with a calibration set. Its coverage guarantee is finite-sample but needs exchangeability, which can fail on clinical data, so measure achieved coverage on a test split disjoint from the calibration split and report it with its binomial CI — never report it as guaranteed.
  • Deep ensemble — K ≥ 2 independent members (distinct seeds/inits); shared seeds underestimate epistemic uncertainty.
  • MC-dropout — dropout active at inference, T passes; off, every pass is identical and the estimate collapses to a point prediction.
  • Bayesian / last-layer Laplace — a light option.
  • OOD guard — energy score, feature Mahalanobis, ODIN or max-softmax, evaluated on a held-out OOD set (different scanner/site/pathology) with detection AUROC and the operating point.
  • Selective prediction — abstain at a pre-specified coverage/risk target; report the risk–coverage curve. A post-hoc threshold inflates accuracy-at-coverage.
  • Under shift — report calibration/coverage on shifted or external data, not in-distribution only (Ovadia 2019).

Phase 11 — Emit and gate the uncertainty manifest

Write uncertainty_manifest.json:

{
  "task": "classification",
  "deployment_claim": true,
  "uncertainty_method": "conformal",
  "coverage_target": 0.90,
  "coverage_validated": true,
  "ood_method": "mahalanobis",
  "ood_heldout_set": "external-ood-cohort",
  "selective_prediction": true,
  "selective_target": 0.95,
  "calibration_under_shift": true
}
python3 ${CLAUDE_SKILL_DIR}/scripts/check_uncertainty_reporting.py --manifest uncertainty_manifest.json \
  --out qc/uncertainty_reporting.json --strict

Verdicts: POINT_PREDICTION_NO_UNCERTAINTY, CONFORMAL_NO_COVERAGE_VALIDATION, OOD_NO_HELDOUT_SET (Major); ENSEMBLE_NOT_INDEPENDENT, MCDROPOUT_DISABLED_AT_INFERENCE, SELECTIVE_NO_TARGET, NO_CALIBRATION_UNDER_SHIFT (Minor). It audits the declared spec; it complements, not replaces, Phase 8's executed calibration. Report TRIPOD+AI / DECIDE-AI deployment-monitoring fit via /check-reporting.

Part D — Explainability

A saliency / Grad-CAM map is the most over-interpreted artifact in imaging AI: Adebayo et al. (NeurIPS 2018) showed many methods produce convincing maps independent of the model's weights and labels. Read ${CLAUDE_SKILL_DIR}/references/explainability_guide.md for method by architecture, sanity checks, localisation metrics and framing.

Phase 12 — Produce, sanity-check and quantify the maps

Choose the method for the architecture — Grad-CAM / Grad-CAM++ for CNNs, attention rollout for ViTs, integrated gradients / SHAP for attribution — wired through captum or pytorch-grad-cam. Run the Adebayo model-parameter and data (label) randomisation tests; a faithful map degrades when they are randomised, and both axes are the minimum bar. If the map is claimed to localise the finding, compute IoU / pointing game / Dice against ground-truth masks over the cohort — not eyeballed, cherry-picked cases. Frame a map as attribution ("where signal is attributed"), never as proof the model is correct or of causation.

Phase 13 — Emit and gate the explainability report

Write explainability_report.json:

{
  "method": "grad-cam++",
  "n_examples": 200,
  "cohort_level": true,
  "localization_metric": "iou",
  "localization_value": 0.63,
  "sanity_checks": ["model_randomization", "data_randomization"],
  "interpretation": "localization"
}

interpretation: attribution / localization / faithfulness — never validation / causal.

python3 ${CLAUDE_SKILL_DIR}/scripts/check_explainability_report.py --manifest explainability_report.json \
  --out qc/explainability_report.json --strict

Verdicts: SALIENCY_AS_VALIDATION, NO_SANITY_CHECK, NO_LOCALIZATION_METRIC (Major); INSUFFICIENT_SANITY, CHERRY_PICKED_EXAMPLES, MISSING_METHOD (Minor).

Outputs and hand-off

  • Part A: validation-design decision notes (leakage, tier, comparator, metric, sizing handoff, reporting fit) and qc/split_leakage.json.
  • Part B: results.md, the per-case CSV, and qc/metric_reporting.json.
  • Part C: uncertainty_manifest.json + qc/uncertainty_reporting.json.
  • Part D: explainability_report.json + qc/explainability_report.json.

The per-case table → /analyze-stats (paired ΔAUC of frozen models on the same test patients — DeLong or bootstrap; added value over a baseline per incremental_value.md; decision curves; publication tables); figures → /make-figures; numbers and subgroup performance → /model-card; Methods/Results → /write-paper; compliance → /check-reporting; sizing → /calc-sample-size; the reviewer-side audit of the draft → /self-review, whose ai_overclaiming / image_synthesis probes also check saliency claims. Gate regression (${CLAUDE_SKILL_DIR}/): scripts/check_split_leakage_challenge/verify.sh, scripts/metric_reporting_challenge/verify.sh, scripts/check_uncertainty_reporting_challenge/verify.sh, scripts/check_explainability_report_challenge/verify.sh, and tests/test_*.sh.

Model-Card Skill

Purpose

This skill produces the documentation an engineer-built medical-imaging model must carry: a Model Card (intended use, out-of-scope use, training data, per-subgroup performance, caveats), a Datasheet for its dataset (provenance, composition, collection, labelling, consent), and a METRIC-informed data-quality pass. It fills the templates from facts the user supplies — it never invents a number, a provenance detail, a consent status, or a licence — and ships a deterministic gate that no required section is missing or left as an unfilled [NEEDS INPUT] placeholder.

It is the reporting seam of the model-engineering lane: after /model-assessment audits the design and produces the numbers, this skill records them in a portable, auditable card that /write-paper and /check-reporting consume. It mirrors /version-dataset structurally (generate + deterministic verify).

When to use

  • A trained model needs a Model Card / Datasheet for a repo, Hugging Face card, or manuscript supplement.

When NOT to use

  • Auditing the validation design / metrics → /model-assessment.
  • Versioning the dataset bytes → /version-dataset; tabular variable docs → /generate-codebook.
  • Item-by-item reporting-guideline compliance of the manuscript → /check-reporting.
  • Building / training the model → /model-scaffold.

Workflow

Phase 1 — Collect the facts

Gather, from the user / the model's developers: task + architecture + provenance + licence; intended use and out-of-scope use; training and evaluation cohorts; the reference standard and inter-reader agreement; overall and per-subgroup performance; data collection, consent, and de-identification. Anything not supplied stays [NEEDS INPUT] — never guess.

Phase 2 — Fill the Model Card

Copy ${CLAUDE_SKILL_DIR}/references/model_card_template.md to MODEL_CARD.md and fill each section from the facts. Keep the headings. Numbers come only from /model-assessment / executed results.

Phase 3 — Fill the Datasheet

Copy ${CLAUDE_SKILL_DIR}/references/datasheet_template.md to DATASHEET.md and fill the seven question groups (Motivation, Composition, Collection, Preprocessing/Labeling, Uses, Distribution, Maintenance).

Phase 4 — METRIC data-quality pass

Walk ${CLAUDE_SKILL_DIR}/references/metric_dimensions.md (completeness, correctness, consistency, representativeness, timeliness, provenance, label provenance, fairness/coverage, leakage safety) and record each finding in the Datasheet. Anything that affects the headline metric's validity is also a /model-assessment finding — cross-check there.

Phase 5 — Verify completeness (deterministic gate)

python3 ${CLAUDE_SKILL_DIR}/scripts/check_model_card_complete.py \
  --card MODEL_CARD.md --datasheet DATASHEET.md --strict

MISSING_SECTION / EMPTY_REQUIRED_SECTION / UNFILLED_FIELD must be zero before the card ships. UNFILLED_FIELD names each field of a required section still left as [NEEDS INPUT] / [VERIFY] (e.g. License or subgroup performance), even when a sibling field is filled. An explicit N/A / None counts as an answer only as a field's whole value, and never in Intended Use, Training Data, Evaluation Data, Metrics or Quantitative Analyses. Only the template tokens as written ([NEEDS INPUT ...], [VERIFY], [VERIFY: ...], upper case) count; markdown link text such as [verify the protocol](https://...) or [Verify][ref], and anything inside an HTML comment or a fenced code block, is not a placeholder. Known limits: a field left as a hand-written TODO / TBD / <...> / XXXX is caught only when the whole section is unfilled; keep the template's bracketed [NEEDS INPUT] markers for open fields.

Phase 6 — Hand off

Carry the card into /write-paper (the Methods / supplement reference it), /check-reporting (CLAIM 2024 / TRIPOD+AI item audit of the manuscript), and /self-review.

Anti-Hallucination

  • Never invent evaluation numbers, subgroup results, or dataset provenance. Every figure comes from /model-assessment or the user's executed results; every provenance / consent / licence statement is user-confirmed. Unknown → [NEEDS INPUT], which the gate flags.
  • Never mark a section complete without user-supplied content, and never auto-fill a placeholder to pass the gate.
  • Never assert a licence or consent status the user did not confirm.
  • The gate checks presence, not truth — a complete card can still contain a wrong number; validity is /model-assessment and the human's responsibility.

Deterministic gate

scripts/check_model_card_complete.py — verifies every required Model Card / Datasheet section is present and non-empty (stdlib, network-free). Reproducible challenge: bash ${CLAUDE_SKILL_DIR}/scripts/check_model_card_complete_challenge/verify.sh.

Note on classification

Model Cards (Mitchell et al. 2019) and Datasheets (Gebru et al. 2021) are documentation standards, not clinical reporting guidelines, so they live here as references/ templates (uncounted), not in /check-reporting's counted checklist set — the same way appraisal_tools/METRICS.md is kept separate. /check-reporting still owns the manuscript-level CLAIM 2024 / TRIPOD+AI item audit.

Boundaries

model-assessment (audit design + metrics)
  └─ model-card (this skill: Model Card + Datasheet + METRIC pass, completeness-gated)
       └─ write-paper + check-reporting (manuscript) ; version-dataset (dataset bytes)

/model-evaluation was renamed to /model-assessment in v6.0; this redirect is removed in v7. Run /model-assessment now, passing the same arguments the user gave /model-evaluation. Start at /model-assessment Part B, Phases 8–9 (task-correct held-out metrics and the metric-reporting gate).

Model-Scaffold Skill

Purpose

This skill stamps out a runnable PyTorch training repo for a medical-imaging task — --task segmentation (U-Net), classification (CNN / timm backbone), detection (torchvision Faster R-CNN / FPN), synthesis (Pix2Pix generator + PatchGAN), ssl (SimCLR encoder), or finetune (transfer-learning a pretrained backbone with a frozen→unfrozen schedule + a provenance record) — with the reproducibility guarantees baked in by construction — so the build is leakage-safe and reproducible before a single epoch runs. It is the imaging analogue of how /analyze-stats generates runnable statistical code: the generator produces the repo, you run the training on your GPU / Colab, and the lane's deterministic gates verify the network-free parts.

It is the missing middle link in the lane: /model-selection (choose) → model-scaffold (build) → /model-assessment (validate the split / design, compute metrics) → /analyze-stats → /write-paper + /check-reporting (publish). It integrates MONAI / nnU-Net / TorchIO (referenced in the generated requirements.txt); it does not reimplement them.

When to use

  • You have a data manifest (one row per image, with a patient/subject ID) and want a reproducible, leakage-safe starting repo for a segmentation, classification, detection, synthesis or self-supervised model.
  • You want to fine-tune a pretrained backbone (transfer learning — the common clinician workflow: a timm / MONAI / MedSAM checkpoint adapted to your collected clinical data) with the freeze schedule, discriminative learning rates, and pretrained-weight provenance recorded (--task finetune).

When NOT to use

  • Auditing an already-trained model's validation design → /model-assessment.
  • Held-out metrics / calibration / bootstrap CIs → /model-assessment then /analyze-stats.
  • Choosing the architecture for the research question → /model-selection.
  • Reimplementing MONAI / nnU-Net → out of scope (the scaffold integrates them).
  • LLM / MLLM evaluation → /mllm-eval.

Workflow

Phase 1 — Prepare the manifest

A CSV with one row per image and a patient/subject ID column (patient_id / subject_id / case_id), plus image and label path columns. The ID column is load-bearing: the split is done at the patient level off this column. IDs are compared after stripping surrounding whitespace (P01 and P01 are one patient), in the split and in the generated dataset.py alike.

Phase 2 — Generate the repo

python3 ${CLAUDE_SKILL_DIR}/scripts/scaffold.py \
  --manifest <manifest.csv> --task segmentation --out model_repo --seed 42 \
  --in-channels 1 --out-channels 1
# --task = segmentation | classification | detection | synthesis | ssl | finetune
#   (out-channels = num classes for classification/finetune, target channels for synthesis;
#    finetune uses a softmax CrossEntropy head, so it refuses --out-channels < 2 — binary = 2)
# fine-tuning a pretrained backbone (transfer learning) on collected clinical data:
python3 ${CLAUDE_SKILL_DIR}/scripts/scaffold.py \
  --manifest <manifest.csv> --task finetune --out model_repo --seed 42 \
  --out-channels <num_classes> --from-pretrained timm:resnet50.a1_in1k
#   emits PRETRAINED.md (provenance) + a frozen→unfrozen train.py with discriminative LRs;
#   record the exact pretrained source so the fine-tune is reproducible. build_model(pretrained=True)
#   raises if timm is missing (never a silent random-init stand-in); best.pt records backbone_class.
# reuse the split /imaging-data's preprocessing gate checked (do not draw a new one):
python3 ${CLAUDE_SKILL_DIR}/scripts/scaffold.py \
  --manifest <manifest.csv> --preprocessing-manifest preprocessing_manifest.json --out model_repo
#   copies its split_assignment + split_seed into splits/, reading rows with the gate's own rules
#   (patient key patient_id/subject_id/patient/id; split synonyms such as training/validation/holdout);
#   exits 2 if a manifest patient has no split there, a patient sits in two splits, a split does not
#   map to train/val/test, or split_seed is absent.

The imaging-data QC handoff is enforced, not advisory. With --preprocessing-manifest the scaffold also reads /imaging-data's gate reports (check_dataset_profile, check_preprocessing_leakage, check_normalizer_domain JSON) from <manifest dir>/qc/ and <manifest dir>/../qc/ (the manifest path is resolved first). --imaging-qc <file|dir> (repeatable) replaces that search: pointed at an empty directory, every gate is recorded NOT ASSESSED. A leakage report is skipped only when its recorded manifest has a different file name, resolves to an existing file, and no candidate path holds a byte-identical copy of the scaffolded manifest (the reason is shown); anything else — a copied project, a same-named manifest elsewhere — is read.

  • A Major claim — or any severity that is not plainly Minor/Flag, or a report whose summary.n_major exceeds its listed Majors — refuses: exit 1, nothing written, each code + report listed. Resolve it upstream and re-run the gate, or pass --ack-qc CODE='reason' once per code.
  • Minor / Flag claims never block; they are carried forward as warnings.
  • A gate with no readable report is NOT ASSESSED; an unparseable or off-shape file in qc/ is listed as UNREADABLE (stderr + record), never dropped.

All of it lands in model_repo/IMAGING_QC.md, referenced from config.yaml (imaging_qc:) and REPRODUCIBILITY.md, so training, evaluation and the Methods read what /imaging-data found. Never write an --ack-qc reason the user has not given. Without either flag the output is unchanged. This writes model_repo/ with config.yaml, model.py (the task's model — U-Net / CNN / Faster R-CNN / Pix2Pix / SimCLR encoder), dataset.py (reads the frozen split), losses.py (task-appropriate), train.py, evaluate.py, requirements.txt, REPRODUCIBILITY.md, methods_stub.md (+ IMAGING_QC.md when imaging-data outputs are given), and — the key artifact — splits/split_assignment.csv + splits/split_seed.txt. The split is patient-disjoint by construction (a deterministic group split) and the emitted code seeds every RNG, sets cuDNN deterministic, builds the training loader from the train split only, and infers under model.eval() + torch.no_grad().

Phase 3 — Verify the build (network-free)

# this skill's own training-hygiene gate
python3 ${CLAUDE_SKILL_DIR}/scripts/check_training_hygiene.py --repo model_repo --strict
# the split-leakage gate (proves patient disjointness) — owned by /model-assessment

Route the emitted splits/split_assignment.csv to /model-assessment (check_split_leakage.py --splits model_repo/splits/split_assignment.csv --strict) for the patient-disjointness proof, and (optionally, locally with torch installed) bash ${CLAUDE_SKILL_DIR}/scripts/scaffold_challenge/verify.sh to smoke the forward pass.

Phase 4 — Plug in your data and train

Implement dataset.py's _load_image / _load_label for your modality (DICOM / NIfTI / TIFF via nibabel / pydicom / tifffile / TorchIO / MONAI transforms). For production, swap model.py for MONAI UNet / SegResNet or an nnU-Net plan (see ${CLAUDE_SKILL_DIR}/references/training_guide.md). For a fine-tuning repo (--task finetune), fill PRETRAINED.md and set the freeze schedule / discriminative learning rates (see ${CLAUDE_SKILL_DIR}/references/finetuning_guide.md, which also covers MedSAM/SAM adaptation and train-only diffusion augmentation). Run python train.py (best model selected on the val split), then python evaluate.py (predictions on the test split, touched once).

Phase 5 — Validate, evaluate, publish

Hand off to /model-assessment (validation-tier + comparator + metric-selection audit; Dice + HD95/NSD with CIs) + /analyze-stats, /make-figures, and /write-paper (fill the methods_stub.md [VERIFY] placeholders) + /check-reporting (CLAIM 2024 / TRIPOD+AI). For reproducibility-safe wiring of experiment tracking (W&B / MLflow), config / data / environment versioning, and the MLOps reporting checklist, see ${CLAUDE_SKILL_DIR}/references/mlops_guide.md (a wiring + reporting reference — it points to the frameworks, it does not replace them).

Runnability — honest contract

The generated repo is runnable, but runnability is not a CI guarantee. The default gates prove the network-free properties (the emitted split is patient-disjoint + seeded; the emitted training code is hygienic) by parsing the produced artifacts — no torch is executed. A torch forward-pass smoke (build + forward shape + gradients flow + reproducible loss) is a self-skipping tier in the challenge verify.sh and a documented local command; it is never counted as CI coverage of runnability.

Anti-Hallucination

  • Never fabricate training or evaluation metrics. The scaffold emits [VERIFY] placeholders; every number must come from the user's executed run and from /model-assessment + /analyze-stats.
  • Never emit a split that is not patient-disjoint or not seed-locked. The generator does this by construction; do not hand-edit the split table to introduce overlap or remove the seed.
  • Never claim the generated repo was trained or that it achieved a result — it is a starting point the user runs.
  • If a library API, default, or architecture detail is uncertain, flag [VERIFY] and ask rather than guessing.

Deterministic gates

  • scripts/scaffold.py — the generator (stdlib + numpy; deterministic given manifest + seed); also the imaging-data QC handoff (tests/test_imaging_qc_handoff.sh).
  • scripts/check_training_hygiene.py — AST linter: all RNGs seeded, cuDNN deterministic, eval() + no_grad() inference, no training on a non-train split, and (fine-tuning) a recorded pretrained-weight provenance when pretrained weights are loaded (PRETRAINED_PROVENANCE_MISSING).
  • scripts/scaffold_challenge/verify.sh — the build → validate chain, network-free (torch tier self-skips).

Known limits of check_training_hygiene.py

  • It counts seeding / cuDNN / eval() / no_grad() only in code a run reaches from the script's import-time statements. A file with no import-time call into its own functions is entered through every public top-level function nothing references, so an uncalled public seed_everything in such a file still counts. Every decorated function or method (a click/typer command, a route, a @staticmethod) is also an entry point, so an uncalled decorated seeding helper still counts.
  • It does not check statement order: inference placed before model.eval() in the same function is not detected.
  • --repo without train.py / evaluate.py reports those checks as NOT CHECKED; with --strict it exits 2 rather than clearing them.
  • Dataset variables are resolved by one name-to-split map for the whole file (later assignment in source order wins, across functions), not per scope; a shuffled loader that combines train with val only (a train+val refit) is not flagged.

Boundaries

model-selection (choose)
  └─ model-scaffold (this skill: generate the reproducible repo)
       ├─ check_training_hygiene.py   (training-code hygiene)
       ├─ model-assessment -> analyze-stats   (split-leakage proof, validation design, metrics + CIs)
       └─ write-paper + check-reporting        (Methods stub -> compliant manuscript)

Model-Selection Skill

Two questions, in this order. Phases 1–3 answer a literature question with a stable answer: which architecture family suits the task. Phases 4–6 answer a provenance question: which concrete artifact — repository, revision, checkpoint — will be run, and what its numbers may claim. The skill writes decision notes and runs one stdlib gate; it never downloads, runs, trains or benchmarks a model, and it describes archetypes, not a live SOTA leaderboard.

Elsewhere: building the repo → /model-scaffold; profiling data and planning preprocessing → /imaging-data; validation design, metrics, uncertainty, explainability → /model-assessment; documenting a model you built → /model-card; an LLM/MLLM, including benchmark contamination in that setting → /mllm-eval; AI-vs-expert reader study → /design-ai-benchmarking.

Workflow

Phase 1 — Frame the question

State the task (classification / segmentation / detection / synthesis / transfer), the modality and dimensionality (2-D vs 3-D volume), the labelled-data scale (events / structures, not just images), label availability (lots / few / unlabelled pool), and the constraints (class imbalance, small structures, interpretability, deployment compute).

Phase 2 — Walk the decision tree, read the family card

Follow ${CLAUDE_SKILL_DIR}/references/index.md (task → constraints → default pick). It routes to one card; each gives the source paper, core idea, when-to-use, medical-imaging use, reference implementation, and the typical validation setup for that class.

CardFamilies
${CLAUDE_SKILL_DIR}/references/classification.mdResNet / DenseNet / EfficientNet / Inception / ConvNeXt / ViT / Swin / DeiT
${CLAUDE_SKILL_DIR}/references/segmentation.mdU-Net / 3-D U-Net / V-Net / Attention & Residual U-Net / nnU-Net (+ ResEnc, MedNeXt, STU-Net) / SegResNet / Swin-UNETR / Mask R-CNN
${CLAUDE_SKILL_DIR}/references/detection.mdnnDetection / Faster R-CNN + FPN / Mask R-CNN / RetinaNet / YOLO / DETR
${CLAUDE_SKILL_DIR}/references/synthesis.mdPix2Pix / CycleGAN / SPADE / diffusion (DDPM, latent) / VAE / fastMRI reconstruction
${CLAUDE_SKILL_DIR}/references/foundation_models.mdSAM / MedSAM / MedSAM2 / nnInteractive / VISTA3D / TotalSegmentator / SegVol / BiomedCLIP / DINO / MAE / SimCLR / MoCo
${CLAUDE_SKILL_DIR}/references/graph.mdGCN / GraphSAGE / GAT / GIN / BrainGNN for connectomes and population graphs (PyTorch Geometric / DGL; not scaffolded by /model-scaffold)

Never recommend an architecture for a modality or data scale it does not suit (a from-scratch ViT on a few hundred images, 2-D slices for a volumetric structure) — the decision-tree constraints exist to prevent exactly that. If asked for "the best" model, say the zoo is a curated archetype map, not a current SOTA ranking.

Phase 3 — Write the architecture decision note

Write decisions/architecture_choice.md: the task, the chosen architecture, its source paper (mandatory — the Methods cite it), the reason against the constraints, the runner-up and why not, and the matching /model-scaffold template. Never invent a benchmark number or paper claim: cite it (verify via /search-lit), or write [VERIFY] and ask. A reference without a /search-lit-confirmed DOI/PMID is marked [UNVERIFIED - NEEDS MANUAL CHECK].

Phase 4 — Write the model dossier for the concrete artifact

When a concrete candidate exists (a GitHub repo, a Hugging Face checkpoint, a paper's released weights), write model_dossier.json recording what is known; leave unknowns unstated, never guessed — the gate turns an absence into a finding.

{
  "model": "OrganSeg-3D v2.5.1",
  "source": {"kind": "github", "url": "...", "version": "v2.5.1", "commit": "abc1234"},
  "licence": {"spdx": "Apache-2.0", "verified_from": "LICENSE at commit abc1234"},
  "intended_use": "research",
  "weights": {"pretrained": false},
  "task": {"model": "3d_ct_organ_segmentation", "study": "3d_ct_organ_segmentation"},
  "reported_validation": [{"dataset": "ExampleBench", "metric": "Dice", "source": "J Ex 2021"}],
  "developed_on": ["ExampleBench"],
  "evaluation_arms": [{"name": "external", "dataset": "OtherCohort-2026"}],
  "hardware": {"claimed": "any CUDA GPU", "verified_on": "GTX 1080 Ti", "verified": true}
}

Read each field from the artifact, not from memory:

  • Licence — only from the licence file at the pinned revision, and record which file (verified_from). Never from a README badge, a model-card summary, or memory; an unstated licence is LICENCE_UNSTATED, never "probably MIT".
  • developed_on — the paper's own account of where the method was built and tuned. It is the field people skip and the one the gate needs: a method that won a challenge was tuned on it.
  • hardware.verified — true only after it actually ran. A support matrix is a different claim: a CUDA capability one compiler accepts may still be refused by another in the same stack.

Phase 5 — Gate the dossier

python3 ${CLAUDE_SKILL_DIR}/scripts/check_model_provenance.py --dossier model_dossier.json \
    --out qc/model_provenance.json --strict

Stdlib-only and network-free: nothing is fetched and no licence resolved online.

VerdictSeverityFires when
BENCHMARK_PROVENANCE_CONFLICTMajoran evaluation arm uses a dataset the model was developed or tuned on
EVAL_DATA_IN_TRAININGMajoran evaluation arm's dataset is inside the pretraining corpus
LICENCE_UNSTATEDMajorno licence recorded — not the same as a permissive one
LICENCE_INCOMPATIBLEMajornon-commercial / research-only licence under commercial or deployment intent
WEIGHTS_PROVENANCE_UNKNOWNMajorpretrained weights whose training corpus is not stated
DEVELOPED_ON_UNSTATEDMajorthe developed_on key is absent, so the conflict check could not run
EVALUATION_ARMS_UNSTATEDMajorthe evaluation_arms key is absent, so neither provenance check could run
INTENDED_USE_UNSTATEDMajorintended_use is absent, so licence compatibility could not be checked
TASK_MISMATCHMinorthe model's task is not the study's task
NO_VERSION_PINMinorno commit, tag or revision
VALIDATION_UNREPORTEDMinorno reported validation (dataset + metric + source)
HARDWARE_UNVERIFIEDMinorhardware support claimed but never executed
LICENCE_UNVERIFIEDMinora licence is named but not the file it was read from

The gate flags a relationship, not a reputation: developed_on: ExampleBench passes cleanly as long as no evaluation arm uses ExampleBench (the clean fixture proves this). Dataset names match as token sequences with a small family-alias table — MSD Task09 Spleen matches MSD, MS Cohort 2026 does not — never by substring.

An explicit empty list ("developed_on": []) states "none" and is not a finding; an absent key is. intended_use must be research, commercial or clinical_deployment, and developed_on / weights.trained_on must be lists of strings; anything else is an input error (exit 2), never a pass.

Known limits. LICENCE_INCOMPATIBLE recognises non-commercial / research-only markers in the licence string (NC, non-commercial, research-only, in SPDX or spaced form). A free-text licence that restricts use without those markers (e.g. "academic use only"), or a copyleft licence under closed deployment, is not detected: read the licence file and record the conclusion yourself.

Phase 6 — Turn a Major into a study decision

A BENCHMARK_PROVENANCE_CONFLICT rarely means abandoning the model (it is often the best-engineered option precisely because it was tuned hard). It changes what the arm may claim:

  1. Report that arm as a demonstration that the pipeline runs end to end, not as evidence the method works.
  2. Put the evidential weight on an arm whose data post-dates the model, and say so with dates.
  3. State the conflict in Methods and Limitations rather than leaving a reviewer to find it.

Never report a flagged arm as independent validation. An EVAL_DATA_IN_TRAINING arm is different in kind: it produces a training-set score and cannot be reported as validation at all. Write this arm-by-arm decision into the study record.

Outputs and hand-off

  • decisions/architecture_choice.md — the architecture decision note (Phase 3).
  • model_dossier.json — the provenance record (Phase 4); qc/model_provenance.json — the audit (Phase 5).
  • The arm-by-arm decision (Phase 6).

Carry them to /model-scaffold (instantiate the template), /model-assessment (arm and validation design, what each arm may claim, metrics), /model-card (provenance section), and /write-paper (Methods cite the source paper; Limitations state any conflict). Gate regression: bash ${CLAUDE_SKILL_DIR}/scripts/check_model_provenance_challenge/verify.sh and bash ${CLAUDE_SKILL_DIR}/tests/test_model_provenance.sh.

/model-sourcing was renamed to /model-selection in v6.0; this redirect is removed in v7. Run /model-selection now, passing the same arguments the user gave /model-sourcing. Start at /model-selection Phases 4–6 (write and gate the model dossier for the concrete repository or checkpoint).

/model-validation was renamed to /model-assessment in v6.0; this redirect is removed in v7. Run /model-assessment now, passing the same arguments the user gave /model-validation. Start at /model-assessment Part A, Phases 1–7 (validation design and the split-leakage gate).

obsidian-paper-vaultorchestratepeer-reviewpolish-languagepreprocess-imagingpresent-paperprofile-imagingpublish-skillradiomics-mlrender-pdf-docreplicate-studyreview-paperrevisesearch-litself-reviewsetup-medscisync-submissionuncertainty-imagingverify-refsversion-datasetwrite-paperwrite-protocol

Sponsors

Details

Marketplace
Aperivue/medsci-skills
Author
Aperivue
Category
Community
Keywords
medical-research, clinical-research, biostatistics, meta-analysis, figures

More from Aperivue/medsci-skills

All 9