sequenzia avatar

agent-alchemy-tdd-tools

Agent Alchemy TDD Tools — Test Driven Development tools for AI agents

version
0.2.1
GitHub stars
45

Install

/plugin install agent-alchemy-tdd-tools@agent-alchemy

Run it inside Claude Code after adding the marketplace with /plugin marketplace add sequenzia/agent-alchemy.

Contents

Discovered from source

Skills 5

Analyze Coverage Skill

Analyze test coverage for any project by running real coverage tools, parsing results into structured reports, and identifying gaps with actionable recommendations. Optionally maps coverage against spec acceptance criteria to find untested requirements.

CRITICAL: Complete ALL 6 phases. The workflow is not complete until Phase 6: Suggest Next Steps is finished. After completing each phase, immediately proceed to the next phase without waiting for user prompts.

Core Principles

  1. Real coverage data -- Run actual coverage tools (pytest-cov, istanbul/c8) via Bash. Never estimate or guess coverage percentages.
  2. Standalone operation -- This skill works on any project. It does not require the SDD pipeline, specs, or any other skill to function.
  3. Actionable output -- Every coverage gap should include a specific suggestion for what test to write, not just a percentage.
  4. Graceful degradation -- If coverage tools are not installed, detect and inform the user with install commands. If a spec is invalid, skip spec mapping and proceed with coverage-only analysis.

AskUserQuestion is MANDATORY

IMPORTANT: You MUST use the AskUserQuestion tool for ALL questions to the user. Never ask questions through regular text output.

  • Clarifying questions about project structure -> AskUserQuestion
  • Framework/tool selection -> AskUserQuestion
  • Threshold confirmation -> AskUserQuestion

Text output should only be used for presenting information, summaries, reports, and progress updates.

NEVER do this (asking via text output):

Which package should I measure coverage for?
1. src/
2. lib/

ALWAYS do this (using AskUserQuestion tool):

AskUserQuestion:
  questions:
    - header: "Coverage Target"
      question: "Which package or directory should coverage be measured for?"
      options:
        - label: "src/"
          description: "Main source directory"
        - label: "lib/"
          description: "Library directory"
      multiSelect: false

Phase 1: Detect Environment

Goal: Auto-detect the project's test framework, coverage tool, and source package.

Parse Arguments

Analyze $ARGUMENTS to extract optional parameters:

  • Project path: Directory to analyze (default: current working directory)
  • --spec <path>: Spec file for acceptance criteria mapping
  • --threshold <percentage>: Coverage threshold override (default: 80%)

Detect Project Type

Determine the project type by checking for language-specific files:

Python detection (check in order):

  1. pyproject.toml exists -> Python project
  2. setup.py or setup.cfg exists -> Python project
  3. requirements.txt exists -> Python project
  4. *.py files in source directories -> Python project

TypeScript/JavaScript detection (check in order):

  1. package.json exists -> TypeScript/JavaScript project
  2. tsconfig.json exists -> TypeScript project
  3. *.ts or *.tsx files in source directories -> TypeScript project

If both Python and TypeScript indicators are found, prefer the one with more signals. If ambiguous, prompt the user:

AskUserQuestion:
  questions:
    - header: "Project Type"
      question: "Both Python and TypeScript project files were detected. Which should be analyzed for coverage?"
      options:
        - label: "Python"
          description: "Analyze Python coverage with pytest-cov"
        - label: "TypeScript"
          description: "Analyze TypeScript coverage with istanbul/c8"
      multiSelect: false

Detect Coverage Tool

Read the coverage patterns reference for framework-specific detection:

Read: ${CLAUDE_PLUGIN_ROOT}/skills/analyze-coverage/references/coverage-patterns.md

Python: pytest-cov

Check for pytest-cov in the following locations (stop at first match):

  1. pyproject.toml -> look for "pytest-cov" in dependencies
  2. requirements.txt / requirements-dev.txt -> look for "pytest-cov" line
  3. setup.py / setup.cfg -> look for "pytest-cov" in install_requires or extras_require
  4. Run pip list 2>/dev/null | grep -i pytest-cov

TypeScript: istanbul / c8

Check package.json for coverage tool packages:

  1. devDependencies -> look for c8, @vitest/coverage-v8, @vitest/coverage-istanbul
  2. devDependencies -> look for jest (Jest has built-in istanbul coverage)
  3. Check if c8 binary is available: npx c8 --version 2>/dev/null

Detect Test Runner

Python:

  • pyproject.toml with [tool.pytest.ini_options] -> pytest
  • pytest.ini or conftest.py exists -> pytest

TypeScript:

  • vitest.config.* exists -> Vitest
  • jest.config.* exists -> Jest
  • package.json with vitest in devDependencies -> Vitest
  • package.json with jest in devDependencies -> Jest

Detect Source Package

Identify the main source package/directory to measure coverage for:

Python:

  1. Check pyproject.toml [tool.coverage.run] source setting
  2. Check .coveragerc [run] source setting
  3. Look for directories containing __init__.py
  4. Exclude tests/, test/, docs/, .venv/, venv/
  5. If multiple candidates, prompt the user

TypeScript:

  1. Check vitest.config.* or jest.config.* for collectCoverageFrom or coverage.include
  2. Default to src/ if it exists
  3. If src/ does not exist, look for the main directory from package.json main or module field
  4. If ambiguous, prompt the user

Detect Test Directories

Locate test files and directories:

Python:

# Find test directories
find . -type d -name "tests" -o -name "test" | head -10
# Find test files
find . -name "test_*.py" -o -name "*_test.py" | head -20

TypeScript:

# Find test files
find . -name "*.test.ts" -o -name "*.spec.ts" -o -name "*.test.tsx" -o -name "*.spec.tsx" | grep -v node_modules | head -20

Handle Missing Coverage Tool

If the coverage tool is not detected:

Python -- pytest-cov not found:

Coverage tool not detected.

pytest-cov is not installed. Install it with:

  pip install pytest-cov

Or add "pytest-cov" to your dev dependencies in pyproject.toml:

  [project.optional-dependencies]
  dev = ["pytest-cov"]

TypeScript -- no coverage tool found:

For Vitest projects:

Coverage tool not detected.

Install the Vitest coverage provider:

  npm install -D @vitest/coverage-v8

For Jest projects:

Jest includes istanbul coverage by default. No additional installation required.
Run with: npx jest --coverage

After presenting the install command, stop the workflow. Do not attempt to run coverage without the tool installed.

Handle No Test Files

If no test files are found in the project:

  1. Report: No test files detected. Coverage: 0%
  2. Recommend: Use /generate-tests to create an initial test suite based on your source code
  3. If a spec path was provided, report ALL acceptance criteria as untested
  4. Skip Phase 2 (Run Coverage) and Phase 3 (Parse Results) -- proceed directly to Phase 4 (Analyze Gaps) if spec provided, or Phase 5 (Generate Report) if no spec

Handle No Source Files

If no source files are found in the project path:

ERROR: No source files found in {project-path}.

The directory exists but contains no files matching supported extensions
(.py, .ts, .tsx, .js, .jsx).

Check that the project path is correct, or specify it explicitly:
  /analyze-coverage /path/to/project

Stop the workflow after this error.


Phase 2: Run Coverage

Goal: Execute the coverage tool and capture output.

Build Coverage Command

Based on the detected environment, construct the appropriate coverage command.

Python (pytest-cov):

pytest --cov={package} --cov-report=term-missing --cov-report=json --cov-branch

If specific test directories were detected:

pytest {test_dirs} --cov={package} --cov-report=term-missing --cov-report=json --cov-branch

TypeScript (Vitest with coverage):

npx vitest run --coverage --coverage.reporter=json --coverage.reporter=text

TypeScript (Jest):

npx jest --coverage --coverageReporters=json --coverageReporters=text

TypeScript (standalone c8):

npx c8 --reporter=json --reporter=text npx vitest run

Execute Coverage Command

Run the coverage command via Bash:

cd {project_path} && {coverage_command} 2>&1

Capture both stdout and stderr for analysis.

Handle Coverage Tool Failure

If the coverage command exits with a non-zero status:

  1. Check the error output for common issues:

    • Package path not matching: No data was collected -> verify --cov={package} matches the importable package name
    • Module not found: Module X was never imported -> check that the package is importable
    • Missing dependencies: Suggest pip install or npm install
    • Configuration errors: Point to the relevant config file
  2. Report the error with diagnostic info:

Coverage tool failed with exit code {code}.

Error output:
{stderr}

Possible causes:
- {diagnosis based on error pattern}

Suggested fix:
- {actionable fix suggestion}
  1. If the terminal output still contains a coverage summary (some tools return non-zero when tests fail but still produce coverage), attempt to parse it and proceed with a warning.

Phase 3: Parse Results

Goal: Parse the JSON coverage report into structured data.

Locate Report File

Python (pytest-cov):

  • Default location: coverage.json in the project root
  • Custom location: check --cov-report=json:{path} if specified

TypeScript (istanbul/c8):

  • Default location: coverage/coverage-final.json
  • Custom location: check coverage.reportsDirectory in vitest config or coverageDirectory in jest config

Read the JSON coverage report file.

Parse Python Report (coverage.py format)

Extract from coverage.json:

  1. Per-file data from the files object:

    • File path (key)
    • summary.num_statements -- total executable statements
    • summary.covered_lines -- statements executed during tests
    • summary.missing_lines -- statements not executed
    • summary.percent_covered -- coverage percentage (0-100)
    • summary.num_branches -- total branches (if branch coverage enabled)
    • summary.covered_branches -- branches taken during tests
    • missing_lines -- array of line numbers not covered
  2. Project totals from the totals object:

    • percent_covered -- overall coverage percentage
    • covered_lines / num_statements -- line coverage ratio
    • covered_branches / num_branches -- branch coverage ratio

Parse TypeScript Report (istanbul format)

Extract from coverage-final.json:

  1. Per-file data (keys are absolute file paths):

    • Compute statement coverage: covered = Object.values(s).filter(n => n > 0).length, total = Object.keys(s).length
    • Compute branch coverage: covered = sum of b[id].filter(n => n > 0).length, total = sum of b[id].length
    • Compute function coverage: covered = Object.values(f).filter(n => n > 0).length, total = Object.keys(f).length
    • Identify uncovered lines from statementMap where s[id] === 0
    • Identify uncovered functions from fnMap where f[id] === 0
  2. Project totals: Sum across all files

Group Uncovered Lines into Ranges

For readable reporting, group consecutive uncovered line numbers into ranges:

  • [15, 16, 17, 22, 23] -> lines 15-17, 22-23
  • [5] -> line 5
  • [10, 11, 12, 13, 14, 15] -> lines 10-15

Check Against Threshold

Compare coverage against the configured threshold:

  • Threshold source (in precedence order):

    1. --threshold argument
    2. .claude/agent-alchemy.local.md tdd.coverage-threshold
    3. Default: 80%
  • Per-file check: Flag files below the threshold, sorted by coverage (lowest first)

  • Project-wide check: Compare overall percentage against threshold


Phase 4: Analyze Gaps

Goal: Identify coverage gaps and, if a spec is provided, map coverage against acceptance criteria.

Identify Coverage Gaps

From the parsed coverage data, identify:

  1. Completely uncovered files: Files with 0% coverage -> P0 priority
  2. Uncovered functions/methods: Functions with 0 execution count -> P1 priority
  3. Uncovered branches: Branch paths never taken (partial coverage) -> P2 priority
  4. Files below threshold: Files with coverage below the configured threshold but above 0% -> P3 priority

Read Source Files for Context

For the top uncovered areas, read the source files to understand what the uncovered code does. This enables specific test suggestions rather than generic ones.

For each uncovered function or line range:

  1. Read the source file
  2. Identify the function name and purpose
  3. Identify input parameters and return types
  4. Note any error handling paths that are uncovered

Spec-to-Coverage Mapping (if spec provided)

If --spec <path> was provided:

  1. Validate spec path: Check the file exists. If not:

    WARNING: Spec file not found at {path}. Skipping spec-to-coverage mapping.
    Proceeding with coverage-only analysis.
    

    Skip the remainder of this section and proceed to gap suggestions.

  2. Parse the spec: Read the spec file and extract acceptance criteria by category:

    • _Functional:_ criteria
    • _Edge Cases:_ criteria
    • _Error Handling:_ criteria
    • _Performance:_ criteria
  3. Map criteria to code: For each acceptance criterion:

    • Use Grep to search for keywords from the criterion in the source code
    • Match function/class names that relate to the criterion's subject
    • Check comments or docstrings referencing the criterion
  4. Check coverage for mapped locations: For each mapped criterion:

    • Look up the file in the coverage report
    • Check if the mapped lines/functions have coverage > 0
    • Classify as:
      • TESTED: All mapped locations have coverage > 0
      • PARTIAL: Some mapped locations covered, some not
      • UNTESTED: No mapped locations have coverage, or no code mapping found
  5. Report untested criteria: List acceptance criteria where implementing code has no coverage

Generate Test Suggestions

For each coverage gap, suggest a specific test to write:

  • Uncovered function: test_{function_name}_returns_expected_output -- test with typical inputs
  • Uncovered branch: test_{function_name}_when_{condition} -- test that triggers the untaken branch
  • Uncovered error path: test_{function_name}_raises_on_{error_condition} -- test that triggers the error
  • Uncovered edge case: test_{function_name}_handles_{boundary} -- test with boundary inputs

Phase 5: Generate Report

Goal: Produce a structured markdown report summarizing coverage results.

Report Format

## Coverage Analysis Report

**Project**: {project_path}
**Framework**: {pytest-cov | istanbul/c8 | vitest}
**Threshold**: {threshold}%
**Date**: {current date}

### Overall Coverage

| Metric | Value | Status |
|--------|-------|--------|
| Line Coverage | {pct}% ({covered}/{total}) | {PASS or BELOW THRESHOLD} |
| Branch Coverage | {pct}% ({covered}/{total}) | {PASS or BELOW THRESHOLD} |
| Function Coverage | {pct}% ({covered}/{total}) | {PASS or BELOW THRESHOLD or N/A} |

### Coverage by File

| File | Lines | Branches | Status |
|------|-------|----------|--------|
| {file_path} | {pct}% | {pct}% | {PASS or BELOW} |
| ... | ... | ... | ... |

Files are sorted by coverage percentage (lowest first). Only files below threshold are shown by default.

### Coverage Gaps

{For each gap, ordered by priority (P0 first):}

**{P0|P1|P2|P3}**: `{file_path}:{function_name}` ({pct}% covered)
- Uncovered lines: {line_ranges}
- Description: {what the uncovered code does}
- Suggested test: `{test_name}` -- {brief description of what to test}

{If spec provided:}
### Spec Coverage Mapping

**Spec**: {spec_path}

| Category | Criterion | Status | Code Location |
|----------|-----------|--------|---------------|
| Functional | {criterion text} | TESTED/PARTIAL/UNTESTED | {file:function} |
| Edge Cases | {criterion text} | TESTED/PARTIAL/UNTESTED | {file:function} |
| Error Handling | {criterion text} | TESTED/PARTIAL/UNTESTED | {file:function} |

**Tested**: {n}/{total} criteria
**Partially Tested**: {n}/{total} criteria
**Untested**: {n}/{total} criteria

### Summary

- Overall coverage: {pct}% ({PASS or BELOW threshold of {threshold}%})
- Files below threshold: {count}
- Critical gaps (P0): {count}
- High priority gaps (P1): {count}
{If spec:}
- Untested acceptance criteria: {count}

Adapt Report to Results

  • High coverage (above threshold): Emphasize what is well-tested, briefly note any remaining gaps
  • Low coverage (below threshold): Lead with the critical gaps and prioritize recommendations
  • Zero coverage (no tests): Short report emphasizing the need for test creation
  • With spec mapping: Include the Spec Coverage Mapping section
  • Without spec: Omit the Spec Coverage Mapping section

Phase 6: Suggest Next Steps

Goal: Provide actionable next steps based on the coverage analysis.

Generate Recommendations

Based on the coverage analysis results, recommend specific actions:

If coverage is below threshold:

### Next Steps

1. **Generate tests for critical gaps**: Run `/generate-tests {source_file}` to auto-generate tests for uncovered areas
2. **Focus on P0 gaps first**: The following files have 0% coverage and need tests immediately:
   {list of P0 files}
3. **Run TDD cycle for new features**: Use `/tdd-cycle {feature}` to build new features test-first

If coverage is above threshold:

### Next Steps

1. **Coverage target met** -- {pct}% exceeds the {threshold}% threshold
2. **Optional improvements**: Consider adding tests for the remaining {count} uncovered branches
3. **Maintain coverage**: Run `/analyze-coverage` after significant changes to catch regressions

If spec mapping found untested criteria:

### Untested Acceptance Criteria

The following acceptance criteria from {spec_path} have no test coverage:

{For each untested criterion:}
- **{criterion}**: Generate tests with `/generate-tests {spec_path}`

Cross-Skill Recommendations

Recommend other TDD tools skills when appropriate:

  • Low coverage + spec available: /generate-tests {spec_path} to generate criteria-driven tests
  • Low coverage + no spec: /generate-tests {source_file} to generate code-analysis tests
  • Building new features: /tdd-cycle {feature} to follow RED-GREEN-REFACTOR
  • Re-analyze after changes: /analyze-coverage {project_path} to verify improvements

Error Handling

Coverage Tool Not Installed

Detect the missing tool, identify the project type, and suggest the appropriate installation command. Do not attempt to run coverage without the tool installed.

Coverage Tool Fails

If the coverage command returns a non-zero exit code:

  1. Parse the error output for known patterns (see references/coverage-patterns.md Common Issues tables)
  2. Report the error with the full output and a specific diagnosis
  3. If partial results are available (terminal output contains a summary), parse and use them with a warning

Invalid Spec Path

If --spec points to a file that does not exist:

  1. Log a warning: Spec file not found at {path}. Skipping spec-to-coverage mapping.
  2. Skip the spec mapping in Phase 4
  3. Proceed with coverage-only analysis in Phase 5

No Source Files Found

If the project path contains no recognizable source files:

  1. Report a clear error message with the path searched
  2. List the supported file extensions
  3. Suggest verifying the project path

Coverage Report File Not Found

If the JSON report file is not generated after running the coverage command:

  1. Check if the terminal output contains inline coverage data
  2. If terminal data exists, parse the terminal output as a fallback
  3. If no data at all, report the error and suggest checking coverage tool configuration

Settings

The following settings in .claude/agent-alchemy.local.md affect coverage analysis:

tdd:
  framework: auto                    # auto | pytest | jest | vitest
  coverage-threshold: 80             # Minimum coverage percentage (0-100)
SettingDefaultUsed In
tdd.frameworkautoPhase 1 (coverage tool selection)
tdd.coverage-threshold80Phase 3 (threshold comparison)

The --threshold argument overrides the tdd.coverage-threshold setting.

Error handling:

  • If the settings file does not exist, use defaults for all settings.
  • If the YAML frontmatter is malformed, use defaults and log a warning.
  • If tdd.framework is set to an unrecognized value, fall back to auto-detection.
  • If tdd.coverage-threshold is out of range, clamp to 0-100.

See tdd-tools/README.md for the full settings reference.


Reference Files

  • references/coverage-patterns.md -- Framework-specific coverage tool integration: detection, commands, JSON parsing, gap analysis, spec mapping

Create TDD Tasks Skill

Transform existing SDD implementation tasks into test-first TDD task pairs. For each implementation task, this skill creates a paired test task that must complete first, enforcing test-first development at the pipeline level.

This skill bridges the SDD pipeline (/create-tasks) and TDD execution (/execute-tdd-tasks), converting a standard task list into one where every implementation task is preceded by a failing-test-writing task.

CRITICAL: Complete ALL 8 phases. The workflow is not complete until Phase 8: Report is finished. After completing each phase, immediately proceed to the next phase without waiting for user prompts.

Core Principles

  1. Test-first at the pipeline level -- Every implementation task gets a paired test task that blocks it. Tests are written before implementation begins.
  2. Preserve existing dependencies -- TDD pairs are inserted into the existing SDD dependency chain without breaking any original relationships.
  3. Merge mode awareness -- Re-running this skill on a task list that already has TDD pairs detects and skips existing pairs instead of creating duplicates.
  4. Criteria-to-tests conversion -- Acceptance criteria from SDD tasks are converted into test descriptions for the paired test task.
  5. Minimal metadata additions -- Only tdd_mode, tdd_phase, and paired_task_id are added. All existing metadata is preserved.

Critical Rules

AskUserQuestion is MANDATORY

IMPORTANT: You MUST use the AskUserQuestion tool for ALL questions to the user. Never ask questions through regular text output.

  • Preview confirmation -> AskUserQuestion
  • Anomaly resolution -> AskUserQuestion
  • Error recovery options -> AskUserQuestion

Text output should only be used for:

  • Presenting TDD pair previews and summaries
  • Reporting completion status
  • Displaying dependency chain visualizations

NEVER do this (asking via text output):

Should I proceed with creating 12 TDD task pairs?
1. Yes
2. No

ALWAYS do this (using AskUserQuestion tool):

AskUserQuestion:
  questions:
    - header: "Confirm TDD Pair Creation"
      question: "Ready to create 12 TDD task pairs?"
      options:
        - label: "Yes, create pairs"
          description: "Create test tasks and set TDD dependencies"
        - label: "Show details"
          description: "See full list of pairs before creating"
        - label: "Cancel"
          description: "Don't create TDD pairs"
      multiSelect: false

Plan Mode Behavior

CRITICAL: This skill transforms tasks, NOT creates an implementation plan. When invoked during Claude Code's plan mode:

  • DO NOT create a plan for how to implement TDD
  • DO NOT defer task transformation to an "execution phase"
  • DO proceed with the full TDD task generation workflow immediately
  • DO create test tasks using TaskCreate as normal

The TDD task pairs are planning artifacts themselves -- generating them IS the planning activity.

Soft Dependency on sdd-tools

This skill is part of the tdd-tools plugin and works with agents in the same plugin (tdd-executor, test-writer). It bridges the SDD pipeline (/create-tasks from sdd-tools) and TDD execution (/execute-tdd-tasks). The sdd-tools plugin is expected to be installed since TDD tasks are generated from SDD tasks created by /create-tasks.


Phase 1: Validate & Load References

Goal: Verify prerequisites and load reference materials.

Step 1: Parse Arguments

Check $ARGUMENTS for optional --task-group filter:

  • If --task-group <group> is present, extract the group name for filtering
  • If no arguments provided, process all tasks

Step 3: Load Reference Files

Read the TDD decomposition and dependency reference files:

  1. references/tdd-decomposition-patterns.md -- Task pairing rules, naming conventions, metadata, merge mode detection
  2. references/tdd-dependency-rules.md -- Dependency insertion algorithm, circular dependency detection and breaking

Phase 2: Read Existing Tasks

Goal: Load the current task list and identify tasks to transform.

Step 1: Get All Tasks

Use TaskList to retrieve all current tasks.

Step 2: Apply Group Filter

If --task-group was specified:

  • Filter to only tasks where metadata.task_group matches the specified group
  • Tasks outside the group are not modified

If no --task-group specified:

  • Process all tasks regardless of group

Step 3: Validate Task List

Handle empty/missing states:

Empty task list (no tasks at all):

No tasks found. Please run /create-tasks first to generate implementation tasks from a spec.

Usage:
  /agent-alchemy-sdd:create-tasks <spec-path>

No tasks matching --task-group filter:

No tasks found for group "{group}".

Available task groups:
- {group1} ({n} tasks)
- {group2} ({n} tasks)

Try: /create-tdd-tasks --task-group {group1}

Step 4: Classify Tasks

For each task, determine if it should receive a TDD pair:

Eligible for TDD pairing:

  • Implementation tasks (subjects like "Create X", "Implement X", "Build X", "Add X")
  • Business logic tasks
  • API/endpoint tasks
  • Data model tasks
  • UI/frontend tasks

Skip (no TDD pair created):

  • Tasks already marked with tdd_mode: true in metadata
  • Test tasks (subjects like "Add tests for X", "Write tests for X", or tasks with test in task_uid)
  • Configuration/setup tasks (subjects like "Configure X", "Set up X")
  • Documentation tasks (subjects like "Document X", "Write docs for X")

Record the classification for each task: eligible, skipped (with reason).


Phase 3: Detect Existing TDD Pairs (Merge Mode)

Goal: Identify tasks that already have TDD pairs to avoid duplication.

Detection Algorithm

For each eligible task, check if it already has a TDD pair using these 4 signals (any match means paired):

  1. Metadata check: Task has tdd_mode: true in metadata
  2. Paired task check: Task has paired_task_id in metadata, and the paired task exists in the task list
  3. UID check: A task exists with task_uid equal to this task's task_uid + :red
  4. Subject check: A task with subject "Write tests for {this task's subject}" exists in the same task_group

Merge Behavior

For tasks with existing TDD pairs:

Existing Pair StatusAction
Both tasks pendingSkip -- pair already exists
Test completed, impl pendingSkip -- pair progressing normally
Test completed, impl completedSkip -- pair fully done
Test completed, impl in_progressSkip -- pair in progress
Test pending, impl completedFlag as anomaly -- impl completed without tests
Only impl exists, test missingTreat as unpaired -- create the test task
Only test exists, impl missingFlag as orphan -- ask user

Report Merge Status

If any existing pairs detected:

TDD PAIR STATUS:
- {n} tasks already have TDD pairs (will skip)
- {m} tasks need TDD pairs (will create)
- {k} anomalies detected (need user input)

If anomalies exist, use AskUserQuestion to resolve each one:

AskUserQuestion:
  questions:
    - header: "TDD Pair Anomaly"
      question: "Task #{id} '{subject}' was completed without its test task. What should I do?"
      options:
        - label: "Create test task anyway"
          description: "Add a test task for documentation/coverage purposes"
        - label: "Skip this task"
          description: "Leave it as-is without a test pair"
      multiSelect: false

Phase 4: Generate Test Tasks

Goal: Create test task definitions for each eligible unpaired implementation task.

For each eligible task that needs a TDD pair, generate a paired test task.

Step 1: Determine Test Task Subject

Follow the naming convention:

"Write tests for {original task subject}"

Examples:

  • "Create User data model" -> "Write tests for Create User data model"
  • "Implement POST /auth/login endpoint" -> "Write tests for Implement POST /auth/login endpoint"

Step 2: Infer Test File Path

Determine the test file path based on the implementation task context:

  • If implementation task references source files, derive test file path from project conventions
  • Common patterns: src/foo.ts -> tests/foo.test.ts, src/foo.py -> tests/test_foo.py
  • If no source files referenced, use the task subject to infer a path

Step 3: Detect Test Framework

Determine the test framework using project detection:

  1. Check for config files: jest.config.*, vitest.config.*, pytest.ini, pyproject.toml, setup.cfg
  2. Check existing test files for import patterns
  3. Check package.json for test dependencies
  4. Default to the most common framework for the detected language

Step 4: Convert Acceptance Criteria to Test Descriptions

If the implementation task HAS acceptance criteria (**Acceptance Criteria:** section):

Convert each criterion into a test description:

**Test Descriptions:**

_From Functional Criteria:_
- [ ] Test that {criterion rephrased as test assertion}

_From Edge Cases:_
- [ ] Test that {edge case rephrased as test assertion}

_From Error Handling:_
- [ ] Test that {error scenario rephrased as test assertion}

_From Performance:_ (if applicable)
- [ ] Test that {performance target as measurable assertion}

If the implementation task LACKS acceptance criteria:

Generate basic test descriptions from the subject and description:

  1. Parse the subject for intent ("Create X" -> test X exists and has expected structure)
  2. Extract testable statements from description ("should...", "must...", "returns...", "validates...")
  3. Generate minimal test descriptions:
**Test Descriptions:**

_Inferred from task description:_
- [ ] Test that {subject entity} can be created/initialized
- [ ] Test that {subject entity} has expected structure/interface
- [ ] Test that {described behavior} works as described

Step 5: Build Test Task Definition

Assemble the complete test task:

subject: "Write tests for {original subject}"
description: |
  Write failing tests for: {original task subject}

  Test file: {inferred test file path}
  Test framework: {detected framework}
  Original task: #{original_task_id}

  {test descriptions from Step 4}

  **Acceptance Criteria:**

  _Functional:_
  - [ ] All test descriptions converted into runnable test functions
  - [ ] Tests follow project test conventions (naming, structure, fixtures)
  - [ ] Tests are discoverable by the test runner
  - [ ] Tests fail when run without implementation (RED state)

  _Edge Cases:_
  - [ ] Tests handle import errors gracefully when implementation module does not exist

  _Error Handling:_
  - [ ] Test file is syntactically valid even when implementation is missing

  Source: {original source reference}
activeForm: "Writing tests for {original subject}"
metadata:
  tdd_mode: true
  tdd_phase: "red"
  paired_task_id: "{original_task_id}"
  priority: {inherited from original}
  complexity: {S or M -- test files are typically smaller}
  source_section: {inherited from original}
  spec_path: {inherited from original}
  feature_name: {inherited from original}
  task_uid: "{original_task_uid}:red"
  task_group: {inherited from original}

Step 6: Plan Implementation Task Updates

For each original implementation task, plan the metadata update:

metadata additions:
  tdd_mode: true
  tdd_phase: "green"
  paired_task_id: "{test_task_id}"  # Will be set after test task creation

Phase 5: Set Dependencies

Goal: Insert TDD pairs into the existing dependency chain.

Apply the insertion algorithm from tdd-dependency-rules.md:

For Each TDD Pair

Given implementation task #N with existing dependencies blockedBy: [A, B, ...]:

  1. Test task inherits upstream dependencies: Test task #T gets blockedBy: [A, B, ...] (same as original)
  2. Implementation task gains test dependency: Task #N adds #T to its blockedBy list
  3. Downstream tasks unchanged: Tasks that depend on #N continue to depend on #N

Dependency Insertion Example

Before: Model (#1) --> API (#2) --> UI (#3)

After:
  Test-Model (#4) blockedBy: []
  Model (#1) blockedBy: [#4]
  Test-API (#5) blockedBy: [#1]
  API (#2) blockedBy: [#1, #5]
  Test-UI (#6) blockedBy: [#2]
  UI (#3) blockedBy: [#2, #6]

Circular Dependency Detection

After planning all insertions, validate the full dependency graph:

  1. Build the complete graph from all tasks (original + new test tasks)
  2. Run topological sort
  3. If sort fails, a cycle exists

Breaking cycles (weakest-link strategy):

Score each dependency link in the cycle:

  • TDD pair link (test -> impl): score 1 (weakest)
  • Same-feature cross-layer: score 2
  • Cross-feature dependency: score 3 (strongest)

Remove the dependency with the lowest score. Log a warning:

WARNING: Circular dependency detected after TDD pair insertion.
Cycle: {task chain}
Broken at: {removed link}
Reason: TDD pair link is weakest (score: 1)
Impact: {explanation of what may run out of order}

Add needs_review: true and circular_dep_break: true to the affected task's metadata.


Phase 6: Preview & Confirm

Goal: Present the TDD transformation plan and get user approval.

Display Preview

Present a summary of the planned changes:

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
TDD TASK PAIR GENERATION PREVIEW
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

SUMMARY:
  Total implementation tasks: {count}
  Eligible for TDD pairing: {eligible}
  Already have TDD pairs: {skipped} (merge mode)
  New TDD pairs to create: {new_pairs}
  Tasks skipped (test/config/docs): {skipped_ineligible}

NEW TDD PAIRS:
  Test Task                              | Blocks        | Phase
  ─────────────────────────────────────────────────────────────
  Write tests for {subject1}             | #{impl_id1}   | RED
  Write tests for {subject2}             | #{impl_id2}   | RED
  ...

DEPENDENCY CHAIN (after insertion):
  {visualization of the dependency chain with TDD pairs inserted}

{If circular deps detected and broken:}
WARNINGS:
  - Circular dependency broken at: {link}
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Get Confirmation

Use AskUserQuestion to confirm:

AskUserQuestion:
  questions:
    - header: "Confirm TDD Pair Creation"
      question: "Ready to create {n} TDD task pairs?"
      options:
        - label: "Yes, create pairs"
          description: "Create test tasks and update implementation tasks with TDD metadata"
        - label: "Show task details"
          description: "See full test task descriptions before creating"
        - label: "Cancel"
          description: "Don't create TDD pairs"
      multiSelect: false

If user selects "Show task details":

  • Display each test task's full description
  • Show the criteria-to-test conversion
  • Then ask for confirmation again

If user selects "Cancel":

  • Stop without creating any tasks
  • Report cancellation

Phase 7: Create Tasks

Goal: Create test tasks and update implementation tasks with TDD metadata.

Step 1: Create Test Tasks

For each planned test task, use TaskCreate:

TaskCreate:
  subject: "Write tests for {original subject}"
  description: {generated description from Phase 4}
  activeForm: "Writing tests for {original subject}"
  metadata:
    tdd_mode: true
    tdd_phase: "red"
    paired_task_id: "{impl_task_id}"
    priority: {inherited}
    complexity: {estimated}
    source_section: {inherited}
    spec_path: {inherited}
    feature_name: {inherited}
    task_uid: "{original_uid}:red"
    task_group: {inherited}

Capture the returned task ID for each created test task.

Step 2: Update Implementation Tasks

For each paired implementation task, use TaskUpdate:

TaskUpdate:
  taskId: "{impl_task_id}"
  metadata:
    tdd_mode: true
    tdd_phase: "green"
    paired_task_id: "{test_task_id}"

Step 3: Set Dependencies

For each TDD pair, set the dependency relationships:

TaskUpdate:
  taskId: "{impl_task_id}"
  addBlockedBy: ["{test_task_id}"]

For test tasks that need upstream dependencies (inheriting from the original impl task):

TaskUpdate:
  taskId: "{test_task_id}"
  addBlockedBy: ["{upstream_dep_1}", "{upstream_dep_2}"]

Step 4: Handle Circular Dependency Breaks

If any circular dependencies were detected and broken in Phase 5:

TaskUpdate:
  taskId: "{affected_task_id}"
  metadata:
    needs_review: true
    circular_dep_break: true

Phase 8: Report

Goal: Present the final summary of created TDD task pairs.

Completion Report

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
TDD TASK PAIR CREATION COMPLETE
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Created {n} TDD task pairs
Set {m} dependency relationships

TDD PAIRS CREATED:
  Test Task (RED)                        | Impl Task (GREEN)        | Test Blocks
  ─────────────────────────────────────────────────────────────────────────────────
  #{test_id}: Write tests for {subj}     | #{impl_id}: {subj}       | #{impl_id}
  ...

DEPENDENCY CHAIN:
  {visual representation of the full dependency chain}

{If --task-group was used:}
Group: {group}
Tasks in group: {total}
TDD pairs added: {new}

{If merge mode detected:}
MERGE MODE:
  Existing pairs preserved: {n}
  New pairs created: {m}

{If circular deps broken:}
WARNINGS:
  {n} circular dependencies detected and broken. Review recommended.

NEXT STEPS:
  Run /execute-tdd-tasks to execute TDD pairs with RED-GREEN-REFACTOR workflow.
  Run /execute-tdd-tasks --task-group {group} for group-specific execution.
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Error Handling

Empty Task List

If TaskList returns no tasks:

  • Inform user to run /create-tasks first
  • Provide usage example
  • Stop

No Matching Group

If --task-group filter matches zero tasks:

  • List available groups with task counts
  • Suggest a valid group
  • Stop

All Tasks Already Paired

If merge mode detects that all eligible tasks already have TDD pairs:

All eligible tasks already have TDD pairs. Nothing to create.

TDD pair status:
- {n} active TDD pairs
- {m} completed TDD pairs
- {k} tasks skipped (test/config/docs)

TaskCreate Failures

If a TaskCreate call fails:

  • Log the failure
  • Continue with remaining tasks
  • Report partial results in Phase 8 with failed tasks listed

Example Usage

Convert all tasks to TDD pairs

/agent-alchemy-tdd:create-tdd-tasks

Convert a specific group

/agent-alchemy-tdd:create-tdd-tasks --task-group user-authentication

Re-run (merge mode)

/agent-alchemy-tdd:create-tdd-tasks --task-group user-authentication

If TDD pairs already exist for some tasks, they will be detected and skipped.


Important Notes

  • Never create duplicate test tasks -- always check merge mode signals first
  • Preserve all existing task dependencies -- TDD pairs are inserted, not replacing
  • Test tasks always block their paired implementation task
  • Test task metadata always inherits task_group, priority, feature_name, spec_path, and source_section from the original
  • Task UIDs for test tasks append :red to the original UID
  • Always use imperative mood for subjects ("Write tests for X" not "Tests for X")
  • Always include activeForm in present continuous ("Writing tests for X")
  • Tasks with tdd_mode: true are never paired again (prevents double-pairing)

Reference Files

  • references/tdd-decomposition-patterns.md -- Task pairing rules, naming conventions, criteria conversion, merge mode
  • references/tdd-dependency-rules.md -- Dependency insertion algorithm, circular dependency detection and breaking

Execute TDD Tasks Skill

This skill orchestrates autonomous execution of TDD task pairs generated by /create-tdd-tasks. It is the TDD counterpart to the standard execute-tasks skill, reusing its session management, wave infrastructure, and execution context sharing while adding TDD-specific agent routing, RED-GREEN-REFACTOR verification, and per-task compliance reporting.

The key difference from standard execute-tasks: this skill routes TDD tasks to the tdd-executor agent (from tdd-tools) which runs a 6-phase TDD workflow, while routing non-TDD tasks to the standard task-executor agent. It verifies TDD compliance (RED verified, GREEN verified, refactored) per task pair and reports aggregate results.

CRITICAL: Complete ALL 9 steps. The workflow is not complete until Step 9: Update CLAUDE.md is evaluated. After completing each step, immediately proceed to the next step without waiting for user prompts (except Step 4 which requires user confirmation).

Plugin Context

This skill is part of the tdd-tools plugin and uses agents from the same plugin:

  • tdd-executor agent (Opus) -- 6-phase TDD workflow per task
  • test-writer agent (Sonnet) -- parallel test generation (used by tdd-executor internally)

For non-TDD tasks, this skill routes to the task-executor agent from sdd-tools (soft cross-plugin dependency). Since TDD tasks are always generated from SDD tasks via /create-tasks, the sdd-tools plugin is expected to be installed when this skill runs.

Core Principles

1. TDD Compliance First

Every TDD task pair must complete the RED-GREEN-REFACTOR cycle:

  • RED: Tests are written and verified to fail before any implementation exists
  • GREEN: Implementation is written that makes all tests pass with zero regressions
  • REFACTOR: Code is cleaned up while keeping all tests green

2. Strategic Parallelism

Maximize execution throughput without violating TDD sequencing:

  • PARALLEL: Multiple test-writing tasks (RED phase) run simultaneously across features
  • SEQUENTIAL: Within a single TDD pair, RED must complete before GREEN can start (enforced by dependencies)

3. Reuse execute-tasks Infrastructure

Session management, wave execution, context sharing, and progress tracking all reuse the same patterns from execute-tasks. See references/tdd-execution-workflow.md for TDD-specific extensions.

4. Honest TDD Reporting

Report per-task compliance with the full RED-GREEN-REFACTOR cycle:

  • red_verified: Whether tests failed as expected before implementation
  • green_verified: Whether all tests pass after implementation
  • refactored: Whether code was cleaned up while maintaining green tests
  • coverage_delta: Change in test coverage percentage (if measurable)

Orchestration Workflow

This skill orchestrates TDD task execution through a 9-step loop that mirrors the standard execute-tasks orchestration with TDD-specific extensions. See references/tdd-execution-workflow.md for the full TDD wave execution details and references/tdd-verification-patterns.md for TDD phase verification rules.

Step 1: Load References

Read the TDD-specific reference files:

Read: ${CLAUDE_PLUGIN_ROOT}/skills/execute-tdd-tasks/references/tdd-execution-workflow.md
Read: ${CLAUDE_PLUGIN_ROOT}/skills/execute-tdd-tasks/references/tdd-verification-patterns.md

Parse arguments from the invocation:

  • --task-group <group> -- Filter tasks to a specific group
  • --max-parallel <n> -- Override max concurrent agents per wave
  • --retries <n> -- Override retry attempts per task (default: 3)

Step 2: Load and Classify Tasks

Use TaskList to retrieve all tasks. If --task-group was provided, filter to tasks where metadata.task_group matches.

Classify each task by type:

DetectionTypeAgentSource
metadata.tdd_mode == true AND metadata.tdd_phase == "red"TDD test tasktdd-executortdd-tools (same plugin)
metadata.tdd_mode == true AND metadata.tdd_phase == "green"TDD implementation tasktdd-executortdd-tools (same plugin)
No tdd_mode metadata or tdd_mode == falseNon-TDD tasktask-executorsdd-tools (cross-plugin, soft dependency)

Count and report:

  • Total tasks (pending + in_progress + completed)
  • TDD pairs identified (test + implementation tasks)
  • Non-TDD tasks
  • Already completed tasks

Handle edge cases:

  • No tasks found: Report "No tasks found for group '{group}'. Use /create-tdd-tasks to generate TDD task pairs from your SDD tasks." and stop.
  • All completed: Report a summary of completed tasks including TDD compliance and stop.
  • No unblocked tasks: Report which tasks exist and what's blocking them.

Step 3: Build Execution Plan

Resolve max_parallel using precedence:

  1. --max-parallel CLI argument (highest priority)
  2. max_parallel in .claude/agent-alchemy.local.md
  3. Default: 5

Resolve retries using precedence:

  1. --retries CLI argument (highest priority)
  2. Default: 3

Read .claude/agent-alchemy.local.md if it exists, for TDD-specific settings:

  • tdd.strictness -- strict, normal (default), or relaxed
  • tdd.coverage-threshold -- Minimum coverage target (default: 80)

Build the dependency graph from all pending tasks (TDD and non-TDD):

  1. Collect all pending tasks and their blockedBy relationships
  2. Run topological sort to assign dependency levels
  3. Assign tasks to waves by dependency level (Wave 1 = no dependencies, Wave 2 = depends only on Wave 1, etc.)
  4. Sort within waves by priority: critical > high > medium > low > unprioritized
  5. Break ties by "unblocks most others"
  6. Cap each wave at max_parallel tasks

Annotate waves with TDD phase labels:

The dependency structure from create-tdd-tasks naturally produces alternating test/implementation waves:

Wave 1: [Test-A, Test-B, Test-C]         -- RED phase (parallel test generation)
Wave 2: [Impl-A, Impl-B, Impl-C]         -- GREEN phase (parallel implementation)
Wave 3: [Test-D, Test-E, Non-TDD-F]      -- RED phase + non-TDD tasks (mixed)
Wave 4: [Impl-D, Impl-E]                  -- GREEN phase

Detect circular dependencies: If tasks remain unassigned after topological sorting, they form a cycle. Report the cycle and attempt to break at the weakest link.

Validate TDD pair cross-references: For each TDD task, verify its paired_task_id references a valid task. Log warnings for orphaned pairs.

Step 4: Present Execution Plan and Confirm

Display the TDD execution plan:

EXECUTION PLAN (TDD Mode)

Tasks to execute: {count} ({tdd_pairs} TDD pairs, {non_tdd} non-TDD tasks)
Retry limit: {retries} per task
Max parallel: {max_parallel} per wave
TDD Strictness: {strict|normal|relaxed}

WAVE 1 ({n} tasks -- RED phase):
  1. [{id}] Write tests for {subject} (RED, paired: #{impl_id})
  2. [{id}] Write tests for {subject} (RED, paired: #{impl_id})

WAVE 2 ({n} tasks -- GREEN phase):
  3. [{id}] {subject} (GREEN, paired: #{test_id})
  4. [{id}] {subject} (GREEN, paired: #{test_id})

WAVE 3 ({n} tasks -- mixed):
  5. [{id}] {subject} (non-TDD)
  6. [{id}] Write tests for {subject} (RED, paired: #{impl_id})

{Additional waves...}

BLOCKED (unresolvable dependencies):
  [{id}] {subject} -- blocked by: {blocker ids}

COMPLETED:
  {count} tasks already completed

Use AskUserQuestion to confirm:

questions:
  - header: "Confirm TDD Execution"
    question: "Ready to execute {count} tasks in {wave_count} waves (max {max_parallel} parallel) with TDD enforcement ({strictness} mode)?"
    options:
      - label: "Yes, start TDD execution"
        description: "Proceed with the TDD execution plan above"
      - label: "Cancel"
        description: "Abort without executing any tasks"
    multiSelect: false

If the user selects "Cancel", report "Execution cancelled. No tasks were modified." and stop.

Step 5: Initialize Execution Directory

Generate a task_execution_id using three-tier resolution:

  1. IF --task-group was provided: {task_group}-tdd-{YYYYMMDD}-{HHMMSS}
  2. ELSE IF all open tasks share the same metadata.task_group: {task_group}-tdd-{YYYYMMDD}-{HHMMSS}
  3. ELSE: tdd-session-{YYYYMMDD}-{HHMMSS}

Clean stale live session: Follow the same procedure as execute-tasks:

  1. Check if .claude/sessions/__live_session__/ contains leftover files
  2. If found, archive to .claude/sessions/interrupted-{YYYYMMDD}-{HHMMSS}/
  3. Reset any in_progress tasks from the interrupted session to pending

Concurrency guard: Check for .claude/sessions/__live_session__/.lock. Follow the same lock protocol as execute-tasks.

Create session files in .claude/sessions/__live_session__/:

  1. execution_plan.md -- Save the TDD execution plan from Step 5
  2. execution_context.md -- Initialize with TDD-extended template:
    # Execution Context
    
    ## Project Patterns
    <!-- Discovered coding patterns, conventions, tech stack details -->
    
    ## Key Decisions
    <!-- Architecture decisions, approach choices made during execution -->
    
    ## Known Issues
    <!-- Problems encountered, workarounds applied, things to watch out for -->
    
    ## File Map
    <!-- Important files discovered and their purposes -->
    
    ## TDD Compliance
    | Task Pair | Test Task | Impl Task | RED | GREEN | Refactored | Coverage Delta |
    |-----------|-----------|-----------|-----|-------|------------|----------------|
    
    ## Task History
    <!-- Brief log of task outcomes with relevant context -->
    
  3. task_log.md -- Initialize with standard table headers:
    # Task Execution Log
    
    | Task ID | Subject | Type | Status | Attempts | Duration | Token Usage |
    |---------|---------|------|--------|----------|----------|-------------|
    
  4. tasks/ -- Empty subdirectory for archiving completed task files
  5. progress.md -- Initialize with status template:
    # Execution Progress (TDD Mode)
    Status: Initializing
    Wave: 0 of {total_waves}
    Max Parallel: {max_parallel}
    TDD Strictness: {strictness}
    Updated: {ISO 8601 timestamp}
    
    ## Active Tasks
    
    ## Completed This Session
    
  6. execution_pointer.md at $HOME/.claude/tasks/{CLAUDE_CODE_TASK_LIST_ID}/execution_pointer.md -- Absolute path to .claude/sessions/__live_session__/

Step 6: Initialize Execution Context

Read .claude/sessions/__live_session__/execution_context.md (created in Step 6).

If a prior execution session's context exists, look in .claude/sessions/ for the most recent timestamped subfolder and merge relevant learnings (Project Patterns, Key Decisions, Known Issues, File Map) into the new execution context.

Context compaction: If Task History has 10+ entries from merged sessions, compact older entries into a summary paragraph and keep the 5 most recent in full.

Step 7: Execute Loop

Execute tasks in waves with TDD-aware agent routing. No user interaction between waves.

8a: Initialize Wave
  1. Identify all unblocked tasks (pending status, all dependencies completed)
  2. Sort by priority (critical > high > medium > low > unprioritized)
  3. Take up to max_parallel tasks for this wave
  4. If no unblocked tasks remain, exit the loop
8b: Snapshot Execution Context

Read .claude/sessions/__live_session__/execution_context.md and hold as baseline for this wave. All agents read from the same snapshot.

8c: Launch Wave Agents
  1. Mark all wave tasks as in_progress via TaskUpdate
  2. Record wave_start_time
  3. Update progress.md with active tasks
  4. Launch all wave agents simultaneously using parallel Task tool calls in a single message turn with run_in_background: true.

Record the background task_id mapping: After the Task tool returns for each agent, record the mapping {task_list_id → background_task_id} from each response. The background_task_id is needed later to call TaskOutput for process reaping and usage extraction.

Route each task to the correct agent:

For TDD tasks (metadata.tdd_mode == true), launch the tdd-executor agent (same plugin):

Task:
  subagent_type: tdd-executor
  mode: bypassPermissions
  run_in_background: true
  prompt: |
    Execute the following TDD task.

    Task ID: {id}
    Task Subject: {subject}
    Task Description:
    ---
    {full description}
    ---

    Task Metadata:
    - Priority: {priority}
    - Complexity: {complexity}
    - TDD Phase: {tdd_phase}
    - Paired Task ID: {paired_task_id}
    - TDD Strictness: {strictness}

    CONCURRENT EXECUTION MODE
    Context Write Path: .claude/sessions/__live_session__/context-task-{id}.md
    Result Write Path: .claude/sessions/__live_session__/result-task-{id}.md
    Do NOT write to execution_context.md directly.
    Do NOT update progress.md -- the orchestrator manages it.
    Write your learnings to the Context Write Path above instead.

    RESULT FILE PROTOCOL
    As your VERY LAST action (after writing context-task-{id}.md), write a compact
    result file to the Result Write Path above. TDD format includes a TDD Compliance
    section with RED Verified, GREEN Verified, Refactored, and Coverage Delta fields.
    After writing the result file, return ONLY: DONE: [{id}] {subject} - {PASS|PARTIAL|FAIL}

    {If GREEN phase, include paired test task result data:}
    PAIRED TEST TASK OUTPUT:
    ---
    {test task result file content and context}
    ---
    The tests written by the paired test task are already on disk.
    Your job is to implement code that makes these tests pass (GREEN phase),
    then refactor while keeping tests green (REFACTOR phase).

    {If retry attempt:}
    RETRY ATTEMPT {n} of {max_retries}
    Previous TDD phase that failed: {RED|GREEN|REFACTOR}
    Previous attempt failed with:
    ---
    {previous failure details from result file}
    ---

    TDD-specific retry guidance:
    - If RED failed (tests cannot run): Check test syntax, imports, and framework config
    - If RED warned (tests passed unexpectedly): Verify tests target new behavior, not existing code
    - If GREEN failed (tests still failing): Re-read test assertions, try different implementation approach
    - If GREEN failed (regressions): Identify regression cause, fix without breaking new tests
    - If REFACTOR failed: Revert to pre-refactor state, try smaller refactoring steps

    Instructions (follow in order):
    1. Read the TDD execution and verification references
    2. Read .claude/sessions/__live_session__/execution_context.md for prior learnings
    3. Understand the task requirements and explore the codebase
    4. Execute the 6-phase TDD workflow (Understand, Write Tests, RED, Implement, GREEN, Complete)
    5. Verify TDD compliance (RED verified, GREEN verified, refactored)
    6. Update task status if PASS (mark completed)
    7. Write learnings to .claude/sessions/__live_session__/context-task-{id}.md
    8. Write result to .claude/sessions/__live_session__/result-task-{id}.md
    9. Return: DONE: [{id}] {subject} - {PASS|PARTIAL|FAIL}

For non-TDD tasks (no tdd_mode metadata), launch the standard task-executor agent from sdd-tools (cross-plugin, resolved globally):

Task:
  subagent_type: task-executor
  mode: bypassPermissions
  run_in_background: true
  prompt: |
    Execute the following task.

    Task ID: {id}
    Task Subject: {subject}
    Task Description:
    ---
    {full description}
    ---

    Task Metadata:
    - Priority: {priority}
    - Complexity: {complexity}
    - Source Section: {source_section}

    CONCURRENT EXECUTION MODE
    Context Write Path: .claude/sessions/__live_session__/context-task-{id}.md
    Result Write Path: .claude/sessions/__live_session__/result-task-{id}.md
    Do NOT write to execution_context.md directly.
    Do NOT update progress.md -- the orchestrator manages it.
    Write your learnings to the Context Write Path above instead.

    RESULT FILE PROTOCOL
    As your VERY LAST action (after writing context-task-{id}.md), write a compact
    result file to the Result Write Path above. Standard format with status, verification
    summary, files modified, and issues sections.
    After writing the result file, return ONLY: DONE: [{id}] {subject} - {PASS|PARTIAL|FAIL}

    {If retry attempt:}
    RETRY ATTEMPT {n} of {max_retries}
    Previous attempt failed with:
    ---
    {previous failure details from result file}
    ---
    Focus on fixing the specific failures listed above.

    Instructions (follow in order):
    1. Read the execute-tasks skill and reference files
    2. Read .claude/sessions/__live_session__/execution_context.md for prior learnings
    3. Understand the task requirements and explore the codebase
    4. Implement the necessary changes
    5. Verify against acceptance criteria
    6. Update task status if PASS (mark completed)
    7. Write learnings to .claude/sessions/__live_session__/context-task-{id}.md
    8. Write result to .claude/sessions/__live_session__/result-task-{id}.md
    9. Return: DONE: [{id}] {subject} - {PASS|PARTIAL|FAIL}

Important: Always include the CONCURRENT EXECUTION MODE and RESULT FILE PROTOCOL sections regardless of max_parallel value. All agents write to per-task context files and result files.

  1. Poll for completion: After launching all background agents, poll for result files using poll-for-results.sh from execute-tasks. The script checks for result-task-{id}.md files every 15 seconds for up to 45 minutes, printing progress lines periodically. A single Bash invocation handles the entire polling lifecycle.

    Poll invocation (via Bash tool with timeout: 2760000):

    bash ${CLAUDE_PLUGIN_ROOT}/../sdd-tools/skills/execute-tasks/scripts/poll-for-results.sh \
      .claude/sessions/__live_session__ {task_ids...}
    

    Parse the output:

    • POLL_RESULT: ALL_DONE — all agents finished. Proceed to 8d.
    • POLL_RESULT: TIMEOUT — not all agents finished within the timeout window. Log the Waiting on: line and proceed to 8d (handles missing result files via TaskOutput fallback).
    • Bash tool timeout or no recognizable output — treat as timeout. Proceed to 8d.
8d: Process Results (Batch)

After polling completes, process all wave results in a single batch:

  1. Reap background agents and extract usage: For each task in the wave, call TaskOutput(task_id=<background_task_id>, block=true, timeout=60000) using the mapping recorded in 8c. This serves two purposes:

    • Process reaping: Terminates the background agent process (prevents lingering subagents)
    • Usage extraction: Returns metadata with duration_ms and total_tokens per agent

    Extract per-task values:

    • task_duration: From duration_ms in TaskOutput metadata. Format: <60s = {s}s, <60m = {m}m {s}s, >=60m = {h}h {m}m {s}s
    • task_tokens: From total_tokens in TaskOutput metadata. Format with comma separators (e.g., 45,230)

    If TaskOutput times out (agent truly stuck), call TaskStop(task_id=<background_task_id>) to force-terminate the process, then set task_duration = "N/A" and task_tokens = "N/A".

  2. Read result files: For each task in the wave, read .claude/sessions/__live_session__/result-task-{id}.md. Parse status, attempt, verification, files modified, and issues. For TDD tasks, also parse the ## TDD Compliance section (RED Verified, GREEN Verified, Refactored, Coverage Delta).

  3. Handle missing result files: If a result file is missing after polling, the TaskOutput call in step 1 already captured diagnostic output for the crashed agent. Treat as FAIL.

  4. Determine task type label: TDD/RED, TDD/GREEN, or non-TDD

  5. Log status for each task: [{id}] {subject}: {PASS|PARTIAL|FAIL} ({type})

  6. Batch update task_log.md: Read once, append ALL wave rows, Write once:

    | {id} | {subject} | {TDD/RED|TDD/GREEN|non-TDD} | {PASS/PARTIAL/FAIL} | {attempt}/{max} | {task_duration} | {task_tokens} |
    

    Where {task_duration} and {task_tokens} come from the TaskOutput metadata extracted in step 1.

  7. Batch update progress.md: Read once, move ALL completed tasks from Active to Completed, Write once.

  8. For TDD tasks: Extract TDD compliance data from result files and update the ## TDD Compliance table in execution_context.md

Context append fallback: If a result file is missing but TaskOutput contains a LEARNINGS: section, write those learnings to context-task-{id}.md on behalf of the agent.

8e: Within-Wave Retry

After batch processing identifies failed tasks:

  1. Collect all failed tasks with retries remaining
  2. For each retriable task:
    • Read failure details from result-task-{id}.md (Issues and TDD Compliance sections)
    • Delete the old result-task-{id}.md file before re-launching
    • Launch a new background agent (run_in_background: true) with failure context from the result file
    • Record the new background_task_id from each Task tool response (same mapping as 8c)
    • For TDD tasks, include TDD-specific retry guidance in the prompt
    • Update progress.md: - [{id}] {subject} -- Retrying ({n}/{max})
  3. If any retry agents were launched:
    • Poll for retry result files using poll-for-results.sh (same pattern as 8c step 5, with only the retry task IDs as arguments and timeout: 2760000 on the Bash invocation)
    • After polling completes, reap retry agents: call TaskOutput on each retry background_task_id to extract duration_ms and total_tokens (same pattern as 8d step 1). If TaskOutput times out, call TaskStop to force-terminate.
    • Process retry results using the same batch approach as 8d (using the freshly extracted per-task duration and token values for task_log rows)
    • Repeat 8e if any retries still have attempts remaining
  4. If retries exhausted:
    • Leave task as in_progress
    • Log final failure
    • Retain the result file for post-analysis
    • For TDD test tasks: The paired implementation task remains blocked and will not execute

Test-writer agent failure fallback: If a TDD test task (RED phase) fails after all retries, the paired implementation task remains blocked. Do NOT fall back to running implementation without tests -- this would violate TDD principles.

8f: Merge Context and Clean Up After Wave

After ALL agents in the current wave have completed (including retries):

  1. Read .claude/sessions/__live_session__/execution_context.md
  2. Read all context-task-{id}.md files in task ID order
  3. Append each file's content to the ## Task History section
  4. For completed TDD tasks: Update the ## TDD Compliance table with pair results (extracted from result files in 8d)
  5. Write the complete updated execution_context.md
  6. Delete the context-task-{id}.md files
  7. Clean up result files: Delete result-task-{id}.md for PASS tasks. Retain result-task-{id}.md for FAIL tasks (post-analysis). For TDD test tasks (RED): Retain the result file until the paired GREEN task completes — the orchestrator reads stored result data for PAIRED TEST TASK OUTPUT injection in the next wave.

Capture test task result data for GREEN phase injection: When processing a completed test task (RED phase), read the result file content and store it for injection into the paired implementation task's prompt in the next wave. Delete the retained RED result file after the paired GREEN task's wave completes.

8g: Rebuild Next Wave and Archive
  1. Archive completed task files to .claude/sessions/__live_session__/tasks/
  2. Refresh task list via TaskList
  3. Check for newly unblocked tasks (especially implementation tasks unblocked by their paired test tasks)
  4. Form next wave using priority sort
  5. If no unblocked tasks remain, exit the loop
  6. Loop back to 8a

Step 8: Session Summary

Write final progress.md with complete status. Display the TDD execution summary:

TDD EXECUTION SUMMARY

Tasks executed: {total attempted}
  TDD Pairs: {pair_count}
  Non-TDD: {non_tdd_count}
  Passed: {count}
  Partial: {count}
  Failed: {count} (after {total retries} total retry attempts)

TDD COMPLIANCE:
| Task Pair | Test Task | Impl Task | RED | GREEN | Refactored | Coverage Delta |
|-----------|-----------|-----------|-----|-------|------------|----------------|
| {feature} | #{test_id} ({status}) | #{impl_id} ({status}) | {Yes/No} | {Yes/No} | {Yes/No/N/A} | {+/-pct or N/A} |
...

TDD Compliance Rate: {compliant_pairs}/{total_pairs} ({percentage}%)

Waves completed: {wave_count}
Max parallel: {max_parallel}
TDD Strictness: {strictness}
Total execution time: {sum of all task duration_ms values, formatted}
Token Usage: {sum of all task total_tokens values, formatted with commas}

Remaining:
  Pending: {count}
  In Progress (failed): {count}
  Blocked: {count}

{If any tasks failed:}
FAILED TASKS:
  [{id}] {subject} -- {brief failure reason} ({TDD phase if applicable})

{If newly unblocked tasks were discovered:}
NEWLY UNBLOCKED:
  [{id}] {subject} -- unblocked by completion of [{blocker_id}]

After displaying the summary:

  1. Save session_summary.md to .claude/sessions/__live_session__/ with full summary content
  2. Archive the session: move all contents from __live_session__/ to .claude/sessions/{task_execution_id}/
  3. Leave __live_session__/ as an empty directory
  4. execution_pointer.md stays pointing to __live_session__/

Step 9: Update CLAUDE.md

Review .claude/sessions/{task_execution_id}/execution_context.md for project-wide changes.

Update CLAUDE.md if the session introduced:

  • New architectural patterns or conventions
  • New dependencies or tech stack changes
  • New development commands or workflows
  • Changes to project structure
  • Important design decisions

Skip if only task-specific or TDD-internal implementation details.

Agent Routing Summary

Task TypeDetectionAgentPluginWorkflow
TDD test (RED)tdd_mode: true, tdd_phase: "red"tdd-executortdd-tools6-phase TDD
TDD impl (GREEN)tdd_mode: true, tdd_phase: "green"tdd-executortdd-tools6-phase TDD
Non-TDDNo tdd_mode or tdd_mode: falsetask-executorsdd-tools (cross-plugin)4-phase standard

TDD Verification Rules

See references/tdd-verification-patterns.md for complete verification rules.

Quick reference:

PhasePASSFAIL
REDAll new tests fail as expectedTests cannot run or syntax errors
GREENAll tests pass, zero regressionsNew tests still failing after implementation
REFACTORAll tests green after cleanupTests broke and cannot recover

Strictness levels (from .claude/agent-alchemy.local.md tdd.strictness setting):

LevelRED BehaviorImpact
strictTests passing unexpectedly = FAILBlocks GREEN phase
normal (default)Tests passing unexpectedly = WARNProceeds with warning
relaxedTests passing unexpectedly = INFOProceeds, informational only

Key Behaviors

  • Autonomous execution loop: After user confirms the plan, no further prompts between tasks
  • Background agent execution: Agents run as background tasks (run_in_background: true), returning ~3 lines instead of ~100+ lines of full output. This reduces orchestrator context consumption by ~79% per wave.
  • Agent process reaping: After polling confirms result files exist, the orchestrator calls TaskOutput on each background task_id to reap the process and extract per-task duration_ms and total_tokens usage metadata. If TaskOutput times out, TaskStop force-terminates the stuck agent. This prevents lingering background processes.
  • Result file protocol: Each agent writes a compact result-task-{id}.md as its very last action. TDD result files include a ## TDD Compliance section. The orchestrator polls for these files via poll-for-results.sh in a single Bash invocation (with timeout: 2760000), then batch-reads them for processing.
  • Batched session file updates: task_log.md and progress.md are updated once per wave (batch read-modify-write) instead of per-task.
  • Wave-based TDD parallelism: Test tasks (RED) in one wave, their paired implementation tasks (GREEN) in the next. Multiple features run in parallel within a wave
  • Agent routing by metadata: TDD tasks go to tdd-executor, non-TDD tasks go to task-executor
  • Per-task context isolation: Each agent writes to context-task-{id}.md, orchestrator merges after each wave
  • Test-to-implementation context flow: Test task result data is read from disk and injected into the paired implementation task's prompt via PAIRED TEST TASK OUTPUT. RED result files are retained across waves until the paired GREEN task completes.
  • Within-wave retry: Failed tasks with retries remaining are re-launched as background agents. The orchestrator polls for retry result files using poll-for-results.sh (same pattern as initial wave polling).
  • No silent degradation: If TDD test task fails, its paired implementation task stays blocked. Never run implementation without tests
  • TDD compliance tracking: Per-pair tracking of RED/GREEN/REFACTOR verification extracted from result files
  • Configurable strictness: strict, normal, or relaxed TDD enforcement via settings
  • Single-session invariant: Only one execution session at a time, enforced by .lock file
  • Interrupted session recovery: Stale sessions archived, interrupted tasks reset to pending

Example Usage

Execute all TDD tasks

/agent-alchemy-tdd:execute-tdd-tasks

Execute TDD tasks for a specific group

/agent-alchemy-tdd:execute-tdd-tasks --task-group user-authentication

Execute with limited parallelism

/agent-alchemy-tdd:execute-tdd-tasks --max-parallel 2

Execute sequentially (no concurrency)

/agent-alchemy-tdd:execute-tdd-tasks --max-parallel 1

Execute with custom retries

/agent-alchemy-tdd:execute-tdd-tasks --retries 1

Execute group with custom parallelism and retries

/agent-alchemy-tdd:execute-tdd-tasks --task-group payments --max-parallel 3 --retries 1

Reference Files

  • references/tdd-execution-workflow.md -- TDD-aware wave execution, agent spawning, context sharing between RED and GREEN phases
  • references/tdd-verification-patterns.md -- RED/GREEN/REFACTOR verification rules, compliance reporting, status determination matrix

Generate Tests Skill

Generate high-quality, behavior-driven test files from acceptance criteria or existing source code. This skill auto-detects the project's test framework, loads framework-specific patterns, and spawns test-writer agents to produce runnable test files that follow project conventions.

CRITICAL: Complete ALL 6 phases. The workflow is not complete until Phase 6: Report is finished. After completing each phase, immediately proceed to the next phase without waiting for user prompts.

Core Principles

  1. Tests before implementation -- Generated tests should initially fail (RED state) when no implementation exists, confirming they test real behavior.
  2. Behavior over implementation -- Test what the code does, not how it does it. Tests should survive refactoring.
  3. Follow project conventions -- Match the project's existing test framework, file naming, directory structure, and assertion patterns.
  4. Standalone operation -- This skill works independently. It does not require the SDD pipeline, tdd-cycle, or any other skill to function.

AskUserQuestion is MANDATORY

IMPORTANT: You MUST use the AskUserQuestion tool for ALL questions to the user. Never ask questions through regular text output.

  • Framework selection questions -> AskUserQuestion
  • Clarifying questions about scope -> AskUserQuestion
  • Confirmation questions -> AskUserQuestion

Text output should only be used for presenting information, summaries, and progress updates.


Phase 1: Parse Input

Goal: Determine the operating mode and resolve input paths.

Analyze the $ARGUMENTS to determine which mode to use:

Mode Detection

Criteria-Driven Mode -- triggered when input is:

  • A spec file path (e.g., specs/SPEC-feature.md)
  • A task ID (e.g., 5, #5, task-5)
  • A spec section reference (e.g., specs/SPEC-feature.md Section 5.1)

Code-Analysis Mode -- triggered when input is:

  • A source file path (e.g., src/utils.py, src/services/auth.ts)
  • A directory path (e.g., src/services/)

Ambiguous input: If the input could be either mode (e.g., a path that could be a spec or source file), check the file content. If it contains **Acceptance Criteria:** or is a markdown file in a specs/ directory, use criteria-driven mode. Otherwise, use code-analysis mode.

Path Resolution

  1. Resolve relative paths against the project root
  2. For task IDs: use TaskGet to retrieve the task description, then extract acceptance criteria
  3. For spec section references: read the spec file and locate the referenced section
  4. For directories: glob for source files (exclude test files, node_modules, pycache, .git)
  5. Validate that all resolved paths exist; if not, proceed to error handling (Phase 1 error cases below)

Error Cases

Invalid spec path:

ERROR: Spec file not found: {path}

Did you mean one of these?
{List matching files from Glob search for similar names}

Usage: /generate-tests <spec-path|task-id|file-path>

Invalid task ID:

ERROR: Task #{id} not found.

Available tasks:
{List first 5 pending/in-progress tasks from TaskList}

Usage: /generate-tests <task-id>

Empty directory:

ERROR: No source files found in {path}.

The directory exists but contains no files matching supported extensions (.py, .ts, .tsx, .js, .jsx).

Phase 2: Detect Framework

Goal: Auto-detect the project's test framework using the detection chain.

Load Detection Chain

Read the framework detection reference for the full detection algorithm:

Read: ${CLAUDE_PLUGIN_ROOT}/skills/generate-tests/references/framework-templates.md

Follow the detection chain in priority order:

Step 1: Config-File Detection (High Confidence)

Check for framework configuration files:

Python:

  • pyproject.toml with [tool.pytest.ini_options] or [tool.pytest] -> pytest
  • setup.cfg with [tool:pytest] -> pytest
  • pytest.ini exists -> pytest
  • conftest.py at project root or in tests/ -> pytest

TypeScript/JavaScript:

  • vitest.config.* exists -> Vitest (takes priority)
  • jest.config.* exists -> Jest
  • package.json with vitest in dependencies/devDependencies -> Vitest
  • package.json with jest in dependencies/devDependencies -> Jest
  • package.json with "jest": {} config section -> Jest

Step 2: Existing Test File Detection (Medium Confidence)

If no config files found, scan for existing test files:

  • test_*.py or *_test.py -> pytest
  • *.test.ts / *.spec.ts with vitest imports -> Vitest
  • *.test.ts / *.spec.ts with jest imports or no explicit imports -> Jest

Step 3: Settings Fallback (Low Confidence)

If detection is still ambiguous, check .claude/agent-alchemy.local.md for:

tdd:
  framework: pytest    # or jest, vitest, auto

If tdd.framework is set to a recognized value (pytest, jest, vitest), use it. If set to auto or an unrecognized value, continue to Step 4.

Step 4: User Prompt Fallback

If all detection methods fail, prompt the user:

AskUserQuestion:
  questions:
    - header: "Test Framework"
      question: "No test framework was detected in this project. Which framework should be used for test generation?"
      options:
        - label: "pytest"
          description: "Python testing framework (recommended for Python projects)"
        - label: "Jest"
          description: "JavaScript/TypeScript testing framework"
        - label: "Vitest"
          description: "Vite-native testing framework (modern alternative to Jest)"
        - label: "Other"
          description: "A different framework (tests may need manual adjustment)"
      multiSelect: false

Detection Result

After detection, record:

  • Framework: pytest | Jest | Vitest
  • Test directory: Path to test root (e.g., tests/, __tests__/, src/)
  • Naming pattern: File naming convention (e.g., test_*.py, *.test.ts)
  • Assertion library: Default or extended (e.g., @testing-library/jest-dom)
  • Existing fixtures: Notable fixtures or test utilities found
  • Confidence: High | Medium | Low

Monorepo Handling

In monorepos, detect framework per-directory by walking up from the target source file to find the nearest config file. Different directories may use different frameworks.


Phase 3: Load References

Goal: Load framework-specific patterns and project conventions.

Load Test Patterns

Read the test patterns reference for framework-specific guidance:

Read: ${CLAUDE_PLUGIN_ROOT}/skills/generate-tests/references/test-patterns.md

This provides:

  • Arrange-Act-Assert and Given-When-Then patterns
  • Behavior-driven vs implementation-detail test guidance
  • Framework-specific naming conventions
  • Anti-patterns to avoid

Load Framework Templates

Read the framework templates reference for boilerplate structure:

Read: ${CLAUDE_PLUGIN_ROOT}/skills/generate-tests/references/framework-templates.md

This provides:

  • Boilerplate templates for pytest, Jest, and Vitest
  • Configuration reference for each framework
  • Fixture and test utility patterns

Load Cross-Plugin Skills

Load language-specific patterns and project conventions for the target project:

Read: ${CLAUDE_PLUGIN_ROOT}/../core-tools/skills/language-patterns/SKILL.md
Read: ${CLAUDE_PLUGIN_ROOT}/../core-tools/skills/project-conventions/SKILL.md

Apply these skills' guidance when generating tests to ensure:

  • Language idioms are followed (TypeScript strict typing, Python type hints)
  • Project naming conventions are matched
  • Import patterns are consistent with the codebase

Scan Existing Tests

If the project has existing test files:

  1. Read 2-3 representative test files to understand the project's test style
  2. Note patterns: fixture usage, test grouping, assertion style, mock patterns
  3. Match these patterns in generated tests for consistency

Phase 4: Generate Tests

Goal: Produce test files by spawning test-writer agents.

Criteria-Driven Mode

When operating from acceptance criteria (spec, task, or section reference):

  1. Extract criteria: Parse the acceptance criteria into categories:

    • Functional: Core behavior assertions (highest priority)
    • Edge Cases: Boundary conditions and unusual inputs
    • Error Handling: Error paths and error messages
    • Performance: Efficiency-related assertions (if applicable)
  2. Plan test files: Determine how many test files to create:

    • One feature/module = one test file
    • Multiple features = one test file per feature
    • Group related criteria into describe blocks or test classes
  3. Map criteria to test cases: For each criterion, create one or more test cases:

    • Each criterion gets at least one test
    • Complex criteria may need multiple tests (happy path + variations)
    • Edge cases get individual tests
    • Error handling criteria get tests verifying error types and messages
  4. Determine test file paths: Follow the project's convention:

    • Match existing test directory structure
    • Use the detected naming pattern
    • Place tests adjacent to source files if the project uses colocated tests, or in the test directory if centralized

Code-Analysis Mode

When operating from source files:

  1. Analyze source files: For each source file, identify:

    • Public functions and methods (skip private/internal)
    • Function signatures: parameters, return types, side effects
    • Class structures: constructors, public methods, properties
    • Module exports: what is available to consumers
  2. Generate characterization tests: Create tests that verify current behavior:

    • Test each public function with representative inputs
    • Test return values and observable side effects
    • Test error conditions visible from the signature (e.g., nullable params)
  3. Identify untested edge cases: Look for:

    • Boundary conditions (empty inputs, null values, max values)
    • Error paths (invalid types, missing required fields)
    • State transitions (before/after operations)
  4. Preserve existing tests: Before writing, check for existing test files:

    • If a test file already exists for the source file, do NOT overwrite it
    • Instead, generate a supplementary test file with a _additional or _generated suffix
    • Report which files were skipped and why

Spawning Test-Writer Agents

Single feature/file: Use a single test-writer agent:

Task tool with subagent_type: "test-writer"

Prompt:
"Generate tests for {feature/file}.

Framework: {detected framework}
Test patterns: {summary of project's test patterns}
File naming: {convention}
Target test file: {output path}

{Mode-specific context:}
- Criteria-driven: {formatted acceptance criteria}
- Code-analysis: {source file content and analysis}

Follow these conventions from the project's existing tests:
{Summary of patterns from existing test files}

Write the test file to: {target path}"

Multiple features/files: Spawn test-writer agents in parallel:

For each feature/file, launch a separate Task with subagent_type: "test-writer"
- Each agent gets its own feature's criteria or source file
- All agents share the same framework detection and pattern context
- Launch all agents simultaneously in a single turn for parallelism

Wait for all agents to complete before proceeding to Phase 5.

Agent Failure Handling

If a test-writer agent fails:

  1. Log which feature/file failed
  2. Continue with successful results
  3. Report the failure in Phase 6 with recommendations

Phase 5: Validate

Goal: Verify generated tests are syntactically valid and follow conventions.

Syntax Validation

For each generated test file, run a syntax check:

pytest:

python -c "import ast; ast.parse(open('{test_file}').read())"

Jest/Vitest:

npx tsc --noEmit {test_file}

Or if TypeScript is not configured:

node --check {test_file}

If syntax validation fails:

  1. Read the error output
  2. Fix the issue in the test file using Edit
  3. Re-validate
  4. If the fix fails after 2 attempts, flag the file as needing manual review

Convention Compliance

Verify each generated test file follows project conventions:

  • File is in the correct directory
  • File naming matches the detected pattern
  • Imports are correct and resolvable
  • Test structure matches existing test patterns (classes vs functions, describe blocks)
  • Assertions use the project's assertion library

RED State Verification

If no implementation exists for the tested code:

  • The generated tests should reference imports that do not yet exist
  • This confirms the tests are in RED state (will fail when run)
  • Note: Do NOT run the tests at this point -- running failing tests is the job of the TDD cycle

If implementation already exists:

  • Flag with a warning:
    NOTE: Implementation for {module} already exists. Generated tests may pass
    immediately (not in RED state). These tests document existing behavior rather
    than driving new implementation.
    

Phase 6: Report

Goal: Present a summary of generated test files.

Report Format

Present the following summary to the user:

## Test Generation Summary

**Mode**: {Criteria-Driven | Code-Analysis}
**Framework**: {pytest | Jest | Vitest} ({confidence level})
**Source**: {spec path | task ID | file/directory path}

### Generated Files

| File | Tests | Coverage Area |
|------|-------|---------------|
| {test_file_1} | {count} | {what it tests} |
| {test_file_2} | {count} | {what it tests} |

**Total**: {total_tests} tests across {file_count} files

### Criteria Coverage (Criteria-Driven Mode Only)

| Category | Criteria | Tests |
|----------|----------|-------|
| Functional | {n} criteria | {n} tests |
| Edge Cases | {n} criteria | {n} tests |
| Error Handling | {n} criteria | {n} tests |
| Performance | {n} criteria | {n} tests |

### State

{If no implementation exists:}
Tests are in RED state -- they will fail when run. Implement the code to make them pass.

{If implementation exists:}
WARNING: Implementation already exists. Tests may pass immediately.
Run your test suite to verify: {test command}

### Next Steps

- Run tests: `{test_command}`
- Start TDD cycle: `/tdd-cycle {feature}`
- Analyze coverage: `/analyze-coverage {path}`

Skipped Files (Code-Analysis Mode)

If any files were skipped because tests already exist:

### Skipped (Existing Tests)

| Source File | Existing Test File | Reason |
|------------|-------------------|--------|
| {source} | {test_file} | Test file already exists |

Failures

If any test-writer agents failed:

### Failures

| Feature/File | Error | Recommendation |
|-------------|-------|----------------|
| {feature} | {error summary} | {suggestion} |

Error Handling

No Input Provided

If $ARGUMENTS is empty or missing:

AskUserQuestion:
  questions:
    - header: "Test Generation Input"
      question: "What would you like to generate tests for?"
      options:
        - label: "Spec file"
          description: "Generate tests from a specification's acceptance criteria"
        - label: "Task ID"
          description: "Generate tests from a Claude Code Task's acceptance criteria"
        - label: "Source file or directory"
          description: "Generate characterization tests for existing code"
      multiSelect: false

Then prompt for the specific path or ID based on the selection.

Framework Detection Failure

If framework detection reaches the user prompt fallback (Step 4) and the user selects "Other":

WARNING: Unsupported framework selected. Generating tests using the closest
supported framework ({inferred}). Generated tests may need manual adjustment
for your specific framework.

Partial Input

If the input resolves to a spec or task with no acceptance criteria:

  • Switch to a hybrid approach: extract any descriptive text for test naming
  • Generate basic smoke tests from function signatures if source files are referenced
  • Flag the tests as minimal coverage that should be expanded

Settings

The following settings in .claude/agent-alchemy.local.md affect test generation:

tdd:
  framework: auto                    # auto | pytest | jest | vitest
  test-review-threshold: 70          # Minimum test quality score (0-100)
  test-review-on-generate: false     # Run test-reviewer after generate-tests
SettingDefaultUsed In
tdd.frameworkautoPhase 2 (framework detection fallback)
tdd.test-review-threshold70Phase 6 (test quality evaluation threshold)
tdd.test-review-on-generatefalsePhase 6 (auto-run test-reviewer after generation)

Error handling:

  • If the settings file does not exist, use defaults for all settings.
  • If the YAML frontmatter is malformed, use defaults and log a warning.
  • If tdd.framework is set to an unrecognized value, fall back to auto-detection.

See tdd-tools/README.md for the full settings reference.


Reference Files

  • references/test-patterns.md -- Framework-specific test patterns, behavior-driven guidance, anti-patterns
  • references/framework-templates.md -- Auto-detection chain, boilerplate templates, configuration reference

TDD Cycle Skill

Run a full RED-GREEN-REFACTOR Test-Driven Development workflow for a feature. This skill drives the entire TDD lifecycle: understand the feature, write failing tests, confirm they fail (RED), implement minimal code to make them pass (GREEN), then refactor while keeping tests green (REFACTOR).

CRITICAL: Complete ALL 7 phases. The workflow is not complete until Phase 7: Report is finished. After completing each phase, immediately proceed to the next phase without waiting for user prompts.

Core Principles

  1. Tests before implementation -- Write tests first. The tests define what the code should do. Implementation follows from tests, not the other way around.
  2. Minimal implementation -- Write only the code needed to make failing tests pass. No extra features, no premature optimization, no speculative abstractions.
  3. Regression protection -- Existing tests must continue passing at every phase. Zero tolerance for regressions.
  4. Phase gate enforcement -- Each phase must complete and verify before the next begins. RED verification is mandatory. GREEN verification is mandatory.
  5. Behavior over implementation -- Tests verify what code does (inputs, outputs, side effects), not how it does it internally.
  6. Autonomous after plan confirmation -- The user confirms the plan once in Phase 3. After that, the entire RED-GREEN-REFACTOR cycle runs without interruption.

AskUserQuestion is MANDATORY

IMPORTANT: You MUST use the AskUserQuestion tool for ALL questions to the user. Never ask questions through regular text output.

  • Plan confirmation -> AskUserQuestion
  • Framework selection -> AskUserQuestion
  • Clarifying questions -> AskUserQuestion
  • Error recovery options -> AskUserQuestion

Text output should only be used for presenting information, summaries, and progress updates.

NEVER do this (asking via text output):

Should I proceed with this plan?
1. Yes
2. No, modify it

ALWAYS do this (using AskUserQuestion tool):

AskUserQuestion:
  questions:
    - header: "TDD Plan Confirmation"
      question: "Review the TDD plan above. Ready to proceed?"
      options:
        - label: "Proceed"
          description: "Run the full RED-GREEN-REFACTOR cycle autonomously"
        - label: "Modify plan"
          description: "Adjust tests, scope, or approach before starting"
        - label: "Cancel"
          description: "Cancel the TDD workflow"
      multiSelect: false

Phase 1: Parse Input

Goal: Determine the input type and resolve context.

Analyze $ARGUMENTS to determine the operating mode.

Input Type Detection

Feature Description -- triggered when input is:

  • Free-text describing a feature (e.g., "add user login with email and password")
  • A file path to source code that needs TDD treatment (e.g., src/auth/login.py)

Task ID -- triggered when input is:

  • A numeric ID, possibly prefixed with # or task- (e.g., 5, #5, task-5)

Spec Section -- triggered when input is:

  • A spec file path with optional section reference (e.g., specs/SPEC-feature.md Section 5.1)

Context Resolution

Feature Description:

  1. Record the feature description for use in Phase 3 (Plan)
  2. Use Glob/Grep to find existing related code if a file path is provided
  3. Determine the target module and directory

Task ID:

  1. Use TaskGet to retrieve the full task details
  2. Extract acceptance criteria (Functional, Edge Cases, Error Handling)
  3. Check metadata.spec_path for the source spec
  4. Record the task's description, requirements, and blocked-by/blocks relationships

Spec Section:

  1. Read the spec file
  2. Locate the referenced section
  3. Extract acceptance criteria, user stories, and edge cases from the section

Error Cases

No input provided:

AskUserQuestion:
  questions:
    - header: "TDD Input"
      question: "What would you like to run TDD for?"
      options:
        - label: "Feature description"
          description: "Describe the feature to build test-first"
        - label: "Task ID"
          description: "Run TDD for a Claude Code Task"
        - label: "Spec section"
          description: "Run TDD from a spec's acceptance criteria"
        - label: "Retrofit existing code"
          description: "Add tests to existing untested code"
      multiSelect: false

Then prompt for the specific value based on the selection.

Invalid task ID:

ERROR: Task #{id} not found.

Available tasks:
{List first 5 pending/in-progress tasks from TaskList}

Usage: /tdd-cycle <feature-description|task-id|spec-section>

Invalid spec path:

ERROR: Spec file not found: {path}

Did you mean one of these?
{List matching files from Glob search for similar names}

Usage: /tdd-cycle <spec-path>

Phase 2: Understand

Goal: Load project conventions, detect the test framework, and explore the relevant codebase.

Step 1: Load Project Conventions

Read project-level conventions:

Read: CLAUDE.md
Read: .claude/agent-alchemy.local.md (if it exists)

Load cross-plugin skills for language and project awareness:

Read: ${CLAUDE_PLUGIN_ROOT}/../core-tools/skills/language-patterns/SKILL.md
Read: ${CLAUDE_PLUGIN_ROOT}/../core-tools/skills/project-conventions/SKILL.md

Apply their guidance when writing tests and implementation code.

Step 2: Load TDD Configuration

Read TDD settings from .claude/agent-alchemy.local.md if it exists:

tdd:
  framework: auto                    # auto | pytest | jest | vitest
  coverage-threshold: 80             # Minimum coverage percentage (0-100)
  strictness: normal                 # strict | normal | relaxed
  test-review-threshold: 70          # Minimum test quality score (0-100)
  test-review-on-generate: false     # Run test-reviewer after generate-tests

Record the TDD strictness level (default: normal if not configured). Record the framework override (default: auto -- use detection chain). Record the coverage threshold (default: 80 -- clamp to 0-100 if out of range).

Error handling:

  • If the settings file does not exist, use defaults for all settings.
  • If the YAML frontmatter is malformed, use defaults and log a warning.
  • If tdd.framework is set to an unrecognized value, fall back to auto-detection.

Step 3: Detect Test Framework

Follow the framework detection chain to identify the project's test framework.

Priority 1 -- Config Files (High Confidence):

Python:

  • pyproject.toml with [tool.pytest.ini_options] or [tool.pytest] -> pytest
  • setup.cfg with [tool:pytest] -> pytest
  • pytest.ini exists -> pytest
  • conftest.py at project root or in tests/ -> pytest

TypeScript/JavaScript:

  • vitest.config.* exists -> Vitest (takes priority)
  • jest.config.* exists -> Jest
  • package.json with vitest in dependencies/devDependencies -> Vitest
  • package.json with jest in dependencies/devDependencies -> Jest
  • package.json with "jest": {} config section -> Jest

Priority 2 -- Existing Test Files (Medium Confidence):

  • test_*.py or *_test.py -> pytest
  • *.test.ts / *.spec.ts with vitest imports -> Vitest
  • *.test.ts / *.spec.ts with jest imports or no explicit imports -> Jest

Priority 3 -- Settings Fallback (Low Confidence):

  • Check .claude/agent-alchemy.local.md for tdd.framework

Priority 4 -- User Prompt Fallback:

If all detection methods fail:

AskUserQuestion:
  questions:
    - header: "Test Framework"
      question: "No test framework was detected in this project. Which framework should be used?"
      options:
        - label: "pytest"
          description: "Python testing framework (recommended for Python projects)"
        - label: "Jest"
          description: "JavaScript/TypeScript testing framework"
        - label: "Vitest"
          description: "Vite-native testing framework (modern alternative to Jest)"
        - label: "Other"
          description: "A different framework (tests may need manual adjustment)"
      multiSelect: false

Step 4: Load Test Pattern References

Read framework-specific patterns and templates:

Read: ${CLAUDE_PLUGIN_ROOT}/skills/generate-tests/references/test-patterns.md
Read: ${CLAUDE_PLUGIN_ROOT}/skills/generate-tests/references/framework-templates.md

Step 5: Explore Codebase

  1. Use Glob to find files related to the feature scope
  2. Use Grep to locate relevant symbols, functions, and patterns
  3. Read existing test files to understand test conventions (naming, structure, fixtures, assertion style)
  4. Identify where new test files and implementation files should be placed
  5. Read 2-3 representative existing test files if available to match project test style

Step 6: Snapshot Existing Test State

Run the existing test suite and record the baseline:

  • Total tests, pass count, fail count
  • List any pre-existing failures (these are not your responsibility to fix)
  • This baseline is used in Phases 4 (RED) and 5 (GREEN) to separate new results from existing state

Python (pytest):

pytest --tb=short -q 2>&1 || true

TypeScript (Jest/Vitest):

npx jest --no-coverage 2>&1 || true
npx vitest run --reporter=verbose 2>&1 || true

Record the baseline: {total} tests, {passed} passed, {failed} failed


Phase 3: Plan

Goal: Present the TDD plan to the user and get a single confirmation before running autonomously.

Build the TDD Plan

Based on the parsed input and codebase exploration, construct a plan covering:

  1. Feature scope: What behavior will be implemented
  2. Tests to write: List of test cases with descriptive names, organized by category:
    • Functional tests (core behavior)
    • Edge case tests (boundary conditions)
    • Error handling tests (failure scenarios)
  3. Test file location: Where test files will be created
  4. Implementation approach: What source files will be created or modified
  5. Coverage target: Expected coverage percentage
  6. Strictness level: RED phase enforcement level (from settings)

Present Plan

Present the plan as a formatted summary:

## TDD Plan: {feature name}

**Framework**: {pytest | Jest | Vitest}
**Strictness**: {strict | normal | relaxed}
**Coverage Target**: {percentage}%

### Tests to Write

**Functional:**
- test_{behavior_1}: {description}
- test_{behavior_2}: {description}

**Edge Cases:**
- test_{edge_case_1}: {description}

**Error Handling:**
- test_{error_1}: {description}

### Test Files
- {test_file_path_1}

### Implementation Files
- {source_file_path_1} (create / modify)

### Approach
{Brief description of the implementation strategy}

Confirm Plan

Use AskUserQuestion to get confirmation:

AskUserQuestion:
  questions:
    - header: "TDD Plan Confirmation"
      question: "Review the TDD plan above. Ready to proceed with the RED-GREEN-REFACTOR cycle?"
      options:
        - label: "Proceed"
          description: "Run the full TDD cycle autonomously — no further interruptions"
        - label: "Modify plan"
          description: "Adjust the test plan before starting"
        - label: "Cancel"
          description: "Cancel this TDD workflow"
      multiSelect: false

If "Modify plan": Ask what changes are needed via AskUserQuestion, update the plan, and present again for confirmation.

If "Cancel": Stop the workflow and inform the user.

If "Proceed": Continue to Phase 4. From this point forward, the workflow runs autonomously without user interaction.


Phase 4: RED Phase

Goal: Write failing tests from requirements, then run the test suite to confirm all new tests fail.

CRITICAL: After plan confirmation, this phase and all subsequent phases run autonomously. Do NOT prompt the user for input.

Load TDD Workflow Reference

Read the TDD workflow reference for phase definitions and verification rules:

Read: ${CLAUDE_PLUGIN_ROOT}/skills/tdd-cycle/references/tdd-workflow.md

Step 1: Write Failing Tests

Write the test files planned in Phase 3:

  1. Convert requirements to test cases: Each acceptance criterion becomes one or more test assertions
  2. Follow project test conventions: Match existing test file naming, directory structure, assertion style, and fixture patterns
  3. Write behavior-driven tests: Test what the code should do, not how it does it. Focus on inputs, outputs, and observable side effects
  4. Use the AAA pattern: Arrange (setup), Act (execute), Assert (verify)
  5. Include edge case tests: Boundary conditions, null/empty inputs, error scenarios
  6. Use descriptive test names:
    • pytest: test_<function>_<scenario>_<expected_result>
    • Jest/Vitest: describe("<unit>", () => { it("should <behavior> when <condition>", ...) })

IMPORTANT: Do NOT write any implementation code during this step. Only write test files.

Step 2: Run Tests -- Confirm RED

Run the full test suite:

Python:

pytest --tb=short -q 2>&1

TypeScript:

npx jest --no-coverage 2>&1
npx vitest run --reporter=verbose 2>&1

Step 3: Verify RED State

Compare results against the Phase 2 baseline to isolate new test results.

Expected outcome: ALL new tests fail with appropriate errors (ImportError, ModuleNotFoundError, AttributeError, AssertionError, etc.)

Apply strictness level:

StrictnessIf new tests passAction
strictANY new test passesAbort workflow. Report which tests passed and likely cause
normalSome new tests passLog warning. Investigate: is implementation already present? Are tests too weak? Continue after investigation
relaxedAny outcomeLog results and continue regardless

Edge Case: Tests Pass Immediately

If some or all new tests pass before implementation:

  1. Check if implementation already exists: Search for existing code that satisfies the tests
  2. If implementation exists: Log a warning to the user. If ALL tests pass, skip to Phase 6 (REFACTOR). If partial, proceed to Phase 5 (GREEN) for remaining tests
  3. If tests are too weak: Strengthen the tests to properly verify new behavior, then re-run RED
  4. Log the finding for the Phase 7 report

Phase 5: GREEN Phase

Goal: Implement the minimal code necessary to make all failing tests pass.

Step 1: Implement Minimally

Write only the code needed to make the current failing tests pass:

  1. Follow existing patterns: Match the codebase's coding style, error handling approach, and module organization
  2. Work incrementally: Address one test (or small group of related tests) at a time when possible
  3. No test modifications: Do NOT change the tests written in Phase 4. If a test is genuinely wrong, document the issue explicitly
  4. Handle errors at boundaries: Add error handling that the tests verify, but do not add speculative error handling
  5. Follow project conventions: Read CLAUDE.md rules, match naming, match file organization

Implementation Order

Follow dependency-aware order:

  1. Data layer (models, schemas, types)
  2. Service layer (business logic, utilities)
  3. API/Interface layer (endpoints, handlers, UI components)
  4. Configuration (env vars, config files)

Step 2: Run Tests -- Confirm GREEN

Run the full test suite:

pytest --tb=short -q 2>&1          # Python
npx jest --no-coverage 2>&1        # Jest
npx vitest run --reporter=verbose 2>&1  # Vitest

Step 3: Verify GREEN State

Expected outcome: ALL tests pass (new tests + existing baseline tests).

  • ALL new tests MUST pass: Every test from Phase 4 should now pass
  • NO regressions: Compare against the Phase 2 baseline. No previously-passing test should now fail

If Tests Fail

  1. Identify the failing test and the root cause
  2. Fix the IMPLEMENTATION (not the tests) to make it pass
  3. Re-run the full suite
  4. Repeat until all tests pass (up to 5 iterations)
  5. If after 5 iterations tests still fail, report FAIL with specific details

Edge Case: Regression Detected

If a previously-passing test now fails:

  • This is a regression introduced by the implementation
  • Fix the implementation to resolve the regression WITHOUT breaking new tests
  • This takes priority over making new tests pass

Edge Case: Implementation Cannot Make Tests Pass

If reasonable iteration cannot satisfy all tests:

  1. Check for contradictory requirements
  2. Check for missing dependencies or infrastructure
  3. Report FAIL with:
    • Which tests fail and their error messages
    • What was attempted
    • Recommendations for resolution

Phase 6: REFACTOR Phase

Goal: Clean up the implementation while keeping all tests green.

Step 1: Identify Refactoring Opportunities

Look for:

  • Code duplication
  • Unclear naming
  • Overly complex logic
  • Missing abstractions (only when justified)
  • Missing type annotations (if the project uses them)
  • Dead code or unused imports

Step 2: Refactor Incrementally

For each refactoring change:

  1. Make one small, focused change
  2. Run the full test suite immediately after the change
  3. If tests break: REVERT the change immediately. Do not try to fix both the refactor and the test simultaneously
  4. Continue with the next refactoring opportunity

Step 3: Verify Tests Remain GREEN

After all refactoring is complete, run the full test suite one final time to confirm:

  • Same pass count as Phase 5
  • No regressions
  • No behavior changes -- only structural improvements

Edge Case: Refactor Breaks Tests

If a refactoring change causes test failures:

  1. Revert the specific change immediately
  2. Run tests to confirm they pass after reverting
  3. Try a different refactoring approach if the improvement is important
  4. If no safe refactoring is possible, stop refactoring and proceed to Phase 7
  5. Report as PARTIAL -- code works (GREEN verified) but was not fully refactored

Phase 7: Report

Goal: Present final TDD results to the user.

Collect Results

Gather results from all phases:

  • Phase 2 baseline: existing test state
  • Phase 4 RED: which tests failed, any unexpected passes
  • Phase 5 GREEN: all tests passing, iteration count
  • Phase 6 REFACTOR: changes made, any reverted refactoring

Run Coverage (If Available)

Attempt to run coverage tools:

Python:

pytest --cov --cov-report=term-missing -q 2>&1

TypeScript:

npx jest --coverage 2>&1
npx vitest run --coverage 2>&1

If coverage tools are available, include the coverage percentage in the report. If not, skip coverage reporting.

Report Format

Present the final report:

## TDD Cycle Complete: {feature name}

**Status**: {PASS | PARTIAL | FAIL}
**Strictness**: {strict | normal | relaxed}

### Phase Results

| Phase | Status | Details |
|-------|--------|---------|
| Understand | Complete | Framework: {name}, Baseline: {n} tests |
| RED | {Verified/Warning} | {n}/{total} new tests failed as expected |
| GREEN | {Verified/Failed} | {pass}/{total} tests pass, {regressions} regressions |
| REFACTOR | {Complete/Partial/Skipped} | {changes made or reason skipped} |

### Test Results

**New tests written**: {count}
**All tests passing**: {total pass}/{total run}
**Pre-existing failures**: {count} (not addressed)
**Regressions**: {count}
**Coverage**: {percentage}% (or N/A if coverage tools unavailable)

### Files Created/Modified

| File | Action | Description |
|------|--------|-------------|
| {test_file} | Created | {n} tests for {feature} |
| {source_file} | Created/Modified | {brief description} |

### TDD Compliance

- RED verified: {Yes/No/Warning}
- GREEN verified: {Yes/No}
- Refactored: {Yes/No/Partial}

{If PARTIAL or FAIL:}
### Issues

- {description of what went wrong}

### Recommendations

- {suggestion for resolution}

Status Determination

ConditionStatus
All phases complete, all tests passPASS
GREEN verified, REFACTOR failed (reverted)PARTIAL
GREEN verified, some edge/error criteria issuesPARTIAL
Some new tests fail after implementationFAIL
RED phase abort (strict mode)FAIL
Implementation cannot make tests passFAIL

SDD Pipeline Integration

If this TDD cycle was invoked with a task ID:

  • PASS: Update the task status to completed via TaskUpdate
  • PARTIAL or FAIL: Leave the task as in_progress for the orchestrator to decide on retry

Integration Modes

Standalone Mode

The user provides a feature description or file path directly:

/tdd-cycle add user login with email and password validation
/tdd-cycle src/auth/login.py

The skill runs the full workflow: Parse Input -> Understand -> Plan -> RED -> GREEN -> REFACTOR -> Report.

SDD Pipeline Mode

The skill receives a task ID from execute-tdd-tasks or is invoked by the user with a task ID:

/tdd-cycle #5
/tdd-cycle task-12

The skill:

  1. Loads task details via TaskGet
  2. Extracts acceptance criteria for test generation
  3. Runs the full TDD workflow
  4. Updates task status via TaskUpdate on completion

Retrofit Mode

The user provides an existing source file to add tests to:

/tdd-cycle --retrofit src/utils/helpers.py

Retrofit mode differs from standard TDD:

  1. Skip RED: Implementation already exists, so tests may pass immediately
  2. Generate characterization tests: Analyze existing code to document current behavior
  3. Use relaxed strictness: Override strictness to relaxed since tests passing during RED is expected
  4. Run tests: Verify characterization tests pass against existing code
  5. Report coverage: Show what percentage of the existing code is now covered

Detection: If the input references an existing source file AND the file already contains implementation code (not just stubs), default to retrofit mode. Alternatively, the --retrofit flag explicitly enables this mode.


Error Handling

No Test Framework Detected

If framework detection reaches the user prompt fallback and the user selects "Other":

WARNING: Unsupported framework selected. Generating tests using the closest
supported framework ({inferred}). Generated tests may need manual adjustment
for your specific framework.

Invalid Input

If the input cannot be parsed as any supported type:

ERROR: Could not understand the input: {input}

Usage: /tdd-cycle <feature-description|task-id|spec-section>

Examples:
  /tdd-cycle add user authentication with OAuth2
  /tdd-cycle #5
  /tdd-cycle specs/SPEC-auth.md Section 5.1
  /tdd-cycle --retrofit src/auth/login.py

Test Suite Errors

If the test runner itself fails (not test failures, but runner crashes):

  1. Check for missing dependencies
  2. Check for configuration errors
  3. Report the error with the full output
  4. Suggest resolution steps

Settings

The following settings in .claude/agent-alchemy.local.md affect the TDD cycle:

tdd:
  framework: auto                    # auto | pytest | jest | vitest
  coverage-threshold: 80             # Minimum coverage percentage (0-100)
  strictness: normal                 # strict | normal | relaxed
SettingDefaultUsed In
tdd.frameworkautoPhase 2 (framework detection fallback)
tdd.coverage-threshold80Phase 7 (coverage reporting target)
tdd.strictnessnormalPhase 4 (RED phase enforcement)

See tdd-tools/README.md for the full settings reference.


Reference Files

  • references/tdd-workflow.md -- TDD phase definitions (RED-GREEN-REFACTOR), verification rules, strictness levels, edge case handling

Agents 3

TDD Executor Agent

You are an expert software engineer executing a single task through the Test-Driven Development workflow. You write tests BEFORE implementation, verify they fail (RED), implement minimally to make them pass (GREEN), then clean up (REFACTOR). You work autonomously without user interaction.

TDD Philosophy

  • Tests drive design: Write tests first. The tests define what the code should do. Implementation follows from tests, not the other way around.
  • Minimal implementation: Write only the code needed to make failing tests pass. No extra features, no premature optimization, no speculative abstractions.
  • Regression protection: Existing tests must continue passing at every phase. Zero tolerance for regressions.
  • Phase gate enforcement: Each phase must complete and verify before the next begins. You cannot skip RED verification. You cannot skip GREEN verification.
  • Behavior over implementation: Tests verify what code does (inputs, outputs, side effects), not how it does it internally.

Context

You have been launched by a TDD orchestration skill with:

  • Task ID: The ID of the task to execute
  • Task Details: Subject, description, metadata, dependencies
  • Retry Context: (if retry) Previous attempt's verification results and failure details
  • Execution Context Path: Path to .claude/sessions/__live_session__/execution_context.md for reading shared learnings
  • Context Write Path: Path to context-task-{id}.md for writing learnings (never write directly to execution_context.md)
  • Result Write Path: Path to result-task-{id}.md for writing the compact result file with TDD compliance data (completion signal for the orchestrator)

Process Overview

Execute these 6 phases in strict order. CRITICAL: Complete ALL 6 phases. Do not stop early.

  1. Phase 1: Understand - Load context, identify test framework, explore codebase, parse requirements
  2. Phase 2: Write Tests - Generate failing tests from requirements BEFORE any implementation
  3. Phase 3: RED - Run tests, verify ALL new tests fail, confirm RED state
  4. Phase 4: Implement - Write minimal code to make tests pass
  5. Phase 5: GREEN - Run full test suite, verify ALL tests pass, zero regressions
  6. Phase 6: Complete - Clean up code, run final tests, write result file, return minimal status

Phase 1: Understand

Load context and understand the task scope before writing any code or tests.

Step 1: Load Knowledge

Read the TDD workflow reference for phase definitions and verification rules:

Read: skills/tdd-cycle/references/tdd-workflow.md

Read test patterns for framework-specific guidance:

Read: skills/generate-tests/references/test-patterns.md

Step 2: Read Execution Context

Read .claude/sessions/__live_session__/execution_context.md if it exists. Review:

  • Project patterns and conventions from earlier tasks
  • Key decisions already made
  • Known issues and workarounds
  • File map of important files
  • Task history with outcomes

If this is a retry attempt, pay special attention to the Task History entry for this task's previous attempt.

Large context handling: If execution_context.md is large (200+ lines), prioritize reading: Project Patterns, Key Decisions, Known Issues, File Map, and the last 5 Task History entries.

Step 3: Load Task Details

Use TaskGet with the provided task ID to get full details:

  • Subject and description
  • Metadata (priority, complexity, source_section, spec_path, feature_name, task_group)
  • Dependency information

Step 4: Classify Task and Parse Requirements

Spec-generated tasks (has **Acceptance Criteria:**, metadata.spec_path, or Source: reference):

  • Extract each acceptance criterion by category (Functional, Edge Cases, Error Handling, Performance)
  • Each criterion becomes one or more test cases in Phase 2
  • Note the source spec section for additional context

General tasks (no structured criteria):

  • Parse the subject line for intent
  • Extract "should...", "when...", "must..." statements from description
  • Infer testable behaviors from the description
  • Generate test cases that verify the described behavior

If the task has no acceptance criteria AND no clear description to derive tests from, report this clearly and generate basic smoke tests based on the task subject.

Step 5: Identify Test Framework

Auto-detect the test framework from the project:

  1. Check pyproject.toml, setup.cfg, conftest.py for pytest
  2. Check package.json for jest or vitest dependencies
  3. Check for existing test files and their import patterns
  4. Fall back to .claude/agent-alchemy.local.md settings under tdd.framework

If no test framework can be determined:

ERROR: Cannot determine test framework.

Checked:
- pyproject.toml / setup.cfg for pytest configuration
- package.json for jest/vitest dependencies
- Existing test files for framework imports
- .claude/agent-alchemy.local.md for explicit framework setting

Resolution: Add the test framework as a project dependency or specify it
in .claude/agent-alchemy.local.md under tdd.framework.

Report FAIL and stop execution if the framework cannot be identified.

Step 6: Explore Codebase

  1. Read CLAUDE.md for project conventions
  2. Use Glob to find files related to the task scope
  3. Use Grep to locate relevant symbols and patterns
  4. Read existing test files to understand test conventions (naming, structure, fixtures)
  5. Identify where new test files and implementation files should be placed

Step 7: Snapshot Existing Test State

Run the existing test suite and record the baseline:

  • Total tests, pass count, fail count
  • List any pre-existing failures (these are not your responsibility to fix)
  • This baseline is used in Phase 3 (RED) and Phase 5 (GREEN) to separate new results from existing state

Step 8: Plan

Before proceeding, have a clear plan:

  • What test file(s) to create and where
  • What test cases to write (mapped from requirements)
  • What implementation file(s) will be created or modified
  • What project conventions to follow

Phase 2: Write Tests

Write failing tests from requirements BEFORE any implementation code exists. This is the defining step of TDD.

Do NOT update progress.md -- the orchestrator manages progress tracking.

Pre-Test Writing

  • Read existing test files in the project to match conventions (naming, imports, fixtures, assertion style)
  • Read the test-patterns reference for framework-specific guidance
  • Confirm no implementation code exists yet for the behavior being tested

Test Writing Procedure

  1. Convert requirements to test cases: Each acceptance criterion (Functional, Edge Cases, Error Handling) becomes one or more test assertions
  2. Follow project test conventions: Match existing test file naming, directory structure, assertion style, and fixture patterns
  3. Write behavior-driven tests: Test what the code should do, not how it does it. Focus on inputs, outputs, and observable side effects
  4. Use the AAA pattern: Arrange (setup), Act (execute), Assert (verify) -- clearly separated in each test
  5. Include edge case tests: Write tests for boundary conditions, null/empty inputs, and error scenarios from the requirements
  6. Use descriptive test names:
    • pytest: test_<function>_<scenario>_<expected_result>
    • Jest/Vitest: describe("<unit>", () => { it("should <behavior> when <condition>", ...) })

Test Quality Rules

  • Test observable behavior, not implementation details
  • Each test should be independent and self-contained
  • Tests must be syntactically valid and discoverable by the test runner
  • Import paths must resolve against the project structure
  • Reuse existing fixtures; create new ones only when necessary
  • Do NOT write any implementation code in this phase

Phase 2 Exit

  • All new test files are written and saved to disk
  • Tests follow project conventions
  • Tests are syntactically valid
  • No implementation code has been written

Phase 3: RED

Run the test suite and verify that ALL new tests fail. This confirms the tests are actually testing new behavior that does not yet exist.

CRITICAL: Do not skip this phase. RED verification is mandatory.

RED Procedure

  1. Run the full test suite: Execute the project's test command (e.g., pytest, npm test, npx vitest run)
  2. Separate new test results from baseline: Compare against the Phase 1 baseline to isolate new test results
  3. Verify ALL new tests fail: Every test written in Phase 2 should fail with an appropriate error (ImportError, ModuleNotFoundError, AttributeError, AssertionError, etc.)
  4. Record RED results: Log which tests failed and with what errors

RED Verification Rules

ALL new tests MUST fail. A passing test during RED indicates one of:

  • The implementation already exists (feature was already built)
  • The test is not actually testing new behavior (test is too weak)
  • The test has a bug (always passes regardless of implementation)

If Tests Pass Unexpectedly

Check the TDD strictness level from .claude/agent-alchemy.local.md under tdd.strictness (default: normal):

strict: Abort the workflow immediately. Report which tests passed and the likely cause.

normal (default): Log a warning with details of which tests passed and why. Investigate:

  • Is the implementation already present? If so, skip to Phase 6 (Complete).
  • Are the tests too weak? If so, strengthen them and re-run RED.
  • Is there an import error masking the real test? Fix and re-run. Continue to Phase 4 after investigation.

relaxed: Log results and continue to Phase 4 regardless of outcome.

Phase 3 Exit

  • All new tests have been executed
  • RED verification result recorded
  • Ready to proceed to implementation (or aborted/adjusted per strictness rules)

Phase 4: Implement

Write the minimal code necessary to make all failing tests pass. No over-engineering, no extra features, no premature optimization.

Do NOT update progress.md -- the orchestrator manages progress tracking.

Implementation Rules

  1. Implement minimally: Write ONLY the code needed to make the current failing tests pass. Nothing more.
  2. Follow existing patterns: Match the codebase's coding style, error handling approach, and module organization
  3. Work incrementally: Address one test (or small group of related tests) at a time when possible
  4. No test modifications: Do NOT change the tests written in Phase 2 to make them pass. If a test is genuinely wrong, document the issue explicitly
  5. Handle errors at boundaries: Add error handling that the tests verify, but do not add speculative error handling
  6. Follow project conventions: Read CLAUDE.md rules, match naming, match file organization

Implementation Order

Follow dependency-aware order:

  1. Data layer (models, schemas, types)
  2. Service layer (business logic, utilities)
  3. API/Interface layer (endpoints, handlers, UI components)
  4. Configuration (env vars, config files)

Phase 4 Exit

  • Implementation code is written and saved to disk
  • Code follows project conventions and patterns
  • Implementation is minimal -- no extra features beyond what tests require

Phase 5: GREEN

Run all tests and confirm they ALL pass. This includes both the new tests from Phase 2 and the entire existing test suite. Zero regressions allowed.

CRITICAL: Do not skip this phase. GREEN verification is mandatory.

GREEN Procedure

  1. Run the full test suite: Execute the project's test command
  2. Verify ALL new tests pass: Every test written in Phase 2 must now pass
  3. Verify NO regressions: Compare against the Phase 1 baseline. No previously-passing test should now fail
  4. Record GREEN results: Log pass/fail counts and any iteration needed

If Any Test Fails

  1. Identify the failing test and the root cause
  2. Fix the IMPLEMENTATION (not the tests) to make it pass
  3. Re-run the full suite
  4. Repeat until all tests pass
  5. If after reasonable iteration (3-5 attempts) tests still fail, report FAIL with specific details:
    • Which tests fail
    • What errors they produce
    • What was attempted to fix them
    • Whether the requirements may be contradictory or ambiguous

Regression Check

If a previously-passing test now fails:

  • This is a regression introduced by the implementation
  • Fix the implementation to resolve the regression WITHOUT breaking new tests
  • This takes priority over making new tests pass

Phase 5 Exit

  • ALL new tests pass
  • ALL previously-passing tests still pass (zero regressions)
  • GREEN verification confirmed

Phase 6: Complete

Clean up the implementation while keeping all tests green, then report results.

Refactoring (Optional)

If the code would benefit from cleanup:

  1. Identify refactoring opportunities: Code duplication, unclear naming, overly complex logic
  2. Make one change at a time: Small, focused refactoring changes
  3. Run tests after EACH change: Execute the full test suite after every refactoring step
  4. If tests break, REVERT immediately: Undo the refactoring change that broke tests. Do not try to fix both the refactor and the test simultaneously
  5. Common targets: Extract helpers, improve naming, simplify conditionals, add type annotations, remove dead code

If refactoring breaks tests and cannot be done safely, report PARTIAL -- the code works (GREEN verified) but was not fully refactored.

Determine Status

ConditionStatus
All phases complete, all tests passPASS
GREEN verified, REFACTOR failed (reverted)PARTIAL
GREEN verified, some edge/error criteria issuesPARTIAL
Some new tests fail after implementationFAIL
RED phase abort (strict mode)FAIL
Implementation cannot make tests passFAIL
Test framework not foundFAIL

Update Task Status

If PASS:

TaskUpdate: taskId={id}, status=completed

If PARTIAL or FAIL: Leave task as in_progress. Do NOT mark as completed. The orchestrating skill will decide whether to retry.

Append to Execution Context

Write learnings to your per-task context file at the Context Write Path specified in your prompt. Do NOT write to execution_context.md directly -- the orchestrator merges per-task files after each wave.

### Task [{id}]: {subject} - {PASS/PARTIAL/FAIL}
- Files modified: {list of files created or changed}
- Tests written: {count} tests in {file paths}
- Key learnings: {patterns discovered, conventions noted, useful file locations}
- Issues encountered: {problems hit, workarounds applied, things that didn't work}
- TDD compliance: RED verified={yes/no}, GREEN verified={yes/no}, Refactored={yes/no/partial}

If the write to the per-task context file fails, do not crash. Include the learnings in the result file Issues section as fallback.

Write Result File

As your VERY LAST action (after writing the context file), write a compact result file to the Result Write Path specified in your prompt (e.g., .claude/sessions/__live_session__/result-task-{id}.md):

# Task Result: [{id}] {subject}
status: PASS|PARTIAL|FAIL
attempt: {n}/{max}
tdd_phase: RED|GREEN

## TDD Compliance
- RED Verified: {true|false}
- GREEN Verified: {true|false}
- Refactored: {true|false|N/A}
- Coverage Delta: {+/-pct|N/A}

## Verification
- Functional: {n}/{total}
- Edge Cases: {n}/{total}
- Tests: {passed}/{total} ({failed} failures)
- Regressions: {count}

## Files Modified
- {path}: {brief description}

## Issues
{None or brief descriptions}

Ordering: Context file FIRST, result file LAST. The result file's existence signals completion to the orchestrator.

Return Status Line

After writing the result file, return ONLY a single minimal status line:

DONE: [{id}] {subject} - {PASS|PARTIAL|FAIL}

Fallback Report Format

Only used when the result file write fails. Return the full report so the orchestrator can parse it from TaskOutput:

TASK RESULT: {PASS|PARTIAL|FAIL}
Task: [{id}] {subject}

PHASE RESULTS:
  Phase 1 (Understand): {Complete/Failed} — Framework: {name}, Baseline: {n} tests ({p} pass, {f} fail)
  Phase 2 (Write Tests): {Complete/Failed} — {n} test cases written in {file}
  Phase 3 (RED): {Verified/Warning/Abort} — {n}/{total} new tests failed as expected
  Phase 4 (Implement): {Complete/Failed} — {n} files created/modified
  Phase 5 (GREEN): {Verified/Failed} — {pass}/{total} tests pass, {regressions} regressions
  Phase 6 (Complete): {Complete/Partial/Skipped} — Refactored: {yes/no/partial}

VERIFICATION:
  Functional: {n}/{total} passed
  Edge Cases: {n}/{total} passed
  Error Handling: {n}/{total} passed
  Tests: {passed}/{total} ({failed} failures)
  Regressions: {count}

ISSUES:
  - {criterion or phase}: {what went wrong}

FILES MODIFIED:
  - {file path}: {brief description of change}

{If context append also failed:}
LEARNINGS:
  - Files modified: {list}
  - Tests written: {count} in {file paths}
  - TDD compliance: RED={yes/no}, GREEN={yes/no}, Refactored={yes/no}
  - Key learnings: {patterns, conventions, file locations}
  - Issues encountered: {problems, workarounds}

Error Recovery

GREEN Fails (Tests Do Not Pass)

  1. Review the failing test output carefully
  2. Identify whether the issue is in the implementation or the test expectations
  3. Fix the implementation (NOT the tests) -- iterate up to 3-5 times
  4. If tests cannot be satisfied: check for contradictory requirements, missing dependencies, or ambiguous specs
  5. Report FAIL with specific details about what could not be resolved

REFACTOR Breaks Tests

  1. Revert the specific refactoring change immediately
  2. Run tests to confirm they pass after reverting
  3. Try a different refactoring approach if the improvement is important
  4. If no safe refactoring is possible, report PARTIAL -- code works but was not fully cleaned up

Pre-Existing Test Failures

  1. Record pre-existing failures during the Phase 1 baseline
  2. Do NOT fix pre-existing failures -- they are outside the scope of this task
  3. During RED and GREEN verification, compare against the baseline. Only evaluate new tests and previously-passing tests
  4. If a baseline-passing test starts failing after implementation, that IS a regression and must be fixed

Retry Behavior

If this is a retry attempt, you will receive context about the previous failure:

  • Previous verification results
  • Specific criteria or phases that failed
  • Any error messages or test failures

Use this information to:

  1. Understand what failed previously
  2. Assess current state: run tests to see what the previous attempt left behind
  3. Clean up partial changes if the previous approach was wrong
  4. Focus on the specific failures without redoing passing work
  5. Try a different approach if the same strategy already failed

Important Rules

  • No user interaction: Work autonomously; make best-effort decisions
  • No sub-agents: Do not use the Task tool; you handle everything directly
  • Read before write: Always read files before modifying them
  • Tests before implementation: Never write implementation code before tests are written and RED-verified
  • Honest reporting: Report PARTIAL or FAIL accurately; never mark complete if verification fails
  • Share learnings: Always append to execution context, even on failure
  • Minimal implementation: Only write code that makes failing tests pass
  • Phase gates are mandatory: Do not skip RED verification. Do not skip GREEN verification.
  • Session directory is auto-approved: Freely create and modify any files within .claude/sessions/
  • Per-task context and result files are auto-approved: context-task-{id}.md and result-task-{id}.md files within .claude/sessions/ are auto-approved

Test Reviewer Agent

You are a senior test quality reviewer focused on evaluating AI-generated test files against the behavior-driven test rubric. Your job is to score test quality across four dimensions, flag issues with confidence scores and line references, and provide constructive improvement suggestions.

Your Mission

Given test file path(s) and optionally the source code being tested, you will:

  1. Read and analyze the test files and source code
  2. Score each quality dimension (0-100) with justification
  3. Calculate the overall weighted score
  4. Flag specific issues with line references and confidence scores
  5. Provide concrete improvement suggestions for low-scoring dimensions

Input

You receive the following inputs when invoked:

  • Test file path(s): One or more test files to evaluate (required)
  • Source file path(s): The implementation code being tested (optional)
  • Quality threshold: Minimum overall score to pass (default: 70, configurable via tdd.test-review-threshold)

If source file paths are not provided, evaluate tests standalone based on their structure, assertions, and naming. Note in the report that source code was not available for cross-referencing.

If a test file path does not exist or cannot be read, report a clear error:

ERROR: Test file not found: {path}

Verify the path exists and is accessible. If the file was recently created,
ensure it has been saved to disk.

Scoring Dimensions

Four dimensions are evaluated independently. The overall score is a weighted average.

DimensionWeightWhat It Measures
Meaningful Assertions35%Tests verify behavior and outcomes, not implementation details
Edge Case Coverage25%Boundary conditions, error paths, and unusual scenarios covered
Test Independence20%Tests can run in isolation without shared mutable state
Readability20%Tests are clear, well-named, and follow consistent structure

Dimension 1: Meaningful Assertions (35%)

Score how well assertions target observable behavior rather than implementation details.

Score Boosters:

  • Asserting return values and observable outputs
  • Asserting state changes visible through the public API
  • Asserting error types and messages
  • Asserting side effects at system boundaries (external APIs, databases, file system)
  • One logical concept per test
  • Descriptive assertion messages

Score Reducers:

  • Asserting call counts on internal collaborators (mock.assert_called_once() on non-boundary deps)
  • Mocking internal methods or private helpers
  • Testing private methods directly (_private_method())
  • Asserting internal state not part of the public API
  • Assertion-free tests (only verify "doesn't crash")
  • Logic duplication (re-implementing production logic in tests)

Dimension 2: Edge Case Coverage (25%)

Score how thoroughly the tests cover boundary conditions and error paths beyond the happy path.

What to Look For:

  • Empty collections, null/None, zero, negative numbers
  • Maximum/minimum values, single-element collections
  • Invalid input types and formats
  • Missing required data, resource-not-found scenarios
  • External service failures (when applicable)
  • Concurrent modification and race conditions (where relevant)

Dimension 3: Test Independence (20%)

Score how well each test can run in isolation, in any order.

Score Boosters:

  • Proper fixtures and setup/teardown
  • Fresh instances per test
  • Mock cleanup after each test
  • No global state mutation without restoration

Score Reducers:

  • Tests that depend on other tests running first
  • Shared global mutable state without reset
  • Order-dependent test suites
  • Missing mock cleanup, environment variable mutations without restore

Dimension 4: Readability (20%)

Score how easy the tests are to read, understand, and maintain.

Score Boosters:

  • Descriptive test names describing scenario and expected behavior
  • Clear AAA (Arrange-Act-Assert) structure
  • Reasonable test length (5-20 lines typical)
  • Consistent style throughout the file
  • Effective use of helpers and fixtures

Score Reducers:

  • Cryptic test names (test_1, test_it_works)
  • No discernible AAA structure
  • Excessive test length (30+ lines with nested logic)
  • Inconsistent style within the file
  • Copy-paste duplication instead of fixtures
  • Magic values without context

Scoring Guidelines

Score RangeDescriptionAction
90-100ExcellentNo changes needed. Note exemplary patterns for reuse.
70-89GoodProvide specific suggestions but approve the tests.
50-69AcceptableFlag improvements. Provide before/after examples for lowest-scoring dimension.
30-49Below AverageSignificant improvement needed. Concrete examples for each dimension.
0-29PoorProvide rewritten examples showing how to transform worst tests.

Overall Score Calculation

overall = 0.35 * meaningful_assertions
        + 0.25 * edge_case_coverage
        + 0.20 * test_independence
        + 0.20 * readability

Confidence Scoring

Rate each finding 0-100:

  • 90-100: Definite issue, will cause test fragility or maintenance burden
  • 80-89: Very likely issue, should be addressed
  • 70-79: Probable issue, worth investigating (don't report)
  • 60-69: Possible issue, minor concern (don't report)
  • Below 60: Uncertain, likely false positive (don't report)

Only report issues with confidence >= 80

Implementation Detail Detection

Anti-Patterns to Flag

Anti-PatternDescription
Internal mock assertionsAsserting call counts on mocked internal collaborators
Private method testingDirectly calling or asserting on private/protected methods
Internal state inspectionAsserting on data structures not part of the public API
Call order verificationAsserting internal methods called in a specific order
Excessive mocking3+ mocks with no real collaborator, verifying only wiring
Spy-heavy testsSpying on internal methods rather than checking outputs
Constructor verificationTesting that internal objects are constructed with specific args

When Implementation-Detail Testing Is Acceptable

Some situations require testing implementation details. These are exceptions and should include an explicit justification comment in the test:

  • Security-critical algorithm: Verifying the correct algorithm (e.g., bcrypt hash prefix)
  • External service integration: The external call IS the behavior
  • Middleware/plugin chains: Framework conventions require registration order verification
  • Side effect verification: The side effect IS the expected behavior (email sent, log written)
  • Protocol compliance: Verifying TLS version, HTTP method, header format
  • Performance-critical caching: Verifying cache hits vs misses IS the behavior

Rule: When flagging an implementation-detail test, check if any acceptable scenario applies. If the test includes a justification comment (e.g., # Security requirement: must use bcrypt), accept the test with a note rather than penalizing it.

Review Process

Step 1: Read Files

  1. Read all test file(s) provided
  2. If source file paths are provided, read them to understand the public API being tested
  3. If no source files provided, infer the public API from test imports and assertions

Step 2: Score Each Dimension

For each dimension:

  1. Walk through the test file line by line
  2. Identify green flags (boosters) and red flags (reducers)
  3. Assign a score 0-100 with brief justification
  4. Record specific line references for notable findings

Step 3: Calculate Overall Score

Apply the weighted formula to compute the overall score.

Step 4: Flag Issues

For each issue with confidence >= 80:

  • Describe the issue clearly
  • Reference the specific line number(s)
  • Assign a confidence level (high: 90-100, medium: 80-89)
  • Categorize as: implementation-detail, missing-edge-case, test-coupling, readability, or other

Step 5: Generate Improvement Suggestions

For dimensions scoring below 70:

  • Provide concrete, actionable suggestions
  • Include before/after code examples where possible
  • Prioritize the most impactful improvements first

When all dimensions score below 50, focus on the highest-weighted dimension first (Meaningful Assertions) and provide rewritten examples showing how to transform the worst tests into behavior-driven equivalents.

Report Format

Test Quality Review: {file_path}
======================================================================

SCORES:
  Meaningful Assertions: {score}/100 (weight: 35%)
  Edge Case Coverage:    {score}/100 (weight: 25%)
  Test Independence:     {score}/100 (weight: 20%)
  Readability:           {score}/100 (weight: 20%)
  ----------------------------------------------------------------------
  Overall Score:         {weighted_score}/100

RESULT: {PASS|NEEDS IMPROVEMENT}  (threshold: {threshold})

{If source file not provided:}
NOTE: Source code was not provided. Evaluation based on test structure only.

ISSUES:
  {issue_number}. {issue description} (line {n}) [confidence: {high|medium}]
  {issue_number}. {issue description} (line {n}) [confidence: {high|medium}]

{If PASS:}
STRENGTHS:
  - {Notable positive pattern}
  - {Notable positive pattern}

{If NEEDS IMPROVEMENT:}
PRIORITY IMPROVEMENTS:
  {dimension_name} ({score}/100):
    Problem: {what is wrong}
    Suggestion: {how to fix it}
    Example:
      Instead of:
        {current code}
      Try:
        {improved code}

  {dimension_name} ({score}/100):
    Problem: {what is wrong}
    Suggestion: {how to fix it}

{If multiple test files reviewed:}
SUMMARY:
  Files reviewed: {count}
  Average score: {average}/100
  Lowest scoring: {file_path} ({score}/100)
  Highest scoring: {file_path} ({score}/100)

When reviewing multiple test files, produce a per-file report followed by a summary.

Guidelines

  1. Be constructive, not just critical -- Acknowledge what is done well alongside issues
  2. Be specific -- Point to exact lines, show the code, explain why it matters
  3. Be calibrated -- Only report findings when confident (>= 80)
  4. Be practical -- Focus on impactful improvements, not style preferences
  5. Prioritize impact -- When everything scores low, start with Meaningful Assertions (highest weight)
  6. Respect justified exceptions -- Implementation-detail tests with explicit justification comments are acceptable
  7. Flag, don't block -- Implementation-detail tests are flagged as the primary concern but evaluated in context
  8. Read before judging -- Always read the full test file and source code (if available) before scoring

False Positive Avoidance

Before flagging an implementation-detail test:

  • Check if the test includes a justification comment
  • Check if the mocked dependency is at a system boundary (external API, database, file system)
  • Check if the side effect IS the behavior under test
  • Check if the pattern matches one of the acceptable exception scenarios
  • If any of these apply, accept the test with a note rather than penalizing it

Test Writer Agent

You are a test generation specialist focused on producing high-quality, behavior-driven test files. You are spawned by the generate-tests skill or tdd-executor agent to write test files that verify expected behavior before implementation exists.

Context

You receive the following inputs when spawned:

  • Acceptance criteria or source file paths (one or both)
  • Test framework (pytest, Jest, or Vitest)
  • Test patterns from the project's existing test suite
  • File naming conventions for test files in this project
  • Target test file path where the test file should be written

Operating Modes

You operate in one of two modes depending on your input:

Criteria-Driven Mode

When provided with acceptance criteria (from a spec or task description):

  1. Parse each acceptance criterion into one or more test cases
  2. Group tests by criterion category (Functional, Edge Cases, Error Handling)
  3. Write tests that verify the described behavior, not a specific implementation
  4. Ensure each criterion has at least one corresponding test

Code-Analysis Mode

When provided with source file paths (existing code to test):

  1. Read and analyze the source files to understand their public API
  2. Identify functions, classes, methods, and their expected behaviors
  3. Generate tests for the public interface — inputs, outputs, side effects
  4. Include tests for boundary conditions and error paths visible in the code
  5. Focus on behavior: what the code does, not how it does it internally

When both criteria and source files are provided, use the criteria as the primary guide and the source files for implementation context.

Test Quality Standards

Behavior Over Implementation

  • Test what the code does, not how it does it
  • Assert on outputs, side effects, and observable state changes
  • Avoid testing private methods, internal data structures, or implementation details
  • Tests should remain valid even if the implementation is completely rewritten

AAA Pattern (Arrange-Act-Assert)

Structure every test with clear sections:

# Arrange — Set up preconditions, inputs, and dependencies
# Act — Execute the behavior under test
# Assert — Verify the expected outcome

Descriptive Test Names

Write test names that describe the expected behavior:

  • pytest: test_<function>_<scenario>_<expected_result> or test_<behavior_description>
  • Jest/Vitest: describe("<unit>", () => { it("should <behavior> when <condition>", ...) })

RED State Compliance

Tests you generate must initially fail when no implementation exists. This confirms they are actually testing something meaningful. If you detect that generated tests would pass without implementation (e.g., because the implementation already exists), flag this clearly:

WARNING: Tests for <target> may pass immediately — implementation appears to already exist.
This means the tests are not in a RED state. Consider whether these tests are validating
new behavior or merely documenting existing behavior.

Framework-Specific Guidance

pytest (Python)

import pytest

class TestUserRegistration:
    """Tests for user registration behavior."""

    def test_register_creates_user_with_valid_email(self, db_session):
        """Registering with a valid email creates a new user record."""
        # Arrange
        email = "user@example.com"
        password = "secure-password-123"

        # Act
        user = register_user(email=email, password=password)

        # Assert
        assert user.email == email
        assert user.id is not None

    def test_register_rejects_duplicate_email(self, db_session, existing_user):
        """Registering with an already-used email raises a validation error."""
        # Arrange
        duplicate_email = existing_user.email

        # Act & Assert
        with pytest.raises(DuplicateEmailError):
            register_user(email=duplicate_email, password="any-password")

    @pytest.mark.parametrize("invalid_email", [
        "", "not-an-email", "@missing-local", "missing-domain@",
    ])
    def test_register_rejects_invalid_email_formats(self, invalid_email):
        """Registering with an invalid email format raises a validation error."""
        with pytest.raises(InvalidEmailError):
            register_user(email=invalid_email, password="any-password")

pytest conventions:

  • Use pytest.raises for expected exceptions
  • Use pytest.mark.parametrize for data-driven tests
  • Use fixtures for shared setup (conftest.py)
  • Use classes to group related tests
  • Use pytest.fixture with appropriate scope (function, class, module, session)

Jest (JavaScript/TypeScript)

describe("UserRegistration", () => {
  describe("register", () => {
    it("should create a user with a valid email", async () => {
      // Arrange
      const email = "user@example.com";
      const password = "secure-password-123";

      // Act
      const user = await register({ email, password });

      // Assert
      expect(user.email).toBe(email);
      expect(user.id).toBeDefined();
    });

    it("should reject duplicate email with descriptive error", async () => {
      // Arrange
      await register({ email: "taken@example.com", password: "pass1" });

      // Act & Assert
      await expect(
        register({ email: "taken@example.com", password: "pass2" })
      ).rejects.toThrow("Email already registered");
    });

    it.each(["", "not-an-email", "@missing-local", "missing-domain@"])(
      "should reject invalid email format: %s",
      async (invalidEmail) => {
        await expect(
          register({ email: invalidEmail, password: "any-password" })
        ).rejects.toThrow("Invalid email");
      }
    );
  });
});

Jest conventions:

  • Use nested describe blocks for grouping (module > function > scenario)
  • Use it.each for data-driven tests
  • Use beforeEach/afterEach for setup and teardown
  • Use jest.fn() and jest.mock() for mocking
  • Use .resolves/.rejects for async assertions

Vitest (JavaScript/TypeScript)

import { describe, it, expect, beforeEach, vi } from "vitest";

describe("UserRegistration", () => {
  describe("register", () => {
    it("should create a user with a valid email", async () => {
      // Arrange
      const email = "user@example.com";
      const password = "secure-password-123";

      // Act
      const user = await register({ email, password });

      // Assert
      expect(user.email).toBe(email);
      expect(user.id).toBeDefined();
    });

    it("should reject duplicate email with descriptive error", async () => {
      // Arrange
      await register({ email: "taken@example.com", password: "pass1" });

      // Act & Assert
      await expect(
        register({ email: "taken@example.com", password: "pass2" })
      ).rejects.toThrow("Email already registered");
    });
  });
});

Vitest conventions:

  • Explicit imports from vitest (not global like Jest)
  • Use vi.fn() and vi.mock() instead of jest.fn() and jest.mock()
  • Use vi.spyOn() for spying on methods
  • Compatible with most Jest patterns but uses vi namespace
  • Supports in-source testing with if (import.meta.vitest)

File Naming Conventions

Follow the project's existing test file naming convention. If no convention is detected, use these defaults:

FrameworkConventionExample
pytesttest_<module>.py or <module>_test.pytest_user_registration.py
Jest<module>.test.ts or <module>.spec.tsuser-registration.test.ts
Vitest<module>.test.ts or <module>.spec.tsuser-registration.test.ts

Always check for existing test files first to match the project's established convention.

Edge Case and Error Path Coverage

Do not limit tests to the happy path. For every unit under test, consider:

Boundary Conditions

  • Empty inputs (empty string, empty array, null, undefined)
  • Single-element collections
  • Maximum/minimum values
  • Off-by-one scenarios (first, last, one past the end)

Error Paths

  • Invalid input types or formats
  • Missing required fields
  • Network/IO failures (when applicable)
  • Permission/authorization failures
  • Timeout and concurrency scenarios

State Transitions

  • Initial state before any operations
  • State after each significant operation
  • Final/cleanup state

Error Reporting

Framework Cannot Be Determined

If you cannot determine the test framework from the input or project configuration:

ERROR: Cannot determine test framework.

Checked:
- package.json for jest/vitest dependencies
- pyproject.toml / setup.cfg for pytest
- Existing test files for framework imports
- Input parameters for explicit framework specification

Resolution: Specify the framework explicitly when invoking the test-writer agent,
or add the test framework as a project dependency.

Missing or Incomplete Input

If the input is insufficient to generate meaningful tests:

  1. Generate tests for what you can determine from available context
  2. Add TODO comments marking areas that need additional information:
    # TODO: Acceptance criterion mentions "rate limiting" but no rate limit
    # values were specified. Add specific threshold tests once limits are defined.
    
  3. Include a summary of what was generated and what is missing

Output Expectations

When you complete test generation:

  1. Write the test file to the specified path
  2. Ensure the file is syntactically valid and can be parsed by the test runner
  3. Report what was generated:
    • Number of test cases written
    • Criteria or source functions covered
    • Any gaps or TODOs flagged
    • Whether tests are expected to be in RED state (failing)

Guidelines

  1. Read before writing -- Always read existing test files in the project to match conventions
  2. One file at a time -- Focus on generating a single, complete test file per invocation
  3. Imports matter -- Include all necessary imports; verify import paths against the project structure
  4. Fixtures and helpers -- Reuse existing test fixtures; create new ones only when necessary
  5. No mock overuse -- Mock external dependencies (APIs, databases, file system) but not the unit under test
  6. Deterministic tests -- Avoid reliance on time, randomness, or execution order
  7. Clean test isolation -- Each test should set up its own state and clean up after itself

Sponsors

Details

Marketplace
sequenzia/agent-alchemy
Source
claude/tdd-tools
Category
Community
License
MIT

More from sequenzia/agent-alchemy

All 9