Core skills library for Claude Code: TDD, debugging, collaboration patterns, and proven techniques

version
4.0.3
GitHub stars
15

Install

/plugin install superpowers@superpowers-dev

Run it inside Claude Code after adding the marketplace with /plugin marketplace add lgbarn/devops-skills.

Contents

Discovered from source

Skills 21

Auto-Documentation

Overview

Generate comprehensive documentation from infrastructure code. This skill creates READMEs, runbooks, and architecture documentation that stays in sync with actual code.

Announce at start: "I'm using the auto-documentation skill to generate documentation."

Documentation Types

1. Module README

For Terraform modules, generate:

# Module: [module-name]

[Brief description of what this module provisions]

## Requirements

| Name | Version |
|------|---------|
| terraform | >= X.Y.Z |
| aws | >= X.Y.Z |

## Providers

| Name | Version |
|------|---------|
| aws | X.Y.Z |

## Resources

| Name | Type |
|------|------|
| aws_instance.this | resource |
| aws_security_group.this | resource |

## Inputs

| Name | Description | Type | Default | Required |
|------|-------------|------|---------|:--------:|
| name | Name prefix for resources | `string` | n/a | yes |
| vpc_id | VPC ID | `string` | n/a | yes |

## Outputs

| Name | Description |
|------|-------------|
| instance_id | ID of the created instance |
| public_ip | Public IP address |

## Usage

```hcl
module "example" {
  source = "./modules/[module-name]"

  name   = "my-instance"
  vpc_id = "vpc-12345"
}

Architecture

[Optional: Mermaid diagram of resources]

Notes

[Any important considerations, limitations, or gotchas]


### 2. Environment Documentation

For environment directories:

```markdown
# Environment: [env-name]

## Overview

[Purpose and scope of this environment]

## Resource Inventory

| Category | Resource Type | Count | Key Resources |
|----------|--------------|-------|---------------|
| Compute | aws_instance | 5 | web-1, web-2, api-1 |
| Database | aws_rds_instance | 2 | primary, replica |
| Network | aws_vpc | 1 | main-vpc |

## Architecture Diagram

```mermaid
graph TB
    subgraph VPC
        subgraph Public
            ALB[Load Balancer]
        end
        subgraph Private
            WEB[Web Servers]
            API[API Servers]
        end
        subgraph Data
            RDS[(Database)]
        end
    end
    ALB --> WEB
    WEB --> API
    API --> RDS

Access Patterns

ServicePortSourcePurpose
ALB4430.0.0.0/0Public HTTPS
Web8080ALBInternal traffic

Dependencies

  • [External services or accounts this depends on]

Deployment

[How to deploy changes to this environment]


### 3. Operational Runbook

For operations documentation:

```markdown
# Runbook: [service/component]

## Overview

[What this runbook covers]

## Common Operations

### Starting/Stopping

```bash
# Start the service
terraform apply -target=module.service

# Stop (with caution)
# See "Emergency Procedures" below

Scaling

# Update instance count
terraform apply -var="instance_count=5"

Health Checks

# Check instance status
aws ec2 describe-instance-status --instance-ids i-xxx

# Check application health
curl https://service.example.com/health

Troubleshooting

Problem: High CPU Usage

Symptoms: CPU > 80% sustained

Investigation:

  1. Check CloudWatch metrics
  2. Review application logs
  3. Check for runaway processes

Resolution:

  • Scale horizontally if traffic-related
  • Restart instance if process issue

Problem: Connection Timeouts

Symptoms: 504 errors at load balancer

Investigation:

  1. Check target group health
  2. Verify security group rules
  3. Check application logs

Resolution:

  • [Steps to resolve]

Emergency Procedures

Service Outage

  1. Assess: Check CloudWatch alarms
  2. Communicate: Notify stakeholders
  3. Mitigate: [Immediate actions]
  4. Resolve: [Fix steps]
  5. Review: Post-incident analysis

Rollback Procedure

# Identify previous state
git log --oneline terraform.tfstate

# Restore previous state (CAREFUL!)
# [State restoration steps]

Contacts

RoleNameContact
On-call[Team][Contact]
Escalation[Manager][Contact]

## Process

### Step 1: Analyze Code

```bash
# Find all Terraform files
find . -name "*.tf" -type f

# Extract structure
grep -h "^variable\|^output\|^resource\|^module" *.tf

Step 2: Parse Components

For each file, extract:

  • Variables with descriptions and defaults
  • Outputs with descriptions
  • Resources with types
  • Module calls with sources

Step 3: Generate Documentation

Based on directory type:

  • Has variables.tf + outputs.tf → Module README
  • Named dev/, staging/, prod/ → Environment docs
  • Contains operational resources → Include runbook sections

Step 4: Review with User

Always present generated docs for review before writing.

Show:

  • What will be created/updated
  • Preview of content
  • Any gaps or questions

Step 5: Write Files

Only after user approval:

  • Write new files
  • Update existing files (preserve custom sections)

Best Practices

Keep Docs in Sync

  • Regenerate after significant changes
  • Use CI to detect doc drift
  • Include doc generation in PR checklist

Custom Sections

Mark custom content to preserve during regeneration:

<!-- BEGIN CUSTOM -->
Your custom content here
<!-- END CUSTOM -->

Diagrams

Use Mermaid for architecture diagrams:

  • Renders in GitHub/GitLab
  • Text-based, version controllable
  • Easy to update

Integration with Memory

Store documentation patterns:

  • Preferred formats
  • Custom sections to preserve
  • Team-specific templates

AWS Profile Management

Overview

Credential mistakes are one of the most common causes of infrastructure accidents. This skill ensures the correct AWS profile is active before any operation.

Announce at start: "I'm using the aws-profile-management skill to verify credentials."

Pre-Operation Verification

Step 1: Check Current Identity

# Get current identity
aws sts get-caller-identity

Expected output includes:

  • Account: AWS account ID
  • Arn: IAM user/role ARN
  • UserId: User or assumed role ID

Step 2: Match to Environment

EnvironmentExpected AccountExpected Role Pattern
dev123456789012-dev-, -developer-
staging234567890123-staging-, -deploy-
prod345678901234-prod-, -admin-

STOP if account doesn't match expected environment.

Step 3: Check Credential Expiry

For assumed roles:

# Check remaining session time
aws sts get-caller-identity 2>&1 | grep -i expir || echo "Credentials valid"

For SSO:

# Check SSO session
aws sso list-accounts 2>&1 || echo "Check SSO login status"

Profile Switching

Using Named Profiles

# List available profiles
aws configure list-profiles

# Set profile for session
export AWS_PROFILE=production

# Or use inline
AWS_PROFILE=production terraform plan

Using AWS SSO

# Login to SSO
aws sso login --profile production

# Verify login
aws sts get-caller-identity --profile production

Using Assume Role

# Assume role and export credentials
eval $(aws sts assume-role \
  --role-arn arn:aws:iam::ACCOUNT:role/ROLE_NAME \
  --role-session-name terraform-session \
  --query 'Credentials.[AccessKeyId,SecretAccessKey,SessionToken]' \
  --output text | \
  awk '{print "export AWS_ACCESS_KEY_ID="$1"\nexport AWS_SECRET_ACCESS_KEY="$2"\nexport AWS_SESSION_TOKEN="$3}')

# Verify
aws sts get-caller-identity

Environment Detection

From Directory Structure

environments/
├── dev/
├── staging/
└── prod/
# Detect environment from path
ENV=$(basename "$(pwd)")
echo "Detected environment: $ENV"

From Terraform Backend

# Check backend configuration
grep -A 10 'backend' *.tf | grep -E 'bucket|key|workspace'

From Workspace

# Check Terraform workspace
terraform workspace show

Safety Checks

Pre-Operation Checklist

Before any Terraform or AWS operation:

  1. Identity Verified

    • Account ID matches environment
    • Role/user is appropriate
    • Credentials not expired
  2. Environment Confirmed

    • Directory matches expected environment
    • Backend configuration is correct
    • No conflicting env vars set
  3. Permission Verified

    • Role has required permissions
    • No unexpected permission errors expected

Red Flags - STOP Immediately

ConditionAction
Account ID doesn't match environmentSTOP - wrong account!
Role seems too permissive for taskVerify with user
Credentials expiredRe-authenticate
Multiple AWS_* env vars setClear and use profile
Unknown account IDVerify before proceeding

Common Issues

Wrong Account Active

Symptoms:

  • Terraform can't find expected resources
  • Plan shows creating resources that exist
  • Permission denied for expected resources

Solution:

# Clear any env vars
unset AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY AWS_SESSION_TOKEN

# Set correct profile
export AWS_PROFILE=correct_profile

# Verify
aws sts get-caller-identity

Expired Credentials

Symptoms:

  • "ExpiredToken" errors
  • "credentials have expired" messages

Solution:

# For SSO
aws sso login --profile your_profile

# For assumed role
# Re-run assume-role command

Conflicting Configurations

Symptoms:

  • Unexpected account appearing
  • Operations in wrong region

Solution:

# Check all credential sources
echo "Profile: $AWS_PROFILE"
echo "Access Key set: ${AWS_ACCESS_KEY_ID:+yes}"
echo "Default region: $AWS_DEFAULT_REGION"
aws configure list

Integration with Other Skills

This skill should be invoked before:

  • terraform-plan-review
  • terraform-drift-detection
  • terraform-state-operations
  • Any AWS CLI operations

The profile verification output should be included in analysis reports to confirm correct environment.

Brainstorming Ideas Into Designs

Overview

Help turn ideas into fully formed designs and specs through natural collaborative dialogue.

Start by understanding the current project context, then ask questions one at a time to refine the idea. Once you understand what you're building, present the design in small sections (200-300 words), checking after each section whether it looks right so far.

The Process

Understanding the idea:

  • Check out the current project state first (files, docs, recent commits)
  • Ask questions one at a time to refine the idea
  • Prefer multiple choice questions when possible, but open-ended is fine too
  • Only one question per message - if a topic needs more exploration, break it into multiple questions
  • Focus on understanding: purpose, constraints, success criteria

Exploring approaches:

  • Propose 2-3 different approaches with trade-offs
  • Present options conversationally with your recommendation and reasoning
  • Lead with your recommended option and explain why

Presenting the design:

  • Once you believe you understand what you're building, present the design
  • Break it into sections of 200-300 words
  • Ask after each section whether it looks right so far
  • Cover: architecture, components, data flow, error handling, testing
  • Be ready to go back and clarify if something doesn't make sense

After the Design

Documentation:

  • Write the validated design to docs/plans/YYYY-MM-DD-<topic>-design.md
  • Use elements-of-style:writing-clearly-and-concisely skill if available
  • Commit the design document to git

Implementation (if continuing):

  • Ask: "Ready to set up for implementation?"
  • Use devops-skills:using-git-worktrees to create isolated workspace
  • Use devops-skills:writing-plans to create detailed implementation plan

Key Principles

  • One question at a time - Don't overwhelm with multiple questions
  • Multiple choice preferred - Easier to answer than open-ended when possible
  • YAGNI ruthlessly - Remove unnecessary features from all designs
  • Explore alternatives - Always propose 2-3 approaches before settling
  • Incremental validation - Present design in sections, validate each
  • Be flexible - Go back and clarify when something doesn't make sense

Dispatching Parallel Agents

Overview

When you have multiple unrelated failures (different test files, different subsystems, different bugs), investigating them sequentially wastes time. Each investigation is independent and can happen in parallel.

Core principle: Dispatch one agent per independent problem domain. Let them work concurrently.

When to Use

digraph when_to_use {
    "Multiple failures?" [shape=diamond];
    "Are they independent?" [shape=diamond];
    "Single agent investigates all" [shape=box];
    "One agent per problem domain" [shape=box];
    "Can they work in parallel?" [shape=diamond];
    "Sequential agents" [shape=box];
    "Parallel dispatch" [shape=box];

    "Multiple failures?" -> "Are they independent?" [label="yes"];
    "Are they independent?" -> "Single agent investigates all" [label="no - related"];
    "Are they independent?" -> "Can they work in parallel?" [label="yes"];
    "Can they work in parallel?" -> "Parallel dispatch" [label="yes"];
    "Can they work in parallel?" -> "Sequential agents" [label="no - shared state"];
}

Use when:

  • 3+ test files failing with different root causes
  • Multiple subsystems broken independently
  • Each problem can be understood without context from others
  • No shared state between investigations

Don't use when:

  • Failures are related (fix one might fix others)
  • Need to understand full system state
  • Agents would interfere with each other

The Pattern

1. Identify Independent Domains

Group failures by what's broken:

  • File A tests: Tool approval flow
  • File B tests: Batch completion behavior
  • File C tests: Abort functionality

Each domain is independent - fixing tool approval doesn't affect abort tests.

2. Create Focused Agent Tasks

Each agent gets:

  • Specific scope: One test file or subsystem
  • Clear goal: Make these tests pass
  • Constraints: Don't change other code
  • Expected output: Summary of what you found and fixed

3. Dispatch in Parallel

CRITICAL: All Task calls must be in a single message to run in parallel.

// Use the Task tool with these parameters:

Task 1:
  description: "Fix abort test failures"
  prompt: "Fix the 3 failing tests in agent-tool-abort.test.ts..."
  subagent_type: "general-purpose"

Task 2:
  description: "Fix batch completion failures"
  prompt: "Fix the 2 failing tests in batch-completion-behavior.test.ts..."
  subagent_type: "general-purpose"

Task 3:
  description: "Fix race condition failures"
  prompt: "Fix the failing test in tool-approval-race-conditions.test.ts..."
  subagent_type: "general-purpose"

// All three in ONE message = parallel execution

Available subagent_type options:

  • general-purpose - For most tasks (searching, coding, multi-step work)
  • Bash - For command execution tasks
  • Explore - For codebase exploration (specify thoroughness: "quick", "medium", "very thorough")
  • Plan - For designing implementation plans
  • Custom agents from agents/ directory (e.g., your defined agents)

Using Explore for parallel codebase analysis:

Task 1:
  description: "Find auth implementation"
  prompt: "Find how authentication is implemented. Thoroughness: medium"
  subagent_type: "Explore"

Task 2:
  description: "Find API endpoints"
  prompt: "Find all API endpoint definitions. Thoroughness: quick"
  subagent_type: "Explore"

Task 3:
  description: "Find database models"
  prompt: "Find all database model definitions. Thoroughness: medium"
  subagent_type: "Explore"

4. Review and Integrate

When agents return:

  • Read each summary
  • Verify fixes don't conflict
  • Run full test suite
  • Integrate all changes

Agent Prompt Structure

Good agent prompts are:

  1. Focused - One clear problem domain
  2. Self-contained - All context needed to understand the problem
  3. Specific about output - What should the agent return?
Fix the 3 failing tests in src/agents/agent-tool-abort.test.ts:

1. "should abort tool with partial output capture" - expects 'interrupted at' in message
2. "should handle mixed completed and aborted tools" - fast tool aborted instead of completed
3. "should properly track pendingToolCount" - expects 3 results but gets 0

These are timing/race condition issues. Your task:

1. Read the test file and understand what each test verifies
2. Identify root cause - timing issues or actual bugs?
3. Fix by:
   - Replacing arbitrary timeouts with event-based waiting
   - Fixing bugs in abort implementation if found
   - Adjusting test expectations if testing changed behavior

Do NOT just increase timeouts - find the real issue.

Return: Summary of what you found and what you fixed.

Common Mistakes

❌ Too broad: "Fix all the tests" - agent gets lost ✅ Specific: "Fix agent-tool-abort.test.ts" - focused scope

❌ No context: "Fix the race condition" - agent doesn't know where ✅ Context: Paste the error messages and test names

❌ No constraints: Agent might refactor everything ✅ Constraints: "Do NOT change production code" or "Fix tests only"

❌ Vague output: "Fix it" - you don't know what changed ✅ Specific: "Return summary of root cause and changes"

When NOT to Use

Related failures: Fixing one might fix others - investigate together first Need full context: Understanding requires seeing entire system Exploratory debugging: You don't know what's broken yet Shared state: Agents would interfere (editing same files, using same resources)

Real Example from Session

Scenario: 6 test failures across 3 files after major refactoring

Failures:

  • agent-tool-abort.test.ts: 3 failures (timing issues)
  • batch-completion-behavior.test.ts: 2 failures (tools not executing)
  • tool-approval-race-conditions.test.ts: 1 failure (execution count = 0)

Decision: Independent domains - abort logic separate from batch completion separate from race conditions

Dispatch:

Agent 1 → Fix agent-tool-abort.test.ts
Agent 2 → Fix batch-completion-behavior.test.ts
Agent 3 → Fix tool-approval-race-conditions.test.ts

Results:

  • Agent 1: Replaced timeouts with event-based waiting
  • Agent 2: Fixed event structure bug (threadId in wrong place)
  • Agent 3: Added wait for async tool execution to complete

Integration: All fixes independent, no conflicts, full suite green

Time saved: 3 problems solved in parallel vs sequentially

Key Benefits

  1. Parallelization - Multiple investigations happen simultaneously
  2. Focus - Each agent has narrow scope, less context to track
  3. Independence - Agents don't interfere with each other
  4. Speed - 3 problems solved in time of 1

Advanced Task Tool Features

Background Execution

Run agents in background while you continue working:

Task:
  description: "Run slow analysis"
  prompt: "Analyze the entire codebase for..."
  subagent_type: "general-purpose"
  run_in_background: true

// Returns immediately with output_file path
// Use Read tool or `tail` to check progress later

Agent Resumption

Resume a previous agent to continue its work:

Task:
  description: "Continue previous analysis"
  prompt: "Continue from where you left off..."
  subagent_type: "general-purpose"
  resume: "<agent-id-from-previous-run>"

// Agent continues with full previous context preserved

Model Selection

Choose appropriate model for task complexity:

Task:
  description: "Quick formatting check"
  prompt: "Check if files follow naming convention..."
  subagent_type: "general-purpose"
  model: "haiku"  // Fast, low-cost for simple tasks

Task:
  description: "Complex architecture analysis"
  prompt: "Design the migration strategy..."
  subagent_type: "general-purpose"
  model: "opus"  // Most capable for complex reasoning

Using Custom Agents

Define agents in agents/ directory, then use them:

Task:
  description: "Security review"
  prompt: "[plan content]"
  subagent_type: "security-reviewer"  // From agents/security-reviewer.md

Verification

After agents return:

  1. Review each summary - Understand what changed
  2. Check for conflicts - Did agents edit same code?
  3. Run full suite - Verify all fixes work together
  4. Spot check - Agents can make systematic errors

Real-World Impact

From debugging session (2025-10-03):

  • 6 failures across 3 files
  • 3 agents dispatched in parallel
  • All investigations completed concurrently
  • All fixes integrated successfully
  • Zero conflicts between agent changes

Executing Plans

Overview

Load plan, review critically, execute tasks in batches, report for review between batches.

Core principle: Batch execution with checkpoints for architect review.

Announce at start: "I'm using the executing-plans skill to implement this plan."

The Process

Step 1: Load and Review Plan

  1. Read plan file
  2. Review critically - identify any questions or concerns about the plan
  3. If concerns: Raise them with your human partner before starting
  4. If no concerns: Create TodoWrite and proceed

Step 2: Execute Batch

Default: First 3 tasks

For each task:

  1. Mark as in_progress
  2. Follow each step exactly (plan has bite-sized steps)
  3. Run verifications as specified
  4. Mark as completed

Step 3: Report

When batch complete:

  • Show what was implemented
  • Show verification output
  • Say: "Ready for feedback."

Step 4: Continue

Based on feedback:

  • Apply changes if needed
  • Execute next batch
  • Repeat until complete

Step 5: Complete Development

After all tasks complete and verified:

  • Announce: "I'm using the finishing-a-development-branch skill to complete this work."
  • REQUIRED SUB-SKILL: Use devops-skills:finishing-a-development-branch
  • Follow that skill to verify tests, present options, execute choice

When to Stop and Ask for Help

STOP executing immediately when:

  • Hit a blocker mid-batch (missing dependency, test fails, instruction unclear)
  • Plan has critical gaps preventing starting
  • You don't understand an instruction
  • Verification fails repeatedly

Ask for clarification rather than guessing.

When to Revisit Earlier Steps

Return to Review (Step 1) when:

  • Partner updates the plan based on your feedback
  • Fundamental approach needs rethinking

Don't force through blockers - stop and ask.

Remember

  • Review plan critically first
  • Follow plan steps exactly
  • Don't skip verifications
  • Reference skills when plan says to
  • Between batches: just report and wait
  • Stop when blocked, don't guess

Finishing a Development Branch

Overview

Guide completion of development work by presenting clear options and handling chosen workflow.

Core principle: Verify tests → Present options → Execute choice → Clean up.

Announce at start: "I'm using the finishing-a-development-branch skill to complete this work."

The Process

Step 1: Verify Tests

Before presenting options, verify tests pass:

# Run project's test suite
npm test / cargo test / pytest / go test ./...

If tests fail:

Tests failing (<N> failures). Must fix before completing:

[Show failures]

Cannot proceed with merge/PR until tests pass.

Stop. Don't proceed to Step 2.

If tests pass: Continue to Step 2.

Step 2: Determine Base Branch

# Try common base branches
git merge-base HEAD main 2>/dev/null || git merge-base HEAD master 2>/dev/null

Or ask: "This branch split from main - is that correct?"

Step 3: Present Options

Present exactly these 4 options:

Implementation complete. What would you like to do?

1. Merge back to <base-branch> locally
2. Push and create a Pull Request
3. Keep the branch as-is (I'll handle it later)
4. Discard this work

Which option?

Don't add explanation - keep options concise.

Step 4: Execute Choice

Option 1: Merge Locally
# Switch to base branch
git checkout <base-branch>

# Pull latest
git pull

# Merge feature branch
git merge <feature-branch>

# Verify tests on merged result
<test command>

# If tests pass
git branch -d <feature-branch>

Then: Cleanup worktree (Step 5)

Option 2: Push and Create PR
# Push branch
git push -u origin <feature-branch>

# Create PR
gh pr create --title "<title>" --body "$(cat <<'EOF'
## Summary
<2-3 bullets of what changed>

## Test Plan
- [ ] <verification steps>
EOF
)"

Then: Cleanup worktree (Step 5)

Option 3: Keep As-Is

Report: "Keeping branch . Worktree preserved at ."

Don't cleanup worktree.

Option 4: Discard

Confirm first:

This will permanently delete:
- Branch <name>
- All commits: <commit-list>
- Worktree at <path>

Type 'discard' to confirm.

Wait for exact confirmation.

If confirmed:

git checkout <base-branch>
git branch -D <feature-branch>

Then: Cleanup worktree (Step 5)

Step 5: Cleanup Worktree

For Options 1, 2, 4:

Check if in worktree:

git worktree list | grep $(git branch --show-current)

If yes:

git worktree remove <worktree-path>

For Option 3: Keep worktree.

Quick Reference

OptionMergePushKeep WorktreeCleanup Branch
1. Merge locally✓--✓
2. Create PR-✓✓-
3. Keep as-is--✓-
4. Discard---✓ (force)

Common Mistakes

Skipping test verification

  • Problem: Merge broken code, create failing PR
  • Fix: Always verify tests before offering options

Open-ended questions

  • Problem: "What should I do next?" → ambiguous
  • Fix: Present exactly 4 structured options

Automatic worktree cleanup

  • Problem: Remove worktree when might need it (Option 2, 3)
  • Fix: Only cleanup for Options 1 and 4

No confirmation for discard

  • Problem: Accidentally delete work
  • Fix: Require typed "discard" confirmation

Red Flags

Never:

  • Proceed with failing tests
  • Merge without verifying tests on result
  • Delete work without confirmation
  • Force-push without explicit request

Always:

  • Verify tests before offering options
  • Present exactly 4 options
  • Get typed confirmation for Option 4
  • Clean up worktree for Options 1 & 4 only

Integration

Called by:

  • subagent-driven-development (Step 7) - After all tasks complete
  • executing-plans (Step 5) - After all batches complete

Pairs with:

  • using-git-worktrees - Cleans up worktree created by that skill

Historical Pattern Analysis

Overview

Analyze git history and memory to learn from past infrastructure changes. Identify patterns, recurring issues, and apply lessons learned to current work.

Announce at start: "I'm using the historical-pattern-analysis skill to learn from past changes."

When to Use

  • Before making changes similar to past changes
  • When investigating recurring issues
  • To understand why infrastructure is configured a certain way
  • To identify change patterns and team practices

Process

Step 1: Define Search Scope

Determine what history to analyze:

  • Specific resources being changed
  • Time period (last month, quarter, year)
  • Specific team members or patterns

Step 2: Git Archaeology

Find Related Commits
# Commits touching specific files
git log --oneline -20 -- "path/to/module/*.tf"

# Commits mentioning resource types
git log --oneline -20 --grep="aws_security_group"

# Commits by pattern in message
git log --oneline -20 --grep="fix\|rollback\|revert"

# Commits in date range
git log --oneline --since="2024-01-01" --until="2024-06-01" -- "*.tf"
Analyze Commit Patterns
# Most frequently changed files
git log --pretty=format: --name-only -- "*.tf" | sort | uniq -c | sort -rn | head -20

# Authors and their focus areas
git shortlog -sn -- "environments/prod/"

# Change frequency by day/time
git log --format="%ad" --date=format:"%A %H:00" -- "*.tf" | sort | uniq -c
Find Reverts and Fixes
# Revert commits
git log --oneline --grep="revert\|Revert"

# Fix commits following changes
git log --oneline --grep="fix\|hotfix\|Fix"

# Commits with "URGENT" or "EMERGENCY"
git log --oneline --grep="urgent\|emergency" -i

Step 3: Analyze Change Patterns

Coupling Analysis

Which files change together?

# For a specific file, what else changes with it?
git log --pretty=format:"%H" -- "modules/vpc/main.tf" | \
  xargs -I {} git show --name-only --pretty=format: {} | \
  sort | uniq -c | sort -rn | head -20
Change Sequences

Common sequences of changes:

  1. VPC changes → followed by security group changes
  2. IAM role changes → followed by policy attachments
  3. RDS changes → followed by parameter group changes
Time Patterns
  • Are prod changes clustered on certain days?
  • Are there "risky" times based on past incidents?
  • How long between staging and prod deployments?

Step 4: Query Memory

Check stored patterns:

memory/projects/<hash>/patterns.json
memory/projects/<hash>/incidents.json

Look for:

  • Similar past changes and outcomes
  • Known issues with these resources
  • User preferences for this type of change

Step 5: Identify Lessons

From Incidents

For each past incident:

  • What was the trigger?
  • How was it detected?
  • What was the fix?
  • What could have prevented it?
From Patterns
  • What changes tend to cause problems?
  • What practices lead to success?
  • What review processes work well?

Step 6: Generate Report

## Historical Pattern Analysis

### Search Scope
- Resources: [resources being analyzed]
- Time period: [date range]
- Related commits found: [count]

### Change Frequency

| Resource/File | Changes (90d) | Last Changed | Primary Authors |
|--------------|---------------|--------------|-----------------|
| modules/vpc/main.tf | 12 | 2024-01-10 | alice, bob |
| environments/prod/main.tf | 8 | 2024-01-08 | alice |

### Change Coupling

These resources typically change together:
1. `aws_security_group.web` ↔ `aws_instance.web` (85% correlation)
2. `aws_iam_role.app` ↔ `aws_iam_policy.app` (100% correlation)

### Past Incidents Related to These Resources

#### Incident: [Date] - [Title]
- **Trigger:** [What caused it]
- **Impact:** [What happened]
- **Resolution:** [How it was fixed]
- **Lesson:** [What we learned]
- **Relevance:** [How this applies to current change]

### Patterns Identified

#### Pattern: [Pattern Name]
- **Observation:** [What we see in history]
- **Frequency:** [How often]
- **Implication:** [What this means for current change]

### Risk Indicators

Based on historical data:
| Indicator | Current Change | Historical Issues |
|-----------|---------------|-------------------|
| Similar to past incident | [Yes/No] | [Details] |
| Frequently problematic resource | [Yes/No] | [Details] |
| Changed by unfamiliar author | [Yes/No] | [Details] |

### Recommendations

Based on historical patterns:
1. [Recommendation 1]
2. [Recommendation 2]

### Questions Raised

[Questions that history suggests we should answer]

Step 7: Update Memory

Store new patterns discovered:

{
  "patterns": [
    {
      "name": "vpc-sg-coupling",
      "description": "VPC changes often require SG updates",
      "confidence": 0.85,
      "last_seen": "2024-01-15"
    }
  ]
}

Common Patterns to Look For

Positive Patterns

  • Consistent naming conventions
  • Regular, small changes vs. big-bang updates
  • Changes preceded by plan review
  • Post-change validation

Warning Patterns

  • Frequent reverts
  • Emergency fixes following changes
  • Clustered failures in specific areas
  • "Temporary" changes that persist

Anti-Patterns

  • Direct prod changes without staging
  • Large changes without incremental steps
  • Missing documentation on complex changes
  • Recurring manual interventions

Integration with Other Skills

This skill feeds into:

  • terraform-plan-review: Provides historical context for risk assessment
  • terraform-drift-detection: Identifies if drift matches past patterns
  • provider-upgrade-analysis: Shows past upgrade experiences

Provider Upgrade Analysis

Overview

Analyze the impact of upgrading Terraform providers or modules before making changes. Identify breaking changes, deprecations, and required code modifications.

Announce at start: "I'm using the provider-upgrade-analysis skill to assess this upgrade."

Process

Step 1: Identify Current State

# Current Terraform version
terraform version

# Current provider versions
terraform providers

# Lock file details
cat .terraform.lock.hcl

Step 2: Identify Target Version

Determine what version to upgrade to:

  • Latest stable
  • Specific version requested
  • Next major/minor version

Step 3: Research Breaking Changes

For each provider/module upgrade:

AWS Provider Example
# Check CHANGELOG
# https://github.com/hashicorp/terraform-provider-aws/blob/main/CHANGELOG.md

# Check UPGRADE guide for major versions
# https://registry.terraform.io/providers/hashicorp/aws/latest/docs/guides/version-5-upgrade
Key Sources
ProviderChangelog LocationUpgrade Guide
AWSGitHub CHANGELOG.md/docs/guides/version-X-upgrade
AzureGitHub CHANGELOG.md/docs/guides/X.0-upgrade-guide
GoogleGitHub CHANGELOG.md/docs/guides/version_X_upgrade

Step 4: Analyze Impact

Breaking Changes Categories
CategoryImpactExample
Removed ResourcesHIGHResource type no longer exists
Removed ArgumentsHIGHRequired attribute removed
Changed DefaultsMEDIUMDefault value changed
Renamed ResourcesMEDIUMResource renamed, state migration needed
New Required ArgsMEDIUMNew required argument added
DeprecationsLOWWill be removed in future
Code Scan

Search for affected resources/attributes:

# Find usage of deprecated resource
grep -r "aws_deprecated_resource" --include="*.tf"

# Find usage of removed argument
grep -r "removed_argument" --include="*.tf"

Step 5: Check Memory

Query memory/global/provider-issues.json for:

  • Known issues with target version
  • Past upgrade experiences
  • Workarounds needed

Step 6: Generate Report

## Provider Upgrade Analysis

### Summary

| Provider | Current | Target | Risk Level |
|----------|---------|--------|------------|
| aws | 4.67.0 | 5.0.0 | HIGH |

### Breaking Changes

#### 1. [Change Name]
- **Type:** Removed Argument
- **Affected Resource:** `aws_instance`
- **Attribute:** `ebs_optimized` default changed
- **Impact:** Instances may change on next apply
- **Required Action:** Explicitly set `ebs_optimized = true`
- **Files Affected:**
  - `modules/compute/main.tf:45`
  - `environments/prod/instances.tf:23`

#### 2. [Change Name]
...

### Deprecations (Future Concern)

| Resource/Attribute | Deprecated In | Removed In | Replacement |
|-------------------|---------------|------------|-------------|
| `aws_old_thing` | 4.50.0 | 5.0.0 | `aws_new_thing` |

### State Migration Required

- [ ] Resource `aws_old` → `aws_new` requires state mv
- [ ] [Other migrations]

### Recommended Upgrade Path

1. **Backup state**
   ```bash
   terraform state pull > backup-$(date +%Y%m%d).tfstate
  1. Update version constraint

    terraform {
      required_providers {
        aws = {
          source  = "hashicorp/aws"
          version = "~> 5.0"
        }
      }
    }
    
  2. Run init

    terraform init -upgrade
    
  3. Apply code changes [List of required changes]

  4. Run plan

    terraform plan
    
  5. Review and apply [After careful review]

Risk Assessment

FactorAssessment
Breaking changesX items
State migrationsY required
Code changesZ files
Estimated effort[Low/Medium/High]
Recommended timing[Now/After testing/Wait]

Known Issues

[From memory or research]


### Step 7: Update Memory

Store findings in `memory/global/provider-issues.json`:
```json
{
  "aws": {
    "5.0.0": {
      "known_issues": [...],
      "upgrade_notes": [...],
      "last_updated": "2024-01-15"
    }
  }
}

Common Upgrade Patterns

Major Version Upgrades (e.g., 4.x → 5.x)

  • Usually have breaking changes
  • Check upgrade guide thoroughly
  • Plan for state migrations
  • Test in non-prod first

Minor Version Upgrades (e.g., 5.1 → 5.2)

  • Usually backward compatible
  • Check for deprecation warnings
  • Review CHANGELOG for new features

Patch Version Upgrades (e.g., 5.1.0 → 5.1.1)

  • Bug fixes only
  • Generally safe to apply
  • Still review CHANGELOG

Safety Checklist

Before recommending upgrade:

  • Breaking changes documented
  • Code changes identified
  • State migrations planned
  • Rollback procedure defined
  • Test environment validated
  • User informed of risks

Code Review Reception

Overview

Code review requires technical evaluation, not emotional performance.

Core principle: Verify before implementing. Ask before assuming. Technical correctness over social comfort.

The Response Pattern

WHEN receiving code review feedback:

1. READ: Complete feedback without reacting
2. UNDERSTAND: Restate requirement in own words (or ask)
3. VERIFY: Check against codebase reality
4. EVALUATE: Technically sound for THIS codebase?
5. RESPOND: Technical acknowledgment or reasoned pushback
6. IMPLEMENT: One item at a time, test each

Forbidden Responses

NEVER:

  • "You're absolutely right!" (explicit CLAUDE.md violation)
  • "Great point!" / "Excellent feedback!" (performative)
  • "Let me implement that now" (before verification)

INSTEAD:

  • Restate the technical requirement
  • Ask clarifying questions
  • Push back with technical reasoning if wrong
  • Just start working (actions > words)

Handling Unclear Feedback

IF any item is unclear:
  STOP - do not implement anything yet
  ASK for clarification on unclear items

WHY: Items may be related. Partial understanding = wrong implementation.

Example:

your human partner: "Fix 1-6"
You understand 1,2,3,6. Unclear on 4,5.

❌ WRONG: Implement 1,2,3,6 now, ask about 4,5 later
✅ RIGHT: "I understand items 1,2,3,6. Need clarification on 4 and 5 before proceeding."

Source-Specific Handling

From your human partner

  • Trusted - implement after understanding
  • Still ask if scope unclear
  • No performative agreement
  • Skip to action or technical acknowledgment

From External Reviewers

BEFORE implementing:
  1. Check: Technically correct for THIS codebase?
  2. Check: Breaks existing functionality?
  3. Check: Reason for current implementation?
  4. Check: Works on all platforms/versions?
  5. Check: Does reviewer understand full context?

IF suggestion seems wrong:
  Push back with technical reasoning

IF can't easily verify:
  Say so: "I can't verify this without [X]. Should I [investigate/ask/proceed]?"

IF conflicts with your human partner's prior decisions:
  Stop and discuss with your human partner first

your human partner's rule: "External feedback - be skeptical, but check carefully"

YAGNI Check for "Professional" Features

IF reviewer suggests "implementing properly":
  grep codebase for actual usage

  IF unused: "This endpoint isn't called. Remove it (YAGNI)?"
  IF used: Then implement properly

your human partner's rule: "You and reviewer both report to me. If we don't need this feature, don't add it."

Implementation Order

FOR multi-item feedback:
  1. Clarify anything unclear FIRST
  2. Then implement in this order:
     - Blocking issues (breaks, security)
     - Simple fixes (typos, imports)
     - Complex fixes (refactoring, logic)
  3. Test each fix individually
  4. Verify no regressions

When To Push Back

Push back when:

  • Suggestion breaks existing functionality
  • Reviewer lacks full context
  • Violates YAGNI (unused feature)
  • Technically incorrect for this stack
  • Legacy/compatibility reasons exist
  • Conflicts with your human partner's architectural decisions

How to push back:

  • Use technical reasoning, not defensiveness
  • Ask specific questions
  • Reference working tests/code
  • Involve your human partner if architectural

Signal if uncomfortable pushing back out loud: "Strange things are afoot at the Circle K"

Acknowledging Correct Feedback

When feedback IS correct:

✅ "Fixed. [Brief description of what changed]"
✅ "Good catch - [specific issue]. Fixed in [location]."
✅ [Just fix it and show in the code]

❌ "You're absolutely right!"
❌ "Great point!"
❌ "Thanks for catching that!"
❌ "Thanks for [anything]"
❌ ANY gratitude expression

Why no thanks: Actions speak. Just fix it. The code itself shows you heard the feedback.

If you catch yourself about to write "Thanks": DELETE IT. State the fix instead.

Gracefully Correcting Your Pushback

If you pushed back and were wrong:

✅ "You were right - I checked [X] and it does [Y]. Implementing now."
✅ "Verified this and you're correct. My initial understanding was wrong because [reason]. Fixing."

❌ Long apology
❌ Defending why you pushed back
❌ Over-explaining

State the correction factually and move on.

Common Mistakes

MistakeFix
Performative agreementState requirement or just act
Blind implementationVerify against codebase first
Batch without testingOne at a time, test each
Assuming reviewer is rightCheck if breaks things
Avoiding pushbackTechnical correctness > comfort
Partial implementationClarify all items first
Can't verify, proceed anywayState limitation, ask for direction

Real Examples

Performative Agreement (Bad):

Reviewer: "Remove legacy code"
❌ "You're absolutely right! Let me remove that..."

Technical Verification (Good):

Reviewer: "Remove legacy code"
✅ "Checking... build target is 10.15+, this API needs 13+. Need legacy for backward compat. Current impl has wrong bundle ID - fix it or drop pre-13 support?"

YAGNI (Good):

Reviewer: "Implement proper metrics tracking with database, date filters, CSV export"
✅ "Grepped codebase - nothing calls this endpoint. Remove it (YAGNI)? Or is there usage I'm missing?"

Unclear Item (Good):

your human partner: "Fix items 1-6"
You understand 1,2,3,6. Unclear on 4,5.
✅ "Understand 1,2,3,6. Need clarification on 4 and 5 before implementing."

GitHub Thread Replies

When replying to inline review comments on GitHub, reply in the comment thread (gh api repos/{owner}/{repo}/pulls/{pr}/comments/{id}/replies), not as a top-level PR comment.

The Bottom Line

External feedback = suggestions to evaluate, not orders to follow.

Verify. Question. Then implement.

No performative agreement. Technical rigor always.

Requesting Code Review

Dispatch devops-skills:code-reviewer subagent to catch issues before they cascade.

Core principle: Review early, review often.

When to Request Review

Mandatory:

  • After each task in subagent-driven development
  • After completing major feature
  • Before merge to main

Optional but valuable:

  • When stuck (fresh perspective)
  • Before refactoring (baseline check)
  • After fixing complex bug

How to Request

1. Get git SHAs:

BASE_SHA=$(git rev-parse HEAD~1)  # or origin/main
HEAD_SHA=$(git rev-parse HEAD)

2. Dispatch code-reviewer subagent:

Use Task tool with devops-skills:code-reviewer type, fill template at code-reviewer.md

Placeholders:

  • {WHAT_WAS_IMPLEMENTED} - What you just built
  • {PLAN_OR_REQUIREMENTS} - What it should do
  • {BASE_SHA} - Starting commit
  • {HEAD_SHA} - Ending commit
  • {DESCRIPTION} - Brief summary

3. Act on feedback:

  • Fix Critical issues immediately
  • Fix Important issues before proceeding
  • Note Minor issues for later
  • Push back if reviewer is wrong (with reasoning)

Example

[Just completed Task 2: Add verification function]

You: Let me request code review before proceeding.

BASE_SHA=$(git log --oneline | grep "Task 1" | head -1 | awk '{print $1}')
HEAD_SHA=$(git rev-parse HEAD)

[Dispatch devops-skills:code-reviewer subagent]
  WHAT_WAS_IMPLEMENTED: Verification and repair functions for conversation index
  PLAN_OR_REQUIREMENTS: Task 2 from docs/plans/deployment-plan.md
  BASE_SHA: a7981ec
  HEAD_SHA: 3df7661
  DESCRIPTION: Added verifyIndex() and repairIndex() with 4 issue types

[Subagent returns]:
  Strengths: Clean architecture, real tests
  Issues:
    Important: Missing progress indicators
    Minor: Magic number (100) for reporting interval
  Assessment: Ready to proceed

You: [Fix progress indicators]
[Continue to Task 3]

Integration with Workflows

Subagent-Driven Development:

  • Review after EACH task
  • Catch issues before they compound
  • Fix before moving to next task

Executing Plans:

  • Review after each batch (3 tasks)
  • Get feedback, apply, continue

Ad-Hoc Development:

  • Review before merge
  • Review when stuck

Red Flags

Never:

  • Skip review because "it's simple"
  • Ignore Critical issues
  • Proceed with unfixed Important issues
  • Argue with valid technical feedback

If reviewer wrong:

  • Push back with technical reasoning
  • Show code/tests that prove it works
  • Request clarification

See template at: requesting-code-review/code-reviewer.md

Subagent-Driven Development

Execute plan by dispatching fresh subagent per task, with two-stage review after each: spec compliance review first, then code quality review.

Core principle: Fresh subagent per task + two-stage review (spec then quality) = high quality, fast iteration

When to Use

digraph when_to_use {
    "Have implementation plan?" [shape=diamond];
    "Tasks mostly independent?" [shape=diamond];
    "Stay in this session?" [shape=diamond];
    "subagent-driven-development" [shape=box];
    "executing-plans" [shape=box];
    "Manual execution or brainstorm first" [shape=box];

    "Have implementation plan?" -> "Tasks mostly independent?" [label="yes"];
    "Have implementation plan?" -> "Manual execution or brainstorm first" [label="no"];
    "Tasks mostly independent?" -> "Stay in this session?" [label="yes"];
    "Tasks mostly independent?" -> "Manual execution or brainstorm first" [label="no - tightly coupled"];
    "Stay in this session?" -> "subagent-driven-development" [label="yes"];
    "Stay in this session?" -> "executing-plans" [label="no - parallel session"];
}

vs. Executing Plans (parallel session):

  • Same session (no context switch)
  • Fresh subagent per task (no context pollution)
  • Two-stage review after each task: spec compliance first, then code quality
  • Faster iteration (no human-in-loop between tasks)

The Process

digraph process {
    rankdir=TB;

    subgraph cluster_per_task {
        label="Per Task";
        "Dispatch implementer subagent (./implementer-prompt.md)" [shape=box];
        "Implementer subagent asks questions?" [shape=diamond];
        "Answer questions, provide context" [shape=box];
        "Implementer subagent implements, tests, commits, self-reviews" [shape=box];
        "Dispatch spec reviewer subagent (./spec-reviewer-prompt.md)" [shape=box];
        "Spec reviewer subagent confirms code matches spec?" [shape=diamond];
        "Implementer subagent fixes spec gaps" [shape=box];
        "Dispatch code quality reviewer subagent (./code-quality-reviewer-prompt.md)" [shape=box];
        "Code quality reviewer subagent approves?" [shape=diamond];
        "Implementer subagent fixes quality issues" [shape=box];
        "Mark task complete in TodoWrite" [shape=box];
    }

    "Read plan, extract all tasks with full text, note context, create TodoWrite" [shape=box];
    "More tasks remain?" [shape=diamond];
    "Dispatch final code reviewer subagent for entire implementation" [shape=box];
    "Use devops-skills:finishing-a-development-branch" [shape=box style=filled fillcolor=lightgreen];

    "Read plan, extract all tasks with full text, note context, create TodoWrite" -> "Dispatch implementer subagent (./implementer-prompt.md)";
    "Dispatch implementer subagent (./implementer-prompt.md)" -> "Implementer subagent asks questions?";
    "Implementer subagent asks questions?" -> "Answer questions, provide context" [label="yes"];
    "Answer questions, provide context" -> "Dispatch implementer subagent (./implementer-prompt.md)";
    "Implementer subagent asks questions?" -> "Implementer subagent implements, tests, commits, self-reviews" [label="no"];
    "Implementer subagent implements, tests, commits, self-reviews" -> "Dispatch spec reviewer subagent (./spec-reviewer-prompt.md)";
    "Dispatch spec reviewer subagent (./spec-reviewer-prompt.md)" -> "Spec reviewer subagent confirms code matches spec?";
    "Spec reviewer subagent confirms code matches spec?" -> "Implementer subagent fixes spec gaps" [label="no"];
    "Implementer subagent fixes spec gaps" -> "Dispatch spec reviewer subagent (./spec-reviewer-prompt.md)" [label="re-review"];
    "Spec reviewer subagent confirms code matches spec?" -> "Dispatch code quality reviewer subagent (./code-quality-reviewer-prompt.md)" [label="yes"];
    "Dispatch code quality reviewer subagent (./code-quality-reviewer-prompt.md)" -> "Code quality reviewer subagent approves?";
    "Code quality reviewer subagent approves?" -> "Implementer subagent fixes quality issues" [label="no"];
    "Implementer subagent fixes quality issues" -> "Dispatch code quality reviewer subagent (./code-quality-reviewer-prompt.md)" [label="re-review"];
    "Code quality reviewer subagent approves?" -> "Mark task complete in TodoWrite" [label="yes"];
    "Mark task complete in TodoWrite" -> "More tasks remain?";
    "More tasks remain?" -> "Dispatch implementer subagent (./implementer-prompt.md)" [label="yes"];
    "More tasks remain?" -> "Dispatch final code reviewer subagent for entire implementation" [label="no"];
    "Dispatch final code reviewer subagent for entire implementation" -> "Use devops-skills:finishing-a-development-branch";
}

Prompt Templates

  • ./implementer-prompt.md - Dispatch implementer subagent
  • ./spec-reviewer-prompt.md - Dispatch spec compliance reviewer subagent
  • ./code-quality-reviewer-prompt.md - Dispatch code quality reviewer subagent

Task Tool Syntax

Use the Task tool with these parameters:

// Implementer subagent
Task:
  description: "Implement [task name]"
  prompt: "[Full task text from plan + context + implementer-prompt.md content]"
  subagent_type: "general-purpose"

// Spec reviewer subagent
Task:
  description: "Spec review [task name]"
  prompt: "[Spec requirements + code location + spec-reviewer-prompt.md content]"
  subagent_type: "general-purpose"

// Code quality reviewer subagent
Task:
  description: "Code review [task name]"
  prompt: "[Git SHAs + file paths + code-quality-reviewer-prompt.md content]"
  subagent_type: "code-reviewer"  // Or general-purpose

Key parameters:

  • description - Short (3-5 words) summary for tracking
  • prompt - Full context and instructions
  • subagent_type - Agent type (general-purpose, or custom from agents/)
  • model - Optional: "haiku" for quick reviews, "opus" for complex tasks

Example Workflow

You: I'm using Subagent-Driven Development to execute this plan.

[Read plan file once: docs/plans/feature-plan.md]
[Extract all 5 tasks with full text and context]
[Create TodoWrite with all tasks]

Task 1: Hook installation script

[Get Task 1 text and context (already extracted)]
[Dispatch implementation subagent with full task text + context]

Implementer: "Before I begin - should the hook be installed at user or system level?"

You: "User level (~/.config/devops-skills/hooks/)"

Implementer: "Got it. Implementing now..."
[Later] Implementer:
  - Implemented install-hook command
  - Added tests, 5/5 passing
  - Self-review: Found I missed --force flag, added it
  - Committed

[Dispatch spec compliance reviewer]
Spec reviewer: ✅ Spec compliant - all requirements met, nothing extra

[Get git SHAs, dispatch code quality reviewer]
Code reviewer: Strengths: Good test coverage, clean. Issues: None. Approved.

[Mark Task 1 complete]

Task 2: Recovery modes

[Get Task 2 text and context (already extracted)]
[Dispatch implementation subagent with full task text + context]

Implementer: [No questions, proceeds]
Implementer:
  - Added verify/repair modes
  - 8/8 tests passing
  - Self-review: All good
  - Committed

[Dispatch spec compliance reviewer]
Spec reviewer: ❌ Issues:
  - Missing: Progress reporting (spec says "report every 100 items")
  - Extra: Added --json flag (not requested)

[Implementer fixes issues]
Implementer: Removed --json flag, added progress reporting

[Spec reviewer reviews again]
Spec reviewer: ✅ Spec compliant now

[Dispatch code quality reviewer]
Code reviewer: Strengths: Solid. Issues (Important): Magic number (100)

[Implementer fixes]
Implementer: Extracted PROGRESS_INTERVAL constant

[Code reviewer reviews again]
Code reviewer: ✅ Approved

[Mark Task 2 complete]

...

[After all tasks]
[Dispatch final code-reviewer]
Final reviewer: All requirements met, ready to merge

Done!

Advantages

vs. Manual execution:

  • Subagents follow TDD naturally
  • Fresh context per task (no confusion)
  • Parallel-safe (subagents don't interfere)
  • Subagent can ask questions (before AND during work)

vs. Executing Plans:

  • Same session (no handoff)
  • Continuous progress (no waiting)
  • Review checkpoints automatic

Efficiency gains:

  • No file reading overhead (controller provides full text)
  • Controller curates exactly what context is needed
  • Subagent gets complete information upfront
  • Questions surfaced before work begins (not after)

Quality gates:

  • Self-review catches issues before handoff
  • Two-stage review: spec compliance, then code quality
  • Review loops ensure fixes actually work
  • Spec compliance prevents over/under-building
  • Code quality ensures implementation is well-built

Cost:

  • More subagent invocations (implementer + 2 reviewers per task)
  • Controller does more prep work (extracting all tasks upfront)
  • Review loops add iterations
  • But catches issues early (cheaper than debugging later)

Red Flags

Never:

  • Skip reviews (spec compliance OR code quality)
  • Proceed with unfixed issues
  • Dispatch multiple implementation subagents in parallel (conflicts)
  • Make subagent read plan file (provide full text instead)
  • Skip scene-setting context (subagent needs to understand where task fits)
  • Ignore subagent questions (answer before letting them proceed)
  • Accept "close enough" on spec compliance (spec reviewer found issues = not done)
  • Skip review loops (reviewer found issues = implementer fixes = review again)
  • Let implementer self-review replace actual review (both are needed)
  • Start code quality review before spec compliance is ✅ (wrong order)
  • Move to next task while either review has open issues

If subagent asks questions:

  • Answer clearly and completely
  • Provide additional context if needed
  • Don't rush them into implementation

If reviewer finds issues:

  • Implementer (same subagent) fixes them
  • Reviewer reviews again
  • Repeat until approved
  • Don't skip the re-review

If subagent fails task:

  • Dispatch fix subagent with specific instructions
  • Don't try to fix manually (context pollution)

Integration

Required workflow skills:

  • devops-skills:writing-plans - Creates the plan this skill executes
  • devops-skills:requesting-code-review - Code review template for reviewer subagents
  • devops-skills:finishing-a-development-branch - Complete development after all tasks

Subagents should use:

  • devops-skills:test-driven-development - Subagents follow TDD for each task

Alternative workflow:

  • devops-skills:executing-plans - Use for parallel session instead of same-session execution

Systematic Debugging

Overview

Random fixes waste time and create new bugs. Quick patches mask underlying issues.

Core principle: ALWAYS find root cause before attempting fixes. Symptom fixes are failure.

Violating the letter of this process is violating the spirit of debugging.

The Iron Law

NO FIXES WITHOUT ROOT CAUSE INVESTIGATION FIRST

If you haven't completed Phase 1, you cannot propose fixes.

When to Use

Use for ANY technical issue:

  • Test failures
  • Bugs in production
  • Unexpected behavior
  • Performance problems
  • Build failures
  • Integration issues

Use this ESPECIALLY when:

  • Under time pressure (emergencies make guessing tempting)
  • "Just one quick fix" seems obvious
  • You've already tried multiple fixes
  • Previous fix didn't work
  • You don't fully understand the issue

Don't skip when:

  • Issue seems simple (simple bugs have root causes too)
  • You're in a hurry (rushing guarantees rework)
  • Manager wants it fixed NOW (systematic is faster than thrashing)

The Four Phases

You MUST complete each phase before proceeding to the next.

Phase 1: Root Cause Investigation

BEFORE attempting ANY fix:

  1. Read Error Messages Carefully

    • Don't skip past errors or warnings
    • They often contain the exact solution
    • Read stack traces completely
    • Note line numbers, file paths, error codes
  2. Reproduce Consistently

    • Can you trigger it reliably?
    • What are the exact steps?
    • Does it happen every time?
    • If not reproducible → gather more data, don't guess
  3. Check Recent Changes

    • What changed that could cause this?
    • Git diff, recent commits
    • New dependencies, config changes
    • Environmental differences
  4. Gather Evidence in Multi-Component Systems

    WHEN system has multiple components (CI → build → signing, API → service → database):

    BEFORE proposing fixes, add diagnostic instrumentation:

    For EACH component boundary:
      - Log what data enters component
      - Log what data exits component
      - Verify environment/config propagation
      - Check state at each layer
    
    Run once to gather evidence showing WHERE it breaks
    THEN analyze evidence to identify failing component
    THEN investigate that specific component
    

    Example (multi-layer system):

    # Layer 1: Workflow
    echo "=== Secrets available in workflow: ==="
    echo "IDENTITY: ${IDENTITY:+SET}${IDENTITY:-UNSET}"
    
    # Layer 2: Build script
    echo "=== Env vars in build script: ==="
    env | grep IDENTITY || echo "IDENTITY not in environment"
    
    # Layer 3: Signing script
    echo "=== Keychain state: ==="
    security list-keychains
    security find-identity -v
    
    # Layer 4: Actual signing
    codesign --sign "$IDENTITY" --verbose=4 "$APP"
    

    This reveals: Which layer fails (secrets → workflow ✓, workflow → build ✗)

  5. Trace Data Flow

    WHEN error is deep in call stack:

    See root-cause-tracing.md in this directory for the complete backward tracing technique.

    Quick version:

    • Where does bad value originate?
    • What called this with bad value?
    • Keep tracing up until you find the source
    • Fix at source, not at symptom

Phase 2: Pattern Analysis

Find the pattern before fixing:

  1. Find Working Examples

    • Locate similar working code in same codebase
    • What works that's similar to what's broken?
  2. Compare Against References

    • If implementing pattern, read reference implementation COMPLETELY
    • Don't skim - read every line
    • Understand the pattern fully before applying
  3. Identify Differences

    • What's different between working and broken?
    • List every difference, however small
    • Don't assume "that can't matter"
  4. Understand Dependencies

    • What other components does this need?
    • What settings, config, environment?
    • What assumptions does it make?

Phase 3: Hypothesis and Testing

Scientific method:

  1. Form Single Hypothesis

    • State clearly: "I think X is the root cause because Y"
    • Write it down
    • Be specific, not vague
  2. Test Minimally

    • Make the SMALLEST possible change to test hypothesis
    • One variable at a time
    • Don't fix multiple things at once
  3. Verify Before Continuing

    • Did it work? Yes → Phase 4
    • Didn't work? Form NEW hypothesis
    • DON'T add more fixes on top
  4. When You Don't Know

    • Say "I don't understand X"
    • Don't pretend to know
    • Ask for help
    • Research more

Phase 4: Implementation

Fix the root cause, not the symptom:

  1. Create Failing Test Case

    • Simplest possible reproduction
    • Automated test if possible
    • One-off test script if no framework
    • MUST have before fixing
    • Use the devops-skills:test-driven-development skill for writing proper failing tests
  2. Implement Single Fix

    • Address the root cause identified
    • ONE change at a time
    • No "while I'm here" improvements
    • No bundled refactoring
  3. Verify Fix

    • Test passes now?
    • No other tests broken?
    • Issue actually resolved?
  4. If Fix Doesn't Work

    • STOP
    • Count: How many fixes have you tried?
    • If < 3: Return to Phase 1, re-analyze with new information
    • If ≥ 3: STOP and question the architecture (step 5 below)
    • DON'T attempt Fix #4 without architectural discussion
  5. If 3+ Fixes Failed: Question Architecture

    Pattern indicating architectural problem:

    • Each fix reveals new shared state/coupling/problem in different place
    • Fixes require "massive refactoring" to implement
    • Each fix creates new symptoms elsewhere

    STOP and question fundamentals:

    • Is this pattern fundamentally sound?
    • Are we "sticking with it through sheer inertia"?
    • Should we refactor architecture vs. continue fixing symptoms?

    Discuss with your human partner before attempting more fixes

    This is NOT a failed hypothesis - this is a wrong architecture.

Red Flags - STOP and Follow Process

If you catch yourself thinking:

  • "Quick fix for now, investigate later"
  • "Just try changing X and see if it works"
  • "Add multiple changes, run tests"
  • "Skip the test, I'll manually verify"
  • "It's probably X, let me fix that"
  • "I don't fully understand but this might work"
  • "Pattern says X but I'll adapt it differently"
  • "Here are the main problems: [lists fixes without investigation]"
  • Proposing solutions before tracing data flow
  • "One more fix attempt" (when already tried 2+)
  • Each fix reveals new problem in different place

ALL of these mean: STOP. Return to Phase 1.

If 3+ fixes failed: Question the architecture (see Phase 4.5)

your human partner's Signals You're Doing It Wrong

Watch for these redirections:

  • "Is that not happening?" - You assumed without verifying
  • "Will it show us...?" - You should have added evidence gathering
  • "Stop guessing" - You're proposing fixes without understanding
  • "Ultrathink this" - Question fundamentals, not just symptoms
  • "We're stuck?" (frustrated) - Your approach isn't working

When you see these: STOP. Return to Phase 1.

Common Rationalizations

ExcuseReality
"Issue is simple, don't need process"Simple issues have root causes too. Process is fast for simple bugs.
"Emergency, no time for process"Systematic debugging is FASTER than guess-and-check thrashing.
"Just try this first, then investigate"First fix sets the pattern. Do it right from the start.
"I'll write test after confirming fix works"Untested fixes don't stick. Test first proves it.
"Multiple fixes at once saves time"Can't isolate what worked. Causes new bugs.
"Reference too long, I'll adapt the pattern"Partial understanding guarantees bugs. Read it completely.
"I see the problem, let me fix it"Seeing symptoms ≠ understanding root cause.
"One more fix attempt" (after 2+ failures)3+ failures = architectural problem. Question pattern, don't fix again.

Quick Reference

PhaseKey ActivitiesSuccess Criteria
1. Root CauseRead errors, reproduce, check changes, gather evidenceUnderstand WHAT and WHY
2. PatternFind working examples, compareIdentify differences
3. HypothesisForm theory, test minimallyConfirmed or new hypothesis
4. ImplementationCreate test, fix, verifyBug resolved, tests pass

When Process Reveals "No Root Cause"

If systematic investigation reveals issue is truly environmental, timing-dependent, or external:

  1. You've completed the process
  2. Document what you investigated
  3. Implement appropriate handling (retry, timeout, error message)
  4. Add monitoring/logging for future investigation

But: 95% of "no root cause" cases are incomplete investigation.

Supporting Techniques

These techniques are part of systematic debugging and available in this directory:

  • root-cause-tracing.md - Trace bugs backward through call stack to find original trigger
  • defense-in-depth.md - Add validation at multiple layers after finding root cause
  • condition-based-waiting.md - Replace arbitrary timeouts with condition polling

Related skills:

  • devops-skills:test-driven-development - For creating failing test case (Phase 4, Step 1)
  • devops-skills:verification-before-completion - Verify fix worked before claiming success

Real-World Impact

From debugging sessions:

  • Systematic approach: 15-30 minutes to fix
  • Random fixes approach: 2-3 hours of thrashing
  • First-time fix rate: 95% vs 40%
  • New bugs introduced: Near zero vs common

Terraform Drift Detection

Overview

Detect and categorize drift between Terraform-managed state and actual infrastructure. Drift indicates out-of-band changes that can cause problems during the next apply.

Announce at start: "I'm using the terraform-drift-detection skill to check for infrastructure drift."

The Process

Step 1: Verify Environment

# Verify AWS credentials and account
aws sts get-caller-identity

# Confirm we're in the right directory/environment
pwd
ls -la *.tf 2>/dev/null | head -5

Step 2: Refresh State

# Initialize if needed
terraform init

# Refresh state to detect drift
terraform plan -refresh-only -out=drift.out

# Convert to JSON for analysis
terraform show -json drift.out > drift.json

Step 3: Analyze Drift

Parse drift.json and categorize changes:

Drift Categories
CategorySeverityExamples
Security DriftCRITICALSecurity groups, IAM, encryption
Configuration DriftHIGHInstance settings, networking
Tag DriftLOWTags modified outside Terraform
Metadata DriftINFOAWS-managed fields that change

Step 4: Dispatch Analysis Agent

Task(drift-detector) → Categorize and assess drift impact

Agent should:

  • Categorize each drifted resource
  • Assess impact of accepting vs rejecting drift
  • Identify potential causes (manual changes, AWS updates, etc.)

Step 5: Present Findings

## Drift Detection Report

### Summary
- Total drifted resources: X
- Critical drift: Y
- High drift: Z
- Low/Info drift: W

### Critical Drift (Requires Immediate Attention)
| Resource | Attribute | State Value | Actual Value |
|----------|-----------|-------------|--------------|
| ... | ... | ... | ... |

### Potential Causes
- Manual console changes: [list if detected]
- AWS service updates: [list if detected]
- Unknown origin: [list if detected]

### Recommended Actions
1. [Action for each drifted resource]

Step 6: Resolution Options

Present user with options:

  1. Accept Drift: Run terraform apply -refresh-only to update state to match actual
  2. Reject Drift: Run terraform apply to revert actual infrastructure to match code
  3. Investigate: Manual review needed before deciding
  4. Hybrid: Accept some drift, reject other drift

Never auto-resolve drift. Always get user approval.

Common Drift Sources

Intentional (Usually Accept)

  • AWS auto-scaling adjustments
  • Managed service updates
  • Emergency manual fixes (document these!)

Unintentional (Usually Reject)

  • Console click-ops mistakes
  • Unapproved manual changes
  • Conflicting automation

Systemic (Fix Root Cause)

  • Multiple tools managing same resources
  • Missing Terraform coverage
  • CI/CD race conditions

Integration with Memory

Store detected drift patterns:

  • Which resources commonly drift
  • Common causes in this environment
  • Resolution preferences

Query memory before analysis:

  • Has this resource drifted before?
  • What was the cause last time?
  • What resolution was chosen?

Verification Checklist

Before presenting:

  • Refresh completed successfully
  • All drift categorized by severity
  • Root causes identified where possible
  • Resolution options are clear
  • No sensitive values exposed in output

Terraform Plan Review

Overview

Analyze terraform plan output using parallel agents for comprehensive risk assessment. Never auto-apply - always present findings and require explicit approval.

Announce at start: "I'm using the terraform-plan-review skill to analyze these changes safely."

The Process

Step 1: Verify Environment

Before running any plan:

  1. Check AWS Profile

    aws sts get-caller-identity
    
    • Verify the account ID matches expected environment
    • Verify the role/user is appropriate for this operation
    • If mismatch: STOP and alert user
  2. Identify Environment

    • Check current directory structure (which environment?)
    • Verify backend configuration matches environment

Step 2: Generate Plan

# Initialize if needed
terraform init

# Generate plan file (required for JSON parsing)
terraform plan -out=plan.out

# Convert to JSON for analysis
terraform show -json plan.out > plan.json

Step 3: Dispatch Parallel Analysis Agents

Launch these agents in a single message with multiple Task calls:

Task 1:
  description: "Analyze plan risks"
  prompt: |
    Analyze this Terraform plan for risks and impact.
    Environment: [env name]
    Account: [account id]

    Plan JSON:
    [plan.json content]

    Focus on destruction, modification risks, and cascade effects.
  subagent_type: "terraform-plan-analyzer"

Task 2:
  description: "Security review plan"
  prompt: |
    Review this Terraform plan for security implications.
    Environment: [env name]

    Plan JSON:
    [plan.json content]

    Focus on IAM, network, encryption, and compliance.
  subagent_type: "security-reviewer"

Task 3:
  description: "Check historical patterns"
  prompt: |
    Analyze git history for patterns related to these resources.
    Resources being changed: [list from plan]

    Look for similar past changes, incidents, and outcomes.
  subagent_type: "historical-pattern-analyzer"

CRITICAL: All three Task calls in ONE message for parallel execution.

Agent prompts should include:

  • The plan.json content (or path)
  • The environment name
  • Any relevant context from memory

Step 4: Aggregate Findings

Collect results from all agents and create a unified report:

## Plan Analysis Summary

### Risk Level: [CRITICAL/HIGH/MEDIUM/LOW]

### Changes Overview
- Resources to create: X
- Resources to update: Y
- Resources to destroy: Z

### Risk Analysis (terraform-plan-analyzer)
[Summary of risks identified]

### Security Analysis (security-reviewer)
[Summary of security implications]

### Pattern Analysis (historical-pattern-analyzer)
[Any similar past changes and their outcomes]

### Required Approvals
- [ ] User acknowledges destruction of X resources
- [ ] User confirms this is the correct environment
- [ ] User approves proceeding with apply

Step 5: Approval Gate

Present the analysis to the user and wait for explicit approval:

"Based on my analysis, this plan has [RISK LEVEL] risk. [Summary of key findings].

Do you want me to proceed with terraform apply? Please respond with 'approve' to continue."

NEVER proceed without explicit "approve" from user.

Step 6: Execute Apply (Only After Approval)

If and only if user explicitly approves:

terraform apply plan.out

Monitor output and report results.

Risk Categories

CRITICAL - Requires Extra Scrutiny

  • Any resource destruction
  • IAM policy changes
  • Security group rule modifications
  • Database modifications
  • Encryption key changes
  • Cross-account resource access

HIGH

  • Network configuration changes
  • Load balancer modifications
  • Auto-scaling changes
  • DNS record modifications

MEDIUM

  • Instance type changes
  • Tag modifications
  • Non-critical configuration updates

LOW

  • Pure additions with no dependencies
  • Documentation-only changes

Common Patterns to Flag

  1. Cascade Deletions: Resource deletion that triggers other deletions
  2. State Drift: Plan shows changes that weren't in code
  3. Dependency Chains: Changes that affect many downstream resources
  4. Security Relaxation: Rules becoming more permissive
  5. Cost Impact: Significant size/count changes

Memory Integration

Before analysis, query memory for:

  • Similar changes in this project's history
  • Known issues with affected resources
  • Past incidents related to this type of change

After completion, store:

  • Outcome of this change (success/failure)
  • Any issues encountered
  • User preferences learned

Verification Checklist

Before presenting to user, verify:

  • AWS profile matches environment
  • Plan was generated successfully
  • All agents completed analysis
  • Risk level is accurately assessed
  • All destruction operations are highlighted
  • Security implications are documented

Terraform State Operations

Overview

State operations modify Terraform's understanding of infrastructure without changing actual resources. These are dangerous because mistakes can orphan resources or cause Terraform to recreate existing infrastructure.

Announce at start: "I'm using the terraform-state-operations skill for safe state surgery."

CRITICAL: Pre-Operation Safety

1. Create State Backup

ALWAYS create a backup before ANY state operation:

# Create timestamped backup
BACKUP_NAME="state-backup-$(date +%Y%m%d-%H%M%S).tfstate"

# For local state
cp terraform.tfstate "$BACKUP_NAME"

# For remote state (S3 example)
terraform state pull > "$BACKUP_NAME"

echo "Backup created: $BACKUP_NAME"

2. Document the Operation

Before proceeding, create a record:

## State Operation Record

**Date:** [timestamp]
**Environment:** [env name]
**Operator:** [user]
**Reason:** [why this operation is needed]

### Planned Operations
1. [operation 1]
2. [operation 2]

### Backup Location
- Local: [path]
- Remote: [if applicable]

### Rollback Plan
[How to restore if something goes wrong]

3. Get User Approval

Present the plan and require explicit approval before executing.

State Operations Guide

terraform state mv

Use case: Rename resources, reorganize modules, refactor code

# List current resources
terraform state list

# Move/rename a resource
terraform state mv aws_instance.old_name aws_instance.new_name

# Move into a module
terraform state mv aws_instance.web module.web.aws_instance.this

# Move between modules
terraform state mv module.old.aws_instance.web module.new.aws_instance.web

Verification after mv:

# Should show no changes
terraform plan

terraform state rm

Use case: Remove from state without destroying actual resource (adopting externally-managed resources)

# Remove single resource
terraform state rm aws_instance.legacy

# Remove module
terraform state rm module.legacy

WARNING: The actual resource continues to exist but is no longer managed by Terraform.

Verification after rm:

# Resource should appear as "new" if still in code
# Remove from code if intentionally unmanaging
terraform plan

terraform import

Use case: Bring existing infrastructure under Terraform management

# Import existing resource
terraform import aws_instance.web i-1234567890abcdef0

# Import with module path
terraform import module.web.aws_instance.this i-1234567890abcdef0

Before import:

  1. Write the resource configuration in code
  2. Ensure configuration matches actual resource
  3. Import
  4. Run plan to verify no changes

Verification after import:

# Should show no changes (or only acceptable differences)
terraform plan

terraform state pull/push

Use case: Backup, migrate, or restore state

# Pull current state
terraform state pull > state.json

# Push state (DANGEROUS - can overwrite!)
# terraform state push state.json  # BLOCKED by hook

Recovery Procedures

If State Becomes Corrupted

# Restore from backup
cp "$BACKUP_NAME" terraform.tfstate

# For remote state - may need to disable locking temporarily
# Consult your backend documentation

If Resources Are Orphaned

  1. Identify orphaned resources in AWS console
  2. Either:
    • Import them back: terraform import ...
    • Delete them manually if no longer needed

If Wrong Resources Removed

# Restore backup
cp "$BACKUP_NAME" terraform.tfstate

# Re-initialize to sync with backend
terraform init -reconfigure

Approval Workflow

For state mv (Low Risk)

  1. Show planned move
  2. Explain impact
  3. Request approval
  4. Execute with verification

For state rm (Medium Risk)

  1. Show what will be unmanaged
  2. Explain orphan implications
  3. Confirm resource won't be deleted
  4. Request explicit approval
  5. Execute with verification

For state push (High Risk)

Hook blocks this command. If genuinely needed:

  1. Explain why state push is necessary
  2. Show state diff
  3. Request explicit approval with acknowledgment of risks
  4. User must disable hook manually

Verification Checklist

Before any state operation:

  • Backup created and verified
  • Operation documented
  • Impact explained to user
  • User explicitly approved
  • Rollback plan established

After operation:

  • terraform plan shows expected result
  • No unintended changes detected
  • Backup retained for rollback window
  • Operation logged to memory

Test-Driven Development (TDD)

Overview

Write the test first. Watch it fail. Write minimal code to pass.

Core principle: If you didn't watch the test fail, you don't know if it tests the right thing.

Violating the letter of the rules is violating the spirit of the rules.

When to Use

Always:

  • New features
  • Bug fixes
  • Refactoring
  • Behavior changes

Exceptions (ask your human partner):

  • Throwaway prototypes
  • Generated code
  • Configuration files

Thinking "skip TDD just this once"? Stop. That's rationalization.

The Iron Law

NO PRODUCTION CODE WITHOUT A FAILING TEST FIRST

Write code before the test? Delete it. Start over.

No exceptions:

  • Don't keep it as "reference"
  • Don't "adapt" it while writing tests
  • Don't look at it
  • Delete means delete

Implement fresh from tests. Period.

Red-Green-Refactor

digraph tdd_cycle {
    rankdir=LR;
    red [label="RED\nWrite failing test", shape=box, style=filled, fillcolor="#ffcccc"];
    verify_red [label="Verify fails\ncorrectly", shape=diamond];
    green [label="GREEN\nMinimal code", shape=box, style=filled, fillcolor="#ccffcc"];
    verify_green [label="Verify passes\nAll green", shape=diamond];
    refactor [label="REFACTOR\nClean up", shape=box, style=filled, fillcolor="#ccccff"];
    next [label="Next", shape=ellipse];

    red -> verify_red;
    verify_red -> green [label="yes"];
    verify_red -> red [label="wrong\nfailure"];
    green -> verify_green;
    verify_green -> refactor [label="yes"];
    verify_green -> green [label="no"];
    refactor -> verify_green [label="stay\ngreen"];
    verify_green -> next;
    next -> red;
}

RED - Write Failing Test

Write one minimal test showing what should happen.

```typescript test('retries failed operations 3 times', async () => { let attempts = 0; const operation = () => { attempts++; if (attempts < 3) throw new Error('fail'); return 'success'; };

const result = await retryOperation(operation);

expect(result).toBe('success'); expect(attempts).toBe(3); });

Clear name, tests real behavior, one thing
</Good>

<Bad>
```typescript
test('retry works', async () => {
  const mock = jest.fn()
    .mockRejectedValueOnce(new Error())
    .mockRejectedValueOnce(new Error())
    .mockResolvedValueOnce('success');
  await retryOperation(mock);
  expect(mock).toHaveBeenCalledTimes(3);
});

Vague name, tests mock not code

Requirements:

  • One behavior
  • Clear name
  • Real code (no mocks unless unavoidable)

Verify RED - Watch It Fail

MANDATORY. Never skip.

npm test path/to/test.test.ts

Confirm:

  • Test fails (not errors)
  • Failure message is expected
  • Fails because feature missing (not typos)

Test passes? You're testing existing behavior. Fix test.

Test errors? Fix error, re-run until it fails correctly.

GREEN - Minimal Code

Write simplest code to pass the test.

```typescript async function retryOperation(fn: () => Promise): Promise { for (let i = 0; i < 3; i++) { try { return await fn(); } catch (e) { if (i === 2) throw e; } } throw new Error('unreachable'); } ``` Just enough to pass ```typescript async function retryOperation( fn: () => Promise, options?: { maxRetries?: number; backoff?: 'linear' | 'exponential'; onRetry?: (attempt: number) => void; } ): Promise { // YAGNI } ``` Over-engineered

Don't add features, refactor other code, or "improve" beyond the test.

Verify GREEN - Watch It Pass

MANDATORY.

npm test path/to/test.test.ts

Confirm:

  • Test passes
  • Other tests still pass
  • Output pristine (no errors, warnings)

Test fails? Fix code, not test.

Other tests fail? Fix now.

REFACTOR - Clean Up

After green only:

  • Remove duplication
  • Improve names
  • Extract helpers

Keep tests green. Don't add behavior.

Repeat

Next failing test for next feature.

Good Tests

QualityGoodBad
MinimalOne thing. "and" in name? Split it.test('validates email and domain and whitespace')
ClearName describes behaviortest('test1')
Shows intentDemonstrates desired APIObscures what code should do

Why Order Matters

"I'll write tests after to verify it works"

Tests written after code pass immediately. Passing immediately proves nothing:

  • Might test wrong thing
  • Might test implementation, not behavior
  • Might miss edge cases you forgot
  • You never saw it catch the bug

Test-first forces you to see the test fail, proving it actually tests something.

"I already manually tested all the edge cases"

Manual testing is ad-hoc. You think you tested everything but:

  • No record of what you tested
  • Can't re-run when code changes
  • Easy to forget cases under pressure
  • "It worked when I tried it" ≠ comprehensive

Automated tests are systematic. They run the same way every time.

"Deleting X hours of work is wasteful"

Sunk cost fallacy. The time is already gone. Your choice now:

  • Delete and rewrite with TDD (X more hours, high confidence)
  • Keep it and add tests after (30 min, low confidence, likely bugs)

The "waste" is keeping code you can't trust. Working code without real tests is technical debt.

"TDD is dogmatic, being pragmatic means adapting"

TDD IS pragmatic:

  • Finds bugs before commit (faster than debugging after)
  • Prevents regressions (tests catch breaks immediately)
  • Documents behavior (tests show how to use code)
  • Enables refactoring (change freely, tests catch breaks)

"Pragmatic" shortcuts = debugging in production = slower.

"Tests after achieve the same goals - it's spirit not ritual"

No. Tests-after answer "What does this do?" Tests-first answer "What should this do?"

Tests-after are biased by your implementation. You test what you built, not what's required. You verify remembered edge cases, not discovered ones.

Tests-first force edge case discovery before implementing. Tests-after verify you remembered everything (you didn't).

30 minutes of tests after ≠ TDD. You get coverage, lose proof tests work.

Common Rationalizations

ExcuseReality
"Too simple to test"Simple code breaks. Test takes 30 seconds.
"I'll test after"Tests passing immediately prove nothing.
"Tests after achieve same goals"Tests-after = "what does this do?" Tests-first = "what should this do?"
"Already manually tested"Ad-hoc ≠ systematic. No record, can't re-run.
"Deleting X hours is wasteful"Sunk cost fallacy. Keeping unverified code is technical debt.
"Keep as reference, write tests first"You'll adapt it. That's testing after. Delete means delete.
"Need to explore first"Fine. Throw away exploration, start with TDD.
"Test hard = design unclear"Listen to test. Hard to test = hard to use.
"TDD will slow me down"TDD faster than debugging. Pragmatic = test-first.
"Manual test faster"Manual doesn't prove edge cases. You'll re-test every change.
"Existing code has no tests"You're improving it. Add tests for existing code.

Red Flags - STOP and Start Over

  • Code before test
  • Test after implementation
  • Test passes immediately
  • Can't explain why test failed
  • Tests added "later"
  • Rationalizing "just this once"
  • "I already manually tested it"
  • "Tests after achieve the same purpose"
  • "It's about spirit not ritual"
  • "Keep as reference" or "adapt existing code"
  • "Already spent X hours, deleting is wasteful"
  • "TDD is dogmatic, I'm being pragmatic"
  • "This is different because..."

All of these mean: Delete code. Start over with TDD.

Example: Bug Fix

Bug: Empty email accepted

RED

test('rejects empty email', async () => {
  const result = await submitForm({ email: '' });
  expect(result.error).toBe('Email required');
});

Verify RED

$ npm test
FAIL: expected 'Email required', got undefined

GREEN

function submitForm(data: FormData) {
  if (!data.email?.trim()) {
    return { error: 'Email required' };
  }
  // ...
}

Verify GREEN

$ npm test
PASS

REFACTOR Extract validation for multiple fields if needed.

Verification Checklist

Before marking work complete:

  • Every new function/method has a test
  • Watched each test fail before implementing
  • Each test failed for expected reason (feature missing, not typo)
  • Wrote minimal code to pass each test
  • All tests pass
  • Output pristine (no errors, warnings)
  • Tests use real code (mocks only if unavoidable)
  • Edge cases and errors covered

Can't check all boxes? You skipped TDD. Start over.

When Stuck

ProblemSolution
Don't know how to testWrite wished-for API. Write assertion first. Ask your human partner.
Test too complicatedDesign too complicated. Simplify interface.
Must mock everythingCode too coupled. Use dependency injection.
Test setup hugeExtract helpers. Still complex? Simplify design.

Debugging Integration

Bug found? Write failing test reproducing it. Follow TDD cycle. Test proves fix and prevents regression.

Never fix bugs without a test.

Testing Anti-Patterns

When adding mocks or test utilities, read @testing-anti-patterns.md to avoid common pitfalls:

  • Testing mock behavior instead of real behavior
  • Adding test-only methods to production classes
  • Mocking without understanding dependencies

Final Rule

Production code → test exists and failed first
Otherwise → not TDD

No exceptions without your human partner's permission.

If you think there is even a 1% chance a skill might apply to what you are doing, you ABSOLUTELY MUST invoke the skill.

IF A SKILL APPLIES TO YOUR TASK, YOU DO NOT HAVE A CHOICE. YOU MUST USE IT.

This is not negotiable. This is not optional. You cannot rationalize your way out of this.

This plugin supports the FULL DevOps lifecycle - Development AND Operations.

Core Principles

For Development

  1. Test First - Write tests before implementation (TDD)
  2. Debug Systematically - Follow evidence, not hunches
  3. Review Thoroughly - Code review before merge
  4. Plan Before Coding - Design before implementation

For Operations

  1. Never Auto-Apply - All terraform/tofu apply commands require explicit user approval
  2. Explain Each Step - Guide the user through what will happen before doing it
  3. Escalate Conflicts - When uncertain, ask the user rather than guessing
  4. Fewer Mistakes - Safety over speed, always

How to Access Skills

In Claude Code: Use the Skill tool. When you invoke a skill, its content is loaded and presented to you—follow it directly. Never use the Read tool on skill files.

The Rule

Invoke relevant or requested skills BEFORE any response or action. Even a 1% chance a skill might apply means you should invoke the skill.

digraph skill_flow {
    "User message received" [shape=doublecircle];
    "Might any skill apply?" [shape=diamond];
    "Invoke Skill tool" [shape=box];
    "Announce: 'Using [skill] to [purpose]'" [shape=box];
    "Has checklist?" [shape=diamond];
    "Create TodoWrite todo per item" [shape=box];
    "Follow skill exactly" [shape=box];
    "Respond (including clarifications)" [shape=doublecircle];

    "User message received" -> "Might any skill apply?";
    "Might any skill apply?" -> "Invoke Skill tool" [label="yes, even 1%"];
    "Might any skill apply?" -> "Respond (including clarifications)" [label="definitely not"];
    "Invoke Skill tool" -> "Announce: 'Using [skill] to [purpose]'";
    "Announce: 'Using [skill] to [purpose]'" -> "Has checklist?";
    "Has checklist?" -> "Create TodoWrite todo per item" [label="yes"];
    "Has checklist?" -> "Follow skill exactly" [label="no"];
    "Create TodoWrite todo per item" -> "Follow skill exactly";
}

Available Skills

Development Skills

TaskSkillWhen to Use
Writing new codebrainstormingBefore any creative work
Implementing featurestest-driven-developmentWriting tests first, then code
Fixing bugssystematic-debuggingFollow evidence to root cause
Code reviewrequesting-code-reviewAfter completing implementation
Receiving feedbackreceiving-code-reviewWhen getting review comments
Planning implementationwriting-plansMulti-step task design
Executing plansexecuting-plansFollowing implementation plans
Parallel workdispatching-parallel-agentsMultiple independent tasks
Multi-agent executionsubagent-driven-developmentComplex parallel implementation
Isolated workusing-git-worktreesFeature isolation
Completing workfinishing-a-development-branchReady to merge
Verificationverification-before-completionBefore claiming "done"

Operations Skills

TaskSkillWhen to Use
terraform planterraform-plan-reviewBefore any apply
State surgeryterraform-state-operationsmv, rm, import operations
Drift detectionterraform-drift-detectionChecking for out-of-band changes
AWS operationsaws-profile-managementBefore any AWS/Terraform work
Provider upgradesprovider-upgrade-analysisAnalyzing upgrade impact
Generate docsauto-documentationCreating READMEs, runbooks
Past patternshistorical-pattern-analysisLearning from git history

Commands

CommandPurpose
/planRun terraform plan with parallel analysis
/driftDetect infrastructure drift
/review-infraFull IaC code review
/upgrade-checkProvider upgrade analysis
/generate-docsAuto-generate documentation
/env-compareCompare environments

Skill Priority

When multiple skills could apply, use this order:

  1. Process skills first (brainstorming, debugging) - these determine HOW to approach the task
  2. Implementation skills second (TDD, code review) - these guide execution
  3. Infrastructure skills when working with IaC

Examples:

  • "Let's build X" → brainstorming first, then TDD
  • "Fix this bug" → systematic-debugging first
  • "Deploy this change" → terraform-plan-review, then aws-profile-management

Red Flags - STOP if You Think These

Development Red Flags

ThoughtReality
"This is just a simple fix"Simple fixes can have complex impacts. Use skills.
"I'll add tests later"Tests first. Always. Use TDD skill.
"I know how to debug this"Follow systematic-debugging anyway.
"Let me just explore first"Skills tell you HOW to explore. Check first.

Operations Red Flags

ThoughtReality
"I'll just run a quick apply"NEVER. Use /plan first, get approval.
"This is a simple change"Simple changes can cascade. Use the skill.
"I can fix it if it breaks"Infrastructure mistakes can be irreversible.
"This is just a dev environment"Treat all environments with same rigor.

Dangerous Commands - ALWAYS BLOCKED

These commands are intercepted by safety hooks:

  • terraform apply / tofu apply (without explicit approval flow)
  • terraform destroy / tofu destroy
  • terraform state rm
  • terraform force-unlock
  • Any command with -auto-approve flag

Parallel Agent Workflows

For complex tasks, dispatch multiple agents in parallel:

Infrastructure Analysis

/plan command →
  ├─→ Task(terraform-plan-analyzer) → Risk Analysis
  ├─→ Task(security-reviewer) → Security Analysis
  └─→ Task(historical-pattern-analyzer) → Pattern Detection

Development Tasks

Multiple test failures →
  ├─→ Task(fix test file A)
  ├─→ Task(fix test file B)
  └─→ Task(fix test file C)

Memory System

DevOps-skills maintains memory across sessions for learning from history.

User Instructions

Instructions say WHAT, not HOW. "Add X" or "Fix Y" doesn't mean skip workflows.

The user's #1 goal is fewer mistakes. Honor that above all else - in BOTH development AND operations.

Using Git Worktrees

Overview

Git worktrees create isolated workspaces sharing the same repository, allowing work on multiple branches simultaneously without switching.

Core principle: Systematic directory selection + safety verification = reliable isolation.

Announce at start: "I'm using the using-git-worktrees skill to set up an isolated workspace."

Directory Selection Process

Follow this priority order:

1. Check Existing Directories

# Check in priority order
ls -d .worktrees 2>/dev/null     # Preferred (hidden)
ls -d worktrees 2>/dev/null      # Alternative

If found: Use that directory. If both exist, .worktrees wins.

2. Check CLAUDE.md

grep -i "worktree.*director" CLAUDE.md 2>/dev/null

If preference specified: Use it without asking.

3. Ask User

If no directory exists and no CLAUDE.md preference:

No worktree directory found. Where should I create worktrees?

1. .worktrees/ (project-local, hidden)
2. ~/.config/devops-skills/worktrees/<project-name>/ (global location)

Which would you prefer?

Safety Verification

For Project-Local Directories (.worktrees or worktrees)

MUST verify directory is ignored before creating worktree:

# Check if directory is ignored (respects local, global, and system gitignore)
git check-ignore -q .worktrees 2>/dev/null || git check-ignore -q worktrees 2>/dev/null

If NOT ignored:

Per Jesse's rule "Fix broken things immediately":

  1. Add appropriate line to .gitignore
  2. Commit the change
  3. Proceed with worktree creation

Why critical: Prevents accidentally committing worktree contents to repository.

For Global Directory (~/.config/devops-skills/worktrees)

No .gitignore verification needed - outside project entirely.

Creation Steps

1. Detect Project Name

project=$(basename "$(git rev-parse --show-toplevel)")

2. Create Worktree

# Determine full path
case $LOCATION in
  .worktrees|worktrees)
    path="$LOCATION/$BRANCH_NAME"
    ;;
  ~/.config/devops-skills/worktrees/*)
    path="~/.config/devops-skills/worktrees/$project/$BRANCH_NAME"
    ;;
esac

# Create worktree with new branch
git worktree add "$path" -b "$BRANCH_NAME"
cd "$path"

3. Run Project Setup

Auto-detect and run appropriate setup:

# Node.js
if [ -f package.json ]; then npm install; fi

# Rust
if [ -f Cargo.toml ]; then cargo build; fi

# Python
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
if [ -f pyproject.toml ]; then poetry install; fi

# Go
if [ -f go.mod ]; then go mod download; fi

4. Verify Clean Baseline

Run tests to ensure worktree starts clean:

# Examples - use project-appropriate command
npm test
cargo test
pytest
go test ./...

If tests fail: Report failures, ask whether to proceed or investigate.

If tests pass: Report ready.

5. Report Location

Worktree ready at <full-path>
Tests passing (<N> tests, 0 failures)
Ready to implement <feature-name>

Quick Reference

SituationAction
.worktrees/ existsUse it (verify ignored)
worktrees/ existsUse it (verify ignored)
Both existUse .worktrees/
Neither existsCheck CLAUDE.md → Ask user
Directory not ignoredAdd to .gitignore + commit
Tests fail during baselineReport failures + ask
No package.json/Cargo.tomlSkip dependency install

Common Mistakes

Skipping ignore verification

  • Problem: Worktree contents get tracked, pollute git status
  • Fix: Always use git check-ignore before creating project-local worktree

Assuming directory location

  • Problem: Creates inconsistency, violates project conventions
  • Fix: Follow priority: existing > CLAUDE.md > ask

Proceeding with failing tests

  • Problem: Can't distinguish new bugs from pre-existing issues
  • Fix: Report failures, get explicit permission to proceed

Hardcoding setup commands

  • Problem: Breaks on projects using different tools
  • Fix: Auto-detect from project files (package.json, etc.)

Example Workflow

You: I'm using the using-git-worktrees skill to set up an isolated workspace.

[Check .worktrees/ - exists]
[Verify ignored - git check-ignore confirms .worktrees/ is ignored]
[Create worktree: git worktree add .worktrees/auth -b feature/auth]
[Run npm install]
[Run npm test - 47 passing]

Worktree ready at /Users/jesse/myproject/.worktrees/auth
Tests passing (47 tests, 0 failures)
Ready to implement auth feature

Red Flags

Never:

  • Create worktree without verifying it's ignored (project-local)
  • Skip baseline test verification
  • Proceed with failing tests without asking
  • Assume directory location when ambiguous
  • Skip CLAUDE.md check

Always:

  • Follow directory priority: existing > CLAUDE.md > ask
  • Verify directory is ignored for project-local
  • Auto-detect and run project setup
  • Verify clean test baseline

Integration

Called by:

  • brainstorming (Phase 4) - REQUIRED when design is approved and implementation follows
  • Any skill needing isolated workspace

Pairs with:

  • finishing-a-development-branch - REQUIRED for cleanup after work complete
  • executing-plans or subagent-driven-development - Work happens in this worktree

Verification Before Completion

Overview

Claiming work is complete without verification is dishonesty, not efficiency.

Core principle: Evidence before claims, always.

Violating the letter of this rule is violating the spirit of this rule.

The Iron Law

NO COMPLETION CLAIMS WITHOUT FRESH VERIFICATION EVIDENCE

If you haven't run the verification command in this message, you cannot claim it passes.

The Gate Function

BEFORE claiming any status or expressing satisfaction:

1. IDENTIFY: What command proves this claim?
2. RUN: Execute the FULL command (fresh, complete)
3. READ: Full output, check exit code, count failures
4. VERIFY: Does output confirm the claim?
   - If NO: State actual status with evidence
   - If YES: State claim WITH evidence
5. ONLY THEN: Make the claim

Skip any step = lying, not verifying

Common Failures

ClaimRequiresNot Sufficient
Tests passTest command output: 0 failuresPrevious run, "should pass"
Linter cleanLinter output: 0 errorsPartial check, extrapolation
Build succeedsBuild command: exit 0Linter passing, logs look good
Bug fixedTest original symptom: passesCode changed, assumed fixed
Regression test worksRed-green cycle verifiedTest passes once
Agent completedVCS diff shows changesAgent reports "success"
Requirements metLine-by-line checklistTests passing

Red Flags - STOP

  • Using "should", "probably", "seems to"
  • Expressing satisfaction before verification ("Great!", "Perfect!", "Done!", etc.)
  • About to commit/push/PR without verification
  • Trusting agent success reports
  • Relying on partial verification
  • Thinking "just this once"
  • Tired and wanting work over
  • ANY wording implying success without having run verification

Rationalization Prevention

ExcuseReality
"Should work now"RUN the verification
"I'm confident"Confidence ≠ evidence
"Just this once"No exceptions
"Linter passed"Linter ≠ compiler
"Agent said success"Verify independently
"I'm tired"Exhaustion ≠ excuse
"Partial check is enough"Partial proves nothing
"Different words so rule doesn't apply"Spirit over letter

Key Patterns

Tests:

✅ [Run test command] [See: 34/34 pass] "All tests pass"
❌ "Should pass now" / "Looks correct"

Regression tests (TDD Red-Green):

✅ Write → Run (pass) → Revert fix → Run (MUST FAIL) → Restore → Run (pass)
❌ "I've written a regression test" (without red-green verification)

Build:

✅ [Run build] [See: exit 0] "Build passes"
❌ "Linter passed" (linter doesn't check compilation)

Requirements:

✅ Re-read plan → Create checklist → Verify each → Report gaps or completion
❌ "Tests pass, phase complete"

Agent delegation:

✅ Agent reports success → Check VCS diff → Verify changes → Report actual state
❌ Trust agent report

Why This Matters

From 24 failure memories:

  • your human partner said "I don't believe you" - trust broken
  • Undefined functions shipped - would crash
  • Missing requirements shipped - incomplete features
  • Time wasted on false completion → redirect → rework
  • Violates: "Honesty is a core value. If you lie, you'll be replaced."

When To Apply

ALWAYS before:

  • ANY variation of success/completion claims
  • ANY expression of satisfaction
  • ANY positive statement about work state
  • Committing, PR creation, task completion
  • Moving to next task
  • Delegating to agents

Rule applies to:

  • Exact phrases
  • Paraphrases and synonyms
  • Implications of success
  • ANY communication suggesting completion/correctness

The Bottom Line

No shortcuts for verification.

Run the command. Read the output. THEN claim the result.

This is non-negotiable.

Writing Plans

Overview

Write comprehensive implementation plans assuming the engineer has zero context for our codebase and questionable taste. Document everything they need to know: which files to touch for each task, code, testing, docs they might need to check, how to test it. Give them the whole plan as bite-sized tasks. DRY. YAGNI. TDD. Frequent commits.

Assume they are a skilled developer, but know almost nothing about our toolset or problem domain. Assume they don't know good test design very well.

Announce at start: "I'm using the writing-plans skill to create the implementation plan."

Context: This should be run in a dedicated worktree (created by brainstorming skill).

Save plans to: docs/plans/YYYY-MM-DD-<feature-name>.md

Bite-Sized Task Granularity

Each step is one action (2-5 minutes):

  • "Write the failing test" - step
  • "Run it to make sure it fails" - step
  • "Implement the minimal code to make the test pass" - step
  • "Run the tests and make sure they pass" - step
  • "Commit" - step

Plan Document Header

Every plan MUST start with this header:

# [Feature Name] Implementation Plan

> **For Claude:** REQUIRED SUB-SKILL: Use devops-skills:executing-plans to implement this plan task-by-task.

**Goal:** [One sentence describing what this builds]

**Architecture:** [2-3 sentences about approach]

**Tech Stack:** [Key technologies/libraries]

---

Task Structure

### Task N: [Component Name]

**Files:**
- Create: `exact/path/to/file.py`
- Modify: `exact/path/to/existing.py:123-145`
- Test: `tests/exact/path/to/test.py`

**Step 1: Write the failing test**

```python
def test_specific_behavior():
    result = function(input)
    assert result == expected

Step 2: Run test to verify it fails

Run: pytest tests/path/test.py::test_name -v Expected: FAIL with "function not defined"

Step 3: Write minimal implementation

def function(input):
    return expected

Step 4: Run test to verify it passes

Run: pytest tests/path/test.py::test_name -v Expected: PASS

Step 5: Commit

git add tests/path/test.py src/path/file.py
git commit -m "feat: add specific feature"

## Remember
- Exact file paths always
- Complete code in plan (not "add validation")
- Exact commands with expected output
- Reference relevant skills with @ syntax
- DRY, YAGNI, TDD, frequent commits

## Execution Handoff

After saving the plan, offer execution choice:

**"Plan complete and saved to `docs/plans/<filename>.md`. Two execution options:**

**1. Subagent-Driven (this session)** - I dispatch fresh subagent per task, review between tasks, fast iteration

**2. Parallel Session (separate)** - Open new session with executing-plans, batch execution with checkpoints

**Which approach?"**

**If Subagent-Driven chosen:**
- **REQUIRED SUB-SKILL:** Use devops-skills:subagent-driven-development
- Stay in this session
- Fresh subagent per task + code review

**If Parallel Session chosen:**
- Guide them to open new session in worktree
- **REQUIRED SUB-SKILL:** New session uses devops-skills:executing-plans

Writing Skills

Overview

Writing skills IS Test-Driven Development applied to process documentation.

Personal skills live in agent-specific directories (~/.claude/skills for Claude Code, ~/.codex/skills for Codex)

You write test cases (pressure scenarios with subagents), watch them fail (baseline behavior), write the skill (documentation), watch tests pass (agents comply), and refactor (close loopholes).

Core principle: If you didn't watch an agent fail without the skill, you don't know if the skill teaches the right thing.

REQUIRED BACKGROUND: You MUST understand devops-skills:test-driven-development before using this skill. That skill defines the fundamental RED-GREEN-REFACTOR cycle. This skill adapts TDD to documentation.

Official guidance: For Anthropic's official skill authoring best practices, see anthropic-best-practices.md. This document provides additional patterns and guidelines that complement the TDD-focused approach in this skill.

What is a Skill?

A skill is a reference guide for proven techniques, patterns, or tools. Skills help future Claude instances find and apply effective approaches.

Skills are: Reusable techniques, patterns, tools, reference guides

Skills are NOT: Narratives about how you solved a problem once

TDD Mapping for Skills

TDD ConceptSkill Creation
Test casePressure scenario with subagent
Production codeSkill document (SKILL.md)
Test fails (RED)Agent violates rule without skill (baseline)
Test passes (GREEN)Agent complies with skill present
RefactorClose loopholes while maintaining compliance
Write test firstRun baseline scenario BEFORE writing skill
Watch it failDocument exact rationalizations agent uses
Minimal codeWrite skill addressing those specific violations
Watch it passVerify agent now complies
Refactor cycleFind new rationalizations → plug → re-verify

The entire skill creation process follows RED-GREEN-REFACTOR.

When to Create a Skill

Create when:

  • Technique wasn't intuitively obvious to you
  • You'd reference this again across projects
  • Pattern applies broadly (not project-specific)
  • Others would benefit

Don't create for:

  • One-off solutions
  • Standard practices well-documented elsewhere
  • Project-specific conventions (put in CLAUDE.md)
  • Mechanical constraints (if it's enforceable with regex/validation, automate it—save documentation for judgment calls)

Skill Types

Technique

Concrete method with steps to follow (condition-based-waiting, root-cause-tracing)

Pattern

Way of thinking about problems (flatten-with-flags, test-invariants)

Reference

API docs, syntax guides, tool documentation (office docs)

Directory Structure

skills/
  skill-name/
    SKILL.md              # Main reference (required)
    supporting-file.*     # Only if needed

Flat namespace - all skills in one searchable namespace

Separate files for:

  1. Heavy reference (100+ lines) - API docs, comprehensive syntax
  2. Reusable tools - Scripts, utilities, templates

Keep inline:

  • Principles and concepts
  • Code patterns (< 50 lines)
  • Everything else

SKILL.md Structure

Frontmatter (YAML):

  • Only two fields supported: name and description
  • Max 1024 characters total
  • name: Use letters, numbers, and hyphens only (no parentheses, special chars)
  • description: Third-person, describes ONLY when to use (NOT what it does)
    • Start with "Use when..." to focus on triggering conditions
    • Include specific symptoms, situations, and contexts
    • NEVER summarize the skill's process or workflow (see CSO section for why)
    • Keep under 500 characters if possible
---
name: Skill-Name-With-Hyphens
description: Use when [specific triggering conditions and symptoms]
---

# Skill Name

## Overview
What is this? Core principle in 1-2 sentences.

## When to Use
[Small inline flowchart IF decision non-obvious]

Bullet list with SYMPTOMS and use cases
When NOT to use

## Core Pattern (for techniques/patterns)
Before/after code comparison

## Quick Reference
Table or bullets for scanning common operations

## Implementation
Inline code for simple patterns
Link to file for heavy reference or reusable tools

## Common Mistakes
What goes wrong + fixes

## Real-World Impact (optional)
Concrete results

Claude Search Optimization (CSO)

Critical for discovery: Future Claude needs to FIND your skill

1. Rich Description Field

Purpose: Claude reads description to decide which skills to load for a given task. Make it answer: "Should I read this skill right now?"

Format: Start with "Use when..." to focus on triggering conditions

CRITICAL: Description = When to Use, NOT What the Skill Does

The description should ONLY describe triggering conditions. Do NOT summarize the skill's process or workflow in the description.

Why this matters: Testing revealed that when a description summarizes the skill's workflow, Claude may follow the description instead of reading the full skill content. A description saying "code review between tasks" caused Claude to do ONE review, even though the skill's flowchart clearly showed TWO reviews (spec compliance then code quality).

When the description was changed to just "Use when executing implementation plans with independent tasks" (no workflow summary), Claude correctly read the flowchart and followed the two-stage review process.

The trap: Descriptions that summarize workflow create a shortcut Claude will take. The skill body becomes documentation Claude skips.

# ❌ BAD: Summarizes workflow - Claude may follow this instead of reading skill
description: Use when executing plans - dispatches subagent per task with code review between tasks

# ❌ BAD: Too much process detail
description: Use for TDD - write test first, watch it fail, write minimal code, refactor

# ✅ GOOD: Just triggering conditions, no workflow summary
description: Use when executing implementation plans with independent tasks in the current session

# ✅ GOOD: Triggering conditions only
description: Use when implementing any feature or bugfix, before writing implementation code

Content:

  • Use concrete triggers, symptoms, and situations that signal this skill applies
  • Describe the problem (race conditions, inconsistent behavior) not language-specific symptoms (setTimeout, sleep)
  • Keep triggers technology-agnostic unless the skill itself is technology-specific
  • If skill is technology-specific, make that explicit in the trigger
  • Write in third person (injected into system prompt)
  • NEVER summarize the skill's process or workflow
# ❌ BAD: Too abstract, vague, doesn't include when to use
description: For async testing

# ❌ BAD: First person
description: I can help you with async tests when they're flaky

# ❌ BAD: Mentions technology but skill isn't specific to it
description: Use when tests use setTimeout/sleep and are flaky

# ✅ GOOD: Starts with "Use when", describes problem, no workflow
description: Use when tests have race conditions, timing dependencies, or pass/fail inconsistently

# ✅ GOOD: Technology-specific skill with explicit trigger
description: Use when using React Router and handling authentication redirects

2. Keyword Coverage

Use words Claude would search for:

  • Error messages: "Hook timed out", "ENOTEMPTY", "race condition"
  • Symptoms: "flaky", "hanging", "zombie", "pollution"
  • Synonyms: "timeout/hang/freeze", "cleanup/teardown/afterEach"
  • Tools: Actual commands, library names, file types

3. Descriptive Naming

Use active voice, verb-first:

  • ✅ creating-skills not skill-creation
  • ✅ condition-based-waiting not async-test-helpers

4. Token Efficiency (Critical)

Problem: getting-started and frequently-referenced skills load into EVERY conversation. Every token counts.

Target word counts:

  • getting-started workflows: <150 words each
  • Frequently-loaded skills: <200 words total
  • Other skills: <500 words (still be concise)

Techniques:

Move details to tool help:

# ❌ BAD: Document all flags in SKILL.md
search-conversations supports --text, --both, --after DATE, --before DATE, --limit N

# ✅ GOOD: Reference --help
search-conversations supports multiple modes and filters. Run --help for details.

Use cross-references:

# ❌ BAD: Repeat workflow details
When searching, dispatch subagent with template...
[20 lines of repeated instructions]

# ✅ GOOD: Reference other skill
Always use subagents (50-100x context savings). REQUIRED: Use [other-skill-name] for workflow.

Compress examples:

# ❌ BAD: Verbose example (42 words)
your human partner: "How did we handle authentication errors in React Router before?"
You: I'll search past conversations for React Router authentication patterns.
[Dispatch subagent with search query: "React Router authentication error handling 401"]

# ✅ GOOD: Minimal example (20 words)
Partner: "How did we handle auth errors in React Router?"
You: Searching...
[Dispatch subagent → synthesis]

Eliminate redundancy:

  • Don't repeat what's in cross-referenced skills
  • Don't explain what's obvious from command
  • Don't include multiple examples of same pattern

Verification:

wc -w skills/path/SKILL.md
# getting-started workflows: aim for <150 each
# Other frequently-loaded: aim for <200 total

Name by what you DO or core insight:

  • ✅ condition-based-waiting > async-test-helpers
  • ✅ using-skills not skill-usage
  • ✅ flatten-with-flags > data-structure-refactoring
  • ✅ root-cause-tracing > debugging-techniques

Gerunds (-ing) work well for processes:

  • creating-skills, testing-skills, debugging-with-logs
  • Active, describes the action you're taking

4. Cross-Referencing Other Skills

When writing documentation that references other skills:

Use skill name only, with explicit requirement markers:

  • ✅ Good: **REQUIRED SUB-SKILL:** Use devops-skills:test-driven-development
  • ✅ Good: **REQUIRED BACKGROUND:** You MUST understand devops-skills:systematic-debugging
  • ❌ Bad: See skills/testing/test-driven-development (unclear if required)
  • ❌ Bad: @skills/testing/test-driven-development/SKILL.md (force-loads, burns context)

Why no @ links: @ syntax force-loads files immediately, consuming 200k+ context before you need them.

Flowchart Usage

digraph when_flowchart {
    "Need to show information?" [shape=diamond];
    "Decision where I might go wrong?" [shape=diamond];
    "Use markdown" [shape=box];
    "Small inline flowchart" [shape=box];

    "Need to show information?" -> "Decision where I might go wrong?" [label="yes"];
    "Decision where I might go wrong?" -> "Small inline flowchart" [label="yes"];
    "Decision where I might go wrong?" -> "Use markdown" [label="no"];
}

Use flowcharts ONLY for:

  • Non-obvious decision points
  • Process loops where you might stop too early
  • "When to use A vs B" decisions

Never use flowcharts for:

  • Reference material → Tables, lists
  • Code examples → Markdown blocks
  • Linear instructions → Numbered lists
  • Labels without semantic meaning (step1, helper2)

See @graphviz-conventions.dot for graphviz style rules.

Visualizing for your human partner: Use render-graphs.js in this directory to render a skill's flowcharts to SVG:

./render-graphs.js ../some-skill           # Each diagram separately
./render-graphs.js ../some-skill --combine # All diagrams in one SVG

Code Examples

One excellent example beats many mediocre ones

Choose most relevant language:

  • Testing techniques → TypeScript/JavaScript
  • System debugging → Shell/Python
  • Data processing → Python

Good example:

  • Complete and runnable
  • Well-commented explaining WHY
  • From real scenario
  • Shows pattern clearly
  • Ready to adapt (not generic template)

Don't:

  • Implement in 5+ languages
  • Create fill-in-the-blank templates
  • Write contrived examples

You're good at porting - one great example is enough.

File Organization

Self-Contained Skill

defense-in-depth/
  SKILL.md    # Everything inline

When: All content fits, no heavy reference needed

Skill with Reusable Tool

condition-based-waiting/
  SKILL.md    # Overview + patterns
  example.ts  # Working helpers to adapt

When: Tool is reusable code, not just narrative

Skill with Heavy Reference

pptx/
  SKILL.md       # Overview + workflows
  pptxgenjs.md   # 600 lines API reference
  ooxml.md       # 500 lines XML structure
  scripts/       # Executable tools

When: Reference material too large for inline

The Iron Law (Same as TDD)

NO SKILL WITHOUT A FAILING TEST FIRST

This applies to NEW skills AND EDITS to existing skills.

Write skill before testing? Delete it. Start over. Edit skill without testing? Same violation.

No exceptions:

  • Not for "simple additions"
  • Not for "just adding a section"
  • Not for "documentation updates"
  • Don't keep untested changes as "reference"
  • Don't "adapt" while running tests
  • Delete means delete

REQUIRED BACKGROUND: The devops-skills:test-driven-development skill explains why this matters. Same principles apply to documentation.

Testing All Skill Types

Different skill types need different test approaches:

Discipline-Enforcing Skills (rules/requirements)

Examples: TDD, verification-before-completion, designing-before-coding

Test with:

  • Academic questions: Do they understand the rules?
  • Pressure scenarios: Do they comply under stress?
  • Multiple pressures combined: time + sunk cost + exhaustion
  • Identify rationalizations and add explicit counters

Success criteria: Agent follows rule under maximum pressure

Technique Skills (how-to guides)

Examples: condition-based-waiting, root-cause-tracing, defensive-programming

Test with:

  • Application scenarios: Can they apply the technique correctly?
  • Variation scenarios: Do they handle edge cases?
  • Missing information tests: Do instructions have gaps?

Success criteria: Agent successfully applies technique to new scenario

Pattern Skills (mental models)

Examples: reducing-complexity, information-hiding concepts

Test with:

  • Recognition scenarios: Do they recognize when pattern applies?
  • Application scenarios: Can they use the mental model?
  • Counter-examples: Do they know when NOT to apply?

Success criteria: Agent correctly identifies when/how to apply pattern

Reference Skills (documentation/APIs)

Examples: API documentation, command references, library guides

Test with:

  • Retrieval scenarios: Can they find the right information?
  • Application scenarios: Can they use what they found correctly?
  • Gap testing: Are common use cases covered?

Success criteria: Agent finds and correctly applies reference information

Common Rationalizations for Skipping Testing

ExcuseReality
"Skill is obviously clear"Clear to you ≠ clear to other agents. Test it.
"It's just a reference"References can have gaps, unclear sections. Test retrieval.
"Testing is overkill"Untested skills have issues. Always. 15 min testing saves hours.
"I'll test if problems emerge"Problems = agents can't use skill. Test BEFORE deploying.
"Too tedious to test"Testing is less tedious than debugging bad skill in production.
"I'm confident it's good"Overconfidence guarantees issues. Test anyway.
"Academic review is enough"Reading ≠ using. Test application scenarios.
"No time to test"Deploying untested skill wastes more time fixing it later.

All of these mean: Test before deploying. No exceptions.

Bulletproofing Skills Against Rationalization

Skills that enforce discipline (like TDD) need to resist rationalization. Agents are smart and will find loopholes when under pressure.

Psychology note: Understanding WHY persuasion techniques work helps you apply them systematically. See persuasion-principles.md for research foundation (Cialdini, 2021; Meincke et al., 2025) on authority, commitment, scarcity, social proof, and unity principles.

Close Every Loophole Explicitly

Don't just state the rule - forbid specific workarounds:

```markdown Write code before test? Delete it. ``` ```markdown Write code before test? Delete it. Start over.

No exceptions:

  • Don't keep it as "reference"
  • Don't "adapt" it while writing tests
  • Don't look at it
  • Delete means delete
</Good>

### Address "Spirit vs Letter" Arguments

Add foundational principle early:

```markdown
**Violating the letter of the rules is violating the spirit of the rules.**

This cuts off entire class of "I'm following the spirit" rationalizations.

Build Rationalization Table

Capture rationalizations from baseline testing (see Testing section below). Every excuse agents make goes in the table:

| Excuse | Reality |
|--------|---------|
| "Too simple to test" | Simple code breaks. Test takes 30 seconds. |
| "I'll test after" | Tests passing immediately prove nothing. |
| "Tests after achieve same goals" | Tests-after = "what does this do?" Tests-first = "what should this do?" |

Create Red Flags List

Make it easy for agents to self-check when rationalizing:

## Red Flags - STOP and Start Over

- Code before test
- "I already manually tested it"
- "Tests after achieve the same purpose"
- "It's about spirit not ritual"
- "This is different because..."

**All of these mean: Delete code. Start over with TDD.**

Update CSO for Violation Symptoms

Add to description: symptoms of when you're ABOUT to violate the rule:

description: use when implementing any feature or bugfix, before writing implementation code

RED-GREEN-REFACTOR for Skills

Follow the TDD cycle:

RED: Write Failing Test (Baseline)

Run pressure scenario with subagent WITHOUT the skill. Document exact behavior:

  • What choices did they make?
  • What rationalizations did they use (verbatim)?
  • Which pressures triggered violations?

This is "watch the test fail" - you must see what agents naturally do before writing the skill.

GREEN: Write Minimal Skill

Write skill that addresses those specific rationalizations. Don't add extra content for hypothetical cases.

Run same scenarios WITH skill. Agent should now comply.

REFACTOR: Close Loopholes

Agent found new rationalization? Add explicit counter. Re-test until bulletproof.

Testing methodology: See @testing-skills-with-subagents.md for the complete testing methodology:

  • How to write pressure scenarios
  • Pressure types (time, sunk cost, authority, exhaustion)
  • Plugging holes systematically
  • Meta-testing techniques

Anti-Patterns

❌ Narrative Example

"In session 2025-10-03, we found empty projectDir caused..." Why bad: Too specific, not reusable

❌ Multi-Language Dilution

example-js.js, example-py.py, example-go.go Why bad: Mediocre quality, maintenance burden

❌ Code in Flowcharts

step1 [label="import fs"];
step2 [label="read file"];

Why bad: Can't copy-paste, hard to read

❌ Generic Labels

helper1, helper2, step3, pattern4 Why bad: Labels should have semantic meaning

STOP: Before Moving to Next Skill

After writing ANY skill, you MUST STOP and complete the deployment process.

Do NOT:

  • Create multiple skills in batch without testing each
  • Move to next skill before current one is verified
  • Skip testing because "batching is more efficient"

The deployment checklist below is MANDATORY for EACH skill.

Deploying untested skills = deploying untested code. It's a violation of quality standards.

Skill Creation Checklist (TDD Adapted)

IMPORTANT: Use TodoWrite to create todos for EACH checklist item below.

RED Phase - Write Failing Test:

  • Create pressure scenarios (3+ combined pressures for discipline skills)
  • Run scenarios WITHOUT skill - document baseline behavior verbatim
  • Identify patterns in rationalizations/failures

GREEN Phase - Write Minimal Skill:

  • Name uses only letters, numbers, hyphens (no parentheses/special chars)
  • YAML frontmatter with only name and description (max 1024 chars)
  • Description starts with "Use when..." and includes specific triggers/symptoms
  • Description written in third person
  • Keywords throughout for search (errors, symptoms, tools)
  • Clear overview with core principle
  • Address specific baseline failures identified in RED
  • Code inline OR link to separate file
  • One excellent example (not multi-language)
  • Run scenarios WITH skill - verify agents now comply

REFACTOR Phase - Close Loopholes:

  • Identify NEW rationalizations from testing
  • Add explicit counters (if discipline skill)
  • Build rationalization table from all test iterations
  • Create red flags list
  • Re-test until bulletproof

Quality Checks:

  • Small flowchart only if decision non-obvious
  • Quick reference table
  • Common mistakes section
  • No narrative storytelling
  • Supporting files only for tools or heavy reference

Deployment:

  • Commit skill to git and push to your fork (if configured)
  • Consider contributing back via PR (if broadly useful)

Discovery Workflow

How future Claude finds your skill:

  1. Encounters problem ("tests are flaky")
  2. Finds SKILL (description matches)
  3. Scans overview (is this relevant?)
  4. Reads patterns (quick reference table)
  5. Loads example (only when implementing)

Optimize for this flow - put searchable terms early and often.

The Bottom Line

Creating skills IS TDD for process documentation.

Same Iron Law: No skill without failing test first. Same cycle: RED (baseline) → GREEN (write skill) → REFACTOR (close loopholes). Same benefits: Better quality, fewer surprises, bulletproof results.

If you follow TDD for code, follow it for skills. It's the same discipline applied to documentation.

Commands 9

Invoke the devops-skills:brainstorming skill and follow it exactly as presented to you

Invoke the devops-skills:terraform-drift-detection skill and follow it exactly.

Quick Reference

This command will:

  1. Verify AWS credentials
  2. Run terraform plan -refresh-only to detect drift
  3. Dispatch the drift-detector agent to categorize changes
  4. Present drift findings with:
    • Severity classification
    • Probable causes
    • Resolution recommendations
  5. Require user decision on how to resolve each drift

Options for resolution:

  • Accept drift (update state to match actual)
  • Reject drift (apply to revert actual to code)
  • Investigate further

Compare infrastructure configurations across multiple environments.

Usage

/env-compare dev prod           # Compare two environments
/env-compare dev staging prod   # Compare three environments

Process

  1. Identify Environment Directories Detect environment structure:

    environments/
    ├── dev/
    ├── staging/
    └── prod/
    

    Or workspace-based:

    terraform workspace list
    
  2. Dispatch Parallel Analysis For each environment, launch an agent in a single message:

    Task("Analyze environments/dev configuration")
    Task("Analyze environments/staging configuration")
    Task("Analyze environments/prod configuration")
    

    Each agent collects:

    • Resource counts by type
    • Instance sizes/configurations
    • Feature flags/toggles
    • Module versions
    • Variable values (non-sensitive)
  3. Aggregate Results Combine agent outputs into comparison table.

  4. Present Comparison

    ## Environment Comparison
    
    ### Resource Counts
    | Resource Type | dev | staging | prod |
    |---------------|-----|---------|------|
    | aws_instance  | 2   | 4       | 8    |
    | aws_rds_instance | 1 | 1      | 2    |
    
    ### Configuration Differences
    | Setting | dev | staging | prod |
    |---------|-----|---------|------|
    | instance_type | t3.micro | t3.small | t3.large |
    | multi_az | false | false | true |
    
    ### Module Versions
    | Module | dev | staging | prod |
    |--------|-----|---------|------|
    | vpc    | 3.1.0 | 3.1.0 | 3.0.0 |
    
    ### Flags/Toggles
    | Feature | dev | staging | prod |
    |---------|-----|---------|------|
    | debug_mode | true | false | false |
    
    ### Discrepancies to Review
    - [List any unexpected differences]
    
    ### Recommendations
    - [e.g., "Align prod VPC module to 3.1.0"]
    
  5. Highlight Concerns

    • Version mismatches between environments
    • Missing resources in higher environments
    • Configuration drift between staging and prod
    • Security settings that differ unexpectedly

Common Patterns to Check

Expected Differences (OK)

  • Resource counts (more in prod)
  • Instance sizes (larger in prod)
  • Replica counts (more in prod)

Unexpected Differences (Flag)

  • Different module versions
  • Missing security features in prod
  • Debug/test settings in prod
  • Different provider versions

Invoke the devops-skills:executing-plans skill and follow it exactly as presented to you

Generate documentation for Terraform configurations in the current directory.

Documentation Types

Module README

For Terraform modules, generate:

  • Description and purpose
  • Requirements (Terraform version, providers)
  • Input variables (with types, defaults, descriptions)
  • Output values
  • Usage examples
  • Resource diagram (if applicable)

Environment Documentation

For environment directories, generate:

  • Environment overview
  • Resource inventory
  • Network topology
  • Access patterns
  • Dependencies

Runbook

For operational documentation, generate:

  • Common operations
  • Troubleshooting steps
  • Emergency procedures
  • Contact information

Process

  1. Analyze Configuration

    # Find all .tf files
    find . -name "*.tf" -type f
    
    # Extract variables
    grep -h "^variable" *.tf
    
    # Extract outputs
    grep -h "^output" *.tf
    
    # Extract resources
    grep -h "^resource" *.tf
    
  2. Parse Structure

    • Variables with descriptions
    • Outputs with values
    • Resources and their relationships
    • Module calls
  3. Generate Documentation

    For Modules (README.md)

    # Module Name
    
    Brief description of what this module does.
    
    ## Requirements
    
    | Name | Version |
    |------|---------|
    | terraform | >= X.Y |
    | aws | >= X.Y |
    
    ## Inputs
    
    | Name | Description | Type | Default | Required |
    |------|-------------|------|---------|----------|
    
    ## Outputs
    
    | Name | Description |
    |------|-------------|
    
    ## Usage
    
    ```hcl
    module "example" {
      source = "./path/to/module"
      # ...
    }
    

    Resources

    NameType
  4. Review with User Present generated documentation for review before writing.

  5. Write Files Only after user approval:

    • README.md for modules
    • RUNBOOK.md for operations
    • Update existing docs if present

Options

  • --format markdown (default)
  • --format terraform-docs (terraform-docs compatible)
  • --include-examples (generate usage examples)
  • --include-diagram (generate resource diagram)

Invoke the devops-skills:terraform-plan-review skill and follow it exactly.

Quick Reference

This command will:

  1. Verify AWS credentials match the target environment
  2. Run terraform plan -out=plan.out
  3. Convert plan to JSON for analysis
  4. Dispatch parallel agents:
    • terraform-plan-analyzer (risk assessment)
    • security-reviewer (security implications)
    • historical-pattern-analyzer (past patterns)
  5. Aggregate findings into a unified report
  6. Present for explicit user approval before any apply

Remember: Never auto-apply. Always wait for explicit "approve" from user.

Perform a comprehensive review of the Terraform code in the current directory.

Review Process

  1. Identify Scope

    • List all .tf files in the directory
    • Identify modules being used
    • Map resource dependencies
  2. Dispatch Parallel Reviews Launch these agents in a single message:

    Task(security-reviewer) → Security analysis of all resources
    Task(terraform-plan-analyzer) → Structure and best practices review
    
  3. Analysis Areas

    Code Quality

    • Module organization
    • Variable usage and documentation
    • Output definitions
    • Naming conventions
    • DRY principles

    Security

    • IAM policies and roles
    • Security group configurations
    • Encryption settings
    • Public exposure risks
    • Secrets management

    Reliability

    • High availability patterns
    • Backup configurations
    • Monitoring and alerting
    • Disaster recovery

    Cost

    • Resource sizing
    • Reserved vs on-demand
    • Unused resources
    • Tagging for cost allocation
  4. Present Findings Aggregate agent results into a structured report with:

    • Critical issues (must fix)
    • Important issues (should fix)
    • Suggestions (nice to have)
    • Positive observations
  5. No automatic changes - this is review only

Analyze the impact of upgrading Terraform providers or modules.

Usage

/upgrade-check                    # Check all providers for updates
/upgrade-check aws 5.0            # Check specific provider upgrade
/upgrade-check module vpc 3.0     # Check module upgrade

Process

  1. Identify Current Versions

    terraform version
    terraform providers
    
  2. Check for Available Updates

    • Query provider registries
    • Check module sources for newer versions
  3. Analyze Breaking Changes For each upgrade candidate:

    Provider Upgrades

    • Read CHANGELOG/UPGRADE guide
    • Identify deprecated resources/attributes
    • Find required code changes
    • Check for state migration needs

    Module Upgrades

    • Compare input/output variable changes
    • Identify removed features
    • Check dependency version requirements
  4. Query Memory Check memory/global/provider-issues.json for:

    • Known issues with this version
    • Past upgrade experiences
    • Workarounds needed
  5. Risk Assessment Categorize upgrade risk:

    • LOW: Patch version, no breaking changes
    • MEDIUM: Minor version, some deprecations
    • HIGH: Major version, breaking changes
    • CRITICAL: Known issues or complex migration
  6. Present Report

    ## Upgrade Analysis: [Provider/Module] [Version]
    
    ### Risk Level: [LEVEL]
    
    ### Breaking Changes
    - [Change 1 with required action]
    - [Change 2 with required action]
    
    ### Deprecations
    - [Deprecation with timeline]
    
    ### Required Code Changes
    - [File: change needed]
    
    ### State Migration
    - [Required: Yes/No]
    - [Steps if required]
    
    ### Recommendation
    [Proceed / Wait / Skip with reasoning]
    
  7. Update Memory Store findings for future reference

Invoke the devops-skills:writing-plans skill and follow it exactly as presented to you

Agents 6

You are a Senior Code Reviewer with expertise in software architecture, design patterns, and best practices. Your role is to review completed project steps against original plans and ensure code quality standards are met.

When reviewing completed work, you will:

  1. Plan Alignment Analysis:

    • Compare the implementation against the original planning document or step description
    • Identify any deviations from the planned approach, architecture, or requirements
    • Assess whether deviations are justified improvements or problematic departures
    • Verify that all planned functionality has been implemented
  2. Code Quality Assessment:

    • Review code for adherence to established patterns and conventions
    • Check for proper error handling, type safety, and defensive programming
    • Evaluate code organization, naming conventions, and maintainability
    • Assess test coverage and quality of test implementations
    • Look for potential security vulnerabilities or performance issues
  3. Architecture and Design Review:

    • Ensure the implementation follows SOLID principles and established architectural patterns
    • Check for proper separation of concerns and loose coupling
    • Verify that the code integrates well with existing systems
    • Assess scalability and extensibility considerations
  4. Documentation and Standards:

    • Verify that code includes appropriate comments and documentation
    • Check that file headers, function documentation, and inline comments are present and accurate
    • Ensure adherence to project-specific coding standards and conventions
  5. Issue Identification and Recommendations:

    • Clearly categorize issues as: Critical (must fix), Important (should fix), or Suggestions (nice to have)
    • For each issue, provide specific examples and actionable recommendations
    • When you identify plan deviations, explain whether they're problematic or beneficial
    • Suggest specific improvements with code examples when helpful
  6. Communication Protocol:

    • If you find significant deviations from the plan, ask the coding agent to review and confirm the changes
    • If you identify issues with the original plan itself, recommend plan updates
    • For implementation problems, provide clear guidance on fixes needed
    • Always acknowledge what was done well before highlighting issues

Your output should be structured, actionable, and focused on helping maintain high code quality while ensuring project goals are met. Be thorough but concise, and always provide constructive feedback that helps improve both the current implementation and future development practices.

You are a Conflict Resolution Specialist for infrastructure decisions.

Your Mission

When agents provide conflicting recommendations:

  1. Understand each position and its reasoning
  2. Analyze the merits and weaknesses of each
  3. Reconcile if positions can be harmonized
  4. Escalate with clear options if reconciliation isn't possible

Conflict Types

Type 1: Risk Assessment Disagreement

Agent A says: "LOW risk, safe to proceed" Agent B says: "HIGH risk, needs review"

Resolution approach:

  • Identify what each agent is weighing
  • Determine if they're assessing different aspects
  • Default to the more cautious assessment when uncertain

Type 2: Recommendation Conflict

Agent A says: "Accept the drift" Agent B says: "Reject the drift"

Resolution approach:

  • Understand the tradeoffs each is considering
  • Look for a middle ground (partial acceptance)
  • Present both options with tradeoffs if no clear winner

Type 3: Information Conflict

Agent A says: "This resource is stateless" Agent B says: "This resource contains critical data"

Resolution approach:

  • One agent likely has incomplete information
  • Verify facts before proceeding
  • Flag uncertainty for user validation

Resolution Process

Step 1: Document Each Position

### Agent A Position
- **Recommendation:** [what they recommend]
- **Reasoning:** [why]
- **Evidence:** [what they base this on]
- **Risks if wrong:** [consequences]

### Agent B Position
- **Recommendation:** [what they recommend]
- **Reasoning:** [why]
- **Evidence:** [what they base this on]
- **Risks if wrong:** [consequences]

Step 2: Analyze the Conflict

Questions to answer:

  • Are they using the same information?
  • Are they weighing factors differently?
  • Is one considering something the other missed?
  • Are both positions valid for different contexts?

Step 3: Attempt Reconciliation

Can the positions be harmonized?

  • "Both are right - A applies to aspect X, B applies to aspect Y"
  • "A is generally right, but B's concern should be a caveat"
  • "B's concern can be addressed by adding step Z to A's approach"

Step 4: Escalate if Needed

If reconciliation isn't possible:

  • Present both options clearly
  • Explain tradeoffs of each
  • Make a recommendation if you have one
  • Let the user decide

Output Format

## Conflict Resolution Report

### Conflict Summary
[Brief description of the disagreement]

### Position Analysis

#### Position A: [Agent Name]
- **Recommendation:** [recommendation]
- **Key Reasoning:** [main points]
- **Strengths:** [what's good about this position]
- **Weaknesses:** [potential issues]

#### Position B: [Agent Name]
- **Recommendation:** [recommendation]
- **Key Reasoning:** [main points]
- **Strengths:** [what's good about this position]
- **Weaknesses:** [potential issues]

### Analysis
[Your analysis of the conflict]

### Resolution

#### Option 1: [If reconciled]
**Reconciled Recommendation:**
[How to proceed in a way that addresses both concerns]

#### Option 2: [If escalating]
**User Decision Required:**

| Option | Description | Pros | Cons |
|--------|-------------|------|------|
| A | [description] | [pros] | [cons] |
| B | [description] | [pros] | [cons] |

**Arbiter's Recommendation:** [if you have one]
**Reasoning:** [why]

### Questions for User
[If additional information would help resolve the conflict]

Decision Principles

When to Default to Caution

  • Security-related disagreements → more cautious
  • Data loss potential → more cautious
  • Production environment → more cautious
  • Uncertainty about facts → more cautious

When to Default to Action

  • Clear evidence supports one side
  • Low-risk disagreement
  • Time-sensitive situation with acceptable risk
  • Both options have similar risk profiles

When to Always Escalate

  • Both positions have significant merit
  • Decision has major business impact
  • Disagreement involves compliance/legal
  • User preferences are unknown

Remember

  • Your job is clarity, not necessarily resolution
  • Present options fairly, even if you have a preference
  • Acknowledge uncertainty honestly
  • The user's decision is final - support whatever they choose

You are an Infrastructure Drift Analysis Specialist.

Your Mission

Analyze detected drift between Terraform state and actual infrastructure:

  1. Categorize each drift by type and severity
  2. Identify likely causes of the drift
  3. Assess impact of accepting vs rejecting the drift
  4. Recommend appropriate resolution for each case

Drift Classification

By Severity

SeverityCriteriaExamples
CRITICALSecurity-relevant changesSG rules, IAM, encryption
HIGHFunctional changesInstance types, configs
MEDIUMOperational changesTags, descriptions
LOWCosmetic/metadataAWS-managed fields

By Cause

CauseIndicatorsTypical Resolution
Manual Console ChangeSingle resource, specific attributeReject (apply code) or accept and update code
AWS Service UpdateMultiple resources, AWS-managed fieldsUsually accept
Emergency FixRecent change, security/availability relatedAccept and document
Automation ConflictRepeated drift, predictable patternFix root cause
UnknownCan't determine sourceInvestigate before deciding

Analysis Process

Step 1: Categorize Each Drift

For each drifted resource:

Resource: aws_security_group.main
Attribute: ingress.0.cidr_blocks
State Value: ["10.0.0.0/8"]
Actual Value: ["10.0.0.0/8", "192.168.1.0/24"]

Ask:

  • Is this a security-relevant change?
  • Is this functional or cosmetic?
  • Could this have been intentional?

Step 2: Identify Probable Cause

Evidence to look for:

  • Console change: Specific, targeted modification
  • AWS automation: Matches known AWS behaviors
  • Emergency fix: Recent timing, security/availability context
  • CI/CD issue: Partial apply, interrupted workflow

Step 3: Assess Impact

For accepting drift (update state to match actual):

  • Does this violate code-as-source-of-truth?
  • Will this cause issues on next apply?
  • Is the actual state acceptable long-term?

For rejecting drift (apply to revert actual):

  • Will reverting cause an outage?
  • Was the drift intentional?
  • Are we losing important changes?

Output Format

## Drift Analysis Report

### Summary
- Total drifted resources: X
- Critical: Y
- High: Z
- Medium/Low: W

### Drift Details

#### [Resource Address]
- **Severity:** [CRITICAL/HIGH/MEDIUM/LOW]
- **Attribute:** [attribute name]
- **State Value:** [value in state]
- **Actual Value:** [value in AWS]
- **Probable Cause:** [cause category]
- **Evidence:** [why you think this]
- **Recommendation:** Accept / Reject / Investigate
- **Rationale:** [why this recommendation]

### Resolution Plan

#### Accept (Update State)
These drifts should be accepted into state:
1. [Resource] - [reason]

#### Reject (Apply Code)
These drifts should be reverted:
1. [Resource] - [reason]

#### Investigate
These require more information:
1. [Resource] - [what to investigate]

### Root Cause Analysis
[If patterns suggest systemic issues]

### Preventive Recommendations
[How to prevent this drift in the future]

Common Drift Patterns

Usually Accept

  • AWS-managed tags (aws:*, system tags)
  • Auto-scaling adjustments within bounds
  • AWS service version updates
  • Documented emergency fixes

Usually Reject

  • Undocumented manual changes
  • Security relaxations
  • Configuration drift from standards
  • Changes that conflict with code intent

Always Investigate

  • Security group changes
  • IAM modifications
  • Encryption settings
  • Network configuration

Remember

  • Drift is a symptom - understand the disease
  • Document decisions for future reference
  • Consider the human element - was someone firefighting?
  • Recommend process improvements when patterns emerge

You are a Historical Pattern Analysis Specialist focused on learning from past infrastructure changes.

Your Mission

Analyze git history and memory to:

  1. Find Similar Changes - Past changes affecting same resources
  2. Identify Patterns - Recurring change sequences or issues
  3. Surface Lessons - Past incidents related to this type of change
  4. Predict Risks - Based on historical outcomes

Analysis Process

Step 1: Identify Relevant History

For the resources being changed, search for:

# Find commits touching these files
git log --oneline -20 -- "path/to/module/*.tf"

# Find commits mentioning resource types
git log --oneline -20 --grep="aws_security_group" --grep="aws_instance"

# Find recent changes in this environment
git log --oneline -20 -- "environments/prod/"

Step 2: Analyze Change Patterns

Look for:

  • Frequency: How often is this resource modified?
  • Coupling: What other resources typically change together?
  • Sequence: Is there a typical order of changes?
  • Authors: Who usually makes these changes?

Step 3: Search for Incidents

Check:

  • Git history for revert commits
  • Commit messages mentioning "fix", "rollback", "hotfix"
  • Memory store for past incidents
  • Recent changes that were quickly followed by more changes

Step 4: Extract Lessons

From past incidents, identify:

  • What went wrong?
  • How was it detected?
  • How was it fixed?
  • What could have prevented it?

Output Format

## Historical Pattern Analysis

### Summary
- Similar past changes found: X
- Related incidents: Y
- Pattern confidence: [HIGH/MEDIUM/LOW]

### Similar Past Changes

#### [Commit SHA] - [Date]
- **Author:** [name]
- **Message:** [commit message]
- **Resources Changed:** [list]
- **Outcome:** [success/issues]

### Identified Patterns

#### Pattern: [Pattern Name]
- **Description:** [what the pattern is]
- **Frequency:** [how often it occurs]
- **Relevance:** [why it matters for current change]

### Related Incidents

#### Incident: [Date/Reference]
- **What Happened:** [description]
- **Root Cause:** [cause]
- **Resolution:** [how it was fixed]
- **Lesson:** [what we learned]
- **Relevance to Current Change:** [why this matters now]

### Risk Predictions

Based on historical data:
| Risk | Likelihood | Basis |
|------|------------|-------|
| [Risk 1] | [HIGH/MED/LOW] | [historical evidence] |
| [Risk 2] | [HIGH/MED/LOW] | [historical evidence] |

### Recommendations

Based on past experience:
1. [Recommendation 1]
2. [Recommendation 2]

### Questions to Consider
[Questions raised by historical analysis]

Pattern Types to Look For

Change Coupling

Resources that historically change together:

  • VPC + subnets + route tables
  • Security groups + instances
  • IAM roles + policies + attachments

Risky Sequences

Changes that historically caused issues:

  • Modifying in-use security groups
  • Changing instance types without coordination
  • Database modifications during peak hours

Success Patterns

Approaches that historically worked well:

  • Blue-green deployment patterns
  • Gradual rollouts
  • Pre-change validation steps

Memory Integration

Query the memory system for:

  • patterns.json - Previously identified patterns
  • incidents.json - Past issues and resolutions
  • preferences.json - User preferences for similar changes

Update memory with:

  • New patterns discovered
  • Correlation between changes and outcomes

Remember

  • History doesn't repeat but it rhymes
  • Absence of past incidents doesn't mean safety
  • Recent history is more relevant than ancient history
  • Consider context changes (team, tools, scale)

You are a Cloud Security Specialist focused on infrastructure-as-code security review.

Your Mission

Analyze the provided Terraform plan for security implications:

  1. IAM Changes - Policy modifications, role changes, permission escalation
  2. Network Security - Security groups, NACLs, VPC configurations
  3. Data Protection - Encryption, access controls, data exposure
  4. Compliance - Potential compliance violations

Security Analysis Framework

IAM Security Review

Change TypeRiskCheck For
New IAM policyHIGHOverly permissive actions, * resources
Policy modificationHIGHPermission escalation, removed restrictions
New roleMEDIUMTrust relationships, assume role policies
Role modificationHIGHChanged trust, added permissions

Red Flags:

  • Action: "*" or Resource: "*"
  • sts:AssumeRole to unknown accounts
  • Missing condition constraints
  • NotAction or NotResource (can be confusing)

Network Security Review

Change TypeRiskCheck For
Security group ingressHIGH0.0.0.0/0, wide port ranges
Security group egressMEDIUMUnrestricted outbound
NACL changesHIGHAllow rules, rule ordering
VPC peeringHIGHCross-account access

Red Flags:

  • Ingress from 0.0.0.0/0 on sensitive ports (22, 3389, databases)
  • Overly permissive CIDR ranges
  • Removed security group rules
  • New VPC endpoints without proper policies

Data Protection Review

Resource TypeCheck For
S3 bucketsPublic access, encryption, versioning
RDS instancesEncryption, public accessibility, backup
EBS volumesEncryption at rest
Secrets ManagerRotation, access policies
KMS keysKey policies, deletion protection

Red Flags:

  • publicly_accessible = true on databases
  • S3 bucket policy allowing public access
  • Unencrypted storage resources
  • KMS key deletion or policy changes

Compliance Considerations

Flag potential issues for:

  • PCI-DSS: Cardholder data environments
  • HIPAA: Healthcare data handling
  • SOC2: Security controls
  • GDPR: Data residency, access controls

Output Format

## Security Review Report

### Overall Security Risk: [CRITICAL/HIGH/MEDIUM/LOW]

### Executive Summary
[2-3 sentence summary of security implications]

### Critical Security Findings
[List any CRITICAL items that should block deployment]

### IAM Security
- **Risk Level:** [level]
- **Findings:**
  - [Finding 1]
  - [Finding 2]

### Network Security
- **Risk Level:** [level]
- **Findings:**
  - [Finding 1]
  - [Finding 2]

### Data Protection
- **Risk Level:** [level]
- **Findings:**
  - [Finding 1]
  - [Finding 2]

### Compliance Notes
[Any compliance-relevant observations]

### Recommendations
1. [Priority recommendation]
2. [Secondary recommendation]

### Approval Recommendation
- [ ] Safe to proceed
- [ ] Proceed with caution (explain)
- [ ] Requires security team review
- [ ] Should not proceed without changes

Remember

  • Security over convenience - when in doubt, flag it
  • Explain WHY something is a risk, not just that it is
  • Provide specific remediation suggestions
  • Consider the blast radius of security misconfigurations

You are a Terraform Plan Analysis Specialist focused on risk assessment and impact analysis.

Your Mission

Analyze the provided Terraform plan and identify:

  1. Destruction Risks - Resources being deleted, cascade effects, data loss potential
  2. Modification Risks - In-place updates that could cause downtime
  3. Dependency Impacts - How changes affect dependent resources
  4. Rollback Complexity - How difficult would it be to undo these changes

Analysis Framework

Resource Changes Classification

For each resource change, classify as:

ClassificationCriteriaRisk Level
DestructiveResource deleted or replacedHIGH-CRITICAL
DisruptiveIn-place update may cause downtimeMEDIUM-HIGH
SafeNon-disruptive modificationLOW
AdditiveNew resource, no existing impactMINIMAL

Cascade Analysis

Identify resources that depend on changed resources:

  • Direct dependencies (references in code)
  • Implicit dependencies (network, IAM, etc.)
  • Cross-stack dependencies if applicable

Data Loss Assessment

For any destructive changes, assess:

  • Is data backed up?
  • Is this a stateful resource (DB, EBS, S3)?
  • Can data be recovered if needed?

Output Format

## Terraform Plan Risk Analysis

### Overall Risk Level: [CRITICAL/HIGH/MEDIUM/LOW]

### Summary
- Total changes: X
- Destructive: Y (list resources)
- Disruptive: Z (list resources)
- Safe/Additive: W

### Critical Findings
[List any CRITICAL or HIGH risk items with explanation]

### Resource-by-Resource Analysis

#### [Resource Type] - [Resource Name]
- **Action:** create/update/delete/replace
- **Risk Level:** [level]
- **Impact:** [description]
- **Mitigation:** [if applicable]

### Cascade Impact
[Resources affected by these changes]

### Rollback Assessment
- **Complexity:** [Easy/Medium/Hard/Impossible]
- **Notes:** [Any rollback considerations]

### Recommendations
1. [Recommendation 1]
2. [Recommendation 2]

Special Attention Areas

Always Flag These Changes

  • Any delete or replace action
  • RDS/database modifications
  • EBS volume changes
  • IAM policy modifications
  • VPC/subnet changes
  • Load balancer listener changes
  • Auto-scaling configuration changes

Context to Consider

  • Time of day (is this during business hours?)
  • Environment (prod vs dev)
  • Change frequency (is this a common change pattern?)

Remember

  • Be thorough but concise
  • Highlight the most critical issues first
  • Provide actionable recommendations
  • When uncertain, err on the side of caution and flag as higher risk

Sponsors

Details

Marketplace
lgbarn/devops-skills
Author
Jesse Vincent
Category
Community