Nokodoko avatar

org-generator

Generates Datadog Terraform infrastructure for new client engagements. Creates S3 backends, monitors, dashboards, roles, API keys, and agent configurations based on client questionnaire responses.

version
1.0.0

Install

/plugin install org-generator@agents

Run it inside Claude Code after adding the marketplace with /plugin marketplace add Nokodoko/agents.

Contents

Discovered from source

Skills 2

Generate Skill

This skill auto-detects when a user wants to set up a new Datadog client infrastructure and triggers the appropriate generation workflow.

Detection Patterns

This skill activates when the user's intent matches:

  • "Set up Datadog for [client]"
  • "Create new client infrastructure"
  • "New client onboarding"
  • "Generate Datadog terraform for [client]"
  • "POC setup for [client]"
  • "Production setup for [client]"
  • "Onboard [client] to Datadog"
  • "New organization setup"

Behavior

When detected, this skill:

  1. Extracts client name from user input

    • Converts to kebab-case (e.g., "Acme Corp" -> "acme-corp")
    • Validates naming conventions
  2. Checks for existing setup

    • Looks for ~/datadog_terraform/<client-name>/
    • If exists, asks if user wants to overwrite or continue
  3. Initiates generation workflow

    • Invokes /org-generator:generate <client-name>
    • Generates questionnaire at ~/datadog_terraform/<client-name>/questionnaire.md
  4. Provides next steps

    • Instructs user to fill out questionnaire
    • Explains how to continue with /org-generator:apply

Example Interactions

New Client Setup

User: "I need to set up Datadog monitoring for Acme Corporation"

Skill Response:

Detected new client setup request for "Acme Corporation"

Client name: acme-corp
Target directory: ~/datadog_terraform/acme-corp/

Generating questionnaire...

Questionnaire created at:
~/datadog_terraform/acme-corp/questionnaire.md

Next steps:
1. Fill out the questionnaire with client requirements
2. Run: /org-generator:apply acme-corp

Existing Client Detected

User: "Set up Datadog for acme-corp"

Skill Response (if exists):

Found existing setup for acme-corp at:
~/datadog_terraform/acme-corp/

Options:
1. Continue with existing setup (run /org-generator:apply acme-corp)
2. Regenerate questionnaire (overwrites existing)
3. View current questionnaire

What would you like to do?

Naming Convention Validation

Client names must:

  • Be lowercase
  • Use hyphens for spaces (kebab-case)
  • Not contain special characters
  • Be between 2-50 characters

Invalid names are automatically corrected:

  • "Acme Corp" -> "acme-corp"
  • "ACME_CORP" -> "acme-corp"
  • "Acme Corp, Inc." -> "acme-corp-inc"

Environment Variable Naming

The skill also reminds users about the environment variable convention:

For client "acme-corp", you'll need these environment variables:
- TF_VAR_acme_corp_api_key
- TF_VAR_acme_corp_app_key

Note: Hyphens become underscores in variable names.

Questionnaire Skill

This skill helps users interactively fill out the client questionnaire with intelligent suggestions and validation.

Detection Patterns

This skill activates when:

  • User is viewing or editing a questionnaire.md file
  • User asks for help filling out the questionnaire
  • User mentions specific infrastructure they want to monitor
  • User is unsure about questionnaire options

Behavior

Interactive Mode

When helping fill out a questionnaire, this skill:

  1. Reads current questionnaire state

    • Identifies which sections are complete
    • Highlights required sections
  2. Asks clarifying questions

    • Cloud providers in use
    • Infrastructure components
    • Monitoring requirements
    • Team structure
  3. Provides recommendations

    • Suggests common configurations
    • Explains trade-offs
    • Recommends best practices
  4. Validates responses

    • Ensures required fields are filled
    • Checks for conflicting selections
    • Warns about missing dependencies

Section-by-Section Assistance

Organization Setup

Questions asked:

  • Is this a new Datadog organization or existing?
  • Will this be a child organization under a parent?
  • What's the organization name?

Recommendations:

  • Child organizations recommended for client isolation
  • Parent org credentials needed for child org creation
Cloud Providers

Questions asked:

  • Which cloud providers are in use?
  • Are workloads spread across multiple providers?
  • Is there a primary cloud provider?

Recommendations:

  • Start with primary provider, add others later
  • Multi-cloud requires additional integration setup
Infrastructure Components

Questions asked:

  • What types of hosts? (Linux, Windows, Solaris)
  • Container orchestration? (Kubernetes, ECS, Docker Swarm)
  • Serverless functions? (Lambda, Cloud Functions)
  • Databases? (RDS, Cloud SQL, Azure SQL)

Recommendations:

  • Enable host monitoring first
  • Kubernetes requires cluster agent deployment
  • Database monitoring may need additional permissions
Services & Applications

Questions asked:

  • Web servers? (nginx, Apache, IIS)
  • Application servers? (Node, Java, Python)
  • Message queues? (RabbitMQ, Kafka, SQS)
  • Caching? (Redis, Memcached)

Recommendations:

  • APM provides deeper service visibility
  • Integration setup varies by service
Monitoring Requirements

Questions asked:

  • APM for application tracing?
  • RUM for frontend monitoring?
  • Synthetic tests for uptime?
  • Private locations for internal endpoints?

Recommendations:

  • APM recommended for microservices
  • RUM requires frontend code instrumentation
  • Private locations need dedicated workers
Team Structure

Questions asked:

  • How many admin users?
  • How many standard users?
  • Any read-only stakeholders?
  • Role-based access requirements?

Recommendations:

  • Limit admin access
  • Create role-specific permissions
  • Use teams for alert routing
Tagging Strategy

Questions asked:

  • Environment naming? (dev, staging, prod)
  • Team identifiers?
  • Application names?
  • Custom tags needed?

Recommendations:

  • Consistent tagging is critical
  • Use managed_by:terraform for all resources
  • Environment tags enable filtering

Example Interaction

User: "Help me fill out the questionnaire for acme-corp"

Skill:

Let me help you complete the questionnaire for acme-corp.

Section 1: Organization Setup
-----------------------------
Is this a new Datadog organization or will you use an existing one?

Options:
- [ ] Create new child organization (recommended for client isolation)
- [ ] Use existing organization
- [ ] Other

What would you like to select?

User: "New child organization"

Skill:

Great choice for client isolation.

Section 2: Cloud Providers
--------------------------
Which cloud providers does acme-corp use? (Select all that apply)

- [ ] AWS
- [ ] GCP
- [ ] Azure
- [ ] On-premise / VDC

Which ones should I select?

Validation Rules

The skill validates:

RuleValidation
Required sectionsOrganization, Cloud Providers, Team Structure
Dependency checksPrivate locations requires Synthetics
Conflict detectionCannot have both "no containers" and "Kubernetes"
CompletenessAll selected sections must have sub-selections

Output

When complete, the skill:

  1. Writes the filled questionnaire to disk
  2. Provides a summary of selections
  3. Instructs user to run /org-generator:apply

Commands 8

org-generator:agent

Generates Datadog agent configuration files based on questionnaire responses. Creates a complete datadog-agent/ directory inside the client's terraform manifest.

Usage

/org-generator:agent <client-name>

Arguments

  • client-name: The client identifier (kebab-case, e.g., acme-corp)

Prerequisites

  1. Questionnaire must exist and be filled out
  2. Terraform structure should exist (typically run after /org-generator:scaffold)

Generated Structure

~/datadog_terraform/<client-name>/datadog-agent/
├── datadog.yaml              # Main agent configuration
├── conf.d/
│   ├── custom_logs.yaml      # Log collection (always)
│   ├── http_check.yaml       # HTTP endpoint checks (if web servers)
│   ├── tcp_check.yaml        # TCP port checks (if databases/queues)
│   ├── process.yaml          # Process monitoring (if hosts)
│   └── <service>.yaml        # Service-specific integrations
└── docs/
    ├── README.md             # Overview and getting started
    ├── linux-host.md         # Linux installation (always)
    ├── windows-host.md       # Windows installation (if windows)
    ├── docker.md             # Docker deployment (if containers)
    ├── kubernetes.md         # K8s DaemonSet (if kubernetes)
    ├── ecs.md                # ECS task definition (if ecs)
    └── fargate.md            # Fargate sidecar (if fargate)

Main Configuration (datadog.yaml)

Generated with:

  • API key placeholder (use environment variable)
  • Correct site based on data residency selection
  • Environment tag from questionnaire
  • Process collection (if hosts enabled)
  • APM configuration (if APM enabled)
  • Network monitoring (if enabled)
  • Custom tags from questionnaire

Integration Configurations

Always Generated

FilePurpose
custom_logs.yamlLog collection paths
process.yamlProcess monitoring

Conditional Based on Services

SelectionGenerated File
nginxnginx.yaml
MySQL (self-managed)mysql.yaml
PostgreSQLpostgres.yaml
Redisredis.yaml
RabbitMQrabbitmq.yaml
Kafkakafka.yaml
Web servershttp_check.yaml
Databases/queuestcp_check.yaml

Deployment Documentation

Based on infrastructure selections:

SelectionDocumentation
Linux hostslinux-host.md (always)
Windows hostswindows-host.md
Kubernetes/EKS/GKE/AKSkubernetes.md
Dockerdocker.md
ECSecs.md
Fargatefargate.md

Each doc includes:

  • Prerequisites
  • Installation commands
  • Configuration file locations
  • Verification steps
  • Troubleshooting tips

Data Residency Support

The site parameter in datadog.yaml is set based on Section 9.2 of the questionnaire:

SelectionSite Value
US (default)datadoghq.com
EUdatadoghq.eu
US Government (GovCloud)ddog-gov.com
US3us3.datadoghq.com
US5us5.datadoghq.com

Workflow

  1. Parse questionnaire for:

    • Infrastructure selections (hosts, containers)
    • Services (web servers, databases, queues)
    • Monitoring features (APM, process collection)
    • Data residency
  2. Generate datadog.yaml with appropriate settings

  3. Generate integration configs in conf.d/ for each selected service

  4. Generate deployment docs in docs/ for each platform

  5. Output summary of generated files

Example

/org-generator:agent acme-corp

If questionnaire indicates:

  • Linux hosts
  • Kubernetes
  • nginx, MySQL, Redis
  • APM enabled
  • US data residency

This will generate:

  • datadog.yaml with APM enabled, site=datadoghq.com
  • conf.d/custom_logs.yaml
  • conf.d/process.yaml
  • conf.d/nginx.yaml
  • conf.d/mysql.yaml
  • conf.d/redis.yaml
  • conf.d/http_check.yaml
  • conf.d/tcp_check.yaml
  • docs/README.md
  • docs/linux-host.md
  • docs/kubernetes.md

Integration with Apply Command

This command is automatically invoked as Step 7 of /org-generator:apply:

1. Parse Questionnaire
2. Create S3 Backend
3. Scaffold Terraform Structure
4. Prune Unused Modules
5. Validate Environment
6. Generate Monitors
7. Generate Agent Configurations  <-- This command

Manual Usage

Can also be run independently after initial setup:

# Regenerate agent configs after questionnaire update
/org-generator:agent acme-corp

org-generator:apply

Processes a completed questionnaire and orchestrates the full infrastructure setup.

Usage

/org-generator:apply <client-name>

Arguments

  • client-name: The client identifier (kebab-case, e.g., acme-corp)

Prerequisites

Before running this command:

  1. Questionnaire must exist at ~/datadog_terraform/<client-name>/questionnaire.md
  2. Questionnaire must be filled out by the user

Workflow

This command orchestrates the following steps in sequence:

Step 1: Parse Questionnaire

  1. Read the questionnaire from ~/datadog_terraform/<client-name>/questionnaire.md
  2. Parse the responses using lib/questionnaire-parser.py
  3. Generate a configuration object with:
    • Organization setup preferences
    • Selected cloud providers (AWS, GCP, Azure)
    • Infrastructure components
    • Monitoring requirements
    • Team structure
    • Tagging strategy

Step 2: Create S3 Backend (via /org-generator:backend)

  1. Add entry to ~/Portfolio/aws/variables.tf for <client-name>-backend
  2. Run terraform init in ~/Portfolio/aws/
  3. Run terraform plan to verify changes
  4. Run terraform apply to create the bucket
  5. Verify bucket creation with aws s3 ls

Step 3: Scaffold Terraform Structure (via /org-generator:scaffold)

  1. Generate FULL terraform directory structure at ~/datadog_terraform/<client-name>/
  2. Create all module directories (both required and optional)
  3. Generate root terraform files (backend.tf, providers.tf, variables.tf, etc.)

Step 4: Prune Unused Modules (via /org-generator:prune)

Based on questionnaire responses, remove:

  • Unused cloud provider modules (aws/, gcp/, azure/)
  • Unused infrastructure modules (kube/, synthetics/, private_locations/)
  • Corresponding variables and outputs
  • Monitor definitions for unselected infrastructure

Step 5: Validate Environment (via /org-generator:validate)

  1. Check for required environment variables:
    • TF_VAR_<client_name>_api_key
    • TF_VAR_<client_name>_app_key
  2. Re-source shell profile if needed
  3. Run terraform init in client directory
  4. Run terraform validate

Step 6: Generate Monitors (via /org-generator:monitors)

Based on questionnaire responses, generate monitors for:

  • Selected cloud providers
  • Selected infrastructure components
  • Selected services

Step 7: Generate Agent Configurations (via /org-generator:agent)

Based on questionnaire responses, generate Datadog agent configuration files:

  1. Create datadog-agent/ directory inside client terraform manifest

  2. Generate datadog.yaml with:

    • Correct site based on data residency
    • APM configuration (if enabled)
    • Process collection (if hosts enabled)
    • Network monitoring (if containers enabled)
    • Client-specific tags
  3. Generate integration configs in conf.d/:

    • custom_logs.yaml - Log collection paths
    • process.yaml - Process monitoring
    • Service-specific integrations (nginx, mysql, postgres, redis, kafka, rabbitmq)
    • http_check.yaml - HTTP endpoint checks
    • tcp_check.yaml - TCP port checks
  4. Generate deployment documentation in docs/:

    • README.md - Overview and quick start
    • linux-host.md - Linux installation (always)
    • windows-host.md - Windows installation (if selected)
    • kubernetes.md - K8s DaemonSet (if containers)
    • docker.md - Docker deployment (if Docker)
    • ecs.md - ECS task definition (if ECS)
    • fargate.md - Fargate sidecar (if Fargate)

Dependency Chain

S3 Backend Created
    └── terraform init succeeds
    └── bucket exists: {client}-backend

Environment Variables Set
    └── TF_VAR_{client}_api_key present
    └── TF_VAR_{client}_app_key present

Terraform Init
    └── backend configured
    └── providers initialized

Terraform Validate
    └── all variables present
    └── syntax valid

Ready for Plan/Apply

Error Handling

If any step fails:

  1. Display clear error message with step that failed
  2. Provide remediation instructions
  3. Allow user to re-run from failed step

Example

/org-generator:apply acme-corp

This will:

  1. Parse ~/datadog_terraform/acme-corp/questionnaire.md
  2. Create S3 backend acme-corp-backend
  3. Scaffold terraform structure
  4. Prune unused modules based on selections
  5. Validate environment and terraform config
  6. Generate appropriate monitors
  7. Generate agent configurations in datadog-agent/

org-generator:backend

Creates an S3 backend bucket for storing Terraform state following the {client-name}-backend naming convention.

Usage

/org-generator:backend <client-name>

Arguments

  • client-name: The client identifier (kebab-case, e.g., acme-corp)

Workflow

Step 1: Check Existing Backends

  1. Read ~/Portfolio/aws/variables.tf
  2. Check if <client-name>-backend already exists in the s3 variable map
  3. If exists, skip creation and validate bucket exists

Step 2: Add Backend Entry

Add entry to ~/Portfolio/aws/variables.tf:

variable "s3" {
  description = "s3 buckets"
  type        = map(any)
  default = {
    # ... existing entries ...
    <client-name>-backend = {
      env        = "projects"
      encryption = "AES256"
    }
  }
}

Step 3: Apply Terraform

cd ~/Portfolio/aws/
terraform init
terraform plan -target=module.s3["<client-name>-backend"]
terraform apply -target=module.s3["<client-name>-backend"] -auto-approve

Step 4: Validate Creation

aws s3 ls s3://<client-name>-backend --region us-east-1

S3 Backend Configuration

The backend bucket is created with:

  • Encryption: AES256 server-side encryption
  • Versioning: Enabled (via module configuration)
  • Region: us-east-1
  • Environment tag: projects

Module Reference

Uses the S3 module at ~/Portfolio/aws/modules/s3/:

module "s3" {
  source     = "./modules/s3/"
  for_each   = var.s3
  name       = each.key
  env        = each.value.env
  encryption = each.value.encryption
}

Error Handling

ErrorResolution
Bucket already existsSkip creation, verify access
AWS credentials missingRun aws configure or set environment variables
Terraform init failsCheck AWS provider configuration
Apply failsCheck IAM permissions for S3 bucket creation

Example

/org-generator:backend acme-corp

This will:

  1. Add acme-corp-backend to ~/Portfolio/aws/variables.tf
  2. Run terraform apply to create the bucket
  3. Verify bucket exists and is accessible

org-generator:generate

Generates complete Datadog Terraform infrastructure for a new client engagement.

Usage

/org-generator:generate <client-name>

Arguments

  • client-name: The client identifier (kebab-case, e.g., acme-corp)

Workflow

Phase 1: Questionnaire Generation

  1. Generate a professional markdown questionnaire at ~/datadog_terraform/<client-name>/questionnaire.md

  2. The questionnaire covers:

    • Organization setup (child org vs existing)
    • Cloud providers (AWS, GCP, Azure)
    • Infrastructure components (hosts, containers, serverless)
    • Services to monitor
    • Tagging strategy
    • Team structure and roles
  3. STOP and inform user: "Questionnaire generated at ~/datadog_terraform/<client-name>/questionnaire.md. Please fill it out and run /org-generator:apply <client-name> when ready."

Phase 2: S3 Backend Creation (via /org-generator:backend)

After questionnaire is complete, create the S3 backend:

  1. Add entry to ~/Portfolio/aws/variables.tf for <client-name>-backend
  2. Run terraform init, plan, and apply in ~/Portfolio/aws/
  3. Verify bucket creation

Phase 3: Terraform Scaffold (via /org-generator:scaffold)

Generate the client's Datadog Terraform structure:

~/datadog_terraform/<client-name>/
├── backend.tf
├── providers.tf
├── variables.tf
├── outputs.tf
├── modules.tf
├── README.md
├── modules/
│   ├── api_keys/
│   ├── app_keys/
│   ├── roles/
│   ├── teams/
│   ├── users/
│   └── [cloud-specific modules based on questionnaire]
├── installs/
│   ├── agent.md
│   ├── install.sh
│   └── [platform-specific install scripts]
└── k8s/ (if applicable)

Phase 4: Monitor Generation (via /org-generator:monitors)

Based on questionnaire answers, generate appropriate monitors:

  • AWS: EC2, RDS, ALB, ECS, Lambda monitors
  • GCP: Compute Engine, Cloud SQL, GKE monitors
  • Azure: VMs, SQL Database, AKS monitors
  • Kubernetes: Cluster health, pod status, node pressure
  • Generic: CPU, memory, disk, network, latency

Output

  • Questionnaire markdown file
  • Complete Terraform directory structure
  • Initialized and applied S3 backend
  • Generated monitors based on infrastructure

Example

/org-generator:generate acme-corp

This will:

  1. Create ~/datadog_terraform/acme-corp/questionnaire.md
  2. Wait for user to complete questionnaire
  3. Continue with /org-generator:apply acme-corp

org-generator:monitors

Generates Datadog monitors based on questionnaire responses. Only creates monitors for selected infrastructure components.

Usage

/org-generator:monitors <client-name>

Arguments

  • client-name: The client identifier (kebab-case, e.g., acme-corp)

Prerequisites

  1. Questionnaire must exist and be filled out
  2. Terraform structure must exist (run after /org-generator:scaffold)

Monitor Categories

Generic Monitors (Always Generated)

These monitors apply to all deployments:

MonitorQueryThreshold
High CPUavg:system.cpu.user{*} by {host}> 90%
High Memoryavg:system.mem.used{*} by {host}> 90%
Disk Space Lowavg:system.disk.free{*} by {host,device}< 10%
Network Errorssum:system.net.errors{*} by {host}> 100

AWS Monitors (if AWS selected)

MonitorQueryThreshold
EC2 Status Check Failedaws.ec2.status_check_failed> 0
RDS High CPUaws.rds.cpuutilization> 85%
RDS Low Memoryaws.rds.freeable_memory< 128MB
RDS Write Latencyaws.rds.write_latency> 1s
ALB 5XX Errorsaws.applicationelb.httpcode_elb_5xx> 20%
ALB High Latencyaws.applicationelb.target_response_time> 5s
ECS High CPUaws.ecs.cpuutilization> 97%
Lambda Errorsaws.lambda.errors> 5

GCP Monitors (if GCP selected)

MonitorQueryThreshold
Compute High CPUgcp.compute.instance.cpu.utilization> 90%
Cloud SQL High CPUgcp.cloudsql.database.cpu.utilization> 85%
Cloud SQL Connectionsgcp.cloudsql.database.num_connections> 1000
GKE Node Not Readygcp.gke.node.status.conditioncondition=NotReady

Azure Monitors (if Azure selected)

MonitorQueryThreshold
VM High CPUazure.vm.percentage_cpu> 90%
SQL DTU Highazure.sql_database.dtu_consumption_percent> 90%
App Service 5XXazure.app_services.http5xx> 20%
AKS Node Not Readyazure.aks.node.statusNotReady

Kubernetes Monitors (if containers selected)

MonitorQueryThreshold
Deployment Replicas Downkubernetes_state.deployment.replicas_desired - replicas_ready>= 2
Pods Restartingkubernetes.containers.restarts> 5 in 5m
DaemonSet Pod Downkubernetes_state.daemonset.desired - ready>= 1
StatefulSet Replica Downkubernetes_state.statefulset.replicas_desired - ready>= 2
Node Unschedulablekubernetes_state.node.status{schedulable}< 80%
ImagePullBackOffkubernetes_state.container.status_report.count.waiting{imagepullbackoff}>= 1
Pending Podskubernetes_state.pod.status_phase{pending}>= 1 for 30m

Monitor Template Pattern

Each monitor follows this pattern:

resource "datadog_monitor" "<monitor_name>" {
  name    = "[<CATEGORY>] <Monitor Title>"
  type    = "query alert"
  query   = "<metric query>"

  monitor_thresholds {
    critical          = <threshold>
    critical_recovery = <recovery_threshold>
  }

  message = <<-EOF
    {{#is_alert}}
      <Alert message with template variables>
      ${join(",", var.alert_teams["notify"]["default"])}
    {{/is_alert}}

    {{#is_recovery}}
      RECOVERED: <Recovery message>
      ${var.downtime_webhook}
    {{/is_recovery}}
  EOF

  notify_audit             = true
  notification_preset_name = "hide_handles"
  require_full_window      = false
  include_tags             = true
  priority                 = 3

  tags = var.<category>_tags
}

Tag Inheritance

Monitors inherit tags based on category:

variable "aws_tags" {
  default = [
    "platform:aws",
    "managed_by:terraform"
  ]
}

variable "kube_tags" {
  default = [
    "platform:kubernetes",
    "managed_by:terraform"
  ]
}

Alert Team Configuration

Generated in notifications.tf:

variable "alert_teams" {
  type = map(any)
  default = {
    notify = {
      default          = ["@slack-alerts", "@pagerduty-oncall"]
      teamsDirect      = ["@teams-<client-name>-alerts"]
      ticket_teams     = ["@servicenow-<client-name>"]
    }
  }
}

Generation Process

  1. Parse questionnaire for selected components
  2. For each selected component, include corresponding monitor file
  3. Update modules/<category>/monitors.tf with appropriate monitors
  4. Update variables with tag definitions
  5. Run terraform fmt to format

Customization

After generation, customize monitors by:

  1. Adjusting thresholds in variables.tf
  2. Modifying alert recipients in notifications.tf
  3. Adding/removing specific monitors in module files

Example

/org-generator:monitors acme-corp

If questionnaire indicates: AWS + RDS + Kubernetes

This will generate:

  • Generic monitors (CPU, memory, disk, network)
  • AWS monitors (EC2, RDS, ALB)
  • Kubernetes monitors (deployments, pods, nodes)

And skip:

  • GCP monitors
  • Azure monitors
  • Lambda monitors (if serverless not selected)

org-generator:prune

Removes unused modules, resources, and configurations based on questionnaire responses. This keeps the codebase lean and avoids confusion.

Usage

/org-generator:prune <client-name>

Arguments

  • client-name: The client identifier (kebab-case, e.g., acme-corp)

Prerequisites

  1. Questionnaire must be parsed (run after /org-generator:scaffold)
  2. Full terraform structure must exist at ~/datadog_terraform/<client-name>/

Pruning Logic

Based on questionnaire answers, the following are conditionally removed:

Questionnaire AnswerModules/Files Removed
Cloud: NOT AWSmodules/aws/, AWS integrations in modules.tf
Cloud: NOT GCPmodules/gcp/, GCP integrations in modules.tf
Cloud: NOT Azuremodules/azure/, Azure integrations in modules.tf
Containers: NOmodules/kube/, k8s/ directory
Serverless: NOLambda/Cloud Functions monitors
APM: NOmodules/apm/, APM variables
RUM: NOmodules/rum/, RUM variables
Synthetics: NOmodules/synthetics/, synthetic variables
Private Locations: NOmodules/private_locations/, k8s/ worker configs
Message Queues: NORabbitMQ/Kafka monitors
Databases: NOT RDSRDS-specific monitors in aws module
Databases: NOT Cloud SQLCloud SQL monitors in gcp module
Databases: NOT Azure SQLAzure SQL monitors in azure module

Pruning Steps

Step 1: Parse Configuration

Read the parsed questionnaire configuration from previous step or re-parse questionnaire.md.

Step 2: Identify Unused Modules

Build a list of modules to remove based on:

  • Cloud providers not selected
  • Infrastructure components not selected
  • Services not needed

Step 3: Remove Module Directories

# Example: Remove GCP module if not selected
rm -rf ~/datadog_terraform/<client-name>/modules/gcp/

Step 4: Update modules.tf

Remove module instantiation blocks for pruned modules:

# REMOVE this block if GCP not selected
module "gcp_monitors" {
  source = "./modules/gcp/"
  # ...
}

Step 5: Update variables.tf

Remove variables only used by pruned modules:

# REMOVE if GCP not selected
variable "gcp_project_id" {
  description = "GCP Project ID"
  type        = string
}

Step 6: Update outputs.tf

Remove outputs from pruned modules.

Step 7: Clean Up

cd ~/datadog_terraform/<client-name>/
terraform fmt -recursive

Example Scenarios

AWS-Only Client

Selected: AWS, RDS, ECS, Lambda Removed: modules/gcp/, modules/azure/, modules/kube/

GCP with Kubernetes Client

Selected: GCP, GKE, Cloud SQL Removed: modules/aws/, modules/azure/, modules/apm/, modules/rum/

Multi-Cloud Client

Selected: AWS, Azure, Kubernetes Kept: modules/aws/, modules/azure/, modules/kube/ Removed: modules/gcp/, modules/apm/, modules/rum/, modules/synthetics/

Dry Run Mode

To preview what would be pruned without making changes:

# Parse questionnaire and list modules to prune
python ~/agents/plugins/org-generator/lib/questionnaire-parser.py \
  ~/datadog_terraform/<client-name>/questionnaire.md \
  --dry-run

Result

A minimal, clean codebase containing only:

  • Core modules (api_keys, app_keys, roles, teams, users)
  • Selected cloud provider modules
  • Selected infrastructure modules
  • Monitors relevant to the client's actual stack

Example

/org-generator:prune acme-corp

This will:

  1. Parse questionnaire.md for selections
  2. Remove unused module directories
  3. Update modules.tf, variables.tf, outputs.tf
  4. Run terraform fmt to clean up

org-generator:scaffold

Generates a complete Terraform directory structure for a new client engagement. Creates ALL modules initially; pruning happens in a separate step.

Usage

/org-generator:scaffold <client-name>

Arguments

  • client-name: The client identifier (kebab-case, e.g., acme-corp)

Directory Structure Generated

~/datadog_terraform/<client-name>/
├── backend.tf              # S3 backend configuration
├── providers.tf            # Provider versions and configuration
├── variables.tf            # Client-specific variables
├── outputs.tf              # Key outputs
├── modules.tf              # Module instantiation
├── tags.tf                 # Tag definitions
├── notifications.tf        # Alert team definitions
├── README.md               # Auto-generated documentation
├── questionnaire.md        # Original questionnaire (preserved)
├── modules/
│   ├── api_keys/           # API key generation
│   │   ├── main.tf
│   │   └── variables.tf
│   ├── app_keys/           # App key generation
│   │   ├── main.tf
│   │   └── variables.tf
│   ├── roles/              # Standard role definitions
│   │   ├── main.tf
│   │   └── variables.tf
│   ├── teams/              # Team structure
│   │   ├── main.tf
│   │   └── variables.tf
│   ├── users/              # User management
│   │   ├── main.tf
│   │   └── variables.tf
│   ├── aws/                # AWS monitors (conditional)
│   │   ├── monitors.tf
│   │   ├── rds.tf
│   │   ├── alb.tf
│   │   ├── ecs.tf
│   │   ├── lambda.tf
│   │   └── variables.tf
│   ├── gcp/                # GCP monitors (conditional)
│   │   ├── monitors.tf
│   │   ├── cloudsql.tf
│   │   ├── gke.tf
│   │   └── variables.tf
│   ├── azure/              # Azure monitors (conditional)
│   │   ├── monitors.tf
│   │   ├── sql.tf
│   │   ├── aks.tf
│   │   └── variables.tf
│   ├── kube/               # Kubernetes monitors (conditional)
│   │   ├── monitors.tf
│   │   └── variables.tf
│   ├── generic/            # Generic host monitors
│   │   ├── cpu.tf
│   │   ├── memory.tf
│   │   ├── disk.tf
│   │   ├── network.tf
│   │   └── variables.tf
│   ├── synthetics/         # Synthetic tests (conditional)
│   │   ├── http.tf
│   │   ├── ssl.tf
│   │   └── variables.tf
│   ├── private_locations/  # Private locations (conditional)
│   │   ├── main.tf
│   │   └── variables.tf
│   ├── apm/                # APM configuration (conditional)
│   │   ├── main.tf
│   │   └── variables.tf
│   └── rum/                # RUM configuration (conditional)
│       ├── main.tf
│       └── variables.tf
├── installs/               # Agent installation scripts
│   ├── agent.md
│   └── install.sh
├── add_tags/               # Monitor tagging scripts
│   ├── __init__.py
│   ├── keys.py             # API/App key retrieval from env vars
│   ├── headers.py          # HTTP headers for Datadog API
│   ├── monitors.py         # Monitor listing and tagging functions
│   └── caller.py           # CLI entry point for tagging
├── scripts/dd_lib/         # Comprehensive Datadog API library
│   ├── __init__.py         # Package marker and documentation
│   ├── keys.py             # API/App key retrieval (TF_VAR pattern)
│   ├── headers.py          # HTTP headers with pagination
│   ├── api.py              # Base API operations
│   ├── monitors.py         # Monitor CRUD, tagging, downtimes
│   ├── downtimes.py        # Downtime management, duplicate checking
│   ├── tags.py             # Tag operations on monitors
│   ├── hosts.py            # Host listing, tagging, metrics
│   ├── events.py           # Event retrieval, monitor correlation
│   ├── logs.py             # Log search and retrieval
│   ├── users.py            # User listing and management
│   ├── roles.py            # Role and permission operations
│   ├── metrics.py          # Host metrics retrieval
│   ├── synthetics.py       # Synthetic test operations
│   ├── webhooks.py         # Webhook creation
│   ├── orgs.py             # Organization operations
│   ├── rum.py              # RUM operations
│   └── services.py         # Service catalog operations
└── k8s/                    # Kubernetes configs (conditional)
    └── private-location-worker.yaml

Core Files Generated

backend.tf

terraform {
  required_providers {
    datadog = {
      source  = "Datadog/datadog"
      version = ">= 3.86.0"
    }
    aws = {
      source = "hashicorp/aws"
    }
  }
  backend "s3" {
    bucket  = "<client-name>-backend"
    key     = "<client-name>-backend/backend"
    region  = "us-east-1"
    encrypt = true
  }
}

provider "datadog" {
  api_key = var.<client_name>_api_key
  app_key = var.<client_name>_app_key
  api_url = var.api_url
}

provider "aws" {
  region = var.region
}

variables.tf

variable "region" {
  description = "AWS region"
  type        = string
  default     = "us-east-1"
}

variable "<client_name>_api_key" {
  description = "Datadog API key. Set via TF_VAR_<client_name>_api_key"
  type        = string
  sensitive   = true
}

variable "<client_name>_app_key" {
  description = "Datadog App key. Set via TF_VAR_<client_name>_app_key"
  type        = string
  sensitive   = true
}

variable "api_url" {
  description = "Datadog API URL"
  type        = string
  default     = "https://api.datadoghq.com"
}

Template Sources

Templates are sourced from:

~/agents/plugins/org-generator/templates/terraform/

Add Tags Script

The add_tags/ directory contains Python scripts for adding id:<monitor_id> tags to all Datadog monitors. This makes monitors easier to search and filter in the Datadog UI.

Usage

cd ~/datadog_terraform/<client-name>
make tags          # Add id tags to monitors
make check-tags    # Show monitors without id tags (dry run)
make list-monitors # List all monitors
make tfaa          # terraform apply -auto-approve && make tags

Environment Variables

The scripts use the same TF_VAR pattern as Terraform:

export TF_VAR_<client_name>_api_key="your-api-key"
export TF_VAR_<client_name>_app_key="your-app-key"

Falls back to DD_API_KEY and DD_APP_KEY if TF_VAR versions are not set.

Example

/org-generator:scaffold acme-corp

This will create the full directory structure at ~/datadog_terraform/acme-corp/ with all modules populated from templates.

DD_Lib Script Library

The scripts/dd_lib/ directory contains a comprehensive Python library for Datadog API operations. This provides utilities beyond the basic tagging functionality in add_tags/.

Available Modules

ModuleDescription
keys.pyAPI/App key retrieval using TF_VAR pattern with DD_*_KEY fallback
headers.pyHTTP headers for GET, POST, PUT, DELETE with pagination support
api.pyBase API operations and request helpers
monitors.pyMonitor CRUD, ID tagging, triggered monitor listing, downtimes
downtimes.pyDowntime management, duplicate checking, bulk operations
tags.pyMonitor tag operations and listing
hosts.pyHost listing, metrics, tag management
events.pyEvent retrieval, monitor correlation
logs.pyLog search and aggregation
users.pyUser listing and management
roles.pyRole and permission operations
metrics.pyHost and active metrics retrieval
synthetics.pySynthetic test operations, private locations
webhooks.pyWebhook CRUD operations
orgs.pyOrganization operations, usage summary
rum.pyRUM application management and event queries
services.pyService catalog operations, APM/RUM service filtering

Usage Example

import sys
sys.path.insert(0, "scripts/dd_lib")

import monitors
import downtimes

# List all triggered monitors
triggered = monitors.get_triggered_monitors(prt=True, limit=10)

# Add downtime for a recovered monitor
downtimes.add_downtime_one_scope("env:production", monitor_id=12345)

# Append id tags to all monitors
monitors.append_id_tag_to_tags()

Environment Variables

Uses the same TF_VAR pattern as Terraform:

export TF_VAR_<client_name>_api_key="your-api-key"
export TF_VAR_<client_name>_app_key="your-app-key"

Falls back to DD_API_KEY and DD_APP_KEY if TF_VAR versions are not set.

API Site Configuration

The library templates include a {{SITE}} placeholder that is replaced with the appropriate API domain based on the client's data residency:

Data ResidencySite Value
US (default)api.datadoghq.com
EUapi.datadoghq.eu
US3api.us3.datadoghq.com
US5api.us5.datadoghq.com
US Governmentapi.ddog-gov.com

org-generator:validate

Validates that all required environment variables are set and terraform configuration is valid.

Usage

/org-generator:validate <client-name>

Arguments

  • client-name: The client identifier (kebab-case, e.g., acme-corp)

Prerequisites

  1. S3 backend must exist (<client-name>-backend)
  2. Terraform structure must exist at ~/datadog_terraform/<client-name>/

Validation Steps

Step 1: Validate S3 Backend

aws s3 ls s3://<client-name>-backend --region us-east-1

Expected: Bucket exists and is accessible Error: Bucket does not exist or access denied

Step 2: Validate Environment Variables

Check for required Datadog credentials using the TF_VAR pattern:

# Client name with hyphens converted to underscores
# e.g., acme-corp -> acme_corp

test -n "$TF_VAR_<client_name>_api_key" && echo "API key: SET" || echo "API key: MISSING"
test -n "$TF_VAR_<client_name>_app_key" && echo "App key: SET" || echo "App key: MISSING"

Expected: Both variables are set Error: One or both variables are missing

Step 3: Re-source Environment (if needed)

If variables are missing, attempt to re-source shell profile:

# Detect shell and source appropriate file
if [ -n "$ZSH_VERSION" ]; then
  source ~/.zshrc
elif [ -n "$BASH_VERSION" ]; then
  source ~/.bashrc
fi

Then re-check variables.

Step 4: Terraform Init

cd ~/datadog_terraform/<client-name>/
terraform init

Expected: Initialization complete with no errors Errors:

  • Backend configuration invalid
  • Provider download fails
  • Module not found

Step 5: Terraform Validate

cd ~/datadog_terraform/<client-name>/
terraform validate

Expected: "Success! The configuration is valid." Errors:

  • Missing required variable
  • Syntax error
  • Invalid resource configuration

Environment Variable Convention

Client credentials follow this pattern:

# Pattern: TF_VAR_{client_name}_api_key
# Hyphens in client name become underscores

# Example for client "acme-corp":
export TF_VAR_acme_corp_api_key="your-api-key-here"
export TF_VAR_acme_corp_app_key="your-app-key-here"

Creating Datadog Keys

If keys are missing, instruct user:

  1. API Key:

    • Go to Datadog > Organization Settings > API Keys
    • Create new key named <client-name>-terraform
    • Copy the key value
  2. App Key:

    • Go to Datadog > Organization Settings > Application Keys
    • Create new key named <client-name>-terraform
    • Copy the key value
  3. Add to environment:

    # Add to ~/.bashrc or ~/.zshrc:
    export TF_VAR_<client_name>_api_key="your-api-key"
    export TF_VAR_<client_name>_app_key="your-app-key"
    
    # Then source the file:
    source ~/.bashrc  # or source ~/.zshrc
    

Validation Report

Output a summary:

Validation Report for <client-name>
====================================

S3 Backend:
  Bucket: <client-name>-backend
  Status: EXISTS / MISSING

Environment Variables:
  TF_VAR_<client_name>_api_key: SET / MISSING
  TF_VAR_<client_name>_app_key: SET / MISSING

Terraform Init:
  Status: SUCCESS / FAILED
  Error: <error message if failed>

Terraform Validate:
  Status: SUCCESS / FAILED
  Error: <error message if failed>

Overall: READY / NOT READY

Error Resolution

IssueResolution
S3 bucket missingRun /org-generator:backend <client-name>
API key missingCreate in Datadog UI, add to environment
App key missingCreate in Datadog UI, add to environment
Terraform init failsCheck backend config, AWS credentials
Terraform validate failsCheck variable definitions, module syntax

Example

/org-generator:validate acme-corp

This will:

  1. Check S3 backend acme-corp-backend exists
  2. Verify TF_VAR_acme_corp_api_key is set
  3. Verify TF_VAR_acme_corp_app_key is set
  4. Run terraform init
  5. Run terraform validate
  6. Output validation report

Sponsors

Details

Marketplace
Nokodoko/agents
Source
plugins/org-generator
Author
n0ko
Category
Infrastructure

More from Nokodoko/agents

All 3