
org-generator
Generates Datadog Terraform infrastructure for new client engagements. Creates S3 backends, monitors, dashboards, roles, API keys, and agent configurations based on client questionnaire responses.
- version
- 1.0.0
Install
/plugin install org-generator@agentsRun it inside Claude Code after adding the marketplace with /plugin marketplace add Nokodoko/agents.
Contents
Discovered from sourceSkills 2
Generate Skill
This skill auto-detects when a user wants to set up a new Datadog client infrastructure and triggers the appropriate generation workflow.
Detection Patterns
This skill activates when the user's intent matches:
- "Set up Datadog for [client]"
- "Create new client infrastructure"
- "New client onboarding"
- "Generate Datadog terraform for [client]"
- "POC setup for [client]"
- "Production setup for [client]"
- "Onboard [client] to Datadog"
- "New organization setup"
Behavior
When detected, this skill:
-
Extracts client name from user input
- Converts to kebab-case (e.g., "Acme Corp" -> "acme-corp")
- Validates naming conventions
-
Checks for existing setup
- Looks for
~/datadog_terraform/<client-name>/ - If exists, asks if user wants to overwrite or continue
- Looks for
-
Initiates generation workflow
- Invokes
/org-generator:generate <client-name> - Generates questionnaire at
~/datadog_terraform/<client-name>/questionnaire.md
- Invokes
-
Provides next steps
- Instructs user to fill out questionnaire
- Explains how to continue with
/org-generator:apply
Example Interactions
New Client Setup
User: "I need to set up Datadog monitoring for Acme Corporation"
Skill Response:
Detected new client setup request for "Acme Corporation"
Client name: acme-corp
Target directory: ~/datadog_terraform/acme-corp/
Generating questionnaire...
Questionnaire created at:
~/datadog_terraform/acme-corp/questionnaire.md
Next steps:
1. Fill out the questionnaire with client requirements
2. Run: /org-generator:apply acme-corp
Existing Client Detected
User: "Set up Datadog for acme-corp"
Skill Response (if exists):
Found existing setup for acme-corp at:
~/datadog_terraform/acme-corp/
Options:
1. Continue with existing setup (run /org-generator:apply acme-corp)
2. Regenerate questionnaire (overwrites existing)
3. View current questionnaire
What would you like to do?
Naming Convention Validation
Client names must:
- Be lowercase
- Use hyphens for spaces (kebab-case)
- Not contain special characters
- Be between 2-50 characters
Invalid names are automatically corrected:
- "Acme Corp" -> "acme-corp"
- "ACME_CORP" -> "acme-corp"
- "Acme Corp, Inc." -> "acme-corp-inc"
Environment Variable Naming
The skill also reminds users about the environment variable convention:
For client "acme-corp", you'll need these environment variables:
- TF_VAR_acme_corp_api_key
- TF_VAR_acme_corp_app_key
Note: Hyphens become underscores in variable names.
Questionnaire Skill
This skill helps users interactively fill out the client questionnaire with intelligent suggestions and validation.
Detection Patterns
This skill activates when:
- User is viewing or editing a questionnaire.md file
- User asks for help filling out the questionnaire
- User mentions specific infrastructure they want to monitor
- User is unsure about questionnaire options
Behavior
Interactive Mode
When helping fill out a questionnaire, this skill:
-
Reads current questionnaire state
- Identifies which sections are complete
- Highlights required sections
-
Asks clarifying questions
- Cloud providers in use
- Infrastructure components
- Monitoring requirements
- Team structure
-
Provides recommendations
- Suggests common configurations
- Explains trade-offs
- Recommends best practices
-
Validates responses
- Ensures required fields are filled
- Checks for conflicting selections
- Warns about missing dependencies
Section-by-Section Assistance
Organization Setup
Questions asked:
- Is this a new Datadog organization or existing?
- Will this be a child organization under a parent?
- What's the organization name?
Recommendations:
- Child organizations recommended for client isolation
- Parent org credentials needed for child org creation
Cloud Providers
Questions asked:
- Which cloud providers are in use?
- Are workloads spread across multiple providers?
- Is there a primary cloud provider?
Recommendations:
- Start with primary provider, add others later
- Multi-cloud requires additional integration setup
Infrastructure Components
Questions asked:
- What types of hosts? (Linux, Windows, Solaris)
- Container orchestration? (Kubernetes, ECS, Docker Swarm)
- Serverless functions? (Lambda, Cloud Functions)
- Databases? (RDS, Cloud SQL, Azure SQL)
Recommendations:
- Enable host monitoring first
- Kubernetes requires cluster agent deployment
- Database monitoring may need additional permissions
Services & Applications
Questions asked:
- Web servers? (nginx, Apache, IIS)
- Application servers? (Node, Java, Python)
- Message queues? (RabbitMQ, Kafka, SQS)
- Caching? (Redis, Memcached)
Recommendations:
- APM provides deeper service visibility
- Integration setup varies by service
Monitoring Requirements
Questions asked:
- APM for application tracing?
- RUM for frontend monitoring?
- Synthetic tests for uptime?
- Private locations for internal endpoints?
Recommendations:
- APM recommended for microservices
- RUM requires frontend code instrumentation
- Private locations need dedicated workers
Team Structure
Questions asked:
- How many admin users?
- How many standard users?
- Any read-only stakeholders?
- Role-based access requirements?
Recommendations:
- Limit admin access
- Create role-specific permissions
- Use teams for alert routing
Tagging Strategy
Questions asked:
- Environment naming? (dev, staging, prod)
- Team identifiers?
- Application names?
- Custom tags needed?
Recommendations:
- Consistent tagging is critical
- Use
managed_by:terraformfor all resources - Environment tags enable filtering
Example Interaction
User: "Help me fill out the questionnaire for acme-corp"
Skill:
Let me help you complete the questionnaire for acme-corp.
Section 1: Organization Setup
-----------------------------
Is this a new Datadog organization or will you use an existing one?
Options:
- [ ] Create new child organization (recommended for client isolation)
- [ ] Use existing organization
- [ ] Other
What would you like to select?
User: "New child organization"
Skill:
Great choice for client isolation.
Section 2: Cloud Providers
--------------------------
Which cloud providers does acme-corp use? (Select all that apply)
- [ ] AWS
- [ ] GCP
- [ ] Azure
- [ ] On-premise / VDC
Which ones should I select?
Validation Rules
The skill validates:
| Rule | Validation |
|---|---|
| Required sections | Organization, Cloud Providers, Team Structure |
| Dependency checks | Private locations requires Synthetics |
| Conflict detection | Cannot have both "no containers" and "Kubernetes" |
| Completeness | All selected sections must have sub-selections |
Output
When complete, the skill:
- Writes the filled questionnaire to disk
- Provides a summary of selections
- Instructs user to run
/org-generator:apply
Commands 8
org-generator:agent
Generates Datadog agent configuration files based on questionnaire responses. Creates a complete datadog-agent/ directory inside the client's terraform manifest.
Usage
/org-generator:agent <client-name>
Arguments
client-name: The client identifier (kebab-case, e.g.,acme-corp)
Prerequisites
- Questionnaire must exist and be filled out
- Terraform structure should exist (typically run after /org-generator:scaffold)
Generated Structure
~/datadog_terraform/<client-name>/datadog-agent/
├── datadog.yaml # Main agent configuration
├── conf.d/
│ ├── custom_logs.yaml # Log collection (always)
│ ├── http_check.yaml # HTTP endpoint checks (if web servers)
│ ├── tcp_check.yaml # TCP port checks (if databases/queues)
│ ├── process.yaml # Process monitoring (if hosts)
│ └── <service>.yaml # Service-specific integrations
└── docs/
├── README.md # Overview and getting started
├── linux-host.md # Linux installation (always)
├── windows-host.md # Windows installation (if windows)
├── docker.md # Docker deployment (if containers)
├── kubernetes.md # K8s DaemonSet (if kubernetes)
├── ecs.md # ECS task definition (if ecs)
└── fargate.md # Fargate sidecar (if fargate)
Main Configuration (datadog.yaml)
Generated with:
- API key placeholder (use environment variable)
- Correct site based on data residency selection
- Environment tag from questionnaire
- Process collection (if hosts enabled)
- APM configuration (if APM enabled)
- Network monitoring (if enabled)
- Custom tags from questionnaire
Integration Configurations
Always Generated
| File | Purpose |
|---|---|
custom_logs.yaml | Log collection paths |
process.yaml | Process monitoring |
Conditional Based on Services
| Selection | Generated File |
|---|---|
| nginx | nginx.yaml |
| MySQL (self-managed) | mysql.yaml |
| PostgreSQL | postgres.yaml |
| Redis | redis.yaml |
| RabbitMQ | rabbitmq.yaml |
| Kafka | kafka.yaml |
| Web servers | http_check.yaml |
| Databases/queues | tcp_check.yaml |
Deployment Documentation
Based on infrastructure selections:
| Selection | Documentation |
|---|---|
| Linux hosts | linux-host.md (always) |
| Windows hosts | windows-host.md |
| Kubernetes/EKS/GKE/AKS | kubernetes.md |
| Docker | docker.md |
| ECS | ecs.md |
| Fargate | fargate.md |
Each doc includes:
- Prerequisites
- Installation commands
- Configuration file locations
- Verification steps
- Troubleshooting tips
Data Residency Support
The site parameter in datadog.yaml is set based on Section 9.2 of the questionnaire:
| Selection | Site Value |
|---|---|
| US (default) | datadoghq.com |
| EU | datadoghq.eu |
| US Government (GovCloud) | ddog-gov.com |
| US3 | us3.datadoghq.com |
| US5 | us5.datadoghq.com |
Workflow
-
Parse questionnaire for:
- Infrastructure selections (hosts, containers)
- Services (web servers, databases, queues)
- Monitoring features (APM, process collection)
- Data residency
-
Generate
datadog.yamlwith appropriate settings -
Generate integration configs in
conf.d/for each selected service -
Generate deployment docs in
docs/for each platform -
Output summary of generated files
Example
/org-generator:agent acme-corp
If questionnaire indicates:
- Linux hosts
- Kubernetes
- nginx, MySQL, Redis
- APM enabled
- US data residency
This will generate:
datadog.yamlwith APM enabled, site=datadoghq.comconf.d/custom_logs.yamlconf.d/process.yamlconf.d/nginx.yamlconf.d/mysql.yamlconf.d/redis.yamlconf.d/http_check.yamlconf.d/tcp_check.yamldocs/README.mddocs/linux-host.mddocs/kubernetes.md
Integration with Apply Command
This command is automatically invoked as Step 7 of /org-generator:apply:
1. Parse Questionnaire
2. Create S3 Backend
3. Scaffold Terraform Structure
4. Prune Unused Modules
5. Validate Environment
6. Generate Monitors
7. Generate Agent Configurations <-- This command
Manual Usage
Can also be run independently after initial setup:
# Regenerate agent configs after questionnaire update
/org-generator:agent acme-corp
org-generator:apply
Processes a completed questionnaire and orchestrates the full infrastructure setup.
Usage
/org-generator:apply <client-name>
Arguments
client-name: The client identifier (kebab-case, e.g.,acme-corp)
Prerequisites
Before running this command:
- Questionnaire must exist at
~/datadog_terraform/<client-name>/questionnaire.md - Questionnaire must be filled out by the user
Workflow
This command orchestrates the following steps in sequence:
Step 1: Parse Questionnaire
- Read the questionnaire from
~/datadog_terraform/<client-name>/questionnaire.md - Parse the responses using
lib/questionnaire-parser.py - Generate a configuration object with:
- Organization setup preferences
- Selected cloud providers (AWS, GCP, Azure)
- Infrastructure components
- Monitoring requirements
- Team structure
- Tagging strategy
Step 2: Create S3 Backend (via /org-generator:backend)
- Add entry to
~/Portfolio/aws/variables.tffor<client-name>-backend - Run
terraform initin~/Portfolio/aws/ - Run
terraform planto verify changes - Run
terraform applyto create the bucket - Verify bucket creation with
aws s3 ls
Step 3: Scaffold Terraform Structure (via /org-generator:scaffold)
- Generate FULL terraform directory structure at
~/datadog_terraform/<client-name>/ - Create all module directories (both required and optional)
- Generate root terraform files (backend.tf, providers.tf, variables.tf, etc.)
Step 4: Prune Unused Modules (via /org-generator:prune)
Based on questionnaire responses, remove:
- Unused cloud provider modules (aws/, gcp/, azure/)
- Unused infrastructure modules (kube/, synthetics/, private_locations/)
- Corresponding variables and outputs
- Monitor definitions for unselected infrastructure
Step 5: Validate Environment (via /org-generator:validate)
- Check for required environment variables:
TF_VAR_<client_name>_api_keyTF_VAR_<client_name>_app_key
- Re-source shell profile if needed
- Run
terraform initin client directory - Run
terraform validate
Step 6: Generate Monitors (via /org-generator:monitors)
Based on questionnaire responses, generate monitors for:
- Selected cloud providers
- Selected infrastructure components
- Selected services
Step 7: Generate Agent Configurations (via /org-generator:agent)
Based on questionnaire responses, generate Datadog agent configuration files:
-
Create
datadog-agent/directory inside client terraform manifest -
Generate
datadog.yamlwith:- Correct site based on data residency
- APM configuration (if enabled)
- Process collection (if hosts enabled)
- Network monitoring (if containers enabled)
- Client-specific tags
-
Generate integration configs in
conf.d/:custom_logs.yaml- Log collection pathsprocess.yaml- Process monitoring- Service-specific integrations (nginx, mysql, postgres, redis, kafka, rabbitmq)
http_check.yaml- HTTP endpoint checkstcp_check.yaml- TCP port checks
-
Generate deployment documentation in
docs/:README.md- Overview and quick startlinux-host.md- Linux installation (always)windows-host.md- Windows installation (if selected)kubernetes.md- K8s DaemonSet (if containers)docker.md- Docker deployment (if Docker)ecs.md- ECS task definition (if ECS)fargate.md- Fargate sidecar (if Fargate)
Dependency Chain
S3 Backend Created
└── terraform init succeeds
└── bucket exists: {client}-backend
Environment Variables Set
└── TF_VAR_{client}_api_key present
└── TF_VAR_{client}_app_key present
Terraform Init
└── backend configured
└── providers initialized
Terraform Validate
└── all variables present
└── syntax valid
Ready for Plan/Apply
Error Handling
If any step fails:
- Display clear error message with step that failed
- Provide remediation instructions
- Allow user to re-run from failed step
Example
/org-generator:apply acme-corp
This will:
- Parse
~/datadog_terraform/acme-corp/questionnaire.md - Create S3 backend
acme-corp-backend - Scaffold terraform structure
- Prune unused modules based on selections
- Validate environment and terraform config
- Generate appropriate monitors
- Generate agent configurations in
datadog-agent/
org-generator:backend
Creates an S3 backend bucket for storing Terraform state following the {client-name}-backend naming convention.
Usage
/org-generator:backend <client-name>
Arguments
client-name: The client identifier (kebab-case, e.g.,acme-corp)
Workflow
Step 1: Check Existing Backends
- Read
~/Portfolio/aws/variables.tf - Check if
<client-name>-backendalready exists in thes3variable map - If exists, skip creation and validate bucket exists
Step 2: Add Backend Entry
Add entry to ~/Portfolio/aws/variables.tf:
variable "s3" {
description = "s3 buckets"
type = map(any)
default = {
# ... existing entries ...
<client-name>-backend = {
env = "projects"
encryption = "AES256"
}
}
}
Step 3: Apply Terraform
cd ~/Portfolio/aws/
terraform init
terraform plan -target=module.s3["<client-name>-backend"]
terraform apply -target=module.s3["<client-name>-backend"] -auto-approve
Step 4: Validate Creation
aws s3 ls s3://<client-name>-backend --region us-east-1
S3 Backend Configuration
The backend bucket is created with:
- Encryption: AES256 server-side encryption
- Versioning: Enabled (via module configuration)
- Region: us-east-1
- Environment tag: projects
Module Reference
Uses the S3 module at ~/Portfolio/aws/modules/s3/:
module "s3" {
source = "./modules/s3/"
for_each = var.s3
name = each.key
env = each.value.env
encryption = each.value.encryption
}
Error Handling
| Error | Resolution |
|---|---|
| Bucket already exists | Skip creation, verify access |
| AWS credentials missing | Run aws configure or set environment variables |
| Terraform init fails | Check AWS provider configuration |
| Apply fails | Check IAM permissions for S3 bucket creation |
Example
/org-generator:backend acme-corp
This will:
- Add
acme-corp-backendto~/Portfolio/aws/variables.tf - Run terraform apply to create the bucket
- Verify bucket exists and is accessible
org-generator:generate
Generates complete Datadog Terraform infrastructure for a new client engagement.
Usage
/org-generator:generate <client-name>
Arguments
client-name: The client identifier (kebab-case, e.g.,acme-corp)
Workflow
Phase 1: Questionnaire Generation
-
Generate a professional markdown questionnaire at
~/datadog_terraform/<client-name>/questionnaire.md -
The questionnaire covers:
- Organization setup (child org vs existing)
- Cloud providers (AWS, GCP, Azure)
- Infrastructure components (hosts, containers, serverless)
- Services to monitor
- Tagging strategy
- Team structure and roles
-
STOP and inform user: "Questionnaire generated at
~/datadog_terraform/<client-name>/questionnaire.md. Please fill it out and run/org-generator:apply <client-name>when ready."
Phase 2: S3 Backend Creation (via /org-generator:backend)
After questionnaire is complete, create the S3 backend:
- Add entry to
~/Portfolio/aws/variables.tffor<client-name>-backend - Run
terraform init,plan, andapplyin~/Portfolio/aws/ - Verify bucket creation
Phase 3: Terraform Scaffold (via /org-generator:scaffold)
Generate the client's Datadog Terraform structure:
~/datadog_terraform/<client-name>/
├── backend.tf
├── providers.tf
├── variables.tf
├── outputs.tf
├── modules.tf
├── README.md
├── modules/
│ ├── api_keys/
│ ├── app_keys/
│ ├── roles/
│ ├── teams/
│ ├── users/
│ └── [cloud-specific modules based on questionnaire]
├── installs/
│ ├── agent.md
│ ├── install.sh
│ └── [platform-specific install scripts]
└── k8s/ (if applicable)
Phase 4: Monitor Generation (via /org-generator:monitors)
Based on questionnaire answers, generate appropriate monitors:
- AWS: EC2, RDS, ALB, ECS, Lambda monitors
- GCP: Compute Engine, Cloud SQL, GKE monitors
- Azure: VMs, SQL Database, AKS monitors
- Kubernetes: Cluster health, pod status, node pressure
- Generic: CPU, memory, disk, network, latency
Output
- Questionnaire markdown file
- Complete Terraform directory structure
- Initialized and applied S3 backend
- Generated monitors based on infrastructure
Example
/org-generator:generate acme-corp
This will:
- Create
~/datadog_terraform/acme-corp/questionnaire.md - Wait for user to complete questionnaire
- Continue with
/org-generator:apply acme-corp
org-generator:monitors
Generates Datadog monitors based on questionnaire responses. Only creates monitors for selected infrastructure components.
Usage
/org-generator:monitors <client-name>
Arguments
client-name: The client identifier (kebab-case, e.g.,acme-corp)
Prerequisites
- Questionnaire must exist and be filled out
- Terraform structure must exist (run after /org-generator:scaffold)
Monitor Categories
Generic Monitors (Always Generated)
These monitors apply to all deployments:
| Monitor | Query | Threshold |
|---|---|---|
| High CPU | avg:system.cpu.user{*} by {host} | > 90% |
| High Memory | avg:system.mem.used{*} by {host} | > 90% |
| Disk Space Low | avg:system.disk.free{*} by {host,device} | < 10% |
| Network Errors | sum:system.net.errors{*} by {host} | > 100 |
AWS Monitors (if AWS selected)
| Monitor | Query | Threshold |
|---|---|---|
| EC2 Status Check Failed | aws.ec2.status_check_failed | > 0 |
| RDS High CPU | aws.rds.cpuutilization | > 85% |
| RDS Low Memory | aws.rds.freeable_memory | < 128MB |
| RDS Write Latency | aws.rds.write_latency | > 1s |
| ALB 5XX Errors | aws.applicationelb.httpcode_elb_5xx | > 20% |
| ALB High Latency | aws.applicationelb.target_response_time | > 5s |
| ECS High CPU | aws.ecs.cpuutilization | > 97% |
| Lambda Errors | aws.lambda.errors | > 5 |
GCP Monitors (if GCP selected)
| Monitor | Query | Threshold |
|---|---|---|
| Compute High CPU | gcp.compute.instance.cpu.utilization | > 90% |
| Cloud SQL High CPU | gcp.cloudsql.database.cpu.utilization | > 85% |
| Cloud SQL Connections | gcp.cloudsql.database.num_connections | > 1000 |
| GKE Node Not Ready | gcp.gke.node.status.condition | condition=NotReady |
Azure Monitors (if Azure selected)
| Monitor | Query | Threshold |
|---|---|---|
| VM High CPU | azure.vm.percentage_cpu | > 90% |
| SQL DTU High | azure.sql_database.dtu_consumption_percent | > 90% |
| App Service 5XX | azure.app_services.http5xx | > 20% |
| AKS Node Not Ready | azure.aks.node.status | NotReady |
Kubernetes Monitors (if containers selected)
| Monitor | Query | Threshold |
|---|---|---|
| Deployment Replicas Down | kubernetes_state.deployment.replicas_desired - replicas_ready | >= 2 |
| Pods Restarting | kubernetes.containers.restarts | > 5 in 5m |
| DaemonSet Pod Down | kubernetes_state.daemonset.desired - ready | >= 1 |
| StatefulSet Replica Down | kubernetes_state.statefulset.replicas_desired - ready | >= 2 |
| Node Unschedulable | kubernetes_state.node.status{schedulable} | < 80% |
| ImagePullBackOff | kubernetes_state.container.status_report.count.waiting{imagepullbackoff} | >= 1 |
| Pending Pods | kubernetes_state.pod.status_phase{pending} | >= 1 for 30m |
Monitor Template Pattern
Each monitor follows this pattern:
resource "datadog_monitor" "<monitor_name>" {
name = "[<CATEGORY>] <Monitor Title>"
type = "query alert"
query = "<metric query>"
monitor_thresholds {
critical = <threshold>
critical_recovery = <recovery_threshold>
}
message = <<-EOF
{{#is_alert}}
<Alert message with template variables>
${join(",", var.alert_teams["notify"]["default"])}
{{/is_alert}}
{{#is_recovery}}
RECOVERED: <Recovery message>
${var.downtime_webhook}
{{/is_recovery}}
EOF
notify_audit = true
notification_preset_name = "hide_handles"
require_full_window = false
include_tags = true
priority = 3
tags = var.<category>_tags
}
Tag Inheritance
Monitors inherit tags based on category:
variable "aws_tags" {
default = [
"platform:aws",
"managed_by:terraform"
]
}
variable "kube_tags" {
default = [
"platform:kubernetes",
"managed_by:terraform"
]
}
Alert Team Configuration
Generated in notifications.tf:
variable "alert_teams" {
type = map(any)
default = {
notify = {
default = ["@slack-alerts", "@pagerduty-oncall"]
teamsDirect = ["@teams-<client-name>-alerts"]
ticket_teams = ["@servicenow-<client-name>"]
}
}
}
Generation Process
- Parse questionnaire for selected components
- For each selected component, include corresponding monitor file
- Update
modules/<category>/monitors.tfwith appropriate monitors - Update variables with tag definitions
- Run
terraform fmtto format
Customization
After generation, customize monitors by:
- Adjusting thresholds in variables.tf
- Modifying alert recipients in notifications.tf
- Adding/removing specific monitors in module files
Example
/org-generator:monitors acme-corp
If questionnaire indicates: AWS + RDS + Kubernetes
This will generate:
- Generic monitors (CPU, memory, disk, network)
- AWS monitors (EC2, RDS, ALB)
- Kubernetes monitors (deployments, pods, nodes)
And skip:
- GCP monitors
- Azure monitors
- Lambda monitors (if serverless not selected)
org-generator:prune
Removes unused modules, resources, and configurations based on questionnaire responses. This keeps the codebase lean and avoids confusion.
Usage
/org-generator:prune <client-name>
Arguments
client-name: The client identifier (kebab-case, e.g.,acme-corp)
Prerequisites
- Questionnaire must be parsed (run after /org-generator:scaffold)
- Full terraform structure must exist at
~/datadog_terraform/<client-name>/
Pruning Logic
Based on questionnaire answers, the following are conditionally removed:
| Questionnaire Answer | Modules/Files Removed |
|---|---|
| Cloud: NOT AWS | modules/aws/, AWS integrations in modules.tf |
| Cloud: NOT GCP | modules/gcp/, GCP integrations in modules.tf |
| Cloud: NOT Azure | modules/azure/, Azure integrations in modules.tf |
| Containers: NO | modules/kube/, k8s/ directory |
| Serverless: NO | Lambda/Cloud Functions monitors |
| APM: NO | modules/apm/, APM variables |
| RUM: NO | modules/rum/, RUM variables |
| Synthetics: NO | modules/synthetics/, synthetic variables |
| Private Locations: NO | modules/private_locations/, k8s/ worker configs |
| Message Queues: NO | RabbitMQ/Kafka monitors |
| Databases: NOT RDS | RDS-specific monitors in aws module |
| Databases: NOT Cloud SQL | Cloud SQL monitors in gcp module |
| Databases: NOT Azure SQL | Azure SQL monitors in azure module |
Pruning Steps
Step 1: Parse Configuration
Read the parsed questionnaire configuration from previous step or re-parse questionnaire.md.
Step 2: Identify Unused Modules
Build a list of modules to remove based on:
- Cloud providers not selected
- Infrastructure components not selected
- Services not needed
Step 3: Remove Module Directories
# Example: Remove GCP module if not selected
rm -rf ~/datadog_terraform/<client-name>/modules/gcp/
Step 4: Update modules.tf
Remove module instantiation blocks for pruned modules:
# REMOVE this block if GCP not selected
module "gcp_monitors" {
source = "./modules/gcp/"
# ...
}
Step 5: Update variables.tf
Remove variables only used by pruned modules:
# REMOVE if GCP not selected
variable "gcp_project_id" {
description = "GCP Project ID"
type = string
}
Step 6: Update outputs.tf
Remove outputs from pruned modules.
Step 7: Clean Up
cd ~/datadog_terraform/<client-name>/
terraform fmt -recursive
Example Scenarios
AWS-Only Client
Selected: AWS, RDS, ECS, Lambda Removed: modules/gcp/, modules/azure/, modules/kube/
GCP with Kubernetes Client
Selected: GCP, GKE, Cloud SQL Removed: modules/aws/, modules/azure/, modules/apm/, modules/rum/
Multi-Cloud Client
Selected: AWS, Azure, Kubernetes Kept: modules/aws/, modules/azure/, modules/kube/ Removed: modules/gcp/, modules/apm/, modules/rum/, modules/synthetics/
Dry Run Mode
To preview what would be pruned without making changes:
# Parse questionnaire and list modules to prune
python ~/agents/plugins/org-generator/lib/questionnaire-parser.py \
~/datadog_terraform/<client-name>/questionnaire.md \
--dry-run
Result
A minimal, clean codebase containing only:
- Core modules (api_keys, app_keys, roles, teams, users)
- Selected cloud provider modules
- Selected infrastructure modules
- Monitors relevant to the client's actual stack
Example
/org-generator:prune acme-corp
This will:
- Parse
questionnaire.mdfor selections - Remove unused module directories
- Update modules.tf, variables.tf, outputs.tf
- Run
terraform fmtto clean up
org-generator:scaffold
Generates a complete Terraform directory structure for a new client engagement. Creates ALL modules initially; pruning happens in a separate step.
Usage
/org-generator:scaffold <client-name>
Arguments
client-name: The client identifier (kebab-case, e.g.,acme-corp)
Directory Structure Generated
~/datadog_terraform/<client-name>/
├── backend.tf # S3 backend configuration
├── providers.tf # Provider versions and configuration
├── variables.tf # Client-specific variables
├── outputs.tf # Key outputs
├── modules.tf # Module instantiation
├── tags.tf # Tag definitions
├── notifications.tf # Alert team definitions
├── README.md # Auto-generated documentation
├── questionnaire.md # Original questionnaire (preserved)
├── modules/
│ ├── api_keys/ # API key generation
│ │ ├── main.tf
│ │ └── variables.tf
│ ├── app_keys/ # App key generation
│ │ ├── main.tf
│ │ └── variables.tf
│ ├── roles/ # Standard role definitions
│ │ ├── main.tf
│ │ └── variables.tf
│ ├── teams/ # Team structure
│ │ ├── main.tf
│ │ └── variables.tf
│ ├── users/ # User management
│ │ ├── main.tf
│ │ └── variables.tf
│ ├── aws/ # AWS monitors (conditional)
│ │ ├── monitors.tf
│ │ ├── rds.tf
│ │ ├── alb.tf
│ │ ├── ecs.tf
│ │ ├── lambda.tf
│ │ └── variables.tf
│ ├── gcp/ # GCP monitors (conditional)
│ │ ├── monitors.tf
│ │ ├── cloudsql.tf
│ │ ├── gke.tf
│ │ └── variables.tf
│ ├── azure/ # Azure monitors (conditional)
│ │ ├── monitors.tf
│ │ ├── sql.tf
│ │ ├── aks.tf
│ │ └── variables.tf
│ ├── kube/ # Kubernetes monitors (conditional)
│ │ ├── monitors.tf
│ │ └── variables.tf
│ ├── generic/ # Generic host monitors
│ │ ├── cpu.tf
│ │ ├── memory.tf
│ │ ├── disk.tf
│ │ ├── network.tf
│ │ └── variables.tf
│ ├── synthetics/ # Synthetic tests (conditional)
│ │ ├── http.tf
│ │ ├── ssl.tf
│ │ └── variables.tf
│ ├── private_locations/ # Private locations (conditional)
│ │ ├── main.tf
│ │ └── variables.tf
│ ├── apm/ # APM configuration (conditional)
│ │ ├── main.tf
│ │ └── variables.tf
│ └── rum/ # RUM configuration (conditional)
│ ├── main.tf
│ └── variables.tf
├── installs/ # Agent installation scripts
│ ├── agent.md
│ └── install.sh
├── add_tags/ # Monitor tagging scripts
│ ├── __init__.py
│ ├── keys.py # API/App key retrieval from env vars
│ ├── headers.py # HTTP headers for Datadog API
│ ├── monitors.py # Monitor listing and tagging functions
│ └── caller.py # CLI entry point for tagging
├── scripts/dd_lib/ # Comprehensive Datadog API library
│ ├── __init__.py # Package marker and documentation
│ ├── keys.py # API/App key retrieval (TF_VAR pattern)
│ ├── headers.py # HTTP headers with pagination
│ ├── api.py # Base API operations
│ ├── monitors.py # Monitor CRUD, tagging, downtimes
│ ├── downtimes.py # Downtime management, duplicate checking
│ ├── tags.py # Tag operations on monitors
│ ├── hosts.py # Host listing, tagging, metrics
│ ├── events.py # Event retrieval, monitor correlation
│ ├── logs.py # Log search and retrieval
│ ├── users.py # User listing and management
│ ├── roles.py # Role and permission operations
│ ├── metrics.py # Host metrics retrieval
│ ├── synthetics.py # Synthetic test operations
│ ├── webhooks.py # Webhook creation
│ ├── orgs.py # Organization operations
│ ├── rum.py # RUM operations
│ └── services.py # Service catalog operations
└── k8s/ # Kubernetes configs (conditional)
└── private-location-worker.yaml
Core Files Generated
backend.tf
terraform {
required_providers {
datadog = {
source = "Datadog/datadog"
version = ">= 3.86.0"
}
aws = {
source = "hashicorp/aws"
}
}
backend "s3" {
bucket = "<client-name>-backend"
key = "<client-name>-backend/backend"
region = "us-east-1"
encrypt = true
}
}
provider "datadog" {
api_key = var.<client_name>_api_key
app_key = var.<client_name>_app_key
api_url = var.api_url
}
provider "aws" {
region = var.region
}
variables.tf
variable "region" {
description = "AWS region"
type = string
default = "us-east-1"
}
variable "<client_name>_api_key" {
description = "Datadog API key. Set via TF_VAR_<client_name>_api_key"
type = string
sensitive = true
}
variable "<client_name>_app_key" {
description = "Datadog App key. Set via TF_VAR_<client_name>_app_key"
type = string
sensitive = true
}
variable "api_url" {
description = "Datadog API URL"
type = string
default = "https://api.datadoghq.com"
}
Template Sources
Templates are sourced from:
~/agents/plugins/org-generator/templates/terraform/
Add Tags Script
The add_tags/ directory contains Python scripts for adding id:<monitor_id> tags to all Datadog monitors. This makes monitors easier to search and filter in the Datadog UI.
Usage
cd ~/datadog_terraform/<client-name>
make tags # Add id tags to monitors
make check-tags # Show monitors without id tags (dry run)
make list-monitors # List all monitors
make tfaa # terraform apply -auto-approve && make tags
Environment Variables
The scripts use the same TF_VAR pattern as Terraform:
export TF_VAR_<client_name>_api_key="your-api-key"
export TF_VAR_<client_name>_app_key="your-app-key"
Falls back to DD_API_KEY and DD_APP_KEY if TF_VAR versions are not set.
Example
/org-generator:scaffold acme-corp
This will create the full directory structure at ~/datadog_terraform/acme-corp/ with all modules populated from templates.
DD_Lib Script Library
The scripts/dd_lib/ directory contains a comprehensive Python library for Datadog API operations. This provides utilities beyond the basic tagging functionality in add_tags/.
Available Modules
| Module | Description |
|---|---|
keys.py | API/App key retrieval using TF_VAR pattern with DD_*_KEY fallback |
headers.py | HTTP headers for GET, POST, PUT, DELETE with pagination support |
api.py | Base API operations and request helpers |
monitors.py | Monitor CRUD, ID tagging, triggered monitor listing, downtimes |
downtimes.py | Downtime management, duplicate checking, bulk operations |
tags.py | Monitor tag operations and listing |
hosts.py | Host listing, metrics, tag management |
events.py | Event retrieval, monitor correlation |
logs.py | Log search and aggregation |
users.py | User listing and management |
roles.py | Role and permission operations |
metrics.py | Host and active metrics retrieval |
synthetics.py | Synthetic test operations, private locations |
webhooks.py | Webhook CRUD operations |
orgs.py | Organization operations, usage summary |
rum.py | RUM application management and event queries |
services.py | Service catalog operations, APM/RUM service filtering |
Usage Example
import sys
sys.path.insert(0, "scripts/dd_lib")
import monitors
import downtimes
# List all triggered monitors
triggered = monitors.get_triggered_monitors(prt=True, limit=10)
# Add downtime for a recovered monitor
downtimes.add_downtime_one_scope("env:production", monitor_id=12345)
# Append id tags to all monitors
monitors.append_id_tag_to_tags()
Environment Variables
Uses the same TF_VAR pattern as Terraform:
export TF_VAR_<client_name>_api_key="your-api-key"
export TF_VAR_<client_name>_app_key="your-app-key"
Falls back to DD_API_KEY and DD_APP_KEY if TF_VAR versions are not set.
API Site Configuration
The library templates include a {{SITE}} placeholder that is replaced with the appropriate API domain based on the client's data residency:
| Data Residency | Site Value |
|---|---|
| US (default) | api.datadoghq.com |
| EU | api.datadoghq.eu |
| US3 | api.us3.datadoghq.com |
| US5 | api.us5.datadoghq.com |
| US Government | api.ddog-gov.com |
org-generator:validate
Validates that all required environment variables are set and terraform configuration is valid.
Usage
/org-generator:validate <client-name>
Arguments
client-name: The client identifier (kebab-case, e.g.,acme-corp)
Prerequisites
- S3 backend must exist (
<client-name>-backend) - Terraform structure must exist at
~/datadog_terraform/<client-name>/
Validation Steps
Step 1: Validate S3 Backend
aws s3 ls s3://<client-name>-backend --region us-east-1
Expected: Bucket exists and is accessible Error: Bucket does not exist or access denied
Step 2: Validate Environment Variables
Check for required Datadog credentials using the TF_VAR pattern:
# Client name with hyphens converted to underscores
# e.g., acme-corp -> acme_corp
test -n "$TF_VAR_<client_name>_api_key" && echo "API key: SET" || echo "API key: MISSING"
test -n "$TF_VAR_<client_name>_app_key" && echo "App key: SET" || echo "App key: MISSING"
Expected: Both variables are set Error: One or both variables are missing
Step 3: Re-source Environment (if needed)
If variables are missing, attempt to re-source shell profile:
# Detect shell and source appropriate file
if [ -n "$ZSH_VERSION" ]; then
source ~/.zshrc
elif [ -n "$BASH_VERSION" ]; then
source ~/.bashrc
fi
Then re-check variables.
Step 4: Terraform Init
cd ~/datadog_terraform/<client-name>/
terraform init
Expected: Initialization complete with no errors Errors:
- Backend configuration invalid
- Provider download fails
- Module not found
Step 5: Terraform Validate
cd ~/datadog_terraform/<client-name>/
terraform validate
Expected: "Success! The configuration is valid." Errors:
- Missing required variable
- Syntax error
- Invalid resource configuration
Environment Variable Convention
Client credentials follow this pattern:
# Pattern: TF_VAR_{client_name}_api_key
# Hyphens in client name become underscores
# Example for client "acme-corp":
export TF_VAR_acme_corp_api_key="your-api-key-here"
export TF_VAR_acme_corp_app_key="your-app-key-here"
Creating Datadog Keys
If keys are missing, instruct user:
-
API Key:
- Go to Datadog > Organization Settings > API Keys
- Create new key named
<client-name>-terraform - Copy the key value
-
App Key:
- Go to Datadog > Organization Settings > Application Keys
- Create new key named
<client-name>-terraform - Copy the key value
-
Add to environment:
# Add to ~/.bashrc or ~/.zshrc: export TF_VAR_<client_name>_api_key="your-api-key" export TF_VAR_<client_name>_app_key="your-app-key" # Then source the file: source ~/.bashrc # or source ~/.zshrc
Validation Report
Output a summary:
Validation Report for <client-name>
====================================
S3 Backend:
Bucket: <client-name>-backend
Status: EXISTS / MISSING
Environment Variables:
TF_VAR_<client_name>_api_key: SET / MISSING
TF_VAR_<client_name>_app_key: SET / MISSING
Terraform Init:
Status: SUCCESS / FAILED
Error: <error message if failed>
Terraform Validate:
Status: SUCCESS / FAILED
Error: <error message if failed>
Overall: READY / NOT READY
Error Resolution
| Issue | Resolution |
|---|---|
| S3 bucket missing | Run /org-generator:backend <client-name> |
| API key missing | Create in Datadog UI, add to environment |
| App key missing | Create in Datadog UI, add to environment |
| Terraform init fails | Check backend config, AWS credentials |
| Terraform validate fails | Check variable definitions, module syntax |
Example
/org-generator:validate acme-corp
This will:
- Check S3 backend
acme-corp-backendexists - Verify
TF_VAR_acme_corp_api_keyis set - Verify
TF_VAR_acme_corp_app_keyis set - Run
terraform init - Run
terraform validate - Output validation report
Sponsors
Details
- Marketplace
- Nokodoko/agents
- Source
- plugins/org-generator
- Author
- n0ko
- Category
- Infrastructure