Terraform Cloud workspace management bootstrap and layer reorganization

Author: rohitjha941

README.md

@@ -6,10 +6,11 @@ This project demonstrates a FastAPI application deployed on AWS EKS with compreh
 
 The infrastructure is organized in layers:
 
-- `0_ipam` - IP address management
-- `1_base` - Base infrastructure (EKS, VPC, OpenSearch)
-- `2_argo` - ArgoCD deployment
-- `3_kubernetes_addons` - Additional Kubernetes components
+- `0_tfc_bootstrap` - Terraform Cloud workspace management
+- `1_ipam` - IP address management (global, run once)
+- `2_base` - Base infrastructure (EKS, VPC, OpenSearch)
+- `3_argo` - ArgoCD deployment
+- `4_kubernetes_addons` - Additional Kubernetes components
 
 
 
@@ -21,7 +22,8 @@ The project is organized into two main directories: `app` and `infra`.
     - `.venv/`: A virtual environment for managing Python dependencies locally.
 
 - **`infra/`**: This directory holds the Infrastructure as Code (IaC) definitions using Terraform. The infrastructure is logically separated into layers, with each layer in its own directory.
-    - `0_ipam/`: Manages the VPC IP address space.
-    - `1_base/`: Defines the foundational cloud infrastructure, including the VPC and EKS cluster.
-    - `2_argo/`: Sets up ArgoCD for GitOps-based continuous deployment.
-    - `3_kubernetes_addons/`: Manages Kubernetes add-ons like external-secrets for secret management.
+    - `0_tfc_bootstrap/`: Creates and manages Terraform Cloud workspaces, variable sets, and team access controls.
+    - `1_ipam/`: Manages the VPC IP address space (global, run once).
+    - `2_base/`: Defines the foundational cloud infrastructure, including the VPC and EKS cluster.
+    - `3_argo/`: Sets up ArgoCD for GitOps-based continuous deployment.
+    - `4_kubernetes_addons/`: Manages Kubernetes add-ons like external-secrets for secret management.

infra/0_tfc_bootstrap/README.md

@@ -0,0 +1,69 @@
+# Terraform Cloud Bootstrap
+
+This module manages Terraform Cloud workspaces and configuration for the hello-fastapi-k8s project.
+
+## Overview
+
+This creates a complete Terraform Cloud setup including:
+- Organization project
+- Workspaces for each layer and environment
+- Variable sets for AWS credentials and common configurations
+- Run triggers for dependent workspaces
+- Team access controls
+
+## Prerequisites
+
+1. Terraform Cloud account and organization
+2. GitHub OAuth connection configured in TFC
+3. AWS credentials for TFC to use
+
+## Usage
+
+1. First, create the bootstrap workspace manually or via CLI:
+```bash
+cd infra/0_tfc_bootstrap
+terraform login
+terraform workspace new tfc-bootstrap
+```
+
+2. Configure variables:
+```bash
+# Required variables
+export TF_VAR_tfc_organization="your-org"
+export TF_VAR_aws_access_key_id="AKIA..."
+export TF_VAR_aws_secret_access_key="secret..."
+export TF_VAR_github_repository="yourusername/hello-flask-k8s"
+```
+
+3. Run terraform:
+```bash
+terraform init
+terraform plan
+terraform apply
+```
+
+## Workspace Structure
+
+The following workspaces will be created:
+
+- `ipam-global` - IP Address Management (shared across all environments)
+- `base-{dev,staging,prod}` - Base infrastructure (VPC, EKS)
+- `argo-{dev,staging,prod}` - ArgoCD deployment
+- `addons-{dev,staging,prod}` - Kubernetes addons
+
+## Features
+
+- **Auto-apply**: Enabled for dev workspaces only
+- **Run triggers**: Dependent workspaces trigger automatically
+- **Branch protection**: 
+  - Dev/staging workspaces use respective branches
+  - Prod workspaces use main branch
+- **Access control**: Developers have write access to dev/staging, plan-only for prod
+
+## Customization
+
+Edit `main.tf` to:
+- Add more layers or environments
+- Change auto-apply settings
+- Modify team permissions
+- Add notification channels (Slack, email, webhooks)

infra/0_tfc_bootstrap/data.tf

@@ -0,0 +1,11 @@
+# Data source for current organization
+data "tfe_organization" "main" {
+  name = var.tfc_organization
+}
+
+# GitHub OAuth Client
+data "tfe_oauth_client" "github" {
+  organization     = data.tfe_organization.main.name
+  service_provider = "github"
+  name             = var.github_oauth_client_name
+}

infra/0_tfc_bootstrap/locals.tf

@@ -0,0 +1,53 @@
+locals {
+  environments = ["dev", "staging", "prod"]
+  
+  # Layer definitions with dependencies
+  layers = {
+    base = {
+      path       = "infra/2_base"
+      depends_on = ["ipam-global"]
+      auto_apply = {
+        dev     = true
+        staging = false
+        prod    = false
+      }
+    }
+    argo = {
+      path       = "infra/3_argo"
+      depends_on = ["base"]
+      auto_apply = {
+        dev     = true
+        staging = false
+        prod    = false
+      }
+    }
+    addons = {
+      path       = "infra/4_kubernetes_addons"
+      depends_on = ["base", "argo"]
+      auto_apply = {
+        dev     = true
+        staging = false
+        prod    = false
+      }
+    }
+  }
+  
+  # Flatten workspace configurations
+  workspaces = flatten([
+    for layer_name, layer in local.layers : [
+      for env in local.environments : {
+        name        = "${layer_name}-${env}"
+        layer       = layer_name
+        environment = env
+        path        = layer.path
+        depends_on  = layer.depends_on
+        auto_apply  = layer.auto_apply[env]
+      }
+    ]
+  ])
+  
+  # Create workspace map for easy lookup
+  workspace_map = {
+    for ws in local.workspaces : ws.name => ws
+  }
+}

infra/0_tfc_bootstrap/main.tf

@@ -0,0 +1,18 @@
+# Main configuration file for Terraform Cloud bootstrap
+# This module creates and manages all Terraform Cloud workspaces,
+# variable sets, teams, and access controls for the hello-fastapi-k8s project.
+
+# All configuration is split into logical files:
+# - versions.tf: Terraform and provider version constraints
+# - providers.tf: Provider configurations
+# - data.tf: Data sources
+# - locals.tf: Local values and computed configurations
+# - project.tf: TFC project configuration
+# - variable_sets.tf: Variable sets and variables
+# - workspaces.tf: Workspace definitions
+# - workspace_variables.tf: Workspace-specific variables
+# - run_triggers.tf: Dependencies between workspaces
+# - teams.tf: Team and access control configuration
+# - notifications.tf: Notification settings (placeholder)
+# - variables.tf: Input variables
+# - outputs.tf: Output values

infra/0_tfc_bootstrap/notifications.tf

@@ -0,0 +1,2 @@
+# Notifications configuration placeholder
+# Future enhancements can add Slack, email, or webhook notifications here

infra/0_tfc_bootstrap/outputs.tf

@@ -0,0 +1,29 @@
+output "workspace_ids" {
+  description = "Map of workspace names to their IDs"
+  value = merge(
+    { "ipam-global" = tfe_workspace.ipam.id },
+    { for k, v in tfe_workspace.main : k => v.id }
+  )
+}
+
+output "workspace_urls" {
+  description = "URLs to access each workspace"
+  value = merge(
+    { "ipam-global" = "https://app.terraform.io/app/${var.tfc_organization}/workspaces/${tfe_workspace.ipam.name}" },
+    { for k, v in tfe_workspace.main : k => "https://app.terraform.io/app/${var.tfc_organization}/workspaces/${v.name}" }
+  )
+}
+
+output "project_id" {
+  description = "The ID of the Terraform Cloud project"
+  value       = tfe_project.hello_fastapi.id
+}
+
+output "variable_set_ids" {
+  description = "IDs of the variable sets"
+  value = {
+    aws_credentials = tfe_variable_set.aws_credentials.id
+    common_tags     = tfe_variable_set.common_tags.id
+    environments    = { for k, v in tfe_variable_set.env : k => v.id }
+  }
+}

infra/0_tfc_bootstrap/project.tf

@@ -0,0 +1,6 @@
+# Create project for better organization
+resource "tfe_project" "hello_fastapi" {
+  organization = data.tfe_organization.main.name
+  name         = "hello-fastapi-k8s"
+  description  = "FastAPI application on Kubernetes - DevOps showcase project"
+}

infra/0_tfc_bootstrap/providers.tf

@@ -0,0 +1,3 @@
+provider "tfe" {
+  # TFE_TOKEN environment variable or terraform login
+}

infra/0_tfc_bootstrap/run_triggers.tf

@@ -0,0 +1,12 @@
+# Run triggers for dependencies
+resource "tfe_run_trigger" "dependencies" {
+  for_each = {
+    for ws in local.workspaces : ws.name => ws
+    if length(ws.depends_on) > 0
+  }
+  
+  workspace_id  = tfe_workspace.main[each.key].id
+  sourceable_id = each.value.depends_on[0] == "ipam-global" ? 
+    tfe_workspace.ipam.id : 
+    tfe_workspace.main["${each.value.depends_on[0]}-${each.value.environment}"].id
+}