Skip to content
This repository was archived by the owner on Oct 10, 2025. It is now read-only.

Commit 207c404

Browse files
josecelanojosecelano
committed
docs: [#26] create ADR-007 for two-level environment variable structure
- Add comprehensive ADR documenting security-focused environment variable separation - Level 1: Main environment files for complete deployment configuration - Level 2: Docker environment files with filtered container-only variables - Update ADR table in copilot instructions with new ADR-007 entry - Enhance twelve-factor Release stage documentation with environment structure details - Provide clear guidelines for infrastructure vs container variable management
1 parent 3471409 commit 207c404

File tree

2 files changed

+246
-10
lines changed

2 files changed

+246
-10
lines changed

‎.github/copilot-instructions.md‎

Lines changed: 13 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -186,14 +186,15 @@ torrust-tracker-demo/
186186

187187
Key design decisions are documented in `docs/adr/`. Contributors should review relevant ADRs when working on related features:
188188

189-
| ADR | Title | Description |
190-
|-----|-------|-------------|
191-
| [ADR-001](../docs/adr/001-makefile-location.md) | Makefile Location | Why the main Makefile is at repository root |
192-
| [ADR-002](../docs/adr/002-docker-for-all-services.md) | Docker for All Services | Why we use Docker for all services including UDP tracker |
193-
| [ADR-003](../docs/adr/003-use-mysql-over-mariadb.md) | Use MySQL Over MariaDB | Database backend selection rationale |
194-
| [ADR-004](../docs/adr/004-configuration-approach-files-vs-environment-variables.md) | Configuration Approach | Files vs environment variables for configuration |
195-
| [ADR-005](../docs/adr/005-sudo-cache-management-for-infrastructure-operations.md) | Sudo Cache Management | Managing sudo credentials during infrastructure operations |
196-
| [ADR-006](../docs/adr/006-ssl-certificate-generation-strategy.md) | SSL Certificate Generation Strategy | Approach for SSL certificate management |
189+
| ADR | Title | Description |
190+
| ----------------------------------------------------------------------------------- | ----------------------------------- | ---------------------------------------------------------- |
191+
| [ADR-001](../docs/adr/001-makefile-location.md) | Makefile Location | Why the main Makefile is at repository root |
192+
| [ADR-002](../docs/adr/002-docker-for-all-services.md) | Docker for All Services | Why we use Docker for all services including UDP tracker |
193+
| [ADR-003](../docs/adr/003-use-mysql-over-mariadb.md) | Use MySQL Over MariaDB | Database backend selection rationale |
194+
| [ADR-004](../docs/adr/004-configuration-approach-files-vs-environment-variables.md) | Configuration Approach | Files vs environment variables for configuration |
195+
| [ADR-005](../docs/adr/005-sudo-cache-management-for-infrastructure-operations.md) | Sudo Cache Management | Managing sudo credentials during infrastructure operations |
196+
| [ADR-006](../docs/adr/006-ssl-certificate-generation-strategy.md) | SSL Certificate Generation Strategy | Approach for SSL certificate management |
197+
| [ADR-007](../docs/adr/007-two-level-environment-variable-structure.md) | Two-Level Environment Variable Structure | Security-focused separation of infrastructure and container variables |
197198

198199
## 🛠️ Development Workflow
199200

@@ -297,9 +298,11 @@ The twelve-factor **Build, Release, Run** stages apply to the application deploy
297298

298299
- **Release Stage**: Combine built application with environment-specific configuration
299300

300-
- Apply environment variables and configuration files
301+
- Apply environment variables and configuration files using **two-level structure** (see [ADR-007](../docs/adr/007-two-level-environment-variable-structure.md))
302+
- **Level 1**: Main environment files (`infrastructure/config/environments/`) for all deployment processes
303+
- **Level 2**: Docker environment files (generated from templates) for container runtime only
301304
- Combine application artifacts with runtime configuration
302-
- Prepare deployment-ready releases
305+
- Prepare deployment-ready releases with security-focused variable filtering
303306

304307
- **Run Stage**: Execute the application in the runtime environment
305308
- Start application processes (tracker binary, background jobs)

‎docs/adr/007-two-level-environment-variable-structure.md‎

Lines changed: 233 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,233 @@
1+
# ADR-007: Two-Level Environment Variable Structure
2+
3+
## Status
4+
5+
Accepted
6+
7+
## Context
8+
9+
As part of implementing twelve-factor app methodology, we need a clear approach for
10+
managing environment variables throughout the deployment process. The system has
11+
evolved to use a two-level environment variable structure that serves different
12+
purposes and security requirements.
13+
14+
Currently, the project uses environment variables at two distinct levels:
15+
16+
1. **Main Environment Variables**: Used for the entire deployment process
17+
2. **Docker Compose Environment Variables**: Used only for running containers
18+
19+
This separation has emerged organically but lacks clear documentation, leading to
20+
potential confusion about where variables should be defined and how they flow
21+
through the system.
22+
23+
## Decision
24+
25+
We will formalize and document a **two-level environment variable structure**
26+
with clear separation of concerns:
27+
28+
### Level 1: Main Environment Variables
29+
30+
**Purpose**: Complete deployment configuration
31+
**Location**: `infrastructure/config/environments/`
32+
**Examples**: `local.env`, `production.env`
33+
**Scope**: All deployment processes
34+
35+
**Contents**:
36+
37+
- Infrastructure configuration (VM specs, network settings)
38+
- SSL certificate configuration (domains, Let's Encrypt email)
39+
- Database credentials and connection parameters
40+
- Application API tokens and secrets
41+
- Backup and monitoring configuration
42+
- Build and deployment automation settings
43+
44+
**Usage**:
45+
46+
- Sourced by deployment scripts (`provision-infrastructure.sh`, `deploy-app.sh`)
47+
- Used for template rendering (cloud-init, configuration files)
48+
- Contains variables for infrastructure operations (SSL generation, backups)
49+
- Includes variables that containers never need to see
50+
51+
### Level 2: Docker Compose Environment Variables
52+
53+
**Purpose**: Container runtime configuration
54+
**Template**: `infrastructure/config/templates/docker-compose.env.tpl`
55+
**Generated File**: `.env` (in application directory)
56+
**Scope**: Docker Compose and running containers only
57+
58+
**Contents** (filtered subset from Level 1):
59+
60+
- Database connection strings for application containers
61+
- Application API tokens needed by running services
62+
- Docker runtime configuration (USER_ID)
63+
- Service-specific configuration (Grafana admin credentials)
64+
- Container environment overrides
65+
66+
**Filtering Criteria**:
67+
68+
- **Include**: Variables directly used by containerized applications
69+
- **Exclude**: Infrastructure-only variables (SSL domains, backup settings)
70+
- **Exclude**: Build-time variables not needed at runtime
71+
- **Security**: Minimize attack surface by only exposing necessary variables
72+
73+
## Template Transformation Process
74+
75+
```text
76+
Level 1: Main Environment Variables
77+
├── infrastructure/config/environments/local.env.tpl
78+
├── infrastructure/config/environments/production.env.tpl
79+
└── (user creates) local.env or production.env
80+
│
81+
▼ (template processing)
82+
Level 2: Docker Environment Variables
83+
├── infrastructure/config/templates/docker-compose.env.tpl
84+
└── (generated) application/.env
85+
```
86+
87+
**Processing Flow**:
88+
89+
1. User creates environment file from template (e.g., `local.env`)
90+
2. Deployment script sources the main environment file
91+
3. Template processor generates `docker-compose.env` from template
92+
4. Docker Compose uses the generated `.env` file for container variables
93+
94+
## Rationale
95+
96+
### Security Benefits
97+
98+
- **Principle of Least Privilege**: Containers only receive variables they need
99+
- **Reduced Attack Surface**: Infrastructure secrets not exposed to application containers
100+
- **Separation of Concerns**: Infrastructure and application secrets handled differently
101+
102+
### Operational Benefits
103+
104+
- **Clear Responsibility**: Infrastructure variables vs. application variables
105+
- **Easier Debugging**: Know where to look for specific types of configuration
106+
- **Template Flexibility**: Can generate different container environments from same base config
107+
- **Deployment Isolation**: Infrastructure operations don't leak sensitive data to containers
108+
109+
### Examples
110+
111+
**Level 1 Only (Infrastructure Variables)**:
112+
113+
```bash
114+
# SSL configuration (not needed in containers)
115+
SSL_DOMAIN="tracker.example.com"
116+
SSL_EMAIL="[email protected]"
117+
ENABLE_SSL_AUTOMATION="true"
118+
119+
# Backup configuration (not needed in containers)
120+
ENABLE_DB_BACKUPS="true"
121+
BACKUP_RETENTION_DAYS="30"
122+
123+
# Infrastructure specifications (not needed in containers)
124+
VM_MEMORY="4096"
125+
VM_VCPUS="4"
126+
```
127+
128+
**Level 2 (Container Variables - Filtered from Level 1)**:
129+
130+
```bash
131+
# Database connection (needed by tracker container)
132+
MYSQL_ROOT_PASSWORD="secure_root_password"
133+
MYSQL_PASSWORD="secure_user_password"
134+
TORRUST_TRACKER_CONFIG_OVERRIDE_CORE__DATABASE__PATH="mysql://torrust:${MYSQL_PASSWORD}@mysql:3306/torrust_tracker"
135+
136+
# API tokens (needed by application)
137+
TRACKER_ADMIN_TOKEN="secure_api_token"
138+
139+
# Runtime configuration (needed by containers)
140+
USER_ID="1000"
141+
```
142+
143+
## Implementation Guidelines
144+
145+
### For Infrastructure Scripts
146+
147+
```bash
148+
# Source the main environment file
149+
source "infrastructure/config/environments/${ENVIRONMENT}.env"
150+
151+
# Use all variables for infrastructure operations
152+
generate_ssl_certificates "$SSL_DOMAIN" "$SSL_EMAIL"
153+
configure_backups "$ENABLE_DB_BACKUPS" "$BACKUP_RETENTION_DAYS"
154+
```
155+
156+
### For Template Processing
157+
158+
```bash
159+
# Generate Docker environment file from template
160+
envsubst < "infrastructure/config/templates/docker-compose.env.tpl" > "application/.env"
161+
```
162+
163+
### For Container Configuration
164+
165+
```yaml
166+
# docker-compose.yaml
167+
services:
168+
tracker:
169+
env_file: .env # Only contains container-relevant variables
170+
environment:
171+
- TRACKER_ADMIN_TOKEN=${TRACKER_ADMIN_TOKEN}
172+
```
173+
174+
## Benefits
175+
176+
1. **Security**: Reduced container attack surface
177+
2. **Clarity**: Clear separation between infrastructure and application concerns
178+
3. **Maintainability**: Easier to understand what variables are used where
179+
4. **Flexibility**: Can generate different container environments from same base
180+
5. **Compliance**: Aligns with twelve-factor configuration principles
181+
6. **Debugging**: Easier to troubleshoot configuration issues
182+
183+
## Trade-offs
184+
185+
### Accepted Complexity
186+
187+
- **Two Files to Maintain**: Requires keeping template and source in sync
188+
- **Template Processing**: Additional step in deployment process
189+
- **Learning Curve**: Contributors must understand the two-level structure
190+
191+
### Mitigated Risks
192+
193+
- **Template Drift**: Validation scripts check template consistency
194+
- **Missing Variables**: Docker Compose will fail fast if required variables are missing
195+
- **Documentation**: This ADR and inline comments clarify the structure
196+
197+
## Consequences
198+
199+
### For Contributors
200+
201+
- Must understand which level to modify for different types of changes
202+
- Infrastructure changes: Edit main environment templates
203+
- Container configuration: Edit Docker environment template
204+
- New variables: Consider which level(s) need the variable
205+
206+
### For Deployment
207+
208+
- All deployment scripts use Level 1 (main environment)
209+
- Docker Compose only sees Level 2 (filtered environment)
210+
- Template processing is automatic during deployment
211+
- No manual synchronization required
212+
213+
### For Security
214+
215+
- Infrastructure secrets isolated from application containers
216+
- Container compromise doesn't expose infrastructure configuration
217+
- Easier security auditing of container-exposed variables
218+
219+
## Alternative Considered
220+
221+
**Single-Level Environment Variables**: Using one environment file for everything.
222+
223+
**Rejected because**:
224+
225+
- Security: All variables exposed to containers
226+
- Complexity: Difficult to determine which variables containers actually need
227+
- Maintenance: Changes to infrastructure configuration could affect containers unnecessarily
228+
229+
## References
230+
231+
- [Twelve-Factor App: Config](https://12factor.net/config)
232+
- [ADR-004: Configuration Approach](./004-configuration-approach-files-vs-environment-variables.md)
233+
- [Docker Compose Environment Variables](https://docs.docker.com/compose/environment-variables/)

0 commit comments

Comments
 (0)