Skip to content
This repository was archived by the owner on Oct 10, 2025. It is now read-only.

Commit 372be70

Browse files
committed
refactor: [#21] implement single template approach for environment configuration
- Replace dual templates (local.env.tpl, production.env.tpl) with single base.env.tpl - Add external defaults files (local.defaults, production.defaults) for environment-specific values - Update configure-env.sh to load defaults from files instead of hardcoded values - Improve twelve-factor compliance with single source of truth for configuration - Add generate-secrets command for secure production secret generation - Update documentation and .gitignore for new file structure - Benefits: DRY principle, easier maintenance, version-controlled defaults, no sync issues This addresses the issue where having separate templates could lead to synchronization bugs when adding new variables to only one template. All environment variables now exist in one place with environment-specific values defined in external, version-controlled defaults files.
1 parent 40a32a2 commit 372be70

File tree

10 files changed

+361
-167
lines changed

10 files changed

+361
-167
lines changed

docs/guides/cloud-deployment-guide.md

Lines changed: 36 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -332,7 +332,7 @@ make app-deploy ENVIRONMENT=local
332332
# - HTTP only (no SSL certificates)
333333
# - Local domain names (tracker.local)
334334
# - Basic monitoring
335-
# - SQLite database (for faster setup)
335+
# - MySQL database (same as production)
336336
```
337337

338338
### Production Environment Setup
@@ -341,53 +341,69 @@ Before deploying to production, you must configure secure secrets and environmen
341341

342342
#### Step 1: Generate Secure Secrets
343343

344-
Production deployment requires several secure random secrets. Generate them using GPG:
344+
Production deployment requires several secure random secrets. Use the built-in secret generator:
345345

346346
```bash
347-
# Generate secure secrets (40 characters each)
348-
echo "MYSQL_ROOT_PASSWORD=$(gpg --armor --gen-random 1 40)"
349-
echo "MYSQL_PASSWORD=$(gpg --armor --gen-random 1 40)"
350-
echo "TRACKER_ADMIN_TOKEN=$(gpg --armor --gen-random 1 40)"
351-
echo "GF_SECURITY_ADMIN_PASSWORD=$(gpg --armor --gen-random 1 40)"
347+
# Generate secure secrets using the built-in helper
348+
./infrastructure/scripts/configure-env.sh generate-secrets
352349
```
353350

354351
**Example output**:
355352

356353
```bash
354+
=== TORRUST TRACKER PRODUCTION SECRETS ===
355+
356+
Copy these values into: infrastructure/config/environments/production.env
357+
358+
# === GENERATED SECRETS ===
357359
MYSQL_ROOT_PASSWORD=jcrmbzlGyeP7z53TUQtXmtltMb5TubsIE9e0DPLnS4Ih29JddQw5JA==
358360
MYSQL_PASSWORD=kLp9nReY4vXqA7mZ8wB3QcG6FsE1oNtH5jUiD2fK0zRyS9CxT8V1Mq==
359361
TRACKER_ADMIN_TOKEN=nP6rL2gKbY8xW5zA9mQ4jE3vC7sR1tH0oB9fN6dK5uI8eT2yV1nX4q==
360362
GF_SECURITY_ADMIN_PASSWORD=wQ9tR4nM7bX2zA8kY6pL5sG1oE3vN0cF9eT8jU4dK7hB6rW5iQ2nM==
363+
364+
# === DOMAIN CONFIGURATION (REPLACE WITH YOUR VALUES) ===
365+
DOMAIN_NAME=your-domain.com
366+
361367
```
362368

363369
#### Step 2: Configure Production Environment
364370

365-
Edit the production environment template with your secure secrets:
371+
**Note**: The project now uses a unified configuration template approach following twelve-factor
372+
principles. This eliminates synchronization issues between multiple template files.
366373

367-
```bash
368-
# Copy production template
369-
cp infrastructure/config/environments/production.env.tpl infrastructure/config/environments/production.env
374+
Generate the production configuration template:
370375

371-
# Edit with your secure secrets and domain configuration
372-
vim infrastructure/config/environments/production.env
376+
```bash
377+
# Generate production configuration template with placeholders
378+
make infra-config-production
373379
```
374380

375-
**Required Configuration**:
381+
This will create `infrastructure/config/environments/production.env` with secure placeholder
382+
values that need to be replaced with your actual configuration.
383+
384+
#### Step 3: Replace Placeholder Values
385+
386+
Edit the generated production environment file with your secure secrets and domain configuration:
376387

377388
```bash
378-
# Replace these placeholder values with your actual configuration:
389+
# Edit the production configuration
390+
vim infrastructure/config/environments/production.env
391+
```
379392

380-
# === DOMAIN CONFIGURATION ===
381-
DOMAIN_NAME=your-domain.com # Your actual domain
382-
[email protected] # Your email for Let's Encrypt
393+
**Replace these placeholder values with your actual configuration**:
383394

395+
```bash
384396
# === SECURE SECRETS ===
385397
# Replace with secrets generated above
386398
MYSQL_ROOT_PASSWORD=jcrmbzlGyeP7z53TUQtXmtltMb5TubsIE9e0DPLnS4Ih29JddQw5JA==
387399
MYSQL_PASSWORD=kLp9nReY4vXqA7mZ8wB3QcG6FsE1oNtH5jUiD2fK0zRyS9CxT8V1Mq==
388400
TRACKER_ADMIN_TOKEN=nP6rL2gKbY8xW5zA9mQ4jE3vC7sR1tH0oB9fN6dK5uI8eT2yV1nX4q==
389401
GF_SECURITY_ADMIN_PASSWORD=wQ9tR4nM7bX2zA8kY6pL5sG1oE3vN0cF9eT8jU4dK7hB6rW5iQ2nM==
390402

403+
# === DOMAIN CONFIGURATION ===
404+
DOMAIN_NAME=your-domain.com # Your actual domain
405+
[email protected] # Your email for Let's Encrypt
406+
391407
# === BACKUP CONFIGURATION ===
392408
ENABLE_DB_BACKUPS=true
393409
BACKUP_RETENTION_DAYS=7
@@ -396,12 +412,12 @@ BACKUP_RETENTION_DAYS=7
396412
**⚠️ Security Note**: The `production.env` file contains sensitive secrets and is git-ignored.
397413
Never commit this file to version control.
398414

399-
#### Step 3: Validate Configuration
415+
#### Step 4: Validate Configuration
400416

401417
Validate your production configuration before deployment:
402418

403419
```bash
404-
# Validate configuration
420+
# Validate configuration (will work only after secrets are configured)
405421
make infra-config-production
406422

407423
# Expected output:

docs/issues/21-complete-application-installation-automation.md

Lines changed: 13 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -83,19 +83,19 @@ well-guided.
8383

8484
**Last Updated**: 2025-07-29
8585

86-
| Component | Status | Description | Notes |
87-
| ----------------------------- | ------------------ | -------------------------------------------------- | ------------------------------------------------- |
88-
| **Infrastructure Foundation** |**Complete** | VM provisioning, cloud-init, basic system setup | Fully automated via provision-infrastructure.sh |
89-
| **Application Foundation** |**Complete** | Docker deployment, basic app orchestration | Fully automated via deploy-app.sh |
90-
| **Environment Templates** |**Complete** | SSL/domain/backup variables added to templates | Templates updated with all required variables |
91-
| **Secret Generation Helper** |**Complete** | Helper script for generating secure secrets | generate-secrets.sh implemented |
92-
| **Basic Nginx Templates** |**Complete** | HTTP nginx configuration template exists | nginx.conf.tpl with HTTP + commented HTTPS |
93-
| **configure-env.sh Updates** |**Complete** | SSL/backup variable validation implemented | Comprehensive validation with email/boolean checks |
94-
| **SSL Certificate Scripts** |**Not Started** | Create SSL generation and configuration scripts | Core SSL automation needed |
95-
| **HTTPS Nginx Templates** | 🔄 **Partial** | HTTPS configuration exists but commented out | Current template has HTTPS but needs activation |
96-
| **MySQL Backup Scripts** |**Not Started** | Create MySQL backup automation scripts | Referenced by cron template but doesn't exist |
97-
| **deploy-app.sh Extensions** |**Not Started** | SSL/backup automation not yet integrated | Foundation exists, needs SSL/backup stages |
98-
| **Crontab Templates** | 🔄 **Partial** | Templates exist but reference non-existent scripts | Templates created, scripts and integration needed |
86+
| Component | Status | Description | Notes |
87+
| ----------------------------- | ------------------ | -------------------------------------------------- | -------------------------------------------------- |
88+
| **Infrastructure Foundation** |**Complete** | VM provisioning, cloud-init, basic system setup | Fully automated via provision-infrastructure.sh |
89+
| **Application Foundation** |**Complete** | Docker deployment, basic app orchestration | Fully automated via deploy-app.sh |
90+
| **Environment Templates** |**Complete** | SSL/domain/backup variables added to templates | Templates updated with all required variables |
91+
| **Secret Generation Helper** |**Complete** | Helper script for generating secure secrets | generate-secrets.sh implemented |
92+
| **Basic Nginx Templates** |**Complete** | HTTP nginx configuration template exists | nginx.conf.tpl with HTTP + commented HTTPS |
93+
| **configure-env.sh Updates** |**Complete** | SSL/backup variable validation implemented | Comprehensive validation with email/boolean checks |
94+
| **SSL Certificate Scripts** |**Not Started** | Create SSL generation and configuration scripts | Core SSL automation needed |
95+
| **HTTPS Nginx Templates** | 🔄 **Partial** | HTTPS configuration exists but commented out | Current template has HTTPS but needs activation |
96+
| **MySQL Backup Scripts** |**Not Started** | Create MySQL backup automation scripts | Referenced by cron template but doesn't exist |
97+
| **deploy-app.sh Extensions** |**Not Started** | SSL/backup automation not yet integrated | Foundation exists, needs SSL/backup stages |
98+
| **Crontab Templates** | 🔄 **Partial** | Templates exist but reference non-existent scripts | Templates created, scripts and integration needed |
9999
| **Documentation Updates** | 🔄 **Partial** | ADR-004 updated for deployment automation config | Deployment guides need updates post-implementation |
100100

101101
**Current Progress**: 50% complete (6/12 components fully implemented)

infrastructure/.gitignore

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,10 +10,11 @@
1010
terraform.tfplan
1111
terraform.tfplan.*
1212

13-
# Environment files with secrets (keep templates)
13+
# Environment files with secrets (keep templates and defaults)
1414
config/environments/production.env
1515
config/environments/*.env
1616
!config/environments/*.env.tpl
17+
!config/environments/*.defaults
1718

1819
# Cloud-init generated files
1920
user-data.yaml

infrastructure/config/environments/README.md

Lines changed: 68 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,30 @@
1-
# Environment Configuration Templates
1+
# Environment Configuration
22

3-
This directory contains environment-specific configuration templates that are processed
4-
during deployment to generate the final configuration files.
3+
This directory contains the environment configuration system for the Torrust Tracker Demo.
54

6-
## Files
5+
## Files Overview
76

8-
- `local.env.tpl` - Local development environment template
9-
- `production.env.tpl` - Production environment template (requires manual setup)
7+
### Templates and Configuration
8+
9+
- **`base.env.tpl`** - Single base template for all environments (uses variable substitution)
10+
- **`local.defaults`** - Default values for local development environment
11+
- **`production.defaults`** - Default values for production environment template
12+
13+
### Generated Files (Git-Ignored)
14+
15+
- **`local.env`** - Generated local environment configuration (regenerated automatically)
16+
- **`production.env`** - Generated production environment configuration (manual secrets required)
17+
18+
## How It Works
19+
20+
### Twelve-Factor Compliance
21+
22+
This system follows twelve-factor app principles by:
23+
24+
1. **Single Source of Truth**: One base template (`base.env.tpl`) for all environments
25+
2. **Environment-Specific Configuration**: Default files define environment-specific values
26+
3. **Separation of Concerns**: Configuration (defaults) separated from code (scripts)
27+
4. **Version Control**: Default files are tracked, generated files with secrets are ignored
1028

1129
## Template Processing
1230

@@ -194,6 +212,50 @@ but this is actually a good practice that ensures:
194212
ssh torrust@$VM_IP 'cd torrust-tracker-demo && cat application/.env'
195213
```
196214

215+
## Default Files System (New Approach)
216+
217+
### Configuration Architecture
218+
219+
The environment configuration system now uses a single base template with external default files:
220+
221+
- **`base.env.tpl`**: Single template with variable placeholders (`${VARIABLE_NAME}`)
222+
- **`local.defaults`**: Default values for local development
223+
- **`production.defaults`**: Default placeholder values for production
224+
225+
### Benefits
226+
227+
1. **DRY Principle**: Single source of truth for all environment variables
228+
2. **Maintainability**: Add variables once in base template, define values in defaults
229+
3. **Version Control**: Default values are tracked and can be customized
230+
4. **Consistency**: Same template processing logic for all environments
231+
232+
### Usage
233+
234+
```bash
235+
# Generate local environment (uses local.defaults)
236+
./infrastructure/scripts/configure-env.sh local
237+
238+
# Generate production template (uses production.defaults)
239+
./infrastructure/scripts/configure-env.sh production
240+
241+
# Generate secure production secrets
242+
./infrastructure/scripts/configure-env.sh generate-secrets
243+
```
244+
245+
### Customizing Defaults
246+
247+
Edit the `.defaults` files to change environment-specific values:
248+
249+
```bash
250+
# Change local development domain
251+
vim infrastructure/config/environments/local.defaults
252+
253+
# Change production backup retention
254+
vim infrastructure/config/environments/production.defaults
255+
```
256+
257+
The next time you run configuration generation, your changes will be applied.
258+
197259
## Security Notes
198260

199261
- **Never commit production secrets** - Use placeholder values in templates
Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
# ${ENVIRONMENT_DESCRIPTION}
2+
# ${ENVIRONMENT_INSTRUCTIONS}
3+
4+
ENVIRONMENT=${ENVIRONMENT}
5+
GENERATION_DATE=$(date '+%Y-%m-%d %H:%M:%S')
6+
7+
${TEMPLATE_PROCESSING_VARS}
8+
9+
# === SECRETS (DOCKER SERVICES) ===
10+
${SECRETS_DESCRIPTION}
11+
12+
# Database Secrets
13+
MYSQL_ROOT_PASSWORD=${MYSQL_ROOT_PASSWORD}
14+
MYSQL_DATABASE=torrust_tracker
15+
MYSQL_USER=torrust
16+
MYSQL_PASSWORD=${MYSQL_PASSWORD}
17+
18+
# Tracker API Token${TRACKER_TOKEN_DESCRIPTION}
19+
TRACKER_ADMIN_TOKEN=${TRACKER_ADMIN_TOKEN}
20+
21+
# Grafana Admin Credentials
22+
GF_SECURITY_ADMIN_USER=admin
23+
GF_SECURITY_ADMIN_PASSWORD=${GF_SECURITY_ADMIN_PASSWORD}
24+
25+
# === SSL CERTIFICATE CONFIGURATION ===
26+
# Domain name for SSL certificates${DOMAIN_NAME_DESCRIPTION}
27+
DOMAIN_NAME=${DOMAIN_NAME}
28+
# Email for ${CERTBOT_EMAIL_DESCRIPTION}
29+
CERTBOT_EMAIL=${CERTBOT_EMAIL}
30+
# Enable SSL certificates${ENABLE_SSL_DESCRIPTION}
31+
ENABLE_SSL=${ENABLE_SSL}
32+
33+
# === BACKUP CONFIGURATION ===
34+
# Enable daily database backups${BACKUP_DESCRIPTION}
35+
ENABLE_DB_BACKUPS=${ENABLE_DB_BACKUPS}
36+
# Backup retention period in days
37+
BACKUP_RETENTION_DAYS=${BACKUP_RETENTION_DAYS}
38+
39+
# === DEPLOYMENT AUTOMATION CONFIGURATION ===
40+
# These variables control deployment scripts and automation, not service configuration.
41+
# They are consumed by infrastructure scripts (deploy-app.sh, SSL generation, backup automation)
42+
# rather than individual Docker services. This follows 12-factor principles for deployment automation.
43+
44+
# === DOCKER CONFIGURATION ===
45+
46+
# User ID for file permissions${USER_ID_DESCRIPTION}
47+
USER_ID=${USER_ID}
Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
# Local Development Environment Default Values
2+
# These values are used to generate local.env from the base template
3+
# Safe default values for local development and testing
4+
5+
ENVIRONMENT_DESCRIPTION="Local Development Environment Configuration"
6+
ENVIRONMENT_INSTRUCTIONS="Generated from base template for local development and testing"
7+
ENVIRONMENT="local"
8+
TEMPLATE_PROCESSING_VARS="
9+
# Template processing variables
10+
DOLLAR=\$"
11+
SECRETS_DESCRIPTION=""
12+
MYSQL_ROOT_PASSWORD="root_secret_local"
13+
MYSQL_PASSWORD="tracker_secret_local"
14+
TRACKER_TOKEN_DESCRIPTION=""
15+
TRACKER_ADMIN_TOKEN="MyAccessToken"
16+
GF_SECURITY_ADMIN_PASSWORD="admin_secret_local"
17+
DOMAIN_NAME_DESCRIPTION=" (local testing with fake domains)"
18+
DOMAIN_NAME="test.local"
19+
CERTBOT_EMAIL_DESCRIPTION="certificate registration (test email for local)"
20+
CERTBOT_EMAIL="[email protected]"
21+
ENABLE_SSL_DESCRIPTION=" (false for local testing)"
22+
ENABLE_SSL="false"
23+
BACKUP_DESCRIPTION=" (disabled for local testing)"
24+
ENABLE_DB_BACKUPS="false"
25+
BACKUP_RETENTION_DAYS="3"
26+
USER_ID_DESCRIPTION=""
27+
USER_ID="1000"

infrastructure/config/environments/local.env.tpl

Lines changed: 0 additions & 45 deletions
This file was deleted.
Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
# Production Environment Default Values
2+
# These values are used to generate production.env template from the base template
3+
# Contains placeholder values that must be replaced with secure secrets
4+
5+
ENVIRONMENT_DESCRIPTION="Production Environment Configuration Template"
6+
ENVIRONMENT_INSTRUCTIONS="Copy this file to production.env and replace placeholder values with secure secrets"
7+
ENVIRONMENT="production"
8+
TEMPLATE_PROCESSING_VARS=""
9+
SECRETS_DESCRIPTION="
10+
# IMPORTANT: Replace ALL placeholder values with actual secure secrets before deployment!"
11+
MYSQL_ROOT_PASSWORD="REPLACE_WITH_SECURE_ROOT_PASSWORD"
12+
MYSQL_PASSWORD="REPLACE_WITH_SECURE_PASSWORD"
13+
TRACKER_TOKEN_DESCRIPTION=" (Used for administrative API access)"
14+
TRACKER_ADMIN_TOKEN="REPLACE_WITH_SECURE_ADMIN_TOKEN"
15+
GF_SECURITY_ADMIN_PASSWORD="REPLACE_WITH_SECURE_GRAFANA_PASSWORD"
16+
DOMAIN_NAME_DESCRIPTION=" (required for production)"
17+
DOMAIN_NAME="REPLACE_WITH_YOUR_DOMAIN"
18+
CERTBOT_EMAIL_DESCRIPTION="Let's Encrypt certificate registration (required for production)"
19+
CERTBOT_EMAIL="REPLACE_WITH_YOUR_EMAIL"
20+
ENABLE_SSL_DESCRIPTION=" (true for production, false for testing)"
21+
ENABLE_SSL="true"
22+
BACKUP_DESCRIPTION=" (true/false)"
23+
ENABLE_DB_BACKUPS="true"
24+
BACKUP_RETENTION_DAYS="7"
25+
USER_ID_DESCRIPTION=" (match host user)"
26+
USER_ID="1000"

0 commit comments

Comments
 (0)