Review file-system dependencies

[josecelano]

The application needs some files to work:

• config.toml: configuration file.
• data.db: SQLite database.
• localhost.crt and localhost.key: SSL certificates.

All of them are optional because:

• You can inject the configuration with an environment variable.
• You can use MYSQL.
• You can use only HTTP.

I want to clarify the default location, values and behaviour for those files.

config.toml

Initially, the config.toml was in the root folder. If the file was missing the tracker automatically generated one with the default values and exited with an error code.

The default location is still ./config.toml, but the application does not fail if the file is missing. Instead, it creates a default one and continues the execution with default values.

The location of the config.toml file can now be changed with an env var (TORRUST_TRACKER_CONFIG_PATH). This feature was added after adding support for docker. It was more convenient to mont a volume when you use docker than a single file. So the recommended location was:

./storage/config/config.toml or ./storage/config.toml

The intention was to mount a single storage folder were you can inject app configuration (like the config.yml file or HTTs certificates or persist app data like the database file data.db).

The current default configuration value for the SQLite database location is:

db_path = "./storage/database/data.db"

This means the application, by default, fails when you run it without previously creating these folders/files:

./storage/
└── database
    └── data.db

The error message is:

Loading configuration from configuration file: `./config.toml` ...
2023-08-30T13:34:03.587045844+01:00 [torrust_tracker::bootstrap::logging][INFO] logging initialized.
2023-08-30T13:34:03.587262642+01:00 [r2d2][ERROR] unable to open database file: ./storage/database/data.db
2023-08-30T13:34:03.587268182+01:00 [r2d2][ERROR] unable to open database file: ./storage/database/data.db
2023-08-30T13:34:03.587293932+01:00 [r2d2][ERROR] unable to open database file: ./storage/database/data.db

@da2ce7 @mario-nt I think we should stop the application immediately if it's configured to use a SQLite database and the database is not there.

Or should we create the directories and a new empty SQLite database?

I think the app should adopt one style either:

• Let the user set up all the dependencies and fail if there is a missing system dependency.
• Try to continue the execution with default values, creating dirs and default files if needed.

Another change regarding the config.toml file is that it's now a read-only file. In previous versions, you can change the configuration via the API.

Finally, @da2ce7 is working on a PR where the config.toml file has been renamed to tracker.toml.

data.db

If you use SQLite the default location for the database file was the root folder: data.db.
That has been changed to ./storage/database/data.db.

Pros:

• We can mount a single folder with docker.
• If we want to backup app configuration or data, we only need to copy that folder.

Cons:

• Now, the application has file-system dependency. You need to create the folder ./storage in order to run the app.

As I mentioned above, should we create the dir if it does not exist?

localhost.crt and localhost.key

Those files do not have a default value or a default location. It was always up to the user where to put them. We recommended using the same storage folder.

Questions

Should we normalize where the application stores the configuration and persisted data? I think we have a complex configuration setup because I wanted to keep backward compatibility. But since we are going to release a major version, maybe we could simplify it. On the new PR @da2ce7 is defining these volumes for the docker containers:

--volume ./storage/tracker/lib:/var/lib/torrust/tracker:rw \
--volume ./storage/tracker/log:/var/log/torrust/tracker:rw \
--volume ./storage/tracker/etc:/etc/torrust/tracker:rw \

Should we normalise how the app behaves when a dir/file is missing?

[mario-nt]

Overall I think it is a good idea to have all the files located inside a /storage folder as it will help keeping things organized.

config.toml

In my opinion the app should be able to run without the user creating the config file, I think it is okay that the app (the first time it's run) checks if a config file exists and if not, creates a default one in the /storage/config/ folder, and then runs using that config file that contains the default values.

Another approach would be having some sort of prompt that asks the user if he/she wants to specify a folder for each of the config files, and later, creates those files with default values and tell the user that they can be modified.

(Angular, the front-end framework, for example, ask the user questions about configurations before running for the first time).

We can also give the user the chance to go over the same process in the future if he/she wants to.

data.db

Again, it would be nice to have this file in the /storage/database folder, and I think it might not be necessary to create such directory if the user i using MySql (If I am not wrong, when using mysql there are no files that have to be created in the app directory)

localhost.crt and localhost.key

I would put these in the /storage/certificates folder, and let the app create that folder by default, even though in this case if will not create the certificates files themself by default.

Questions

Having a defined path for config files would help in the organization of the app, and I think it is worth it even if we need to mark that as a "breaking change" in the next version.

[josecelano]

This is deprecated and partially or completely implemented.