How to pass hyperparameters and configs

We use a simple but useful method for setting hyperparameters and environment configuration values for an experiment. We distinguish between these two values.

1. Environment configuration: All values that define an environment. E.g., number of actions, horizon, etc.

2. Hyperparameter constants: All hyperparameter constants that define a model or learning algorithm. E.g., learning rate, batch size, etc.

These are typically defined in data/<environment>/config.json and data/<environment-name>/constant.json.

We follow a three-level hierarchy in defining these values:

1. (Lowest Priority) These are default config and constant values in data/base_config.json and data/base_constant.json

2. (Medium Priority) These are environment-specific config and constant values in data/<environment-name>. These overwrite the default values.

3. (Highest Priority) These are command line values that overwrite even the environment-specific values.

Any key that is present in data/base_config.json and data/base_constant.json can be supplied through the use of command line arguments. At the moment, the same key name cannot be present in both data/base_config.json and data/base_constant.json, to avoid confusion when this value is passed via command line arguments.

How do I set these values?

The most basic hyperparameters that are in general good values can be put in base_constants.json. E.g., a learning rate of 0.001 and a dropout value of 0.5 are generally considered good. If you find hyperparameters that tend to work better for a specific environment, you can put them in the environment folder (you may have to create one if it doesn't exist). Finally, you can provide any remaining values in command line arguments such as the ones you are running a grid search over.