Command Line Hermes
-------------------

.. role:: bash(code)
   :language: bash

In order to understand how hermes works as a command line tool, you need to know what *hermes (execution) files* are and how
they work. They specify what is executed, but also allow to direct hermes to perform specific IO-operations. Here's a simple example:

.. code-block::
   :caption: example_file.hermes

   PRE
   threads:2
   STD-H
   -ca networks/cnn.py
   STD-E
   -gamma 0.995
   --env cartpole
   EXEC
   [[python, train_dqn.py, -lr 0.001],[-ca train_dqn.py]]
   [[python, train_ac.py, -lr_actor 0.003 -lr_critic 0.001],[-ca train_ac.py]]

As you can see, hermes files are typically divided into four sections

* Within :bash:`PRE`, settings for hermes are specified, here we set that at most 2 threads may run in parallel,
* :bash:`STD-H` contains standard options for hermes, in this case :bash:`-ca networks/cnn.py` (more on that later),
* :bash:`STD-E` contains standard script (execution) options, here we specify that each script is provided with :bash:`-gamma` and :bash:`--env cartpole`
* and finally :bash:`EXEC` specifies what is actually executed, namely some imaginary DQN and Actor-Critic training script

In order to execute this hermes file under the name `test_agents`, we call hermes from the command line:

.. code-block:: bash

   hermes exec -f example_file.hermes -n test_agents

Hermes now first compiles the given hermes file and finally executes the following jobs

.. code-block:: bash

   python train_dqn.py 2021_07_25_160008326976_1930e -gamma 0.995 --env cartpole -lr 0.001
   python train_ac.py 2021_07_25_160008438012_1930e -gamma 0.995 --env cartpole -lr_actor 0.003 -lr_critic 0.001

Each job can be thought of as carrying out an *experiment*. Before experiments are executed, hermes creates a unique experiment *identifier*
as well as an experiment *directory* for them. Assuming your :bash:`.hermes_settings` specify :bash:`path_to_experiments:"."`, these directories are

.. code-block:: bash

    experiments/train_dqn/test_agents/2021_07_25_160008326976_1930e
    experiments/train_ac/test_agents/2021_07_25_160008438012_1930e

Because we specified :bash:`-ca networks/cnn.py` at :bash:`STD-H` level and :bash:`-ca train_dqn.py` as well as
:bash:`-ca train_dqn.py` respectively at the :bash:`EXEC` level, hermes is instructed to copy those files into the experiment directories after
the jobs are finished. This can be useful for archiving purposes and works with all kinds of files, not just python files.

Specifying each file that should be moved individually can be highly inconvenient, so Hermes is also able to automatically move files that are
created within the working directory, from which :bash:`hermes exec` is called. For this to work the names under which these files are saved must include
the experiments unique hermes identifier, e.g. :bash:`2021_07_25_160008326976_1930e` in the case of :bash:`train_dqn.py`.

A necessary restriction this imposes on your scripts is that they need to treat their first command line argument as this exact
identifier. For more information on this also see :doc:`the dedicated CLI documentation <../cli/index>`.