The framework has been tested with Python3 version 3.8, but should also work with any higher version. It has been tested under Linux and macOS. It does not work on Windows (at the moment).

To get started:

git clone https://github.com/S2-group/experiment-runner.git
cd experiment-runner/
pip install -r requirements.txt

To verify installation, run:

python experiment-runner/ examples/hello-world/RunnerConfig.py

In this and the following sections, we assume as the current working directory, the root directory of the project.

To run any of the examples, run the following command:

python experiment-runner/ examples/<example-dir>/<RunnerConfig*.py>

Each example is accompanied with a README for further information. It is recommended to start with the hello-world example to also test your installation.

Note that once you successfully run an experiment, the framework will not allow you to run the same experiment again under, giving the message:

[FAIL]: EXPERIMENT_RUNNER ENCOUNTERED AN ERROR!
The experiment was restarted, but all runs are already completed.

This is to prevent you from accidentally overwriting the results of a previously run experiment! In order to run again the experiment, either delete any previously generated data (by default "experiments/" directory), or modify the config's name variable to a different name.

First, generate a config for your experiment:

python experiment-runner/ config-create [directory]

When running this command, where [directory] is an optional argument, a new config file with skeleton code will be generated in the given directory. The default location is the examples/ directory. This config is similar to the hello-world example.

Feel free to move the generated config to any other directory. You can modify its contents and write python code to define your own measurement-based experiment(s). At this stage, you might find useful the linux-ps-profiling example.

Once the experiment has been coded, the experiment can be executed by Experiment Runner. To do this, run the following command:

python experiment-runner/ <MyRunnerConfig.py>

The results of the experiment will be stored in the directory RunnerConfig.results_output_path/RunnerConfig.name as defined by your config variables.