tensorboard-reducer 0.1.3

Compute reduced statistics (mean, std, min, max, median or any other numpy operation) of multiple TensorBoard runs matching a directory glob pattern. This can be used after training multiple identical models to reduce the noise in their loss/accuracy/error curves e.g. when trying to establish a statistically significant improvement in training performance.

Requires PyTorch and TensorBoard. No TensorFlow installation required.

Installation

pip install tensorboard-reducer

Usage

Example:

tb-reducer -i 'glob_pattern/of_dirs_to_reduce*' -o basename_of_output_dir -r mean,std,min,max

tb-reducer has the following flags:

• -i/--indirs-glob (required): Glob pattern of the run directories to reduce.
• -o/--outdir (required): Name of the directory to save the new reduced run data. If --format is tb-events, a separate directory will be created for each reduce op (mean, std, ...) post-fixed by the op's name (outdir-mean, outdir-std, ...). If --format is csv, a single file will created and outdir must end with a .csv extension.
• -r/--reduce-ops (required): Comma-separated names of numpy reduction ops (mean, std, min, max, ...). Default is mean. Each reduction is written to a separate outdir suffixed by its op name, e.g. if outdir='my-new-run, the mean reduction will be written to my-new-run-mean.
• -f/--format: Output format of reduced TensorBoard runs. One of tb-events for regular TensorBoard event files or csv. If csv, -o/--outdir must have .csv extension and all reduction ops will be written to a single CSV file rather than separate directories for each reduce op. Use pandas.read_csv("path/to/file.csv", header=[0, 1], index_col=0) to read data back into memory as a multi-index dataframe.
• -w/--overwrite (optional): Whether to overwrite existing outdirs/CSV files.