ml-logger

Logging utility for ML experiments

Installation

• git clone git@github.com:shagunsodhani/ml-logger.git
• cd ml-logger
• pip install .

Alternatively, pip install "git+https://git@github.com/shagunsodhani/ml-logger.git@master#egg=ml_logger"

If you want to use tensorboardX or Wandb logger, use pip install ".[all]" or pip install "git+https://git@github.com/shagunsodhani/ml-logger.git@master#egg=ml_logger[all]".

Use

• Make a logbook_config:

  from ml_logger import logbook as ml_logbook
  logbook_config = ml_logbook.make_config(
      logger_file_path = <path to write logs>,
      wandb_config = <wandb config or None>,
      tensorboard_config = <tensorboard config or None>,
      mlflow_config = <mlflow config or None>)
  

  The API for make_config can be accessed here.

• Make a LogBook instance:

  logbook = ml_logbook.LogBook(config = logbook_config)
  

• Use the logbook instance:

  log = {
      "epoch": 1,
      "loss": 0.1,
      "accuracy": 0.2
  }
  logbook.write_metric_log(log)
  

  The API for write_metric_log can be accessed here.

Note

• If you are writing to wandb, the log must have a key called step. If your log already captures the step but as a different key (say epoch), you can pass the wandb_key_map argument (set as {epoch: step}). For more details, refer the documentation here.

• If you are writing to mlflow, the log must have a key called step. If your log already captures the step but as a different key (say epoch), you can pass the mlflow_key_map argument (set as {epoch: step}). For more details, refer the documentation here.

• If you are writing to tensorboard, the log must have a key called main_tag or tag which acts as the data Identifier and another key called global_step. These keys are described here. If your log already captures these values but as different key (say mode for main_tag and epoch for global_step), you can pass the tensorboard_key_map argument (set as {mode: main_tag, epoch: global_step}). For more details, refer the documentation here.

• TensorboardX sometimes throws an error when logging metrics whose value is of type string. Currently, such metrics are not logged.

Documentation

https://shagunsodhani.github.io/ml-logger

Dev Setup

• pip install -e .[dev]
• Install pre-commit hooks pre-commit install
• The code is linted using:
  • black
  • flake8
  • mypy
• CI currently only checks for linting.
• Lint tests can be run locally using nox -s lint

Acknowledgements

• Config for circleci, pre-commit, mypy etc are borrowed/modified from Hydra