rlberry 0.0.3

A Reinforcement Learning Library for Research and Education

Try it on Google Colab!

What is rlberry?

Writing reinforcement learning algorithms is fun! But after the fun, we have lots of boring things to implement: run our agents in parallel, average and plot results, optimize hyperparameters, compare to baselines, create tricky environments etc etc!

rlberry is a Python library that makes your life easier by doing all these things with a few lines of code, so that you can spend most of your time developing agents.

Check our documentation or our getting started section to see how!

Contents

Getting started

We provide a handful of notebooks on Google colab as examples to show you how to use rlberry.

Content	Description	Link
Introduction to rlberry	How to create an agent, optimize its hyperparameters and compare to a baseline.
RL Experimental Pipeline	How to define a configuration, run experiments in parallel and save a config.json for reproducibility.

Compatibility with OpenAI Gym

If you want to use gym environments with rlberry, simply do the following:

from rlberry.envs import gym_make

# for example, let's take CartPole
env = gym_make('CartPole-v1')

This way, env behaves exactly the same as the gym environment, we simply replace the seeding function by env.reseed(), which ensures unified seeding and reproducibility when using rlberry.

Seeding

In rlberry, only one global seed is defined, and all the random number generators used by the agents and environments inherit from this seed, ensuring reproducibility and independence between the generators (see NumPy SeedSequence).

Example:

import rlberry.seeding as seeding

seeding.set_global_seed(seed=123)

# From now on, no more seeds are defined by the user, and all the results are reproducible.
...

# If you need a random number generator (rng), call:
rng = seeding.get_rng()   

# which gives a numpy Generator (https://numpy.org/doc/stable/reference/random/generator.html) 
# that is independent of all the previous generators created by seeding.get_rng()
rng.integers(5)
rng.normal()
# etc

Citing rlberry

If you use rlberry in scientific publications, we would appreciate citations using the following Bibtex entry:

@misc{rlberry,
author = {Domingues, Omar Darwiche and ‪Flet-Berliac, Yannis and Leurent, Edouard and M{\'e}nard, Pierre and Shang, Xuedong and Valko, Michal},
title = {{rlberry - A Reinforcement Learning Library for Research and Education}},
year = {2021},
publisher = {GitHub},
journal = {GitHub repository},
howpublished = {\url{https://github.com/rlberry-py/rlberry}},
}

Installation

Cloning & creating virtual environment

It is suggested to create a virtual environment using Anaconda or Miniconda:

git clone https://github.com/rlberry-py/rlberry.git
conda create -n rlberry python=3.7

Basic installation

Install without heavy libraries (e.g. pytorch):

conda activate rlberry
pip install -e .

Full installation

Install with all features:

conda activate rlberry
pip install -e .[full]

which includes:

• Numba for just-in-time compilation of algorithms based on dynamic programming
• PyTorch for Deep RL agents
• Optuna for hyperparameter optimization
• ffmpeg-python for saving videos
• PyOpenGL for more rendering options

Tests

To run tests, install test dependencies with pip install -e .[test] and run pytest. To run tests with coverage, install test dependencies and run bash run_testscov.sh. See coverage report in cov_html/index.html.

Contributing

Want to contribute to rlberry? Please check our contribution guidelines. A list of interesting TODO's will be available soon. If you want to add any new agents or environments, do not hesitate to open an issue!

Implementation notes

• When inheriting from the Agent class, make sure to call Agent.__init__(self, env, **kwargs) using **kwargs in case new features are added to the base class, and to make sure that copy_env and reseed_env are always an option to any agent.

Infos, errors and warnings are printed using the logging library.

• From gym to rlberry:
  • reseed (rlberry) should be called instead of seed (gym). seed keeps compatilibity with gym, whereas reseed uses the unified seeding mechanism of rlberry.