A hyperparameter optimization framework
Project description
Optuna: A hyperparameter optimization framework
Website | Docs | Install Guide | Tutorial
Optuna is an automatic hyperparameter optimization software framework, particularly designed for machine learning. It features an imperative, define-by-run style user API. Thanks to our define-by-run API, the code written with Optuna enjoys high modularity, and the user of Optuna can dynamically construct the search spaces for the hyperparameters.
News
- 2020-12-02 Python 3.9 is now supported. Integration modules are still being worked on and is tracked by #2034
- 2020-09-17
isorthas been incorporated to keep import statements consistent. Read more about it in CONTRIBUTING.md - 2020-08-07 We are welcoming contributions and are working on streamlining the experience. Read more about it in the blog
Key Features
Optuna has modern functionalities as follows:
- Lightweight, versatile, and platform agnostic architecture
- Handle a wide variety of tasks with a simple installation that has few requirements.
- Pythonic search spaces
- Define search spaces using familiar Python syntax including conditionals and loops.
- Efficient optimization algorithms
- Adopt state-of-the-art algorithms for sampling hyperparameters and efficiently pruning unpromising trials.
- Easy parallelization
- Scale studies to tens or hundreds or workers with little or no changes to the code.
- Quick visualization
- Inspect optimization histories from a variety of plotting functions.
Basic Concepts
We use the terms study and trial as follows:
- Study: optimization based on an objective function
- Trial: a single execution of the objective function
Please refer to sample code below. The goal of a study is to find out the optimal set of
hyperparameter values (e.g., classifier and svm_c) through multiple trials (e.g.,
n_trials=100). Optuna is a framework designed for the automation and the acceleration of the
optimization studies.
import ...
# Define an objective function to be minimized.
def objective(trial):
# Invoke suggest methods of a Trial object to generate hyperparameters.
regressor_name = trial.suggest_categorical('classifier', ['SVR', 'RandomForest'])
if regressor_name == 'SVR':
svr_c = trial.suggest_float('svr_c', 1e-10, 1e10, log=True)
regressor_obj = sklearn.svm.SVR(C=svr_c)
else:
rf_max_depth = trial.suggest_int('rf_max_depth', 2, 32)
regressor_obj = sklearn.ensemble.RandomForestRegressor(max_depth=rf_max_depth)
X, y = sklearn.datasets.load_boston(return_X_y=True)
X_train, X_val, y_train, y_val = sklearn.model_selection.train_test_split(X, y, random_state=0)
regressor_obj.fit(X_train, y_train)
y_pred = regressor_obj.predict(X_val)
error = sklearn.metrics.mean_squared_error(y_val, y_pred)
return error # An objective value linked with the Trial object.
study = optuna.create_study() # Create a new study.
study.optimize(objective, n_trials=100) # Invoke optimization of the objective function.
Integrations
Integrations modules, which allow pruning, or early stopping, of unpromising trials are available for the following libraries:
- AllenNLP
- Catalyst
- Catboost
- Chainer
- FastAI (V1, V2)
- Keras
- LightGBM
- MXNet
- PyTorch
- PyTorch Ignite
- PyTorch Lightning
- TensorFlow
- tf.keras
- XGBoost
Web Dashboard (experimental)
The new Web dashboard is under the development at optuna-dashboard. It is still experimental, but much better in many regards. Feature requests and bug reports welcome!
| Manage studies | Visualize with interactive graphs |
|---|---|
 |
 |
Install optuna-dashboard via pip:
$ pip install optuna-dashboard
$ optuna-dashboard sqlite:///db.sqlite3
...
Listening on http://localhost:8080/
Hit Ctrl-C to quit.
Installation
Optuna is available at the Python Package Index and on Anaconda Cloud.
# PyPI
$ pip install optuna
# Anaconda Cloud
$ conda install -c conda-forge optuna
Optuna supports Python 3.6 or newer.
Also, we also provide Optuna docker images on DockerHub.
Communication
- GitHub Issues for bug reports, feature requests and questions.
- Gitter for interactive chat with developers.
- Stack Overflow for questions.
Contribution
Any contributions to Optuna are more than welcome!
If you are new to Optuna, please check the good first issues. They are relatively simple, well-defined and are often good starting points for you to get familiar with the contribution workflow and other developers.
If you already have contributed to Optuna, we recommend the other contribution-welcome issues.
For general guidelines how to contribute to the project, take a look at CONTRIBUTING.md.
Reference
Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. 2019. Optuna: A Next-generation Hyperparameter Optimization Framework. In KDD (arXiv).
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file optuna-2.8.0.dev0.tar.gz.
File metadata
- Download URL: optuna-2.8.0.dev0.tar.gz
- Upload date:
- Size: 215.2 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.59.0 CPython/3.7.10
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | e15ff33722c349af4025c09164ad59c6c05e24413b83fd8856555f3e99d9d982 |
|
| MD5 | eff714d38b8cc44109cbb628a5f9048f |
|
| BLAKE2b-256 | 452fea795fb078f81301bc81c3a0a4470fd4bc21115ae219bedbce918fe5a486 |
File details
Details for the file optuna-2.8.0.dev0-py3-none-any.whl.
File metadata
- Download URL: optuna-2.8.0.dev0-py3-none-any.whl
- Upload date:
- Size: 293.6 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.59.0 CPython/3.7.10
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 91d564e17cfcb079767c6fe82ffaa3f4d7d901181bd47ac855e3acbb8f89ad51 |
|
| MD5 | eaa9f666be3897621deefec5a8b51c8e |
|
| BLAKE2b-256 | 5138efb9aed8e9408255495caabce575a53f0e4902126b6d6c5ace647a1c419e |
