A public and reproducible collection of reference implementations and benchmark suite for distributed machine learning systems.
Project description
====================================================
mlbench_core: Distributed Machine Learning Benchmark
====================================================
.. image:: https://travis-ci.com/mlbench/mlbench.svg?branch=develop
:target: https://travis-ci.com/mlbench/mlbench
.. image:: https://readthedocs.org/projects/mlbench/badge/?version=latest
:target: https://mlbench.readthedocs.io/en/latest/?badge=latest
:alt: Documentation Status
A public and reproducible collection of reference implementations and benchmark suite for distributed machine learning algorithms, frameworks and systems.
* Project website: https://mlbench.github.io/
* Free software: Apache Software License 2.0
* Documentation: https://mlbench.readthedocs.io.
Features
--------
* For reproducibility and simplicity, we currently focus on standard **supervised ML**, including standard deep learning tasks as well as classic linear ML models.
* We provide **reference implementations** for each algorithm, to make it easy to port to a new framework.
* Our goal is to benchmark all/most currently relevant **distributed execution frameworks**. We welcome contributions of new frameworks in the benchmark suite.
* We provide **precisely defined tasks** and datasets to have a fair and precise comparison of all algorithms, frameworks and hardware.
* Independently of all solver implementations, we provide universal **evaluation code** allowing to compare the result metrics of different solvers and frameworks.
* Our benchmark code is easy to run on **public clouds**.
Community
---------
About us: See :doc:`Authors </authors>`
Mailing list: https://groups.google.com/d/forum/mlbench
Contact Email: mlbench-contact@googlegroups.com
# Change Log
## [1.0.0](https://github.com/mlbench/mlbench-core/tree/1.0.0) (2018-11-15)
**Implemented enhancements:**
- Add API Client to mlbench-core [\#6](https://github.com/mlbench/mlbench-core/issues/6)
- Move to google-style docs [\#4](https://github.com/mlbench/mlbench-core/issues/4)
- Add Imagenet Dataset for pytorch [\#3](https://github.com/mlbench/mlbench-core/issues/3)
- Move worker code to mlbench-core repo [\#1](https://github.com/mlbench/mlbench-core/issues/1)
# Change Log
## [0.1.0](https://github.com/mlbench/mlbench/tree/0.1.0) (2018-09-14)
**Implemented enhancements:**
- Add documentation in reference implementation to docs [\#46](https://github.com/mlbench/mlbench/issues/46)
- Replace cAdvisor with Kubernetes stats for Resource usage [\#38](https://github.com/mlbench/mlbench/issues/38)
- Rename folders [\#31](https://github.com/mlbench/mlbench/issues/31)
- Change docker image names [\#30](https://github.com/mlbench/mlbench/issues/30)
- Add continuous output for mpirun [\#27](https://github.com/mlbench/mlbench/issues/27)
- Replace SQlite with Postgres [\#25](https://github.com/mlbench/mlbench/issues/25)
- Fix unittest [\#23](https://github.com/mlbench/mlbench/issues/23)
- Add/Fix CI/Automated build [\#22](https://github.com/mlbench/mlbench/issues/22)
- Cleanup unneeded project files [\#21](https://github.com/mlbench/mlbench/issues/21)
- Remove hardcoded values [\#20](https://github.com/mlbench/mlbench/issues/20)
- Improves Notes.txt [\#19](https://github.com/mlbench/mlbench/issues/19)
- Rename components [\#15](https://github.com/mlbench/mlbench/issues/15)
**Fixed bugs:**
- 504 Error when downloading metrics for long runs [\#61](https://github.com/mlbench/mlbench/issues/61)
**Closed issues:**
- small doc improvements for first release [\#54](https://github.com/mlbench/mlbench/issues/54)
- Check mlbench works on Google Cloud [\#51](https://github.com/mlbench/mlbench/issues/51)
- learning rate scheduler [\#50](https://github.com/mlbench/mlbench/issues/50)
- Add Nvidia k8s-device-plugin to charts [\#48](https://github.com/mlbench/mlbench/issues/48)
- Add Weave to Helm Chart [\#41](https://github.com/mlbench/mlbench/issues/41)
- Allow limiting of resources for experiments [\#39](https://github.com/mlbench/mlbench/issues/39)
- Allow downloading of Run measurements [\#35](https://github.com/mlbench/mlbench/issues/35)
- Worker Details page [\#33](https://github.com/mlbench/mlbench/issues/33)
- Run Visualizations [\#32](https://github.com/mlbench/mlbench/issues/32)
- Show experiment history in Dashboard [\#18](https://github.com/mlbench/mlbench/issues/18)
- Show model progress in Dashboard [\#13](https://github.com/mlbench/mlbench/issues/13)
- Report cluster status in Dashboard [\#12](https://github.com/mlbench/mlbench/issues/12)
- Send metrics from SGD example to metrics api [\#11](https://github.com/mlbench/mlbench/issues/11)
- Add metrics endpoint for experiments [\#10](https://github.com/mlbench/mlbench/issues/10)
- Let Coordinator Dashboard start a distributed Experiment [\#9](https://github.com/mlbench/mlbench/issues/9)
- Add mini-batch SGD model experiment [\#8](https://github.com/mlbench/mlbench/issues/8)
- add benchmark code for MPI [\#7](https://github.com/mlbench/mlbench/issues/7)
- add benchmark code for tensorflow [\#6](https://github.com/mlbench/mlbench/issues/6)
- add benchmark code for apache reef [\#5](https://github.com/mlbench/mlbench/issues/5)
- add benchmark code for apache flink [\#4](https://github.com/mlbench/mlbench/issues/4)
- get initial benchmark numbers \(spark reference implementation and mllib/ml\) [\#3](https://github.com/mlbench/mlbench/issues/3)
- evaluate script \(framework-independent\) and algorithm output format [\#2](https://github.com/mlbench/mlbench/issues/2)
- bench-spark: remove prepare-data for now, comment on solver prequisites [\#1](https://github.com/mlbench/mlbench/issues/1)
\* *This Change Log was automatically generated by [github_changelog_generator](https://github.com/skywinder/Github-Changelog-Generator)*
mlbench_core: Distributed Machine Learning Benchmark
====================================================
.. image:: https://travis-ci.com/mlbench/mlbench.svg?branch=develop
:target: https://travis-ci.com/mlbench/mlbench
.. image:: https://readthedocs.org/projects/mlbench/badge/?version=latest
:target: https://mlbench.readthedocs.io/en/latest/?badge=latest
:alt: Documentation Status
A public and reproducible collection of reference implementations and benchmark suite for distributed machine learning algorithms, frameworks and systems.
* Project website: https://mlbench.github.io/
* Free software: Apache Software License 2.0
* Documentation: https://mlbench.readthedocs.io.
Features
--------
* For reproducibility and simplicity, we currently focus on standard **supervised ML**, including standard deep learning tasks as well as classic linear ML models.
* We provide **reference implementations** for each algorithm, to make it easy to port to a new framework.
* Our goal is to benchmark all/most currently relevant **distributed execution frameworks**. We welcome contributions of new frameworks in the benchmark suite.
* We provide **precisely defined tasks** and datasets to have a fair and precise comparison of all algorithms, frameworks and hardware.
* Independently of all solver implementations, we provide universal **evaluation code** allowing to compare the result metrics of different solvers and frameworks.
* Our benchmark code is easy to run on **public clouds**.
Community
---------
About us: See :doc:`Authors </authors>`
Mailing list: https://groups.google.com/d/forum/mlbench
Contact Email: mlbench-contact@googlegroups.com
# Change Log
## [1.0.0](https://github.com/mlbench/mlbench-core/tree/1.0.0) (2018-11-15)
**Implemented enhancements:**
- Add API Client to mlbench-core [\#6](https://github.com/mlbench/mlbench-core/issues/6)
- Move to google-style docs [\#4](https://github.com/mlbench/mlbench-core/issues/4)
- Add Imagenet Dataset for pytorch [\#3](https://github.com/mlbench/mlbench-core/issues/3)
- Move worker code to mlbench-core repo [\#1](https://github.com/mlbench/mlbench-core/issues/1)
# Change Log
## [0.1.0](https://github.com/mlbench/mlbench/tree/0.1.0) (2018-09-14)
**Implemented enhancements:**
- Add documentation in reference implementation to docs [\#46](https://github.com/mlbench/mlbench/issues/46)
- Replace cAdvisor with Kubernetes stats for Resource usage [\#38](https://github.com/mlbench/mlbench/issues/38)
- Rename folders [\#31](https://github.com/mlbench/mlbench/issues/31)
- Change docker image names [\#30](https://github.com/mlbench/mlbench/issues/30)
- Add continuous output for mpirun [\#27](https://github.com/mlbench/mlbench/issues/27)
- Replace SQlite with Postgres [\#25](https://github.com/mlbench/mlbench/issues/25)
- Fix unittest [\#23](https://github.com/mlbench/mlbench/issues/23)
- Add/Fix CI/Automated build [\#22](https://github.com/mlbench/mlbench/issues/22)
- Cleanup unneeded project files [\#21](https://github.com/mlbench/mlbench/issues/21)
- Remove hardcoded values [\#20](https://github.com/mlbench/mlbench/issues/20)
- Improves Notes.txt [\#19](https://github.com/mlbench/mlbench/issues/19)
- Rename components [\#15](https://github.com/mlbench/mlbench/issues/15)
**Fixed bugs:**
- 504 Error when downloading metrics for long runs [\#61](https://github.com/mlbench/mlbench/issues/61)
**Closed issues:**
- small doc improvements for first release [\#54](https://github.com/mlbench/mlbench/issues/54)
- Check mlbench works on Google Cloud [\#51](https://github.com/mlbench/mlbench/issues/51)
- learning rate scheduler [\#50](https://github.com/mlbench/mlbench/issues/50)
- Add Nvidia k8s-device-plugin to charts [\#48](https://github.com/mlbench/mlbench/issues/48)
- Add Weave to Helm Chart [\#41](https://github.com/mlbench/mlbench/issues/41)
- Allow limiting of resources for experiments [\#39](https://github.com/mlbench/mlbench/issues/39)
- Allow downloading of Run measurements [\#35](https://github.com/mlbench/mlbench/issues/35)
- Worker Details page [\#33](https://github.com/mlbench/mlbench/issues/33)
- Run Visualizations [\#32](https://github.com/mlbench/mlbench/issues/32)
- Show experiment history in Dashboard [\#18](https://github.com/mlbench/mlbench/issues/18)
- Show model progress in Dashboard [\#13](https://github.com/mlbench/mlbench/issues/13)
- Report cluster status in Dashboard [\#12](https://github.com/mlbench/mlbench/issues/12)
- Send metrics from SGD example to metrics api [\#11](https://github.com/mlbench/mlbench/issues/11)
- Add metrics endpoint for experiments [\#10](https://github.com/mlbench/mlbench/issues/10)
- Let Coordinator Dashboard start a distributed Experiment [\#9](https://github.com/mlbench/mlbench/issues/9)
- Add mini-batch SGD model experiment [\#8](https://github.com/mlbench/mlbench/issues/8)
- add benchmark code for MPI [\#7](https://github.com/mlbench/mlbench/issues/7)
- add benchmark code for tensorflow [\#6](https://github.com/mlbench/mlbench/issues/6)
- add benchmark code for apache reef [\#5](https://github.com/mlbench/mlbench/issues/5)
- add benchmark code for apache flink [\#4](https://github.com/mlbench/mlbench/issues/4)
- get initial benchmark numbers \(spark reference implementation and mllib/ml\) [\#3](https://github.com/mlbench/mlbench/issues/3)
- evaluate script \(framework-independent\) and algorithm output format [\#2](https://github.com/mlbench/mlbench/issues/2)
- bench-spark: remove prepare-data for now, comment on solver prequisites [\#1](https://github.com/mlbench/mlbench/issues/1)
\* *This Change Log was automatically generated by [github_changelog_generator](https://github.com/skywinder/Github-Changelog-Generator)*
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
mlbench_core-1.0.0.tar.gz
(41.3 kB
view details)
Built Distribution
File details
Details for the file mlbench_core-1.0.0.tar.gz.
File metadata
- Download URL: mlbench_core-1.0.0.tar.gz
- Upload date:
- Size: 41.3 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.12.1 pkginfo/1.4.2 requests/2.20.0 setuptools/40.4.3 requests-toolbelt/0.8.0 tqdm/4.26.0 CPython/3.6.7
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | ec334d355cc15a724be3d4e387b54f3a3736e85f0c5fdef6204f8bb6dc3b37cf |
|
| MD5 | d5916e1801eb69053973cc764773dfe3 |
|
| BLAKE2b-256 | aa3efea82d52cfaa170c1500767e39c9b74e3138396e905f611a04b195e2f926 |
File details
Details for the file mlbench_core-1.0.0-py3-none-any.whl.
File metadata
- Download URL: mlbench_core-1.0.0-py3-none-any.whl
- Upload date:
- Size: 75.0 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.12.1 pkginfo/1.4.2 requests/2.20.0 setuptools/40.4.3 requests-toolbelt/0.8.0 tqdm/4.26.0 CPython/3.6.7
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 093c233f8d09af14d8be6e93932de93cba9881e22b2dd6b6200820bcc53f374b |
|
| MD5 | 19aaa5dabb26a8055ea979c1836a6a70 |
|
| BLAKE2b-256 | f1853bf1b0119f77975b38dc78bcb7be071ea90bb66267d016f09fdfbd96f479 |
