planemo 0.4.0

===============================
Planemo
===============================

.. image:: https://travis-ci.org/galaxyproject/planemo.png?branch=master
:target: https://travis-ci.org/galaxyproject/planemo

.. image:: https://readthedocs.org/projects/pip/badge/?version=latest
:target: https://planemo.readthedocs.org.

.. .. image:: https://badge.fury.io/py/planemo.png
.. :target: http://badge.fury.io/py/planemo
.. .. image:: https://pypip.in/d/planemo/badge.png
.. :target: https://pypi-hypernode.com/pypi/planemo


Command-line utilities to assist in building tools for the Galaxy_ project.

* Free software: Academic Free License version 3.0
* Documentation: https://planemo.readthedocs.org.
* Code: https://github.com/galaxyproject/planemo

Quick Start
-----------

This quick start demonstrates using ``planemo`` commands to help
develop Galaxy tools. Planemo can quickly be installed via
Homebrew_ or as a more traditional Python project.

To install using Homebrew_ or linuxbrew_:

::

brew tap galaxyproject/tap
brew install planemo

For a more traditional Python installation simply setup a virtualenv
for ``planemo`` (this example creates a new one in ``.venv``) and then
install with ``pip``.

::

% virtualenv .venv; . .venv/bin/activate
% pip install planemo

Planemo is also available a `virtual appliance
<https://planemo.readthedocs.org/en/latest/appliance.html>`_ for Docker_ or Vagrant_ (bundled
with a preconfigured Galaxy server optimized for tool development).

This quick start will assume you will have a directory with one or more
tool XML files. If none is available, one can be quickly create for
demonstrating ``planemo`` as follows ``mkdir mytools; cd mytools; planemo
project_init --template=demo``.

Planemo can check tools containing XML for common problems and best
practices using the ``lint`` `command <http://planemo.readthedocs.org/en/latest/commands.html#lint-command>`_
(also aliased as ``l``). ::

% planemo lint
...

Like many ``planemo`` commands - by default this will search the
current directory and use all tool files it finds. It can be explicitly
passed other tool files or a directory of tool files. ::

% planemo l randomlines.xml

The ``lint`` command takes in a additional options related to
reporting levels, exit code, etc.... These options are descirbed here
or (like all available commands) be accessed by passing it ``--help``.::

% planemo l --help
Usage: planemo lint [OPTIONS] TOOL_PATH
...

Once tools are syntically correct - it is time to test. The ``test``
`command <http://planemo.readthedocs.org/en/latest/commands.html#test-command>`_
can be used to test a tool or directory of tools.::

% planemo test --galaxy_root=../galaxy-central randomlines.xml

If no ``--galaxy_root`` is defined, ``planemo`` will check for a default in
`~/.planemo.yml
<http://planemo.readthedocs.org/en/latest/configuration.html>`_) and finally
search the tool's parent directories for a Galaxy root directory (developing
tools under Galaxy ``tools`` directory is a common development workflow).
Planemo can also download and configure a disposable Galaxy instance just for
testing by passing it ``-install_galaxy`` instead of a Galaxy root.::

% planemo t --install_galaxy

**Warning**: The features of Planemo that require a ``--galaxy_root`` will
only work with the latest ``galaxy-central`` ``default`` branch. Planemo will
not work when used with older versions of Galaxy - even the latest stable
``latest_2014.10.06``. Planemo will stablize with the next release of Galaxy
and serious attempts at backward compatibility going forward will be made at
that time.

Planemo will create a HTML an output report in the current directory named
``tool_test_output.html`` (override with ``--test_output``). `Here <http://galaxyproject.github.io/planemo/tool_test_viewer.html?test_data_url=https://gist.githubusercontent.com/jmchilton/9d4351c9545d34209904/raw/9ed285d3cf98e435fc4a743320363275949ad63c/index>`_ is an
example of such a report for Tophat.

Once tools have been linted and tested - the tools can be viewed in a
Galaxy interface using the ``serve`` (``s``) `command
<http://planemo.readthedocs.org/en/latest/commands.html#serve-command>`_.::

% planemo serve

Like ``test``, ``serve`` requires a Galaxy root and one can be
explicitly specified with ``--galaxy_root`` or installed dynamically
with ``--install_galaxy``.

Finally, when tools are ready to be published to GitHub_, it may be valuable
to setup contineous integration to test changes committed and new pull
requests. `Travis CI <http://travis-ci.org/>`_ is a service providing free
testing and deep integration with GitHub_.

The ``travis_init`` `command
<http://planemo.readthedocs.org/en/latest/commands.html#travis_init-command>`_
will bootstrap a project with files to ease contineous inegration of tools
using a Planemo driven approach inspired by this great `blog post
<http://bit.ly/gxtravisci>`_ by `Peter Cock <https://github.com/peterjc>`_.

::

% planemo travis_init .
% # setup Ubuntu 12.04 w/tool dependencies
% vim .travis/setup_custom_dependencies.bash
% git add .travis.yml .travis
% git commit -m "Add Travis CI testing infrastructure for tools."
% git push # and register repository @ http://travis-ci.org/

Experimental Features
---------------------

While Planemo is very experimental itself - it can also be used to explore
some more experimental features related to Galaxy tooling - including support
for Docker and Brew.

-----------
Docker
-----------

Galaxy has `experimental support
<https://wiki.galaxyproject.org/Admin/Tools/Docker>`_ for running jobs in
Docker_ containers. Planemo contains tools to assist in development of Docker
images for Galaxy tools.

A shell can be launched to explore the Docker enviornment referenced by tools
that are annotated with publically registered Docker images.::

% $(planemo docker_shell bowtie2.xml)

For Docker containers still in development - a Dockerfile can be associated
with a tool by sticking it in the tool's directory. Planemo can then build
and tag a Docker image for this tool and launch a shell into it using the
following commands.::

% planemo docker_build bowtie2.xml # asssumes Dockerfile in same dir
% $(planemo docker_shell --from_tag bowtie2.xml)

For more details see the documentation for the `docker_build
<http://planemo.readthedocs.org/en/latest/commands.html#docker_build-command>`_
and `docker_shell
<http://planemo.readthedocs.org/en/latest/commands.html#docker_shell-command>`_
commands.

-----------
Brew
-----------

The Galaxy development team is exploring different options for integrating
Homebrew_ and linuxbrew_ with Galaxy. One angle is resolving the tool requirements
using ``brew``. An experimental approach for versioning of brew recipes will be
used. See full discussion on the homebrew-science issues page here -
https://github.com/Homebrew/homebrew-science/issues/1191. Information on the
implementation can be found https://github.com/jmchilton/platform-brew until a
more permanent project home is setup.

::

% planemo brew_init # install linuxbrew (only need if not already installed)
% planemo brew # install dependencies for all tools in directory.
% planemo brew bowtie2.xml # install dependencies for one tool
% which bowtie2
bowtie2 not found
% . <(planemo brew_env --shell bowtie2.xml) # shell w/brew deps resolved
(bowtie2) % which bowtie2
/home/john/.linuxbrew/Cellar/bowtie2/2.1.0/bin/bowtie2
(bowtie2) % exit
% . <(planemo brew_env bowtie2.xml) # or just source deps in cur env
% which bowtie2
/home/john/.linuxbrew/Cellar/bowtie2/2.1.0/bin/bowtie2

For more information see the documentation for the `brew
<http://planemo.readthedocs.org/en/latest/commands.html#brew-command>`_
and `brew_env
<http://planemo.readthedocs.org/en/latest/commands.html#brew_env-command>`_ commands.

.. _Galaxy: (http://galaxyproject.org/)
.. _GitHub: https://github.com/
.. _Docker: https://www.docker.com/
.. _Homebrew: http://brew.sh/
.. _linuxbrew: https://github.com/Homebrew/linuxbrew
.. _Vagrant: https://www.vagrantup.com/




History
-------

---------------------
0.4.0 (2015-02-16)
---------------------

* Implement ``tool_init`` command for bootstrapping creation of new
tools (with `tutorial <http://planemo.readthedocs.org/en/latest/writing.html>`_.) 78f8274_
* Implement ``normalize`` command for reorganizing tool XML and macro
debugging. e8c1d45_
* Implement ``tool_factory`` command to spin up Galaxy pre-configured the
`Tool Factory
<http://bioinformatics.oxfordjournals.org/content/early/2012/09/27/bioinformatics.bts573.full.pdf>`_. 9e746b4_
* Added basic linting of ``command`` blocks. b8d90ab_
* Improved linting of ``help`` blocks, including verifying valid
`reStructuredText`. 411a8da_
* Fix bug related to ``serve`` command not killing Galaxy properly when complete. 53a6766_
* Have ``serve`` command display tools at the top level instead of in shallow sections. badc25f_
* Add additional dependencies to ``setup.py`` more functionality works out
of the box. 85b9614_
* Fix terrible error message related to ``bioblend`` being unavailable.
`Issue 70`_
* Various smaller documentation and project structure improvements.

---------------------
0.3.1 (2015-02-15)
---------------------

* Fixes to get PyPI workflow working properly.

---------------------
0.3.0 (2015-02-13)
---------------------

* Add option (``-r``) to the ``shed_upload`` command to recursively upload
subdirectories (thanks to Eric Rasche). `Pull Request 68`_
* Fix diff formatting in test reports (thanks to Eric Rasche).
`Pull Request 63`_
* Grab updated test database to speed up testing (thanks to approach from
Eric Rasche and Dannon Baker). `Issue 61`_, dff4f33_
* Fix test data command-line argument name (was ``test-data`` now it is
``test_data``). 834bfb2_
* Use ``tool_data_table_conf.xml.sample`` file if
``tool_data_table_conf.xml.test`` is unavailable. Should allow some
new tools to be tested without modifying Galaxy's global
``tool_data_table_conf.xml`` file. ac4f828_

---------------------
0.2.0 (2015-01-13)
---------------------

* Improvements to way Planemo loads its own copy of Galaxy modules to prevent
various conflicts when launching Galaxy from Planemo. `Pull Request 56`_
* Allow setting various test output options in ``~/.planemo.yml`` and disabling
JSON output. 21bb463_
* More experimental Brew and Tool Shed options that should not be considered
part of Planemo's stable API. See bit.ly/gxbrew1 for more details.
* Fix ``project_init`` for BSD tar (thanks to Nitesh Turaga for the bug
report.) a4110a8_
* Documentation fixes for tool linting command (thanks to Nicola Soranzo).
`Pull Request 51`_

---------------------
0.1.0 (2014-12-16)
---------------------

* Moved repository URL to https://github.com/galaxyproject/planemo.
* Support for publishing to the Tool Shed. `Pull Request 6`_
* Support for producing diffs (``shed_diff``) between local repositories and
the Tool Shed (based on scripts by Peter Cock). `Pull Request 33`_
* Use tool's local test data when available - add option for configuring
``test-data`` target. `Pull Request 1`_
* Support for testing tool features dependent on cached data. 44de95c_
* Support for generating XUnit tool test reports. 82e8b1f_
* Prettier HTML reports for tool tests. 05cc9f4_
* Implement ``share_test`` command for embedding test result links in pull
requests. `Pull Request 40`_
* Fix for properly resolving links during Tool Shed publishing (thanks to Dave
Bouvier). `Pull Request 29`_
* Fix for citation linter (thanks to Michael Crusoe for the bug report). af39061_
* Fix tool scanning for tool files with fewer than 10 lines (thanks to Dan
Blankenberg). a2c13e4_
* Automate more of Travis CI testing so the scripts added to tool repository
can be smaller. 20a8680_
* Documentation fixes for Travis CI (thanks to Peter Cock). `Pull Request 22`_,
`Pull Request 23`_
* Various documentation fixes (thanks to Martin Čech). 36f7cb11_, b9232e55_
* Various smaller fixes for Docker support, tool linting, and documentation.

---------------------
0.0.1 (2014-10-04)
---------------------

* Initial work on the project - commands for testing, linting, serving Galaxy
tools - and more experimental features involving Docker and Homebrew. 7d07782_

.. _Issue 70: https://github.com/galaxyproject/planemo/issues/70
.. _Pull Request 68: https://github.com/galaxyproject/planemo/pull/68
.. _Issue 61: https://github.com/galaxyproject/planemo/issues/61
.. _Pull Request 63: https://github.com/galaxyproject/planemo/pull/63
.. _Pull Request 56: https://github.com/galaxyproject/planemo/pull/56
.. _Pull Request 51: https://github.com/galaxyproject/planemo/pull/51
.. _Pull Request 40: https://github.com/galaxyproject/planemo/pull/40
.. _Pull Request 29: https://github.com/galaxyproject/planemo/pull/29
.. _Pull Request 22: https://github.com/galaxyproject/planemo/pull/22
.. _Pull Request 23: https://github.com/galaxyproject/planemo/pull/23
.. _Pull Request 33: https://github.com/galaxyproject/planemo/pull/33
.. _Pull Request 6: https://github.com/galaxyproject/planemo/pull/6
.. _Pull Request 1: https://github.com/galaxyproject/planemo/pull/1

.. _85b9614: https://github.com/galaxyproject/planemo/85b961465f46351507f80ddc3758349535060502
.. _53a6766: https://github.com/galaxyproject/planemo/commit/53a6766cdebdddc976189f6dc6a264bb4105c4bf
.. _badc25f: https://github.com/galaxyproject/planemo/commit/badc25fca495b61457ffb2e027f3fe9cf17c798f
.. _411a8da: https://github.com/galaxyproject/planemo/commit/411a8da21c92ba37c7ad95bfce9928d9b8fd998e
.. _b8d90ab: https://github.com/galaxyproject/planemo/commit/b8d90abab8bf53ae2e7cca4317223c01af9ab68c
.. _e8c1d45: https://github.com/galaxyproject/planemo/commit/e8c1d45f0c9a11bcf69ec2967836c3b8f432dd97
.. _78f8274: https://github.com/galaxyproject/planemo/commit/78f82747996e4a28f96c85ad72efe5e54c8c74bd
.. _9e746b4: https://github.com/galaxyproject/planemo/commit/9e746b455e3b15219878cddcdeda722979639401
.. _ac4f828: https://github.com/galaxyproject/planemo/commit/ac4f82898f7006799142503a33c3978428660ce7
.. _834bfb2: https://github.com/galaxyproject/planemo/commit/834bfb2929d367892a3abe9c0b88d5a0277d7905
.. _dff4f33: https://github.com/galaxyproject/planemo/commit/dff4f33c750a8dbe651c38e149a26dd42e706a82
.. _a4110a8: https://github.com/galaxyproject/planemo/commit/a4110a85a770988e5cd3c31ccc9475717897d59c
.. _21bb463: https://github.com/galaxyproject/planemo/commit/21bb463ad6c321bcb669603049a5e89a69766ad9
.. _af39061: https://github.com/galaxyproject/planemo/commit/af390612004dab636d8696839bb723d39f97c85d
.. _20a8680: https://github.com/galaxyproject/planemo/commit/20a86807cb7ea87db2dbc0197ae08a40df3ab2bc
.. _44de95c: https://github.com/galaxyproject/planemo/commit/44de95c0d7087a5822941959f9a062f6382e329b
.. _82e8b1f: https://github.com/galaxyproject/planemo/commit/82e8b1f17eae526aeb341cb4fffb8d09d73bb419
.. _05cc9f4: https://github.com/galaxyproject/planemo/commit/05cc9f485ee87bc344e3f43bb1cfd025a16a6247
.. _32c6e7f: https://github.com/galaxyproject/planemo/commit/32c6e7f78bb8f04d27615cfd8948b0b89f27b4e6
.. _7d07782: https://github.com/galaxyproject/planemo/commit/7d077828559c9c9c352ac814f9e3b86b1b3a2a9f
.. _a2c13e4: https://github.com/galaxyproject/planemo/commit/a2c13e46259e3be35de1ecaae858ba818bb94734
.. _36f7cb11: https://github.com/galaxyproject/planemo/commit/36f7cb114f77731f90860d513a930e10ce5c1ba5
.. _b9232e55: https://github.com/galaxyproject/planemo/commit/b9232e55e713abbd1d9ce8b0b34cbec6c701dc17