polyvers 0.0.2a10

1. Install the tool

And you get the polyvers command:

$ pip install polyvers
...
$ polyvers --version
0.0.0
$ polyvers --help
...

$ polyvers status
polyvers: Neither `setup.py` nor `.polyvers(.json|.py|.salt)` config-files found!

2. Initialize project

Assuming our monorepo project /monorepo.git/ contains two sub-projects:

/monorepo.git/
    +--setup.py:  setup(name='mainprog', ...)
    +--mainprog/__init__.py
    +--...
    +--core-lib/
        +--setup.py: setup(name='core', ...)
        +--core/__init__.py
        +--...

…we let the tool auto-discover the mapping of project folders ↔ project-names and create a traitlets configuration YAML-file named as /monorepo.git/.polyvers.py:

$ cd monorepo.git

$ polyvers init --monorepo
Created new config-file '.polyvers.yaml'.

$ cat .polyvers.yaml
...
PolyversCmd:
  projects:
  - pname: mainprog     # name extracted from `setup.py`.
    basepath: .         # path discovered by the location of `setup.py`
  - pname: core
    basepath: core-lib
...

$ git add .polyvers.yaml
$ git commit -m 'add polyvers config-gile'

And now we can use the polyvers command to inspect the versions of all sub-projects:

$ polyvers status
- mainprog
- core

Indeed there are no tags in in git-history for the tool to derive and display project-versions, so only project-names are shown. With --all option more gets displayed:

$ polyvers status -a
- pname: mainprog
  basepath: .
  gitver:
  history: []
- pname: core
  basepath: core-lib
  gitver:
  history: []

..where gitver would be the result of git-describe.

3. Bump versions

We can now use tool to set the same version to all sub-projects:

$ polyvers bump 0.0.0 -f noengraves   # all projects implied, if no project-name given
00:52:06       |WARNI|polyvers.bumpcmd.BumpCmd|Ignored 1 errors while checking if at least one version-engraving happened:
  ignored (--force=noengraves): CmdException: No version-engravings happened, bump aborted.
00:52:07       |NOTIC|polyvers.bumpcmd.BumpCmd|Bumped projects: mainprog-0.0.0 --> 0.0.0, core-0.0.0 --> 0.0.0

The --force=noengraves disables a safety check that requires at least one file modification for engraving the current version in the leaf “Release” commit (see next step).

 $ polyvers status
 - mainprog-v0.0.0
 - core-v0.0.0

 $ git lg    # Ok, augmented `lg` output a bit here...HEAD --> UPPER branch.
 COMMITS BRANCH TAGS                 REMARKS
 ======= ====== ==================== ========================================
      O  latest mainprog-r0.0.0      - x2 tags on "Release" leaf-commit
     /          core-r0.0.0            outside-of-trunk (not in HEAD).
    O    MASTER mainprog-v0.0.0      - x2 tags on "Version" commit
    |           core-v0.0.0            for bumping both projects to v0.0.0
    O                                - Previous commit, before version bump.

.. Hint::
   Note the difference between ``ABC-v0.0.0`` vs ``ABC-r0.0.0`` tags.

In the source code, it's only the "release" commit that has *engraved* version-ids:

.. code-block:: console

 $ cat mainprog/mainprog/__init__.py    # Untouched!
 import polyvers

 __title__     = "mainprog"
 __version__ = polyvers.version('mainprog')
 ...

 $ git checkout  latest
 $ cat mainprog/mainprog/__init__.py
 import polyvers

 __title__     = "mainprog"
 __version__ = '0.0.0'
 ...

 $ git checkout  -  # to return to master.

4. Engrave version in the sources

Usually programs report their version somehow when run, e.g. with `cmd --version. With polyvers we can derive the latest from the tags created in the previous step, using a code like this, usually in the file /mainprog/mainprog/__init__.py::

import polyvers

__title__ = "mainprog"
__version__ = polyvers.version('mainprog')
...

…and respectively /core-lib/core/__init__.py::

__version__ = polyvers.version('core')

5. Bump sub-projects selectively

Now let’s add another dummy commit and then bump ONLY ONE sub-project:

$ git commit  --allow-empty  -m "some head work"
$ polyvers bump ^1 mainprog
00:53:07       |NOTIC|polyvers.bumpcmd.BumpCmd|Bumped projects: mainprog-0.0.0 --> 0.0.1

$ git lg
COMMITS BRANCH TAGS                 REMARKS
======= ====== ==================== ========================================
     O  latest mainprog-r0.0.1.dev0 - The latest "Release" leaf-commit.
    /                                 branch `latest` was reset non-ff.
   O    MASTER mainprog-v0.0.1.dev0 - The latest "Version" commit.
   O                                - some head work
   | O         mainprog-r0.0.0      - Now it's obvious why "Release" commits
   |/          core-r0.0.0            are called "leafs".
   O           mainprog-v0.0.0
   |           core-v0.0.0
   O

$ git checkout latest
$ cat mainprog/mainprog/__init__.py
import polyvers

__title__     = "mainprog"
__version__ = '0.0.1.dev0'
...

$ cat core/core/__init__.py
import polyvers

__title__ = "core"
__version__ = '0.0.0+mainprog.0.0.1.dev0'
...
$ git checkout -

Notice how the the “local” part of PEP-440 (statring with +...) is used by the engraved version of the un-bumped core project to signify the correlated version of the bumped mainprog. This trick is uneccesary for tags because they apply repo-wide, to all sub-projects.

PEP 440 version ids

While most versioning tools use Semantic versioning, python’s distutils native library supports the quasi-superset, but more versatile, PEP-440 version ids, like that:

• Pre-releases: when working on new features:

  X.YbN               # Beta release
  X.YrcN  or  X.YcN   # Release Candidate
  X.Y                 # Final release

• Post-release:

  X.YaN.postM         # Post-release of an alpha release
  X.YrcN.postM        # Post-release of a release candidate

• Dev-release:

  X.YaN.devM          # Developmental release of an alpha release
  X.Y.postN.devM      # Developmental release of a post-release

Monorepos

When your single project succeeds, problems like these are known only too well:

Changes in web-server part depend on core features that cannot go public because the “official” wire-protocol is freezed.

While downstream projects using core as a library complain about its bloated transitive dependencies (asking why flask library is needed??).

So the time to “split the project has come. But from lerna:

Splitting up large codebases into separate independently versioned packages is extremely useful for code sharing. However, making changes across many repositories is messy and difficult to track, and testing across repositories gets complicated really fast.

So a monorepo [1] [2] is the solution. But as Yarn put it:

OTOH, splitting projects into their own folders is sometimes not enough. Testing, managing dependencies, and publishing multiple packages quickly gets complicated and many such projects adopt tools such as …

Polyvers is such a tool.

Out-of-trunk (leaf) “Release” commit

Even in single-project repos, sharing code across branches may cause merge-conflicts due to the version-ids “engraved” in the sources. In monorepos, the versions proliferate, and so does the conflicts.

Contrary to similar tools, static version-ids are engraved only in out-of-trunk (leaf) commits, and only when the sub-projects are released. In-trunk code is never touched, and version-ids are reported, on runtime, based on Git tags (like git-describe), so they are always up-to-date.

Marking dependent versions across sub-projects

TODO: When bumping the version of a sub-project the “local” part of PEP-440 on all other the dependent sub-projects in the monorepo signify their relationship at the time of the bump.

Lock release trains as “developmental”

TODO: Specific branches can be selected always to be published into PyPi only as PEP-440’s “Developmental” releases, meanining that users need pip install --pre to install from such release-trains. This is a safeguard to avoid accidentally landing half-baked code to users.

Other Features

• Highly configurable using traitlets, with sensible defaults; it’s possible to run without any config file in single-project repos.

• Always accurate version reported on runtime when run from git repos (never again wonder with which version your experimental-data were produced).

Features TODO

Known Limitations, Drawbacks & Workarounds

• PEP440 Epoch handling is not yet working.

• Version-bump’s grammar is not yet as described in “GRAMMAR” section of command’s doc:

  $ polyvers config desc --class BumpCmd
  BumpCmd(_SubCmd)
  ----------------
  Increase or set the version of project(s) to the (relative/absolute) version.
  SYNTAX:
      polyvers config desc [OPTIONS] <version> [<project>]...
  - If no project(s) specified, increase the versions on all projects.
  - Denied if version for some projects is backward-in-time (or has jumped parts?);
    use --force if you might.
  VERSION: - A version specifier, either ABSOLUTE, or RELATIVE to the current
  version og each project:
    - *ABSOLUTE* PEP-440 version samples:
      - Pre-releases: when working on new features:
          X.YbN               # Beta release
          X.YrcN  or  X.YcN   # Release Candidate
          X.Y                 # Final release
  ...

• WARNING: when you build your package for distribution (wheel, correct?) remember to switch to the out-of-trunk (leaf) “Release” commit. This is particularly important if your setup.py file use polyversion() to derive its version.. Because if it fails for whatever reason (git command is missing, project not located in a git-repo, miss-configuration, etc).

  Check also that if you provide a default argument to facilitate development, then you may actually build a package(wheel, ok?) with that “default” version. So, always check you package’s version before uploading it to pypi.

• (not related to this tool) In setup.py script, the kw-argument package_dir={'': <sub-dir>} arg is needed for py_modules to work when packaging sub-projects (also useful with find_packages(), check this project’s sources). But <sub-dir> must be relative to launch cwd, or else, pip install -e <subdir> and/or python setup.py develop break.

• (not related to this tool) When building projects with python setup.py bdist_wheel, you have to clean up your build directory, or else, the distribution package will contain the sources from all previous subprojects. That applies also when rebuilding a project between versions.

• (not related to this tool) If you don’t place a setup.py file at the root of your git-repo, then it becomes more cumbersome to pip install directly from remote URLs, like this:

  pip install -e git+https://repo_url/#egg=pkg&subdirectory=pkg_dir

  You may use package_dir argument to setup() function (see setuptools-docs).

• Set branch latest as default in GitHub to show engraved sub-project version-ids.

0.0.2a10 (2018-05-24): polyvers

• fix: slight change of default engraving for setup.py:version=....

• Remove default versions from the sources of our-own-dog-food (affects installations for developing this tool).

• refact: merged `pvlib.whl and pvlib.run into a single executable and importable standalone wheel in bin/pvlib.run, generated from polyversion-0.0.2a9, release below.

• doc: expand section for installing and contributing into this project.

• chore: tighten various test harnesses.

0.0.2a9.post0 (2018-05-23): polyvers

• feat: add -C option to change project dir before running command.

• init command:

  • fix: were creating invalid .polyvers.yaml configuration-file unless --monorepo/--mono-project flags were given.

  • feat: include config-help in generated file only if the new --doc flag given.

  • feat: inform user of the projects auto-discovered and what type of config-file was generated.

• various fixes.

0.0.2a8 (2018-05-19): polyvers

• FIX(bump): was engraving all projects and not limiting to those specified in the command-line - command’s syntax slightly changed.

• chore: Stop increasing polyversion version from now on.

• doc: fix all sphinx errors and API reference.

0.0.2a6 (2018-05-18)

• bump command:

  • feat: --amend now works

  • feat: --engrave-only.

  • feat: log PRETEND while doing actions.

  • feat: Log which files where engraved in the final message.

• fix(engrave): don’t waste cycles/log-messages on empty-matches (minor).

0.0.2a5 (2018-05-18)

Actually most changes happened in “interim” release v0.0.2a2, below.

• feat: make a standalone polyversion-lib wheel to facilitate bootstrap when installing & building from sources (and the lib is not yet installed).

• Add bin/package.sh that create the pvlib wheel as executable dist/pvlib.run.

• doc: fix rtd & pypi sites.

0.0.2a1 (2018-05-17)

• 2nd release, own “mono-project” splitted into 2-project “monorepo”: - polyvers: cmdline tool - polyversion: library code for program-sources to derive version from git-tags

• init, status, bump and config commands work.

• Read/write YAML config file .polyvers.yaml at the git-root, and can automatically discover used configuration (from existing git tags or projects files).

• Support both --monorepo and --mono-project configurations.

• By default __init__.py, setup.py and README.rst files are engraved with bumped version.

0.0.1a0 (2018-01-29)

• First release on PyPI as mono-project