Skip to main content

Run on multiple platforms

Project description

PyPI version supported Python versions Azure build status codecov coverage status source on GitHub

Run on multiple platforms

I use pip-tools but also want to keep my compiled requirements for multiple platforms. Instead of sharing directories between machines or otherwise shuttling files around I chose to create romp which will let me submit arbitrary work to Azure Pipelines to get access to multiple platforms. Personally, I will use this primarily behind boots.

Below is an example usage. It will run a single job under Linux with 64-bit CPython 3.6. The job will execute echo red > blue.txt and will collect *.txt from the job and save it locally to artifacts.tar.gz.

$ venv/bin/romp --platform linux --interpreter cpython --version 3.6 --architecture x86_64 --command 'echo red > blue.txt' --artifact-paths '*.txt' --artifact artifacts.tar.gz
Requesting build
Waiting for build: https://dev.azure.com/altendky/f1722f91-62fe-4a15-8937-252c96b31292/_build/results?buildId=2938
Handling artifact

Since romp leverages Azure Pipelines to get access to all the platforms, server-side setup is required. You will need an Azure account and to create a pipeline (build) within that account using a romp repository (official or your own fork). Configure the pipeline to use deployed_ci/azure/azure-pipelines-lock.yml. It is configured such that commits do not trigger builds. romp’s own CI and testing is driven by azure-pipelines.yml in the project root but this file can be ignored for regular use. Once the pipeline is setup you will need to create a Personal Access Token (PAT) to use for authentication when running romp. The PAT will need the build read and execute scope enabled.

For local setup, the command line options can be set by environment variables. For many options this will not make sense but for a few it will. Specifically consider the follow options which will often be consistent for all calls.

  • ROMP_BUILD_REQUEST_URL
    • 'https://dev.azure.com/altendky/romp-on/_apis/build/builds?api-version=5.0'

  • ROMP_DEFINITION_ID
    • 5

  • ROMP_PERSONAL_ACCESS_TOKEN
    • fp6al3jxta2zliz6rh5hr6ewd5nw2hsmasse2laiyoyg7otneqjq

  • ROMP_USERNAME
    • altendky

$ venv/bin/romp --help
Usage: romp [OPTIONS]

Options:
  --personal-access-token, --pat TEXT
                                  A personal access token (PAT) with rights to
                                  initiate builds
                                  ($ROMP_PERSONAL_ACCESS_TOKEN)
  --build-request-url TEXT        The URL for submitting a build request
                                  ($ROMP_BUILD_REQUEST_URL)  [default: https:/
                                  /dev.azure.com/altendky/romp/_apis/build/bui
                                  lds?api-version=5.0]
  --command TEXT                  The command to be run for each target
                                  ($ROMP_COMMAND)  [default: python -c 'import
                                  sys; print(sys.version);
                                  print(sys.platform)']
  --username TEXT                 Username for build URL authentication
                                  ($ROMP_USERNAME)  [default: altendky]
  --environments TEXT             Targets to run on.  Mostly use the matrix
                                  options instead.  This may be removed.
                                  ($ROMP_ENVIRONMENTS)
  --check-period INTEGER          The period used to poll the build for
                                  completion ($ROMP_CHECK_PERIOD)  [default:
                                  15]
  --source-branch TEXT            The romp source branch to use for the build
                                  ($ROMP_SOURCE_BRANCH)  [default: develop]
  --definition-id INTEGER         The definition id of the build to be
                                  triggered ($ROMP_DEFINITION_ID)  [default:
                                  3]
  --archive-file FILENAME         The archive to be uploaded to the build
                                  ($ROMP_ARCHIVE)
  --artifact FILENAME             The path at which to save the resulting
                                  artifact ($ROMP_ARTIFACT_PATH)
  --artifact-paths TEXT           Paths on remote system to build the artifact
                                  archive from.  Wildcards are supported via
                                  bash. ($ROMP_ARTIFACT_PATHS)
  --platform [Linux|macOS|Windows]
                                  Platforms to matrix across
                                  ($ROMP_MATRIX_PLATFORMS)  [default: Linux,
                                  macOS, Windows]
  --interpreter [CPython|PyPy]    Interpreters to matrix across
                                  ($ROMP_MATRIX_INTERPRETERS)  [default:
                                  CPython, PyPy]
  --version [2.7|3.4|3.5|3.6|3.7]
                                  Versions to matrix across
                                  ($ROMP_MATRIX_VERSIONS)  [default: 2.7, 3.4,
                                  3.5, 3.6, 3.7]
  --architecture [x86|x86_64]     Architectures to matrix across
                                  ($ROMP_MATRIX_ARCHITECTURES)  [default: x86,
                                  x86_64]
  --include <PLATFORM INTERPRETER VERSION ARCHITECTURE>
                                  Complete environments to include in the
                                  matrix ($ROMP_MATRIX_INCLUDES)
  --exclude <PLATFORM INTERPRETER VERSION ARCHITECTURE>
                                  Complete environments to exclude from the
                                  matrix ($ROMP_MATRIX_EXCLUDES)
  --archive-paths-root DIRECTORY  Files in the uploaded archive will be stored
                                  with paths relative to this path.
                                  ($ROMP_ARCHIVE_PATHS_ROOT)
  --archive-path TEXT             Files to include in the archive which will
                                  be extracted prior on the remote system
                                  prior to running the remote command.
                                  ($ROMP_ARCHIVE_PATHS)
  --verbose                       Increase logging verbosity by up to 2 levels
                                  ($ROMP_VERBOSITY)
  --help                          Show this message and exit.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

romp-2021.11.tar.gz (30.5 kB view details)

Uploaded Source

Built Distribution

romp-2021.11-py2.py3-none-any.whl (15.4 kB view details)

Uploaded Python 2 Python 3

File details

Details for the file romp-2021.11.tar.gz.

File metadata

  • Download URL: romp-2021.11.tar.gz
  • Upload date:
  • Size: 30.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.5.0 importlib_metadata/4.8.1 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.7.12

File hashes

Hashes for romp-2021.11.tar.gz
Algorithm Hash digest
SHA256 a624984f2db5a0922478213f7243d648826b4a53edc7a77c9d7af3f6230d32c1
MD5 6d8c13e79893a823a5753a6bf17f4162
BLAKE2b-256 d3349b29ad7f6f63fd93e61f793014c221e0dff656be12e82a6d623216c1cfb2

See more details on using hashes here.

File details

Details for the file romp-2021.11-py2.py3-none-any.whl.

File metadata

  • Download URL: romp-2021.11-py2.py3-none-any.whl
  • Upload date:
  • Size: 15.4 kB
  • Tags: Python 2, Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.5.0 importlib_metadata/4.8.1 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.7.12

File hashes

Hashes for romp-2021.11-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 e92ee425db340fb276a8fb69be7a501877d4145a6ce6cc982dfe3696ee51996c
MD5 b788bb8fa11aa5c2cb92db17bcd6f703
BLAKE2b-256 771252ae0bc3445af80a5ba2ef97d45a275eed4cb764e85afbe719e5660be827

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page