Run on multiple platforms
Project description
Run on multiple platforms
I use pip-tools but also want to keep my compiled requirements for multiple platforms. Instead of sharing directories between machines or otherwise shuttling files around I chose to create romp which will let me submit arbitrary work to Azure Pipelines to get access to multiple platforms. Personally, I will use this primarily behind boots.
Below is an example usage. It will run a single job under Linux with 64-bit CPython 3.6. The job will execute echo red > blue.txt and will collect *.txt from the job and save it locally to artifacts.tar.gz.
$ venv/bin/romp --platform linux --interpreter cpython --version 3.6 --architecture x86_64 --command 'echo red > blue.txt' --artifact-paths '*.txt' --artifact artifacts.tar.gz
Requesting build
Waiting for build: https://dev.azure.com/altendky/f1722f91-62fe-4a15-8937-252c96b31292/_build/results?buildId=2938
Handling artifact
Since romp leverages Azure Pipelines to get access to all the platforms, server-side setup is required. You will need an Azure account and to create a pipeline (build) within that account using a romp repository (official or your own fork). Configure the pipeline to use deployed_ci/azure/azure-pipelines-lock.yml. It is configured such that commits do not trigger builds. romp’s own CI and testing is driven by azure-pipelines.yml in the project root but this file can be ignored for regular use. Once the pipeline is setup you will need to create a Personal Access Token (PAT) to use for authentication when running romp. The PAT will need the build read and execute scope enabled.
For local setup, the command line options can be set by environment variables. For many options this will not make sense but for a few it will. Specifically consider the follow options which will often be consistent for all calls.
- ROMP_BUILD_REQUEST_URL
'https://dev.azure.com/altendky/romp-on/_apis/build/builds?api-version=5.0'
- ROMP_DEFINITION_ID
5
- ROMP_PERSONAL_ACCESS_TOKEN
fp6al3jxta2zliz6rh5hr6ewd5nw2hsmasse2laiyoyg7otneqjq
- ROMP_USERNAME
altendky
$ venv/bin/romp --help
Usage: romp [OPTIONS]
Options:
--personal-access-token, --pat TEXT
A personal access token (PAT) with rights to
initiate builds
($ROMP_PERSONAL_ACCESS_TOKEN)
--build-request-url TEXT The URL for submitting a build request
($ROMP_BUILD_REQUEST_URL) [default: https:/
/dev.azure.com/altendky/romp/_apis/build/bui
lds?api-version=5.0]
--command TEXT The command to be run for each target
($ROMP_COMMAND) [default: python -c 'import
sys; print(sys.version);
print(sys.platform)']
--username TEXT Username for build URL authentication
($ROMP_USERNAME) [default: altendky]
--environments TEXT Targets to run on. Mostly use the matrix
options instead. This may be removed.
($ROMP_ENVIRONMENTS)
--check-period INTEGER The period used to poll the build for
completion ($ROMP_CHECK_PERIOD) [default:
15]
--source-branch TEXT The romp source branch to use for the build
($ROMP_SOURCE_BRANCH) [default: develop]
--definition-id INTEGER The definition id of the build to be
triggered ($ROMP_DEFINITION_ID) [default:
3]
--archive-file FILENAME The archive to be uploaded to the build
($ROMP_ARCHIVE)
--artifact FILENAME The path at which to save the resulting
artifact ($ROMP_ARTIFACT_PATH)
--artifact-paths TEXT Paths on remote system to build the artifact
archive from. Wildcards are supported via
bash. ($ROMP_ARTIFACT_PATHS)
--platform [Linux|macOS|Windows]
Platforms to matrix across
($ROMP_MATRIX_PLATFORMS) [default: Linux,
macOS, Windows]
--interpreter [CPython|PyPy] Interpreters to matrix across
($ROMP_MATRIX_INTERPRETERS) [default:
CPython, PyPy]
--version [2.7|3.4|3.5|3.6|3.7]
Versions to matrix across
($ROMP_MATRIX_VERSIONS) [default: 2.7, 3.4,
3.5, 3.6, 3.7]
--architecture [x86|x86_64] Architectures to matrix across
($ROMP_MATRIX_ARCHITECTURES) [default: x86,
x86_64]
--include <PLATFORM INTERPRETER VERSION ARCHITECTURE>
Complete environments to include in the
matrix ($ROMP_MATRIX_INCLUDES)
--exclude <PLATFORM INTERPRETER VERSION ARCHITECTURE>
Complete environments to exclude from the
matrix ($ROMP_MATRIX_EXCLUDES)
--archive-paths-root DIRECTORY Files in the uploaded archive will be stored
with paths relative to this path.
($ROMP_ARCHIVE_PATHS_ROOT)
--archive-path TEXT Files to include in the archive which will
be extracted prior on the remote system
prior to running the remote command.
($ROMP_ARCHIVE_PATHS)
--verbose Increase logging verbosity by up to 2 levels
($ROMP_VERBOSITY)
--help Show this message and exit.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file romp-2021.11.tar.gz
.
File metadata
- Download URL: romp-2021.11.tar.gz
- Upload date:
- Size: 30.5 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.5.0 importlib_metadata/4.8.1 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.7.12
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | a624984f2db5a0922478213f7243d648826b4a53edc7a77c9d7af3f6230d32c1 |
|
MD5 | 6d8c13e79893a823a5753a6bf17f4162 |
|
BLAKE2b-256 | d3349b29ad7f6f63fd93e61f793014c221e0dff656be12e82a6d623216c1cfb2 |
File details
Details for the file romp-2021.11-py2.py3-none-any.whl
.
File metadata
- Download URL: romp-2021.11-py2.py3-none-any.whl
- Upload date:
- Size: 15.4 kB
- Tags: Python 2, Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.5.0 importlib_metadata/4.8.1 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.7.12
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | e92ee425db340fb276a8fb69be7a501877d4145a6ce6cc982dfe3696ee51996c |
|
MD5 | b788bb8fa11aa5c2cb92db17bcd6f703 |
|
BLAKE2b-256 | 771252ae0bc3445af80a5ba2ef97d45a275eed4cb764e85afbe719e5660be827 |