Skip to main content

Toy project to test executables defined in a yaml file

Project description

A wrapper around py.test to run your things selectively.

made-with-python master-pipeline license-GPL-2 license-CeCILL-2.1

Installation and running

Main requirements

  • UNIX like

  • Python 3.6, 3.7 or 3.8

From PyPA (pip install)

Install

The most recent packaged version can be installed with pip with options relevant to your usecase.

pip install test_exe_matrix

Run

test_exe_matrix [arguments...]

From source with poetry

Install

Note on poetry: you cannot install it from pip, you need to follow the instructions <https://python-poetry.org/docs/> _ on the official Web site.

git clone https://framagit.org/feth/test_exe_matrix.git
cd test_exe_matrix
poetry install

Run

poetry run test_exe_matrix [arguments...]

Command line options

Typical use:

[poetry run] test_exe_matrix

Synopsys:

usage: test_exe_matrix [-h] [-V] [--collect-only] [-v] [--markers]
                       [-m PYTEST_ARGS]
                       [testsuite [testsuite ...]]

Test command lines listed in yaml files.

positional arguments:
  testsuite       testsuite yaml file -see example
                  test_exe_matrix/matrix.yaml.
                  Argument can be specified multiple times

optional arguments:
  -h, --help      show this help message and exit
  -V, --version   show program's version number and exit
  --collect-only  Only collect tests, without running them.
  -v              Increase verbosity.
  --markers       Lists registered test markers.
  -m PYTEST_ARGS  Only run tests that match selection expression, ie. '-m "not
                  internet"'

test_exe_matrix is mainly a Py.test wrapper

-v adds very nice colors, thanks to py.test.

Parametrizing tests

Put your test suites in a yaml, like matrix.yaml (provided), or in several. Every yaml file will have a separate global config (markers declaration and general timeout).

Minimum viable test

- exe: /path/to/exe

The test will succeed if default values are met: see below (optional values).

Optional values

timeout

Allowed running time in seconds. Program will be killed after this delay. Default to config/timeout or 1.

retcode

Expected return code.

stdout

Expected standard output (full match)

stderr

Expected standard error (full match)

partstdout

Substring expected in standard output.

partstderr

Substring expected in standard error.

args

List of arguments to your program.

name

Description of the test being made.

expect_too_long

Boolean: whether the program is expected to run over the time limit.

markers

List of markers for the test. You can use standard py.test markers (especially ‘xfail’: indicating ‘expected failure’) or custom markers, that _must_ be declared in the config section.

Example

A complete (hopefully) example file is provided as “matrix.yaml”.

Given the examples, you may run

test_exe_matrix matrix.yaml --collect-only

to list the tests, and

test_exe_matrix matrix.yaml -v -m "not internet and not slow"

to deselect all tests requiring Internet and slow tests.

screenshot of the above command and output.

Dev: Build the package

The project uses poetry. Do

poetry build

And a pip installable wheel package will appear under dist/. For instance, you can install with:

pip3 install dist/test_exe_matrix-0.0.18-py3-none-any.whl

Licence

CeCILL 2.1 and GPL 2.1, see files LICENSE_CeCILL_2.1.txt and LICENSE_GPL_2.1.txt.

Thank you

Stéphane Bortzmeyer, who once asked if a simple tool testing the result of commands existed, and then reported bugs/wrote helpful tickets on framagit/gitlab to guide the development.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

test_exe_matrix-0.1.2.tar.gz (23.3 kB view details)

Uploaded Source

Built Distribution

test_exe_matrix-0.1.2-py3-none-any.whl (24.3 kB view details)

Uploaded Python 3

File details

Details for the file test_exe_matrix-0.1.2.tar.gz.

File metadata

  • Download URL: test_exe_matrix-0.1.2.tar.gz
  • Upload date:
  • Size: 23.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.0.3 CPython/2.7.17 Linux/5.4.0-4-amd64

File hashes

Hashes for test_exe_matrix-0.1.2.tar.gz
Algorithm Hash digest
SHA256 a67fea85e8570140a7d66ea128976ed8b38c4ec07622748aad37d531cd509237
MD5 aaf390ee0b4987d867cc7ee979aaf76c
BLAKE2b-256 2f23cb8e893b1fa5f76abc9797604a521947fc057a8ef3461950a198c2d6ea40

See more details on using hashes here.

Provenance

File details

Details for the file test_exe_matrix-0.1.2-py3-none-any.whl.

File metadata

  • Download URL: test_exe_matrix-0.1.2-py3-none-any.whl
  • Upload date:
  • Size: 24.3 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.0.3 CPython/2.7.17 Linux/5.4.0-4-amd64

File hashes

Hashes for test_exe_matrix-0.1.2-py3-none-any.whl
Algorithm Hash digest
SHA256 5fce5a562c50aa5e737a27d52e028e832b67b192e0949d05d813d0ae34cfaa56
MD5 f34506b2a3bda72141ea3e515aac742a
BLAKE2b-256 ec7ee34d925a47d27176deaea83d844ca0182869a3f47869477932220f2ac81d

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page