Skip to main content

parse test results to inform requirements

Project description

testreqstatus

PyPI - Version PyPI - Python Version


pip install testreqstatus

Usage

  1. Establish a set of requirements corresponding to test names:

    {
      "DMS356": [
        "romancal.regtest.test_mos_pipeline.test_level3_mos_pipeline"
      ],
      "DMS373": [
        "romancal.regtest.test_mos_pipeline.test_hlp_mosaic_pipeline"
      ],
      "DMS374": [
        "romancal.regtest.test_mos_pipeline.test_level3_mos_pipeline"
      ],
      "DMS400": [
        "romancal.regtest.test_mos_pipeline.test_level3_mos_pipeline"
      ]
    }
    

[!TIP] To extract test requirements from existing tests decorated with @metrics_logger, use RequirementTests.from_test_directory():

from testreqstatus import RequirementTests

requirements = RequirementTests.from_test_directory("~/projects/romancal/")
requirements.to_file("test_requirements.json")
  1. Run tests and generate a JUnitXML results file:

    <?xml version="1.0" encoding="utf-8"?>
    <testsuites>
      <testsuite name="pytest" errors="0" failures="0" skipped="0" tests="2" time="2021.550" timestamp="2024-08-23T00:23:01.454354" hostname="spacetelescope-runner-2ls89-rrbf2">
        <testcase classname="romancal.regtest.test_mos_pipeline" name="test_level3_mos_pipeline" time="677.728">
        </testcase>
        <testcase classname="romancal.regtest.test_mos_pipeline" name="test_hlp_mosaic_pipeline" time="486.642">
        </testcase>
      </testsuite>
    </testsuites>
    
  2. Use RequirementTestResults.requirements_by_test_result to retrieve the status of each test and its corresponding requirements:

    from testreqstatus import RequirementTestResults
    
    results = RequirementTestResults(
        test_requirements="examples/test_requirements.json",
        test_results="examples/results.xml",
    )
    
    print(results.requirements_by_test_result)
    
    {
      "romancal.regtest.test_mos_pipeline.test_level3_mos_pipeline": {
        "status": "PASS",
        "requirements": [
          "DMS356",
          "DMS374",
          "DMS400"
        ]
      },
      "romancal.regtest.test_mos_pipeline.test_hlp_mosaic_pipeline": {
        "status": "PASS",
        "requirements": [
          "DMS373"
        ]
      }
    }
    

[!TIP] You can alternatively use RequirementTestResults.test_results_by_requirement to obtain the inverse mapping (requirements and their corresponding test statuses):

print(results.test_results_by_requirement)
{
  "DMS356": {
    "romancal.regtest.test_mos_pipeline.test_level3_mos_pipeline": "PASS"
  },
  "DMS373": {
    "romancal.regtest.test_mos_pipeline.test_hlp_mosaic_pipeline": "PASS"
  },
  "DMS374": {
    "romancal.regtest.test_mos_pipeline.test_level3_mos_pipeline": "PASS"
  },
  "DMS400": {
    "romancal.regtest.test_mos_pipeline.test_level3_mos_pipeline": "PASS"
  }
}

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

testreqstatus-0.6.0.tar.gz (290.0 kB view details)

Uploaded Source

Built Distribution

testreqstatus-0.6.0-py3-none-any.whl (29.1 kB view details)

Uploaded Python 3

File details

Details for the file testreqstatus-0.6.0.tar.gz.

File metadata

  • Download URL: testreqstatus-0.6.0.tar.gz
  • Upload date:
  • Size: 290.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.12.6

File hashes

Hashes for testreqstatus-0.6.0.tar.gz
Algorithm Hash digest
SHA256 c14e25eb30fcd987384d3bb337b36e03001100ef59f32b1c7dcc5a6fed401f89
MD5 626d39955bde3a81315252787905f953
BLAKE2b-256 f7f2190c15e1adc66158563fdf297675805937ad21a7c703842c19325fdc0d15

See more details on using hashes here.

File details

Details for the file testreqstatus-0.6.0-py3-none-any.whl.

File metadata

File hashes

Hashes for testreqstatus-0.6.0-py3-none-any.whl
Algorithm Hash digest
SHA256 779be864f8ad4982c0a28fc83f5803f0c55535d40915c859fea523f00efc5c9d
MD5 8836b22ab5c8816d19144e6b604f2a1c
BLAKE2b-256 148e223813e47a724b649d67178dca273b4eef550439ef03ce0e15ae14343a74

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page