Skip to main content

Science processing code for the ViSP instrument on DKIST

Project description

Overview

The dkist-processing-visp library contains the implementation of the visp pipelines as a collection of the dkist-processing-core framework and dkist-processing-common Tasks.

The recommended project structure is to separate tasks and workflows into separate packages. Having the workflows in their own package facilitates using the build_utils to test the integrity of those workflows in the unit test.

Calibration Pipeline

Build

Artifacts are built through Bitbucket Pipelines.

The pipeline can be used in other repos with a modification of the package and artifact locations to use the names relevant to the target repo.

e.g. dkist-processing-test -> dkist-processing-vbi and dkist_processing_test -> dkist_processing_vbi

Deployment

Deployment is done with turtlebot and follows the process detailed in dkist-processing-core

Environment Variables

Only those specified by dkist-processing-core and dkist-processing-common.

Development

git clone git@bitbucket.org:dkistdc/dkist-processing-visp.git
cd dkist-processing-visp
pre-commit install
pip install -e .[test]
pytest -v --cov dkist_processing_visp

Changelog

When you make any change to this repository it MUST be accompanied by a changelog file. The changelog for this repository uses the towncrier package. Entries in the changelog for the next release are added as individual files (one per change) to the changelog/ directory.

Writing a Changelog Entry

A changelog entry accompanying a change should be added to the changelog/ directory. The name of a file in this directory follows a specific template:

<PULL REQUEST NUMBER>.<TYPE>[.<COUNTER>].rst

The fields have the following meanings:

  • <PULL REQUEST NUMBER>: This is the number of the pull request, so people can jump from the changelog entry to the diff on BitBucket.

  • <TYPE>: This is the type of the change and must be one of the values described below.

  • <COUNTER>: This is an optional field, if you make more than one change of the same type you can append a counter to the subsequent changes, i.e. 100.bugfix.rst and 100.bugfix.1.rst for two bugfix changes in the same PR.

The list of possible types is defined in the towncrier section of pyproject.toml, the types are:

  • feature: This change is a new code feature.

  • bugfix: This is a change which fixes a bug.

  • doc: A documentation change.

  • removal: A deprecation or removal of public API.

  • misc: Any small change which doesn’t fit anywhere else, such as a change to the package infrastructure.

Rendering the Changelog at Release Time

When you are about to tag a release first you must run towncrier to render the changelog. The steps for this are as follows:

  • Run towncrier build –version vx.y.z using the version number you want to tag.

  • Agree to have towncrier remove the fragments.

  • Add and commit your changes.

  • Tag the release.

NOTE: If you forget to add a Changelog entry to a tagged release (either manually or automatically with towncrier) then the Bitbucket pipeline will fail. To be able to use the same tag you must delete it locally and on the remote branch:

# First, actually update the CHANGELOG and commit the update
git commit

# Delete tags
git tag -d vWHATEVER.THE.VERSION
git push --delete origin vWHATEVER.THE.VERSION

# Re-tag with the same version
git tag vWHATEVER.THE.VERSION
git push --tags origin main

Science Changelog

Whenever a release involves changes to the scientific quality of L1 data, additional changelog fragment(s) should be created. These fragments are intended to be as verbose as is needed to accurately capture the scope of the change(s), so feel free to use all the fancy RST you want. Science fragments are placed in the same changelog/ directory as other fragments, but are always called:

<PR NUMBER | +>.science[.<COUNTER>].rst

In the case that a single pull request encapsulates the entirety of the scientific change then the first field should be that PR number (same as the normal CHANGELOG). If, however, there is not a simple mapping from a single PR to a scientific change then use the character “+” instead; this will create a changelog entry with no associated PR. For example:

$ ls changelog/
99.bugfix.rst    # This is a normal changelog fragment associated with a bugfix in PR 99
99.science.rst   # Apparently that bugfix also changed the scientific results, so that PR also gets a science fragment
+.science.rst    # This fragment is not associated with a PR

When it comes time to build the SCIENCE_CHANGELOG, use the science_towncrier.sh script in this repo to do so. This script accepts all the same arguments as the default towncrier. For example:

./science_towncrier.sh build --version vx.y.z

This will update the SCIENCE_CHANGELOG and remove any science fragments from the changelog directory.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dkist_processing_visp-2.19.4.tar.gz (469.3 kB view details)

Uploaded Source

Built Distribution

dkist_processing_visp-2.19.4-py3-none-any.whl (495.6 kB view details)

Uploaded Python 3

File details

Details for the file dkist_processing_visp-2.19.4.tar.gz.

File metadata

  • Download URL: dkist_processing_visp-2.19.4.tar.gz
  • Upload date:
  • Size: 469.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.11.10

File hashes

Hashes for dkist_processing_visp-2.19.4.tar.gz
Algorithm Hash digest
SHA256 51e79f2af0114fba2166c7018d8977a48087875679fe64bf549378e64e7f8cac
MD5 8328f3ddbb282ee7b036225d76b3ea4c
BLAKE2b-256 080c51ce0010d12c1c2f66e58d36e505e120f9fe5614639ab77ef36ddd90c477

See more details on using hashes here.

Provenance

File details

Details for the file dkist_processing_visp-2.19.4-py3-none-any.whl.

File metadata

File hashes

Hashes for dkist_processing_visp-2.19.4-py3-none-any.whl
Algorithm Hash digest
SHA256 a77f13311861b873979aca4b73f82d0c6c2fd9b4eeac993b8d1184c6a1c60c06
MD5 5037e5622e17cb6e4dca18ab801a7eb3
BLAKE2b-256 3f5bd76dccd0dbf13bfc7e7102cca47b3d0b7e510e9fa284c3a74f17dfcaabd7

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page