Skip to main content

{{ DESCRIPTION }}

Project description

# datapackage-pipelines-github

Extension for `datapackage-pipelines` for pulling stuff from GitHub.
- **Pipelines and Processor Code** from a specific branch will be pulled
- **Issues** will be presented, using their title and description, as 'unable to start' pipelines.
- **Pull Requests** will be shown and executed, based on a predefined policies defining:
- Should pipeline specs be pulled from PRs
- Should processor code be pulled from PRs
- Which processors are allowed to run, when executing PRs

You can define separate policies for local PRs (from the same fork) or remote PRs (from another fork.)

note: When pulling code from PRs, we will only bring processors referenced from the
pipeline spec and residing in the same directory as the `pipeline-spec.yaml` file.

## Source spec

Place files named `github.source-spec.yaml` in your pipelines directory.
Each one should be of the form:
```yaml
<pipeline-id-prefix>:
repository: <owner/repo>
base-path: <where to look for pipelines and code. default is 'pipelines/'>
code: <if not present, won't fetch code. set to {} to use the defaults>
ref: <which branch/commit to use to get code from. default is 'master>
issues: <if not present, won't fetch issues. set to {} to use the defaults>
closed: <boolean, should fetch closed issues? default is no>
pipeline-id-format: <string, see below>
pull-requests: <if not present, won't fetch prs. set to {} to use the defaults>
local: <policy for local PRs>
specs: <boolean, should pull pipeline specs? default is yes>
code: <boolean, should pull code from PRs? default is yes>
disallow-processors: <which processors are not allowed to run in PRs? default is ["dump\..*"]>
- <regular-expression for processor name>
remote: <policy for remote PRs, same as local>
specs: <boolean, should pull pipeline specs? default is yes>
code: <boolean, should pull code from PRs? default is no>
disallow-processors: <which processors are not allowed to run in PRs? default is ["dump\..*"]>
- <regular-expression for processor name>
pipeline-id-format: <string, see below>
```

`pipeline-id-format` is a Python format string with two placeholders:
- `issue-id`: The issue/pr number
- `title-slug`: The issue title slug

The default format for issues is "issue/{issue-id:03}_{title-slug}"
The default format for pull-requests is "pr/{pr-id:03}_{title-slug}"

#### Example:
```yaml
dpp-github:
repository: firctionlessdata/datapackage-pipelines-github
code:
ref: master
pull-requests:
local:
specs: yes
code: yes
disallow-processors:
- "dump\..*"
remote:
specs: no
pipeline-id-format: "{title-slug}__{issue-id}"
issues:
closed: no
pipeline-id-format: "{title-slug}__{issue-id}"
```

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

datapackage-pipelines-github-0.0.4.tar.gz (85.0 kB view details)

Uploaded Source

File details

Details for the file datapackage-pipelines-github-0.0.4.tar.gz.

File metadata

File hashes

Hashes for datapackage-pipelines-github-0.0.4.tar.gz
Algorithm Hash digest
SHA256 91c1260a3725b888994af9148a32a27cf4f6fa818e9b5854035f638f987fa426
MD5 e984c7bb873e8b2730b4e719cbae6f10
BLAKE2b-256 81fea742710c6955f85476578ba8070611761e29852e068d4ef2a7ab18ecc763

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page