Provider package apache-airflow-providers-apache-spark for Apache Airflow
Project description
Package apache-airflow-providers-apache-spark
Release: 2.1.0rc2
Provider package
This is a provider package for apache.spark provider. All classes for this provider package are in airflow.providers.apache.spark python package.
You can find package information and changelog for the provider in the documentation.
Installation
You can install this package on top of an existing Airflow 2.1+ installation via pip install apache-airflow-providers-apache-spark
The package supports the following python versions: 3.7,3.8,3.9
PIP requirements
PIP package |
Version required |
|---|---|
apache-airflow |
>=2.1.0 |
pyspark |
Changelog
2.1.0
Features
Add more SQL template fields renderers (#21237)
Add optional features in providers. (#21074)
2.0.3
Bug Fixes
Ensure Spark driver response is valid before setting UNKNOWN status (#19978)
2.0.2
Bug Fixes
fix bug of SparkSql Operator log going to infinite loop. (#19449)
2.0.1
Misc
Optimise connection importing for Airflow 2.2.0
2.0.0
Breaking changes
Auto-apply apply_default decorator (#15667)
Bug fixes
Make SparkSqlHook use Connection (#15794)
1.0.3
Bug fixes
Fix 'logging.exception' redundancy (#14823)
1.0.2
Bug fixes
Use apache.spark provider without kubernetes (#14187)
1.0.1
Updated documentation and readme files.
1.0.0
Initial version of the provider.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for apache-airflow-providers-apache-spark-2.1.0rc2.tar.gz
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | c400c5e127ce0ec028f601b110d26692461da13e292c97794a3194ad317c343d |
|
| MD5 | 31c91db2133f0615e68922413ff117f1 |
|
| BLAKE2b-256 | d9eaeee0f52069032b5917c4585bf0a67938d11ee83ac60f8320ac17c65c1ab5 |
Hashes for apache_airflow_providers_apache_spark-2.1.0rc2-py3-none-any.whl
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 04c9199d0b9a87f73b7a80a019aac8ce24e48f933b5d1b60869b0115f104d0ac |
|
| MD5 | db139964b0057cc6dc8661789b9a034d |
|
| BLAKE2b-256 | 3793ee6e08dbfcc4166202f0805e543de20ce2fbf515087bf9e3039f62e91ea1 |
