Provider for Apache Airflow. Implements apache-airflow-providers-apache-hdfs package
Project description
Package apache-airflow-providers-apache-hdfs
Release: 4.0.0
Hadoop Distributed File System (HDFS) and WebHDFS.
Provider package
This is a provider package for apache.hdfs provider. All classes for this provider package are in airflow.providers.apache.hdfs python package.
You can find package information and changelog for the provider in the documentation.
Installation
You can install this package on top of an existing Airflow 2 installation (see Requirements below for the minimum Airflow version supported) via pip install apache-airflow-providers-apache-hdfs
The package supports the following python versions: 3.7,3.8,3.9,3.10
Requirements
PIP package |
Version required |
|---|---|
apache-airflow |
>=2.4.0 |
hdfs[avro,dataframe,kerberos] |
>=2.0.4 |
Changelog
4.0.0
Breaking changes
The original HDFS Hook and sensor has been removed. It used the old HDFS snakebite-py3 library that had no update in years and the protobuf they are using reached end of life.
The 3.* version of the provider is still available and can be used if you need to use the old hooks and sensors.
The HDFSHook, HDFSSensor, HdfsRegexSensor, HdfsRegexSensor that have been removed from this provider and they are not available any more. If you want to continue using them, you can use 3.* version of the provider, but the recommendation is to switch to the new WebHDFSHook and WebHDFSSensor that use the WebHDFS API.
Remove snakebite-py3 based HDFS hooks and sensors (#31262)
Misc
Bump minimum Airflow version in providers (#30917)
3.2.1
Bug Fixes
Fix HDFSHook HAClient is invalid (#30164)
3.2.0
Misc
Move min airflow version to 2.3.0 for all providers (#27196)
3.1.0
Features
Adding Authentication to webhdfs sensor (#25110)
3.0.1
Bug Fixes
'WebHDFSHook' Bugfix/optional port (#24550)
3.0.0
Breaking changes
This release of provider is only available for Airflow 2.2+ as explained in the Apache Airflow providers support policy https://github.com/apache/airflow/blob/main/README.md#support-for-providers
Misc
chore: Refactoring and Cleaning Apache Providers (#24219)
2.2.3
Bug Fixes
Fix mistakenly added install_requires for all providers (#22382)
2.2.2
Misc
Add Trove classifiers in PyPI (Framework :: Apache Airflow :: Provider)
2.2.1
Misc
Support for Python 3.10
Add how-to guide for WebHDFS operators (#21393)
2.2.0
Features
hdfs provider: restore HA support for webhdfs (#19711)
2.1.1
Bug Fixes
fix get_connections deprecation warning in webhdfs hook (#18331)
2.1.0
Features
hdfs provider: allow SSL webhdfs connections (#17637)
Misc
Optimise connection importing for Airflow 2.2.0
2.0.0
Breaking changes
Auto-apply apply_default decorator (#15667)
1.0.1
Updated documentation and readme files.
1.0.0
Initial version of the provider.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for apache-airflow-providers-apache-hdfs-4.0.0.tar.gz
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 5402b41d9e060ca8cd0e3da2191f12c7446ab6e98b27365f14bd1665aaba905e |
|
| MD5 | 2c7fc56345c6407a1928a7b529e5684c |
|
| BLAKE2b-256 | e1f8aad096075218f09688c3eeb82d2dba09649476250fe396979e63dc2d2ce6 |
Hashes for apache_airflow_providers_apache_hdfs-4.0.0-py3-none-any.whl
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 08fab0bd9ded5f514813fc5c7c287ecc5a06780c6f6fc7406fd3874f061bdc2d |
|
| MD5 | c50416e652b8bd5595f7590619694e9e |
|
| BLAKE2b-256 | b1469f2080554f45fe472859553024997a749b41084065d0be90cf2c3688b02e |
