Airflow Providers containing Deferrable Operators & Sensors from Astronomer
Project description
Apache Airflow Providers containing Deferrable Operators & Sensors from Astronomer.
Installation
pip install astronomer-providers
Example Usage
This repo is structured same as the Apache Airflow’s source code, so for example if you want to import Async operators, you can import it as follows:
from astronomer.providers.amazon.aws.sensors.s3 import S3KeySensorAsync as S3KeySensor
waiting_for_s3_key = S3KeySensor(
task_id="waiting_for_s3_key",
bucket_key="sample_key.txt",
wildcard_match=False,
bucket_name="sample-bucket",
)
Development Environment
Run the following commands from the root of the repository:
make dev - To create a development Environment using docker-compose file.
make logs - To view the logs of the all the containers
make stop - To stop all the containers
make clean - To remove all the containers along with volumes
make help - To view the available commands
make build-run - To build the docker image and then run containers
make restart - To restart Scheduler & Triggerer containers
make restart-all - To restart all the containers
make run-tests - Run CI tests
make run-static-checks - Run CI static code checks
Following ports are accessible from the host machine:
8080 - Webserver
5555 - Flower
5432 - Postgres
Dev Directories:
dev/dags/ - DAG Files
dev/logs - Logs files of the Airflow containers
License
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distribution
Hashes for astronomer_providers-1.0.0a1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7cdc88d93449f7b20705b83b39aa1cf729d36cc66b8643d0b24cf58ede88dd48 |
|
MD5 | aefc03e7505ee9a4d666a174e412ab2a |
|
BLAKE2b-256 | 5027421c54c8f760913268aefe6e8b683e6e2aacf0b177e8294b0e0462517c3b |