FlowUI project
Project description
FlowUI Project
FlowUI is an open source workflow management platform, containing:
- an intuitive Graphical User Interface that facilitates creating, editing and supervising any type of Workflows (e.g. data processing, machine learning, etc...)
- a REST API that controls a running Apache Airflow instance
- a standard way of writing Operators which follows good practices for data typing, documentation and distribution
FlowUI Infrastructure
Per Platform:
- Frontend service
- Backend service
- Database
- Airflow services
- Github repository for GitSync of Workflows
Shared storage structure:
Shared workflow data could be stored in a remote source (e.g. S3 bucket) or locally (for dev and tests only).
/shared_storage
..../{dag-id}
......../{run-id}
............/{task-id}
................/results
..................../log.txt
..................../result.npy
..................../result.html
................/report
................/xcom_out
..................../xcom_out.json
Operators
Each Operator will have:
- A
operator.py
file with the source code to be executed, as theoperator_function()
- A
models.py
file containing the Pydantic models that define the input, output and secrets for the Operator - A
metadata.json
file containing the Operators metadata, including frontend node style
Each dependency group from an Operators repository will build an independent Docker image. This dependency group image has the following basic file struture within /home
:
# This path holds the source code from the Operators repository, it comes built in the Image
/operators_repository
..../config.toml
..../operators
......../{OPERATOR-NAME}
............/metadata.json # OPTIONAL
............/model.py # REQUIRED
............/operator.py # REQUIRED
..../.flowui
......../dependencies_map.json
......../compiled_metadata.json
..../dependencies
......../requirements.txt # If dependency group was defined with a requirements.txt file
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
flowui-project-0.0.22.tar.gz
(48.2 kB
view details)
Built Distribution
File details
Details for the file flowui-project-0.0.22.tar.gz
.
File metadata
- Download URL: flowui-project-0.0.22.tar.gz
- Upload date:
- Size: 48.2 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.8.16
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | a7579152efc924da696140c460822a80db0b24d1df33bd36ba402e205b7caeb7 |
|
MD5 | 031005e6da986cb9532189ed8a9e7cc8 |
|
BLAKE2b-256 | d109941be9ffb8bc239586f9092fcae3ae0f76baf1a3c7cae907ec7e60add05d |
Provenance
File details
Details for the file flowui_project-0.0.22-py3-none-any.whl
.
File metadata
- Download URL: flowui_project-0.0.22-py3-none-any.whl
- Upload date:
- Size: 60.0 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.8.16
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 85948b6bb706caa5cc8923d0004fe36fd2deb7438b72bf8a9fa0cfefece4f7c2 |
|
MD5 | 7d7c5ebaa79f652bf45b302d7c1704a5 |
|
BLAKE2b-256 | 6e172692251efd6e72239838d4099a6dafca14856c247b2156ae702ec0496d1c |