Skip to main content

FlowUI project

Project description

FlowUI Project

FlowUI is an open source workflow management platform, containing:

  • an intuitive Graphical User Interface that facilitates creating, editing and supervising any type of Workflows (e.g. data processing, machine learning, etc...)
  • a REST API that controls a running Apache Airflow instance
  • a standard way of writing Operators which follows good practices for data typing, documentation and distribution

FlowUI Infrastructure

Per Platform:

  • Frontend service
  • Backend service
  • Database
  • Airflow services
  • Github repository for GitSync of Workflows

Shared storage structure:

Shared workflow data could be stored in a remote source (e.g. S3 bucket) or locally (for dev and tests only).

/shared_storage
..../{dag-id}
......../{run-id}
............/{task-id}
................/results
..................../log.txt
..................../result.npy
..................../result.html
................/report
................/xcom_out
..................../xcom_out.json



Operators

Each Operator will have:

  • A operator.py file with the source code to be executed, as the operator_function()
  • A models.py file containing the Pydantic models that define the input, output and secrets for the Operator
  • A metadata.json file containing the Operators metadata, including frontend node style

Each dependency group from an Operators repository will build an independent Docker image. This dependency group image has the following basic file struture within /home:

# This path holds the source code from the Operators repository, it comes built in the Image
/operators_repository
..../config.toml
..../operators
......../{OPERATOR-NAME}
............/metadata.json    # OPTIONAL
............/model.py         # REQUIRED
............/operator.py      # REQUIRED
..../.flowui
......../dependencies_map.json
......../compiled_metadata.json
..../dependencies
......../requirements.txt     # If dependency group was defined with a requirements.txt file

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

flowui-project-0.0.22.tar.gz (48.2 kB view details)

Uploaded Source

Built Distribution

flowui_project-0.0.22-py3-none-any.whl (60.0 kB view details)

Uploaded Python 3

File details

Details for the file flowui-project-0.0.22.tar.gz.

File metadata

  • Download URL: flowui-project-0.0.22.tar.gz
  • Upload date:
  • Size: 48.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.8.16

File hashes

Hashes for flowui-project-0.0.22.tar.gz
Algorithm Hash digest
SHA256 a7579152efc924da696140c460822a80db0b24d1df33bd36ba402e205b7caeb7
MD5 031005e6da986cb9532189ed8a9e7cc8
BLAKE2b-256 d109941be9ffb8bc239586f9092fcae3ae0f76baf1a3c7cae907ec7e60add05d

See more details on using hashes here.

Provenance

File details

Details for the file flowui_project-0.0.22-py3-none-any.whl.

File metadata

File hashes

Hashes for flowui_project-0.0.22-py3-none-any.whl
Algorithm Hash digest
SHA256 85948b6bb706caa5cc8923d0004fe36fd2deb7438b72bf8a9fa0cfefece4f7c2
MD5 7d7c5ebaa79f652bf45b302d7c1704a5
BLAKE2b-256 6e172692251efd6e72239838d4099a6dafca14856c247b2156ae702ec0496d1c

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page