Visiongraph is a high level computer vision pipeline.
Project description
Visiongraph 
Visiongraph is a high level computer vision pipeline that includes predefined modules to quickly create and run algorithms on images. It is based on opencv and includes other computer vision frameworks like Intel openVINO and Google MediaPipe.
Here an example on how to start a webcam capture and display the image:
import visiongraph as vg
vg.create_graph(vg.VideoCaptureInput()).then(vg.ImagePreview()).open()
The main goal is to implement a platform independent and high performance framework for day-to-day computer vision tasks.
Installation
To install visiongraph with all dependencies call pip like this:
pip install "visiongraph[all]"
🚨 Please note that visiongraph is in an early alpha phase and the API will still undergo changes.
It is also possible to only install certain packages depending on your needs:
pip install "visiongraph[realsense, openvino, mediapipe, onnx, media, azure, numba]"
Apple Silicon (arm64)
On Apple silicon Macs (arm64 e.g. M1) only the following packages are currently supported.
realsense, openvino, mediapipe
Development
To develop visiograph itself it is recommended to clone this repository and install the dependencies like this:
# in the visiongraph directory
pip install -e ".[all]"
Build
To build a new wheel package of visiongraph run the following command in the root directory.
python setup.py bdist_wheel
Examples
To demonstrate the possibilities of visiongraph there are already implemented examples ready for you to try out. Here is a list of the current examples:
- SimpleVisionGraph - SSD object detection & tracking of live webcam input with
5lines of code. - VisionGraphExample - A face detection and tracking example with custom events.
- InputExample - A basic input example that determines the center if possible.
- RealSenseDepthExample - Display the RealSense or Azure Kinect depth map.
- FaceDetectionExample - A face detection pipeline example.
- FindFaceExample - A face recognition example to find a target face.
- CascadeFaceDetectionExample - A face detection pipeline that also predicts other feature points of the face.
- HandDetectionExample - A hand detection pipeline example.
- PoseEstimationExample - A pose estimation pipeline which annotates the generic pose keypoints.
- ProjectedPoseExample - Project the pose estimation into 3d space with the RealSense camera.
- ObjectDetectionExample - An object detection & tracking example.
- InstanceSegmentationExample - Intance Segmentation based on COCO80 dataset.
- InpaintExample - GAN based inpainting example.
- MidasDepthExample - Realtime depth prediction with the midas-small network.
- RGBDSmoother - Smooth RGB-D depth map videos with a one-euro filter per pixel.
There are even more examples where visiongraph is currently in use:
- Spout/Syphon RGB-D Example - Share RGB-D images over spout or syphon.
Documentation
This documentation is intended to provide an overview of the framework. A full documentation will be available later.
Graph
The core component of visiongraph is the BaseGraph class. It contains and handles all the nodes of the graph. A BaseGraph can run on the same thread as called or a new thread or process. The nodes in the graph are just a list, the graph itself is created by nesting nodes into each other.
Graph Node
A GraphNode is a single step in the graph. It has a input and output type and processes the data within the process() method.
Graph Builder
The graph builder helps to create new graphs on a single line in python. It creates a VisionGraph object which is a child of the BaseGraph. The following code snippet is an example of the graph builder which creates a smooth pose estimation graph.
import visiongraph as vg
graph = vg.create_graph(name="Smooth Pose Estimation",
input_node=vg.VideoCaptureInput(0),
handle_signals=True) \
.apply(ssd=vg.sequence(vg.OpenPoseEstimator.create(), vg.MotpyTracker(), vg.LandmarkSmoothFilter()),
image=vg.passthrough()) \
.then(vg.ResultAnnotator(image="image"), vg.ImagePreview()) \
.open()
Input
Supported are image, video, webcam, RealSense and Azure Kinect input types.
Estimator
Usually an estimator is a graph node which takes an image as an input and estimates an information about the content. This could be a pose estimation or a face detection. It is also possible to have a transformation of the image, for example de-blurring it or estimate the depth map.
Object Detection Tracker
Object detection trackers allow a detected object to be assigned an id that remains the same across successive frames.
DSP (Digital Signal Processing)
To filter noisy estimations or inputs, the DSP package provides different filters which can be applied directly into a graph.
Recorder
To record incoming frames or annotated results, multiple frame recorders are provided.
Assets
Most estimators use big model and weight descriptions for their neural networks. To keep visiongraph small and easy to install, these assets are hosted externally on github. Visiongraph provides a system to directly download and cache these files.
Argparse
To support rapid prototyping many graph and estimator options are already provided to add to the argparse parser.
Roadmap
Next roadmap points:
- Async input and network model (run when ready)
About
Copyright (c) 2022 Florian Bruggisser
Included Libraries
Parts of these libraries are directly included and adapted to work with visiongraph.
- motpy - simple multi object tracking library (MIT License)
- motrackers - Multi-object trackers in Python (MIT License)
- OneEuroFilter-Numpy - (MIT License)
For more information about the dependencies have a look at the requirements.txt.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distribution
File details
Details for the file visiongraph-0.1.23.5-py3-none-any.whl.
File metadata
- Download URL: visiongraph-0.1.23.5-py3-none-any.whl
- Upload date:
- Size: 189.5 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.0 CPython/3.9.13
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | f75359110c2f201c40cb8ac39fe1888c2f6627d6e4bca37e8d9c2676070bba52 |
|
| MD5 | cd5ecaa18c484fc40151437e9b3a2c9b |
|
| BLAKE2b-256 | 44abf35a9a3014fe3c6a1b3f9d4d74ef9364f9b9065c9e8f6ec6532e122d57b1 |
