NebulaGraph AI Suite
Project description
NebulaGraph AI Suite with 4 line code to run Graph Algo on NebulaGraph
Documentation: https://github.com/wey-gu/nebulagraph-ai#documentation
Source Code: https://github.com/wey-gu/nebulagraph-ai
NebulaGraph AI Suite for Python (ng_ai) is a powerful Python library that offers APIs for data scientists to effectively read, write, analyze, and compute data in NebulaGraph.
With the support of a single-machine engine(NetworkX), or distributed computing environment using Spark, we could perform Graph Analysis and Algorithms on top of NebulaGraph in less than 10 lines of code, in unified and intuitive API.
Quick Start in 5 Minutes
Option 1: Try on your Desktop. Go with NebulaGraph Docker Extension!
Option 2: On Linux Server? Go with Nebula-Up ๐๐ป
- Set up env with Nebula-Up following this guide.
- Install ng_ai with pip from the Jupyter Notebook with http://localhost:8888 (password:
nebula
). - Open the demo notebook and run cells one by one.
- Check the API Reference
Installation
pip install ng_ai
Usage
Call from nGQL
See more details in the docs
RETURN ng_ai("pagerank", ["follow"], ["degree"], "spark",
{space: "basketballplayer", max_iter: 10}, {write_mode: "insert"})
Spark Engine Examples
See also: examples/spark_engine.ipynb
Run Algorithm on top of NebulaGraph:
Note, there is also a query mode, refer to examples or docs for more details.
from ng_ai import NebulaReader
# read data with spark engine, scan mode
reader = NebulaReader(engine="spark")
reader.scan(edge="follow", props="degree")
df = reader.read()
# run pagerank algorithm
pr_result = df.algo.pagerank(reset_prob=0.15, max_iter=10)
Write back to NebulaGraph:
from ng_ai import NebulaWriter
from ng_ai.config import NebulaGraphConfig
config = NebulaGraphConfig()
properties = {"louvain": "cluster_id"}
writer = NebulaWriter(
data=df_result, sink="nebulagraph_vertex", config=config, engine="spark")
writer.set_options(
tag="louvain", vid_field="_id", properties=properties,
batch_size=256, write_mode="insert",)
writer.write()
Then we could query the result in NebulaGraph:
MATCH (v:louvain)
RETURN id(v), v.louvain.cluster_id LIMIT 10;
NebulaGraph Engine Examples
Basically the same as Spark Engine, but with engine="nebula"
, refer to examples or docs for more details.
- reader = NebulaReader(engine="spark")
+ reader = NebulaReader(engine="nebula")
Documentation
How it works
ng_ai is a unified abstraction layer for different engines, the current implementation is based on Spark, NetworkX, DGL, and NebulaGraph, but it's easy to extend to other engines like Flink, GraphScope, PyG, etc.
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ Spark Cluster โ
โ .โโโโโ. .โโโโโ. .โโโโโ. .โโโโโ. โ
โ ; : ; : ; : ; : โ
โโโถโ : ; : ; : ; : ; โ
โ โ โฒ โฑ โฒ โฑ โฒ โฑ โฒ โฑ โ
โ โ `โโโ' `โโโ' `โโโ' `โโโ' โ
Algo Spark โ
Engineโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฌโโโโโโโโโโโ
โโโโค โ โ
โ NebulaGraph AI Suite(ngai) โ ngai-api โโโโ
โ โ โ โ
โ โโโโโโโโโโโโค โ
โ โโโโโโโโโโ โโโโโโโโ โโโโโโโโโโ โโโโโโโ โ โ
โ โ Reader โ โ Algo โ โ Writer โ โ GNN โ โ โ
โโโโโโโโโถโ โโโโโโโโโโ โโโโโโโโ โโโโโโโโโโ โโโโโโโ โ โ
โ โ โ โ โ โ โ โ
โ โ โโโโโโโโโโโโโโดโโโโฌโโโโโโโโโดโโโโโโ โโโโโโโโ โ โ
โ โ โผ โผ โผ โผ โ โ
โ โ โโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโโโโโโ โ โ
โ โโโโค โ SparkEngine โ โ NebulaEngine โ โ NetworkX โโ DGLEngineโ โ โ
โ โ โ โโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโโโโโโ โ โ
โ โ โโโโโโโโโโโโฌโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ
โ โ โ Spark โ
โ โ โโโโโโโโโReader โโโโโโโโโโโโโ โ
โ Spark Query Mode โ โ
โ Reader โ โ
โScan Mode โผ โโโโโโโโโโโ
โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฌโโโโโโโโโโค ngai-udfโโโโโโโโโโโโโโโโ
โ โ โ โ โโโโโโโโโโโค โ
โ โ โ NebulaGraph Graph Engine Nebula-GraphD โ ngai-GraphD โ โ
โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโ โ
โ โ โ โ โ โ
โ โ โ NebulaGraph Storage Engine โ โ โ
โ โ โ โ โ โ
โ โโโถโ Nebula-StorageD โ Nebula-Metad โ โ
โ โ โ โ โ
โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโ โ
โ โ
โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ
โ โ RETURN ng_ai("pagerank", ["follow"], ["degree"], "spark", {space:"basketballplayer"}) โโโโ
โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ โ from ng_ai import NebulaReader โ
โ โ โ
โ โ # read data with spark engine, scan mode โ
โ โ reader = NebulaReader(engine="spark") โ
โ โ reader.scan(edge="follow", props="degree") โ
โโโโ df = reader.read() โ
โ โ
โ # run pagerank algorithm โ
โ pr_result = df.algo.pagerank(reset_prob=0.15, max_iter=10) โ
โ โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
Spark Engine Prerequisites
- Spark 2.4, 3.0(not yet tested)
- NebulaGraph 3.4+
- NebulaGraph Spark Connector 3.4+
- NebulaGraph Algorithm 3.1+
NebulaGraph Engine Prerequisites
Contributing
See HACKING.md for details.
License
This project is licensed under the terms of the Apache License 2.0.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file ng-ai-0.2.10.3.tar.gz
.
File metadata
- Download URL: ng-ai-0.2.10.3.tar.gz
- Upload date:
- Size: 29.3 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: pdm/2.18.1 CPython/3.10.12 Linux/6.5.0-1025-azure
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 880a4ae1edc7d5b2c6692b3a9e4a0ef5de4164b7a7a982afb8ce2dd6fe698ffd |
|
MD5 | bd19a100cfb5561f1b02606d30daa140 |
|
BLAKE2b-256 | d63be34468802894470f152e62a657dc3b5536456a2d6788b9f9b2bbefa97d91 |
File details
Details for the file ng_ai-0.2.10.3-py3-none-any.whl
.
File metadata
- Download URL: ng_ai-0.2.10.3-py3-none-any.whl
- Upload date:
- Size: 24.1 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: pdm/2.18.1 CPython/3.10.12 Linux/6.5.0-1025-azure
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | e54e54a662573e29a3c0bd30525a0240330eb760fcae2d72869a4b2f1bf6863b |
|
MD5 | fa5153bacc0faa133ceb9bfabdb95671 |
|
BLAKE2b-256 | 300c19d0beb32342a1d21458c6baafae463e1a951424b1fe0243caf7238d6e0b |