Skip to main content

NebulaGraph AI Suite

Project description

NebulaGraph AI Suite(ng_ai)

NebulaGraph AI Suite with 4 line code to run Graph Algo on NebulaGraph

for NebulaGraph Docker Extension GitHub release (latest by date) License PyPI version pdm-managed Tests Docker Image Docker Image Docker Image Python 3.9


Documentation: https://github.com/wey-gu/nebulagraph-ai#documentation

Source Code: https://github.com/wey-gu/nebulagraph-ai


NebulaGraph AI Suite for Python (ng_ai) is a powerful Python library that offers APIs for data scientists to effectively read, write, analyze, and compute data in NebulaGraph.

With the support of a single-machine engine(NetworkX), or distributed computing environment using Spark, we could perform Graph Analysis and Algorithms on top of NebulaGraph in less than 10 lines of code, in unified and intuitive API.

Quick Start in 5 Minutes

Option 1: Try on your Desktop. Go with NebulaGraph Docker Extension!

Option 2: On Linux Server? Go with Nebula-Up ๐Ÿ‘‡๐Ÿป

Installation

pip install ng_ai

Usage

Call from nGQL

See more details in the docs

RETURN ng_ai("pagerank", ["follow"], ["degree"], "spark",
    {space: "basketballplayer", max_iter: 10}, {write_mode: "insert"})

Spark Engine Examples

See also: examples/spark_engine.ipynb

Run Algorithm on top of NebulaGraph:

Note, there is also a query mode, refer to examples or docs for more details.

from ng_ai import NebulaReader

# read data with spark engine, scan mode
reader = NebulaReader(engine="spark")
reader.scan(edge="follow", props="degree")
df = reader.read()

# run pagerank algorithm
pr_result = df.algo.pagerank(reset_prob=0.15, max_iter=10)

Write back to NebulaGraph:

from ng_ai import NebulaWriter
from ng_ai.config import NebulaGraphConfig

config = NebulaGraphConfig()

properties = {"louvain": "cluster_id"}

writer = NebulaWriter(
    data=df_result, sink="nebulagraph_vertex", config=config, engine="spark")
writer.set_options(
    tag="louvain", vid_field="_id", properties=properties,
    batch_size=256, write_mode="insert",)
writer.write()

Then we could query the result in NebulaGraph:

MATCH (v:louvain)
RETURN id(v), v.louvain.cluster_id LIMIT 10;

NebulaGraph Engine Examples

Basically the same as Spark Engine, but with engine="nebula", refer to examples or docs for more details.

- reader = NebulaReader(engine="spark")
+ reader = NebulaReader(engine="nebula")

Documentation

Environment Setup

API Reference

How it works

ng_ai is a unified abstraction layer for different engines, the current implementation is based on Spark, NetworkX, DGL, and NebulaGraph, but it's easy to extend to other engines like Flink, GraphScope, PyG, etc.

          โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
          โ”‚   Spark Cluster                                   โ”‚
          โ”‚    .โ”€โ”€โ”€โ”€โ”€.    .โ”€โ”€โ”€โ”€โ”€.    .โ”€โ”€โ”€โ”€โ”€.    .โ”€โ”€โ”€โ”€โ”€.       โ”‚
          โ”‚   ;       :  ;       :  ;       :  ;       :      โ”‚
       โ”Œโ”€โ–ถโ”‚   :       ;  :       ;  :       ;  :       ;      โ”‚
       โ”‚  โ”‚    โ•ฒ     โ•ฑ    โ•ฒ     โ•ฑ    โ•ฒ     โ•ฑ    โ•ฒ     โ•ฑ       โ”‚
       โ”‚  โ”‚     `โ”€โ”€โ”€'      `โ”€โ”€โ”€'      `โ”€โ”€โ”€'      `โ”€โ”€โ”€'        โ”‚
  Algo Spark                                                  โ”‚
    Engineโ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
       โ”‚  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
       โ””โ”€โ”€โ”ค                                                    โ”‚          โ”‚
          โ”‚   NebulaGraph AI Suite(ngai)                       โ”‚ ngai-api โ”‚โ—€โ”€โ”
          โ”‚                                                    โ”‚          โ”‚  โ”‚
          โ”‚                                                    โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค  โ”‚
          โ”‚     โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”    โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”    โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”   โ”Œโ”€โ”€โ”€โ”€โ”€โ”            โ”‚  โ”‚
          โ”‚     โ”‚ Reader โ”‚    โ”‚ Algo โ”‚    โ”‚ Writer โ”‚   โ”‚ GNN โ”‚            โ”‚  โ”‚
 โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ–ถโ”‚     โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜    โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”˜    โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜   โ””โ”€โ”€โ”€โ”€โ”€โ”˜            โ”‚  โ”‚
 โ”‚        โ”‚          โ”‚            โ”‚            โ”‚          โ”‚               โ”‚  โ”‚
 โ”‚        โ”‚          โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”    โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”        โ”‚  โ”‚
 โ”‚        โ”‚          โ–ผ                โ–ผ              โ–ผ           โ–ผ        โ”‚  โ”‚
 โ”‚        โ”‚   โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”   โ”‚  โ”‚
 โ”‚     โ”Œโ”€โ”€โ”ค   โ”‚ SparkEngine โ”‚ โ”‚ NebulaEngine โ”‚ โ”‚ NetworkX โ”‚โ”‚ DGLEngineโ”‚   โ”‚  โ”‚
 โ”‚     โ”‚  โ”‚   โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜   โ”‚  โ”‚
 โ”‚     โ”‚  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜  โ”‚
 โ”‚     โ”‚             โ”‚        Spark                                          โ”‚
 โ”‚     โ”‚             โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€Reader โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”                           โ”‚
 โ”‚  Spark                   Query Mode           โ”‚                           โ”‚
 โ”‚  Reader                                       โ”‚                           โ”‚
 โ”‚Scan Mode                                      โ–ผ                      โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
 โ”‚     โ”‚  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค ngai-udfโ”‚โ—€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
 โ”‚     โ”‚  โ”‚                                                   โ”‚         โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค              โ”‚
 โ”‚     โ”‚  โ”‚  NebulaGraph Graph Engine         Nebula-GraphD   โ”‚   ngai-GraphD     โ”‚              โ”‚
 โ”‚     โ”‚  โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜              โ”‚
 โ”‚     โ”‚  โ”‚                              โ”‚                    โ”‚                                  โ”‚
 โ”‚     โ”‚  โ”‚  NebulaGraph Storage Engine  โ”‚                    โ”‚                                  โ”‚
 โ”‚     โ”‚  โ”‚                              โ”‚                    โ”‚                                  โ”‚
 โ”‚     โ””โ”€โ–ถโ”‚  Nebula-StorageD             โ”‚    Nebula-Metad    โ”‚                                  โ”‚
 โ”‚        โ”‚                              โ”‚                    โ”‚                                  โ”‚
 โ”‚        โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜                                  โ”‚
 โ”‚                                                                                               โ”‚
 โ”‚    โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”  โ”‚
 โ”‚    โ”‚ RETURN ng_ai("pagerank", ["follow"], ["degree"], "spark", {space:"basketballplayer"}) โ”‚โ”€โ”€โ”˜
 โ”‚    โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
 โ”‚  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
 โ”‚  โ”‚ from ng_ai import NebulaReader                              โ”‚
 โ”‚  โ”‚                                                             โ”‚
 โ”‚  โ”‚ # read data with spark engine, scan mode                    โ”‚
 โ”‚  โ”‚ reader = NebulaReader(engine="spark")                       โ”‚
 โ”‚  โ”‚ reader.scan(edge="follow", props="degree")                  โ”‚
 โ””โ”€โ”€โ”‚ df = reader.read()                                          โ”‚
    โ”‚                                                             โ”‚
    โ”‚ # run pagerank algorithm                                    โ”‚
    โ”‚ pr_result = df.algo.pagerank(reset_prob=0.15, max_iter=10)  โ”‚
    โ”‚                                                             โ”‚
    โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜  

Spark Engine Prerequisites

NebulaGraph Engine Prerequisites

Contributing

See HACKING.md for details.

License

This project is licensed under the terms of the Apache License 2.0.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ng-ai-0.2.10.3.tar.gz (29.3 kB view details)

Uploaded Source

Built Distribution

ng_ai-0.2.10.3-py3-none-any.whl (24.1 kB view details)

Uploaded Python 3

File details

Details for the file ng-ai-0.2.10.3.tar.gz.

File metadata

  • Download URL: ng-ai-0.2.10.3.tar.gz
  • Upload date:
  • Size: 29.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: pdm/2.18.1 CPython/3.10.12 Linux/6.5.0-1025-azure

File hashes

Hashes for ng-ai-0.2.10.3.tar.gz
Algorithm Hash digest
SHA256 880a4ae1edc7d5b2c6692b3a9e4a0ef5de4164b7a7a982afb8ce2dd6fe698ffd
MD5 bd19a100cfb5561f1b02606d30daa140
BLAKE2b-256 d63be34468802894470f152e62a657dc3b5536456a2d6788b9f9b2bbefa97d91

See more details on using hashes here.

File details

Details for the file ng_ai-0.2.10.3-py3-none-any.whl.

File metadata

  • Download URL: ng_ai-0.2.10.3-py3-none-any.whl
  • Upload date:
  • Size: 24.1 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: pdm/2.18.1 CPython/3.10.12 Linux/6.5.0-1025-azure

File hashes

Hashes for ng_ai-0.2.10.3-py3-none-any.whl
Algorithm Hash digest
SHA256 e54e54a662573e29a3c0bd30525a0240330eb760fcae2d72869a4b2f1bf6863b
MD5 fa5153bacc0faa133ceb9bfabdb95671
BLAKE2b-256 300c19d0beb32342a1d21458c6baafae463e1a951424b1fe0243caf7238d6e0b

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page