No project description provided
Project description
Fused Public Python package
🌎 Geospatial, with Python, at scale.
Fused.io is a Python library to process and store geospatial data - at scale. Express workflows as a set of shareable UDFS (user defined functions) without thinking about the underlying compute. The Fused Python library is maintained by Fused.io.
Prerequisites
Python >= 3.8
Install
The Fused Python package is currently distributed via a private beta. Email info@fused.io for access.
Quickstart
import fused
# Load data
census = 's3://fused-asset/infra/census_bg_us/'
buildings = 's3://fused-asset/infra/building_msft_us/'
# Declare UDF
@fused.udf()
def census_buildings_join(left, right):
import fused
df_joined = fused.utils.geo_join(left.data, right.data)
df_cnt = df_joined.groupby(['fused_index','GEOID']).size().to_frame('count_building').reset_index()
return df_cnt
# Instantiate job configuration that runs the data against the UDF
job = census_buildings_join(census, buildings)
# Run locally
job.run_local()
# Run on remote compute managed by Fused and view logs
job_id = job.run_remote(output_table='s3://my-s3-bucket/census_buildings_join')
job_id.tail_logs()
# Export job to local directory
job.export('census_buildings_join', overwrite=True)
# Re-import job
fused.load_job('census_buildings_join')
Available operations
The following are some of the key functions:
- ingest: Upload a dataset into S3 with the Fused format.
- open_table: Open a Table object given a path to the root of the table
- run_local: Execute data processing tasks locally while you test and debug.
- run_remote: Submit jobs to run on a remote clusters - by changing a single line of code.
- export: Save a job and its configuration as a local directory, zip file, or gist.
- load_job: Open a previously saved job.
- load_udf: Open a previously saved UDF.
- show: Debugger tool.
- render: Render job or UDF to new Notebook cell and edit.
See the Fused documentation for the full list of available functions.
Docs
The documentation is a work in progress. It follows the Diátaxis system:
- Getting Started Tutorial: A hands-on introduction to Fused.
- How-to guides: Simple step-by-step user guides to accomplish specific tasks.
- Reference guide: Commands, modules, classes and methods.
- Explanation: Discussion of key decisions and design philosophy of the library.
Changelog
See the changelog for the latest changes.
Releases
The project manages releases with Semantic Versioning Specification (SemVer).
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file fused-1.1.2.tar.gz.
File metadata
- Download URL: fused-1.1.2.tar.gz
- Upload date:
- Size: 150.5 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.6.1 CPython/3.11.4 Darwin/23.2.0
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | dd05958545b0a638139f2f44ef61b5f3b017b6a6a66492a47969acf11288c740 |
|
| MD5 | 2493fa1117c67a5ec40a29cce9f84b6b |
|
| BLAKE2b-256 | 2e35e40363ec9e8756e8924d2ac69eb51f6ad575fcd2a59f329d9810fab5aac8 |
File details
Details for the file fused-1.1.2-py3-none-any.whl.
File metadata
- Download URL: fused-1.1.2-py3-none-any.whl
- Upload date:
- Size: 195.7 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.6.1 CPython/3.11.4 Darwin/23.2.0
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | a19de38a4920b9d01d91f894ab0f48cfa095a68ce1e7b09c3eb07ab701b6e524 |
|
| MD5 | 019cf8a3c39db7f9ec83c32596812927 |
|
| BLAKE2b-256 | 9be2326658b7ec400b8cd64707ebdc7822f8195aaaefbe355d4ce08fe80f5c58 |
