process data-trees with reconfigurable-paths
Project description
.. image:: doc/_static/pandalone_logo.png
:width: 300 px
:align: center
#######################################################
pandalone: process data-trees with reconfigurable-paths
#######################################################
|dev-status| |travis-status| |appveyor-status| |cover-status| |docs-status| \
|pypi-status| |downloads-count| |github-issues| |proj-license|
:Release: 0.1.0
:Documentation: https://pandalone.readthedocs.org/
:Source: https://github.com/pandalone/pandalone
:PyPI repo: https://pypi-hypernode.com/pypi/pandalone
:Keywords: calculation, data, dependencies, engineering, excel, library,
numpy, pandas, processing, python, resolution, scientific,
simulink, tree, utility
:Copyright: 2015 European Commission (`JRC-IET
<https://ec.europa.eu/jrc/en/institutes/iet>`_)
:License: `EUPL 1.1+ <https://joinup.ec.europa.eu/software/page/eupl>`_
**pandalone** is an open source Python 2/3 library for building
*component-functions* to process *hierarchical-data* using
*reconfigurable-paths*.
Our goal is to facilitate the composition of *engineering-models* from
loosely-coupled *components*.
Initially envisioned as an *indirection-framework* around *pandas* coupled
with a *dependency-resolver*, every such model should auto-adapt and process
only values available, and allow *remapping* of the paths accessing them,
to run on renamed/relocated *value-trees* without component-code modifications.
It is written for *python-3.4* but tested under both *python-2.7* and
*python-3.3+*, for *Windows* and *Linux*.
.. Note::
The project, as of May-2015, is considered at an alpha-stage,
without any released version in *pypi* yet.
.. _end-opening:
.. contents:: Table of Contents
:backlinks: top
.. _begin-intro:
Introduction
============
Overview
--------
At the most fundamental level, an "execution" or a "run" of a processing
can be thought like that ::
.--------------. _____________ .-------------.
; DataTree ; | | ; DataTree ;
;--------------; ==> | <cfunc_1> | ==> ;--------------;
; /some/data ; | <cfunc_2> | ; /some/data ;
; /some/other ; | ... | ; /some/other ;
; /foo/bar ; |_____________| ; /foo/bar ;
'--------------' '--------------.
- The *data-tree* might come from *json*, *hdf5*, *excel-workbooks*, or
plain dictionaries and lists.
Its values are strings and numbers, *numpy-lists*, *pandas* or
*xray-datasets*, etc.
- The *component-functions* must abide to the following simple signature::
cfunc_do_something(pandelone, datatree)
and must not return any value, just read and write into the data-tree.
- Here is a simple component-function:
.. code-block:: python
def cfunc_standardize(pandelone, datatree):
pin, pon = pandelone.paths(),
df = datatree.get(pin.A)
df[pon.A.B_std] = df[pin.A.B] / df[pin.A.B].std()
- Notice the use of the *reconfigurable-paths* marked specifically as input or
output.
- TODO: continue rough example in tutorial...
Quick-start
-----------
.. Note::
The program runs on **Python-2.7+** and **Python-3.3+** (preferred) and requires
**numpy/scipy**, **pandas** and **win32** libraries along with their *native backends* to be installed.
If you do not have such an environment already installed, please read doc(`install`) section below for
suitable distributions such as |winpython|_ or |anaconda|_.
.. _begin-replacements:
.. |virtualenv| replace:: *virtualenv* (isolated Python environment)
.. _virtualenv: http://docs.python-guide.org/en/latest/dev/virtualenvs/
.. |pypi| replace:: *PyPi* repo
.. _pypi: https://pypi-hypernode.com/pypi/pandalone
.. |winpython| replace:: *WinPython*
.. _winpython: http://winpython.github.io/
.. |anaconda| replace:: *Anaconda*
.. _anaconda: http://docs.continuum.io/anaconda/
.. |travis-status| image:: https://travis-ci.org/pandalone/pandalone.svg
:alt: Travis build status
:scale: 100%
:target: https://travis-ci.org/pandalone/pandalone
.. |appveyor-status| image:: https://ci.appveyor.com/api/projects/status/jayah84y3ae7ddfc?svg=true
:alt: Apveyor build status
:scale: 100%
:target: https://ci.appveyor.com/project/ankostis/pandalone
.. |cover-status| image:: https://coveralls.io/repos/pandalone/pandalone/badge.svg
:target: https://coveralls.io/r/pandalone/pandalone
.. |docs-status| image:: https://readthedocs.org/projects/pandalone/badge/
:alt: Documentation status
:scale: 100%
:target: https://readthedocs.org/builds/pandalone/
.. |pypi-status| image:: https://pypip.in/v/pandalone/badge.png
:target: https://pypi-hypernode.com/pypi/pandalone/
:alt: Latest Version in PyPI
.. |python-ver| image:: https://pypip.in/py_versions/pandalone/badge.svg
:target: https://pypi-hypernode.com/pypi/pandalone/
:alt: Supported Python versions
.. |dev-status| image:: https://pypip.in/status/pandalone/badge.svg
:target: https://pypi-hypernode.com/pypi/pandalone/
:alt: Development Status
.. |downloads-count| image:: https://pypip.in/download/pandalone/badge.svg?period=week
:target: https://pypi-hypernode.com/pypi/pandalone/
:alt: Downloads
.. |github-issues| image:: https://img.shields.io/github/issues/pandalone/pandalone.svg
:target: https://github.com/pandalone/pandalone/issues
:alt: Issues count
.. |proj-license| image:: https://img.shields.io/badge/license-EUPL%201.1%2B-blue.svg
:target: https://raw.githubusercontent.com/pandalone/pandalone/master/LICENSE.txt
:alt: Project License
:width: 300 px
:align: center
#######################################################
pandalone: process data-trees with reconfigurable-paths
#######################################################
|dev-status| |travis-status| |appveyor-status| |cover-status| |docs-status| \
|pypi-status| |downloads-count| |github-issues| |proj-license|
:Release: 0.1.0
:Documentation: https://pandalone.readthedocs.org/
:Source: https://github.com/pandalone/pandalone
:PyPI repo: https://pypi-hypernode.com/pypi/pandalone
:Keywords: calculation, data, dependencies, engineering, excel, library,
numpy, pandas, processing, python, resolution, scientific,
simulink, tree, utility
:Copyright: 2015 European Commission (`JRC-IET
<https://ec.europa.eu/jrc/en/institutes/iet>`_)
:License: `EUPL 1.1+ <https://joinup.ec.europa.eu/software/page/eupl>`_
**pandalone** is an open source Python 2/3 library for building
*component-functions* to process *hierarchical-data* using
*reconfigurable-paths*.
Our goal is to facilitate the composition of *engineering-models* from
loosely-coupled *components*.
Initially envisioned as an *indirection-framework* around *pandas* coupled
with a *dependency-resolver*, every such model should auto-adapt and process
only values available, and allow *remapping* of the paths accessing them,
to run on renamed/relocated *value-trees* without component-code modifications.
It is written for *python-3.4* but tested under both *python-2.7* and
*python-3.3+*, for *Windows* and *Linux*.
.. Note::
The project, as of May-2015, is considered at an alpha-stage,
without any released version in *pypi* yet.
.. _end-opening:
.. contents:: Table of Contents
:backlinks: top
.. _begin-intro:
Introduction
============
Overview
--------
At the most fundamental level, an "execution" or a "run" of a processing
can be thought like that ::
.--------------. _____________ .-------------.
; DataTree ; | | ; DataTree ;
;--------------; ==> | <cfunc_1> | ==> ;--------------;
; /some/data ; | <cfunc_2> | ; /some/data ;
; /some/other ; | ... | ; /some/other ;
; /foo/bar ; |_____________| ; /foo/bar ;
'--------------' '--------------.
- The *data-tree* might come from *json*, *hdf5*, *excel-workbooks*, or
plain dictionaries and lists.
Its values are strings and numbers, *numpy-lists*, *pandas* or
*xray-datasets*, etc.
- The *component-functions* must abide to the following simple signature::
cfunc_do_something(pandelone, datatree)
and must not return any value, just read and write into the data-tree.
- Here is a simple component-function:
.. code-block:: python
def cfunc_standardize(pandelone, datatree):
pin, pon = pandelone.paths(),
df = datatree.get(pin.A)
df[pon.A.B_std] = df[pin.A.B] / df[pin.A.B].std()
- Notice the use of the *reconfigurable-paths* marked specifically as input or
output.
- TODO: continue rough example in tutorial...
Quick-start
-----------
.. Note::
The program runs on **Python-2.7+** and **Python-3.3+** (preferred) and requires
**numpy/scipy**, **pandas** and **win32** libraries along with their *native backends* to be installed.
If you do not have such an environment already installed, please read doc(`install`) section below for
suitable distributions such as |winpython|_ or |anaconda|_.
.. _begin-replacements:
.. |virtualenv| replace:: *virtualenv* (isolated Python environment)
.. _virtualenv: http://docs.python-guide.org/en/latest/dev/virtualenvs/
.. |pypi| replace:: *PyPi* repo
.. _pypi: https://pypi-hypernode.com/pypi/pandalone
.. |winpython| replace:: *WinPython*
.. _winpython: http://winpython.github.io/
.. |anaconda| replace:: *Anaconda*
.. _anaconda: http://docs.continuum.io/anaconda/
.. |travis-status| image:: https://travis-ci.org/pandalone/pandalone.svg
:alt: Travis build status
:scale: 100%
:target: https://travis-ci.org/pandalone/pandalone
.. |appveyor-status| image:: https://ci.appveyor.com/api/projects/status/jayah84y3ae7ddfc?svg=true
:alt: Apveyor build status
:scale: 100%
:target: https://ci.appveyor.com/project/ankostis/pandalone
.. |cover-status| image:: https://coveralls.io/repos/pandalone/pandalone/badge.svg
:target: https://coveralls.io/r/pandalone/pandalone
.. |docs-status| image:: https://readthedocs.org/projects/pandalone/badge/
:alt: Documentation status
:scale: 100%
:target: https://readthedocs.org/builds/pandalone/
.. |pypi-status| image:: https://pypip.in/v/pandalone/badge.png
:target: https://pypi-hypernode.com/pypi/pandalone/
:alt: Latest Version in PyPI
.. |python-ver| image:: https://pypip.in/py_versions/pandalone/badge.svg
:target: https://pypi-hypernode.com/pypi/pandalone/
:alt: Supported Python versions
.. |dev-status| image:: https://pypip.in/status/pandalone/badge.svg
:target: https://pypi-hypernode.com/pypi/pandalone/
:alt: Development Status
.. |downloads-count| image:: https://pypip.in/download/pandalone/badge.svg?period=week
:target: https://pypi-hypernode.com/pypi/pandalone/
:alt: Downloads
.. |github-issues| image:: https://img.shields.io/github/issues/pandalone/pandalone.svg
:target: https://github.com/pandalone/pandalone/issues
:alt: Issues count
.. |proj-license| image:: https://img.shields.io/badge/license-EUPL%201.1%2B-blue.svg
:target: https://raw.githubusercontent.com/pandalone/pandalone/master/LICENSE.txt
:alt: Project License
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
pandalone-0.1.0.tar.gz
(214.0 kB
view details)
Built Distribution
File details
Details for the file pandalone-0.1.0.tar.gz
.
File metadata
- Download URL: pandalone-0.1.0.tar.gz
- Upload date:
- Size: 214.0 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3e92a1f6fe675cb30ab9ab217ab260f2782c2412a9d26e67085d0f5f691134e9 |
|
MD5 | 73d1545a000f892678996e72d6e2b820 |
|
BLAKE2b-256 | 346ca499e55959a1b784a9a2b97d5e72c21adbdb8849ff7c4d78cacf5008c336 |
File details
Details for the file pandalone-0.1.0-py2.py3-none-any.whl
.
File metadata
- Download URL: pandalone-0.1.0-py2.py3-none-any.whl
- Upload date:
- Size: 80.7 kB
- Tags: Python 2, Python 3
- Uploaded using Trusted Publishing? No
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4b3f3e9024afde968e4d7f39d9858ed193ab4915bfd56f92a522a8f3430ada1f |
|
MD5 | a0441dcb8a06d82d39e212001bb54a9e |
|
BLAKE2b-256 | ff6fe220de40c27dd976c8652d165a40417e7ae8ce36c7723db55ab89caac4f2 |