Skip to main content

process data-trees with reconfigurable-paths

Project description

.. image:: doc/_static/pandalone_logo.png
:width: 300 px
:align: center

#######################################################
pandalone: process data-trees with reconfigurable-paths
#######################################################
|dev-status| |travis-status| |appveyor-status| |cover-status| |docs-status| \
|pypi-status| |downloads-count| |github-issues| |proj-license|

:Release: 0.1.0
:Documentation: https://pandalone.readthedocs.org/
:Source: https://github.com/pandalone/pandalone
:PyPI repo: https://pypi-hypernode.com/pypi/pandalone
:Keywords: calculation, data, dependencies, engineering, excel, library,
numpy, pandas, processing, python, resolution, scientific,
simulink, tree, utility
:Copyright: 2015 European Commission (`JRC-IET
<https://ec.europa.eu/jrc/en/institutes/iet>`_)
:License: `EUPL 1.1+ <https://joinup.ec.europa.eu/software/page/eupl>`_


**pandalone** is an open source Python 2/3 library for building
*component-functions* to process *hierarchical-data* using
*reconfigurable-paths*.


Our goal is to facilitate the composition of *engineering-models* from
loosely-coupled *components*.
Initially envisioned as an *indirection-framework* around *pandas* coupled
with a *dependency-resolver*, every such model should auto-adapt and process
only values available, and allow *remapping* of the paths accessing them,
to run on renamed/relocated *value-trees* without component-code modifications.

It is written for *python-3.4* but tested under both *python-2.7* and
*python-3.3+*, for *Windows* and *Linux*.

.. Note::
The project, as of May-2015, is considered at an alpha-stage,
without any released version in *pypi* yet.


.. _end-opening:
.. contents:: Table of Contents
:backlinks: top
.. _begin-intro:

Introduction
============

Overview
--------

At the most fundamental level, an "execution" or a "run" of a processing
can be thought like that ::

.--------------. _____________ .-------------.
; DataTree ; | | ; DataTree ;
;--------------; ==> | <cfunc_1> | ==> ;--------------;
; /some/data ; | <cfunc_2> | ; /some/data ;
; /some/other ; | ... | ; /some/other ;
; /foo/bar ; |_____________| ; /foo/bar ;
'--------------' '--------------.


- The *data-tree* might come from *json*, *hdf5*, *excel-workbooks*, or
plain dictionaries and lists.
Its values are strings and numbers, *numpy-lists*, *pandas* or
*xray-datasets*, etc.

- The *component-functions* must abide to the following simple signature::

cfunc_do_something(pandelone, datatree)

and must not return any value, just read and write into the data-tree.

- Here is a simple component-function:

.. code-block:: python

def cfunc_standardize(pandelone, datatree):
pin, pon = pandelone.paths(),
df = datatree.get(pin.A)
df[pon.A.B_std] = df[pin.A.B] / df[pin.A.B].std()

- Notice the use of the *reconfigurable-paths* marked specifically as input or
output.

- TODO: continue rough example in tutorial...


Quick-start
-----------

.. Note::
The program runs on **Python-2.7+** and **Python-3.3+** (preferred) and requires
**numpy/scipy**, **pandas** and **win32** libraries along with their *native backends* to be installed.
If you do not have such an environment already installed, please read doc(`install`) section below for
suitable distributions such as |winpython|_ or |anaconda|_.




.. _begin-replacements:

.. |virtualenv| replace:: *virtualenv* (isolated Python environment)
.. _virtualenv: http://docs.python-guide.org/en/latest/dev/virtualenvs/

.. |pypi| replace:: *PyPi* repo
.. _pypi: https://pypi-hypernode.com/pypi/pandalone

.. |winpython| replace:: *WinPython*
.. _winpython: http://winpython.github.io/

.. |anaconda| replace:: *Anaconda*
.. _anaconda: http://docs.continuum.io/anaconda/

.. |travis-status| image:: https://travis-ci.org/pandalone/pandalone.svg
:alt: Travis build status
:scale: 100%
:target: https://travis-ci.org/pandalone/pandalone

.. |appveyor-status| image:: https://ci.appveyor.com/api/projects/status/jayah84y3ae7ddfc?svg=true
:alt: Apveyor build status
:scale: 100%
:target: https://ci.appveyor.com/project/ankostis/pandalone

.. |cover-status| image:: https://coveralls.io/repos/pandalone/pandalone/badge.svg
:target: https://coveralls.io/r/pandalone/pandalone

.. |docs-status| image:: https://readthedocs.org/projects/pandalone/badge/
:alt: Documentation status
:scale: 100%
:target: https://readthedocs.org/builds/pandalone/

.. |pypi-status| image:: https://pypip.in/v/pandalone/badge.png
:target: https://pypi-hypernode.com/pypi/pandalone/
:alt: Latest Version in PyPI

.. |python-ver| image:: https://pypip.in/py_versions/pandalone/badge.svg
:target: https://pypi-hypernode.com/pypi/pandalone/
:alt: Supported Python versions

.. |dev-status| image:: https://pypip.in/status/pandalone/badge.svg
:target: https://pypi-hypernode.com/pypi/pandalone/
:alt: Development Status

.. |downloads-count| image:: https://pypip.in/download/pandalone/badge.svg?period=week
:target: https://pypi-hypernode.com/pypi/pandalone/
:alt: Downloads

.. |github-issues| image:: https://img.shields.io/github/issues/pandalone/pandalone.svg
:target: https://github.com/pandalone/pandalone/issues
:alt: Issues count

.. |proj-license| image:: https://img.shields.io/badge/license-EUPL%201.1%2B-blue.svg
:target: https://raw.githubusercontent.com/pandalone/pandalone/master/LICENSE.txt
:alt: Project License

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pandalone-0.1.0.tar.gz (214.0 kB view details)

Uploaded Source

Built Distribution

pandalone-0.1.0-py2.py3-none-any.whl (80.7 kB view details)

Uploaded Python 2 Python 3

File details

Details for the file pandalone-0.1.0.tar.gz.

File metadata

  • Download URL: pandalone-0.1.0.tar.gz
  • Upload date:
  • Size: 214.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for pandalone-0.1.0.tar.gz
Algorithm Hash digest
SHA256 3e92a1f6fe675cb30ab9ab217ab260f2782c2412a9d26e67085d0f5f691134e9
MD5 73d1545a000f892678996e72d6e2b820
BLAKE2b-256 346ca499e55959a1b784a9a2b97d5e72c21adbdb8849ff7c4d78cacf5008c336

See more details on using hashes here.

File details

Details for the file pandalone-0.1.0-py2.py3-none-any.whl.

File metadata

File hashes

Hashes for pandalone-0.1.0-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 4b3f3e9024afde968e4d7f39d9858ed193ab4915bfd56f92a522a8f3430ada1f
MD5 a0441dcb8a06d82d39e212001bb54a9e
BLAKE2b-256 ff6fe220de40c27dd976c8652d165a40417e7ae8ce36c7723db55ab89caac4f2

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page