histogrammar 1.0.24

Announcements

Spark 3.0

With Spark 3.0, based on Scala 2.12, make sure to pick up the correct histogrammar jar file:

spark = SparkSession.builder.config("spark.jars.packages", "io.github.histogrammar:histogrammar_2.12:1.0.20,io.github.histogrammar:histogrammar-sparksql_2.12:1.0.20").getOrCreate()

For Spark 2.X compiled against scala 2.11, in the string above simply replace “2.12” with “2.11”.

February, 2021

Example notebooks

Documentation

See histogrammar-docs for a complete introduction to histogrammar. (A bit old but still good.) There you can also find documentation about the Scala implementation of histogrammar.

Check it out

The historgrammar library requires Python 3.6+ and is pip friendly. To get started, simply do:

$ pip install histogrammar

or check out the code from our GitHub repository:

$ git clone https://github.com/histogrammar/histogrammar-python
$ pip install -e histogrammar-python

where in this example the code is installed in edit mode (option -e).

You can now use the package in Python with:

import histogrammar

Congratulations, you are now ready to use the histogrammar library!

Quick run

As a quick example, you can do:

import pandas as pd
import histogrammar as hg
from histogrammar import resources

# open synthetic data
df = pd.read_csv(resources.data('test.csv.gz'), parse_dates=['date'])
df.head()

# create a histogram, tell it to look for column 'age'
# fill the histogram with column 'age' and plot it
hist = hg.Histogram(num=100, low=0, high=100, quantity='age')
hist.fill.numpy(df)
hist.plot.matplotlib()

# generate histograms of all features in the dataframe using automatic binning
# (importing histogrammar automatically adds this functionality to a pandas or spark dataframe)
hists = df.hg_make_histograms()
print(hists.keys())

# multi-dimensional histograms are also supported. e.g. features longitude vs latitude
hists = df.hg_make_histograms(features=['longitude:latitude'])
ll = hists['longitude:latitude']
ll.plot.matplotlib()

# store histogram and retrieve it again
ll.toJsonFile('longitude_latitude.json')
ll2 = hg.Factory().fromJsonFile('longitude_latitude.json')

These examples also work with Spark dataframes (sdf):

from pyspark.sql.functions import col
hist = hg.Histogram(num=100, low=0, high=100, quantity=col('age'))
hist.fill.sparksql(sdf)

For more examples please see the example notebooks and tutorials.

Project contributors

This package was originally authored by DIANA-HEP and is now maintained by volunteers.

Contact and support

• Issues & Ideas & Support: https://github.com/histogrammar/histogrammar-python/issues

Please note that histogrammar is supported only on a best-effort basis.

License

histogrammar is completely free, open-source and licensed under the Apache-2.0 license.