Skip to main content

Virtual large arrays and lazy evaluation

Project description

Build Status

Virtual large arrays and lazy evaluation.

For example, we can combine multiple array data sources into a single virtual array:

>>> first_time_series = OrthoArrayAdapter(hdf_var_a)
>>> second_time_series = OrthoArrayAdapater(hdf_var_b)
>>> print first_time_series.shape, second_time_series.shape
(52000, 800, 600) (56000, 800, 600)
>>> time_series = biggus.LinearMosaic([first_time_series, second_time_series], axis=0)
>>> time_series
<LinearMosaic shape=(108000, 800, 600) dtype=dtype('float32')>

Any biggus Array can then be indexed, independent of underlying data sources:

>>> time_series[51999:52001, 10, 12]
<LinearMosaic shape=(2,) dtype=dtype('float32')>

And an Array can be converted to a numpy ndarray on demand:

>>> time_series[51999:52001, 10, 12].ndarray()
array([ 0.72151309,  0.54654914], dtype=float32)

Get in touch!

We’ve got lots of exciting plans underway for biggus, but we’re also very keen to hear from you.

  • How are you thinking of using biggus?

  • What capabilities does biggus need for it to be useful to you?

  • What capabilities does biggus already have that you find useful?

Further reading

To get more ideas of what Biggus can do, please browse the wiki, and its examples.

If you have any questions or feedback please feel free to post to the discussion group or raise an issue on the issue tracker.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

Biggus-0.4.0.tar.gz (30.9 kB view details)

Uploaded Source

File details

Details for the file Biggus-0.4.0.tar.gz.

File metadata

  • Download URL: Biggus-0.4.0.tar.gz
  • Upload date:
  • Size: 30.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for Biggus-0.4.0.tar.gz
Algorithm Hash digest
SHA256 c95c3d3edff2d4bf852cb379bd9744906edadc79d6d27d3c1eb6d0f37858623e
MD5 8f5919683ad7c530cd0ad168efe7c32c
BLAKE2b-256 5ddb1cf7865b524da9eee21182aa331ab38a3fc2df20493e0112b20708f530df

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page