Skip to main content

High-level Python abstraction layer for Amazon S3

Project description

baiji
=====

High-level Python abstraction layer for Amazon S3:

1. An [`open`][open]-like context handler which allows using S3 keys and
local files interchangeably.
- When reading S3, contents are first written to a temporary local
file.
- When writing S3, contents are written to a temporary local file,
and uploaded on close.
2. An `s3` CLI for listing, copying, syncing, and other common activities.


Features
--------

- Works without an S3 connection (with local files).
- Supports multiprocess parallelism for copying lots of files.
- Supports Python 2.7 and uses boto2.
- Supports OS X, Linux, and Windows.
- Tested and production-hardened.

[open]: https://docs.python.org/2/library/functions.html#open


Examples
--------

```py
with s3.open('s3://example/info.txt', 'w') as f:
f.write('hello')

with s3.open('file:///home/me/info.txt', 'w') as f:
f.write('hello')

with s3.open('s3://example/info.txt', 'r') as f:
contents = f.readlines()

with s3.open('file:///home/me/info.txt', 'r') as f:
contents = f.readlines()
```

```sh
s3 cp foo.txt s3://example/bar.txt
s3 cp s3://example/bar.txt s3://another-example/bazinga.txt
s3 rm s3://example/bar.txt
```


Development
-----------

```sh
pip install -r requirements_dev.txt
rake test
rake lint
```


TODO
----

1. Migrate credentials to `~/.aws/credentials` or env, and deprecate AWS
credential support in `~/.bodylabs`.
2. Move `baiji.util.parallel` into a separate library.
3. Upgrade to boto3.


Contribute
----------

- Issue Tracker: github.com/bodylabs/baiji/issues
- Source Code: github.com/bodylabs/baiji

Pull requests welcome!


Support
-------

If you are having issues, please let us know.


License
-------

The project is licensed under the Apache license, version 2.0.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

baiji-2.6.1.tar.gz (39.0 kB view details)

Uploaded Source

Built Distribution

baiji-2.6.1-py2-none-any.whl (48.8 kB view details)

Uploaded Python 2

File details

Details for the file baiji-2.6.1.tar.gz.

File metadata

  • Download URL: baiji-2.6.1.tar.gz
  • Upload date:
  • Size: 39.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for baiji-2.6.1.tar.gz
Algorithm Hash digest
SHA256 4c902b28140ee688a7383fe5d3e636de9f69972ae832467fa1a3a2e47cf8c8f2
MD5 84397f80e7eb453983a236b9804e3b48
BLAKE2b-256 f1469d2c776562ba186fbb304efda079fe0acd4bae7d8af941f19cb1dacc96c9

See more details on using hashes here.

Provenance

File details

Details for the file baiji-2.6.1-py2-none-any.whl.

File metadata

File hashes

Hashes for baiji-2.6.1-py2-none-any.whl
Algorithm Hash digest
SHA256 09f1d911db88463a5ffa7914904aaab12e47a4d987172829270cf02b4cd4e789
MD5 31281c3a4c5447898b19c87fa3f62a05
BLAKE2b-256 670876135659918a010a7aefdf7dc2c642dfbfd65a23d324c0784a5e8706f8db

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page