Skip to main content

High-level Python abstraction layer for Amazon S3

Project description

baiji
=====

High-level Python abstraction layer for Amazon S3:

1. An [`open`][open]-like context handler which allows using S3 keys and
local files interchangeably.
- When reading S3, contents are first written to a temporary local
file.
- When writing S3, contents are written to a temporary local file,
and uploaded on close.
2. An `s3` CLI for listing, copying, syncing, and other common activities.


Features
--------

- Works without an S3 connection (with local files).
- Supports multiprocess parallelism for copying lots of files.
- Supports Python 2.7 and uses boto2.
- Supports OS X, Linux, and Windows.
- Tested and production-hardened.

[open]: https://docs.python.org/2/library/functions.html#open


Examples
--------

```py
with s3.open('s3://example/info.txt', 'w') as f:
f.write('hello')

with s3.open('file:///home/me/info.txt', 'w') as f:
f.write('hello')

with s3.open('s3://example/info.txt', 'r') as f:
contents = f.readlines()

with s3.open('file:///home/me/info.txt', 'r') as f:
contents = f.readlines()
```

```sh
s3 cp foo.txt s3://example/bar.txt
s3 cp s3://example/bar.txt s3://another-example/bazinga.txt
s3 rm s3://example/bar.txt
```


Development
-----------

```sh
pip install -r requirements_dev.txt
rake test
rake lint
```


TODO
----

1. Migrate credentials to `~/.aws/credentials` or env, and deprecate AWS
credential support in `~/.bodylabs`.
2. Move `baiji.util.parallel` into a separate library.
3. Upgrade to boto3.


Contribute
----------

- Issue Tracker: github.com/bodylabs/baiji/issues
- Source Code: github.com/bodylabs/baiji

Pull requests welcome!


Support
-------

If you are having issues, please let us know.


License
-------

The project is licensed under the Apache license, version 2.0.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

baiji-2.7.2.tar.gz (40.0 kB view details)

Uploaded Source

Built Distribution

baiji-2.7.2-py2-none-any.whl (49.9 kB view details)

Uploaded Python 2

File details

Details for the file baiji-2.7.2.tar.gz.

File metadata

  • Download URL: baiji-2.7.2.tar.gz
  • Upload date:
  • Size: 40.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for baiji-2.7.2.tar.gz
Algorithm Hash digest
SHA256 70814b7338e5d2dd233858a004dba93c962ae6865fca9799c23218b8e3bc12d0
MD5 527c103803982933ef25de07a5285b43
BLAKE2b-256 69dd08d1e3f63733c51e214a94ff085f22afebcf10d3300c06e1b628b63a681b

See more details on using hashes here.

Provenance

File details

Details for the file baiji-2.7.2-py2-none-any.whl.

File metadata

File hashes

Hashes for baiji-2.7.2-py2-none-any.whl
Algorithm Hash digest
SHA256 58bbd7c062e8888f003fa3edde4453662b52e2e672fdf05f13780036b32d8c6f
MD5 e8cfb14e9fbb8e2b525204101add4c1e
BLAKE2b-256 db9c413116f1d41e30a02a3c630d7b8ab575ef3a141438a1e336ff6cb7bf7395

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page