Skip to main content

High-level Python abstraction layer for Amazon S3

Project description

baiji
=====

High-level Python abstraction layer for Amazon S3:

1. An [`open`][open]-like context handler which allows using S3 keys and
local files interchangeably.
- When reading S3, contents are first written to a temporary local
file.
- When writing S3, contents are written to a temporary local file,
and uploaded on close.
2. An `s3` CLI for listing, copying, syncing, and other common activities.


Features
--------

- Works without an S3 connection (with local files).
- Supports multiprocess parallelism for copying lots of files.
- Supports Python 2.7 and uses boto2.
- Supports OS X, Linux, and Windows.
- Tested and production-hardened.

[open]: https://docs.python.org/2/library/functions.html#open


Examples
--------

```py
with s3.open('s3://example/info.txt', 'w') as f:
f.write('hello')

with s3.open('file:///home/me/info.txt', 'w') as f:
f.write('hello')

with s3.open('s3://example/info.txt', 'r') as f:
contents = f.readlines()

with s3.open('file:///home/me/info.txt', 'r') as f:
contents = f.readlines()
```

```sh
s3 cp foo.txt s3://example/bar.txt
s3 cp s3://example/bar.txt s3://another-example/bazinga.txt
s3 rm s3://example/bar.txt
```


Development
-----------

```sh
pip install -r requirements_dev.txt
rake test
rake lint
```


TODO
----

1. Migrate credentials to `~/.aws/credentials` or env, and deprecate AWS
credential support in `~/.bodylabs`.
2. Move `baiji.util.parallel` into a separate library.
3. Upgrade to boto3.


Contribute
----------

- Issue Tracker: github.com/bodylabs/baiji/issues
- Source Code: github.com/bodylabs/baiji

Pull requests welcome!


Support
-------

If you are having issues, please let us know.


License
-------

The project is licensed under the Apache license, version 2.0.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

baiji-2.5.3.tar.gz (39.0 kB view details)

Uploaded Source

Built Distribution

baiji-2.5.3-py2-none-any.whl (48.6 kB view details)

Uploaded Python 2

File details

Details for the file baiji-2.5.3.tar.gz.

File metadata

  • Download URL: baiji-2.5.3.tar.gz
  • Upload date:
  • Size: 39.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for baiji-2.5.3.tar.gz
Algorithm Hash digest
SHA256 ff7e74079924ce6d2e0c7d6c8ab0c8c36acd1f334b6e0f9746ef8be26c8e099f
MD5 3839f8ac354c524e3b37f045e79cfd1b
BLAKE2b-256 207477fb97f177aef442f0aa246162bfaec8ece42dd9a7887d3f15cee2a386d0

See more details on using hashes here.

Provenance

File details

Details for the file baiji-2.5.3-py2-none-any.whl.

File metadata

File hashes

Hashes for baiji-2.5.3-py2-none-any.whl
Algorithm Hash digest
SHA256 7ef8caecf8ad907724fa8c4c983ad43a16baacc18690f3d530ab712b78e6489e
MD5 c6e839b62c08c73e9a05147043a55189
BLAKE2b-256 1fc2c1968ba8a3fdd8698f3c8508ed2661301b5e6b28b13ee9f104b182ca2a01

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page