High-level Python abstraction layer for Amazon S3
Project description
baiji
=====
High-level Python abstraction layer for Amazon S3:
1. An [`open`][open]-like context handler which allows using S3 keys and
local files interchangeably.
- When reading S3, contents are first written to a temporary local
file.
- When writing S3, contents are written to a temporary local file,
and uploaded on close.
2. An `s3` CLI for listing, copying, syncing, and other common activities.
Features
--------
- Works without an S3 connection (with local files).
- Supports multiprocess parallelism for copying lots of files.
- Supports Python 2.7 and uses boto2.
- Supports OS X, Linux, and Windows.
- Tested and production-hardened.
[open]: https://docs.python.org/2/library/functions.html#open
Examples
--------
```py
with s3.open('s3://example/info.txt', 'w') as f:
f.write('hello')
with s3.open('file:///home/me/info.txt', 'w') as f:
f.write('hello')
with s3.open('s3://example/info.txt', 'r') as f:
contents = f.readlines()
with s3.open('file:///home/me/info.txt', 'r') as f:
contents = f.readlines()
```
```sh
s3 cp foo.txt s3://example/bar.txt
s3 cp s3://example/bar.txt s3://another-example/bazinga.txt
s3 rm s3://example/bar.txt
```
Development
-----------
```sh
pip install -r requirements_dev.txt
rake test
rake lint
```
TODO
----
1. Migrate credentials to `~/.aws/credentials` or env, and deprecate AWS
credential support in `~/.bodylabs`.
2. Move `baiji.util.parallel` into a separate library.
3. Upgrade to boto3.
Contribute
----------
- Issue Tracker: github.com/bodylabs/baiji/issues
- Source Code: github.com/bodylabs/baiji
Pull requests welcome!
Support
-------
If you are having issues, please let us know.
License
-------
The project is licensed under the Apache license, version 2.0.
=====
High-level Python abstraction layer for Amazon S3:
1. An [`open`][open]-like context handler which allows using S3 keys and
local files interchangeably.
- When reading S3, contents are first written to a temporary local
file.
- When writing S3, contents are written to a temporary local file,
and uploaded on close.
2. An `s3` CLI for listing, copying, syncing, and other common activities.
Features
--------
- Works without an S3 connection (with local files).
- Supports multiprocess parallelism for copying lots of files.
- Supports Python 2.7 and uses boto2.
- Supports OS X, Linux, and Windows.
- Tested and production-hardened.
[open]: https://docs.python.org/2/library/functions.html#open
Examples
--------
```py
with s3.open('s3://example/info.txt', 'w') as f:
f.write('hello')
with s3.open('file:///home/me/info.txt', 'w') as f:
f.write('hello')
with s3.open('s3://example/info.txt', 'r') as f:
contents = f.readlines()
with s3.open('file:///home/me/info.txt', 'r') as f:
contents = f.readlines()
```
```sh
s3 cp foo.txt s3://example/bar.txt
s3 cp s3://example/bar.txt s3://another-example/bazinga.txt
s3 rm s3://example/bar.txt
```
Development
-----------
```sh
pip install -r requirements_dev.txt
rake test
rake lint
```
TODO
----
1. Migrate credentials to `~/.aws/credentials` or env, and deprecate AWS
credential support in `~/.bodylabs`.
2. Move `baiji.util.parallel` into a separate library.
3. Upgrade to boto3.
Contribute
----------
- Issue Tracker: github.com/bodylabs/baiji/issues
- Source Code: github.com/bodylabs/baiji
Pull requests welcome!
Support
-------
If you are having issues, please let us know.
License
-------
The project is licensed under the Apache license, version 2.0.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
baiji-2.7.2.tar.gz
(40.0 kB
view details)
Built Distribution
baiji-2.7.2-py2-none-any.whl
(49.9 kB
view details)
File details
Details for the file baiji-2.7.2.tar.gz
.
File metadata
- Download URL: baiji-2.7.2.tar.gz
- Upload date:
- Size: 40.0 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 70814b7338e5d2dd233858a004dba93c962ae6865fca9799c23218b8e3bc12d0 |
|
MD5 | 527c103803982933ef25de07a5285b43 |
|
BLAKE2b-256 | 69dd08d1e3f63733c51e214a94ff085f22afebcf10d3300c06e1b628b63a681b |
Provenance
File details
Details for the file baiji-2.7.2-py2-none-any.whl
.
File metadata
- Download URL: baiji-2.7.2-py2-none-any.whl
- Upload date:
- Size: 49.9 kB
- Tags: Python 2
- Uploaded using Trusted Publishing? No
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 58bbd7c062e8888f003fa3edde4453662b52e2e672fdf05f13780036b32d8c6f |
|
MD5 | e8cfb14e9fbb8e2b525204101add4c1e |
|
BLAKE2b-256 | db9c413116f1d41e30a02a3c630d7b8ab575ef3a141438a1e336ff6cb7bf7395 |