High-level Python abstraction layer for Amazon S3
Project description
baiji
=====
High-level Python abstraction layer for Amazon S3:
1. An [`open`][open]-like context handler which allows using S3 keys and
local files interchangeably.
- When reading S3, contents are first written to a temporary local
file.
- When writing S3, contents are written to a temporary local file,
and uploaded on close.
2. An `s3` CLI for listing, copying, syncing, and other common activities.
Features
--------
- Works without an S3 connection (with local files).
- Supports multiprocess parallelism for copying lots of files.
- Supports Python 2.7 and uses boto2.
- Supports OS X, Linux, and Windows.
- Tested and production-hardened.
[open]: https://docs.python.org/2/library/functions.html#open
Examples
--------
```py
with s3.open('s3://example/info.txt', 'w') as f:
f.write('hello')
with s3.open('file:///home/me/info.txt', 'w') as f:
f.write('hello')
with s3.open('s3://example/info.txt', 'r') as f:
contents = f.readlines()
with s3.open('file:///home/me/info.txt', 'r') as f:
contents = f.readlines()
```
```sh
s3 cp foo.txt s3://example/bar.txt
s3 cp s3://example/bar.txt s3://another-example/bazinga.txt
s3 rm s3://example/bar.txt
```
Development
-----------
```sh
pip install -r requirements_dev.txt
rake test
rake lint
```
TODO
----
1. Migrate credentials to `~/.aws/credentials` or env, and deprecate AWS
credential support in `~/.bodylabs`.
2. Move `baiji.util.parallel` into a separate library.
3. Upgrade to boto3.
Contribute
----------
- Issue Tracker: github.com/bodylabs/baiji/issues
- Source Code: github.com/bodylabs/baiji
Pull requests welcome!
Support
-------
If you are having issues, please let us know.
License
-------
The project is licensed under the Apache license, version 2.0.
=====
High-level Python abstraction layer for Amazon S3:
1. An [`open`][open]-like context handler which allows using S3 keys and
local files interchangeably.
- When reading S3, contents are first written to a temporary local
file.
- When writing S3, contents are written to a temporary local file,
and uploaded on close.
2. An `s3` CLI for listing, copying, syncing, and other common activities.
Features
--------
- Works without an S3 connection (with local files).
- Supports multiprocess parallelism for copying lots of files.
- Supports Python 2.7 and uses boto2.
- Supports OS X, Linux, and Windows.
- Tested and production-hardened.
[open]: https://docs.python.org/2/library/functions.html#open
Examples
--------
```py
with s3.open('s3://example/info.txt', 'w') as f:
f.write('hello')
with s3.open('file:///home/me/info.txt', 'w') as f:
f.write('hello')
with s3.open('s3://example/info.txt', 'r') as f:
contents = f.readlines()
with s3.open('file:///home/me/info.txt', 'r') as f:
contents = f.readlines()
```
```sh
s3 cp foo.txt s3://example/bar.txt
s3 cp s3://example/bar.txt s3://another-example/bazinga.txt
s3 rm s3://example/bar.txt
```
Development
-----------
```sh
pip install -r requirements_dev.txt
rake test
rake lint
```
TODO
----
1. Migrate credentials to `~/.aws/credentials` or env, and deprecate AWS
credential support in `~/.bodylabs`.
2. Move `baiji.util.parallel` into a separate library.
3. Upgrade to boto3.
Contribute
----------
- Issue Tracker: github.com/bodylabs/baiji/issues
- Source Code: github.com/bodylabs/baiji
Pull requests welcome!
Support
-------
If you are having issues, please let us know.
License
-------
The project is licensed under the Apache license, version 2.0.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
baiji-2.5.3.tar.gz
(39.0 kB
view details)
Built Distribution
baiji-2.5.3-py2-none-any.whl
(48.6 kB
view details)
File details
Details for the file baiji-2.5.3.tar.gz
.
File metadata
- Download URL: baiji-2.5.3.tar.gz
- Upload date:
- Size: 39.0 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | ff7e74079924ce6d2e0c7d6c8ab0c8c36acd1f334b6e0f9746ef8be26c8e099f |
|
MD5 | 3839f8ac354c524e3b37f045e79cfd1b |
|
BLAKE2b-256 | 207477fb97f177aef442f0aa246162bfaec8ece42dd9a7887d3f15cee2a386d0 |
Provenance
File details
Details for the file baiji-2.5.3-py2-none-any.whl
.
File metadata
- Download URL: baiji-2.5.3-py2-none-any.whl
- Upload date:
- Size: 48.6 kB
- Tags: Python 2
- Uploaded using Trusted Publishing? No
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7ef8caecf8ad907724fa8c4c983ad43a16baacc18690f3d530ab712b78e6489e |
|
MD5 | c6e839b62c08c73e9a05147043a55189 |
|
BLAKE2b-256 | 1fc2c1968ba8a3fdd8698f3c8508ed2661301b5e6b28b13ee9f104b182ca2a01 |