Asynchronous file operations.

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for mosquito from gravatar.com mosquito

Unverified details

These details have not been verified by PyPI
Project links
Meta
Classifiers

Project description

Travis CI Latest Version https://img.shields.io/pypi/wheel/aiofile.svg https://img.shields.io/pypi/pyversions/aiofile.svg https://img.shields.io/pypi/l/aiofile.svg

Real asynchronous file operations with asyncio support.

Status

Development - Stable

Features

  • AIOFile has no internal pointer. You should pass offset and chunk_size for each operation or use helpers (Reader or Writer).

  • For POSIX (MacOS X and Linux) using implementaion based on aio.h (with Cython).

  • For non-POSIX systems using thread-based implementation

Code examples

All code examples requires python 3.5+.

Write and Read

import asyncio
from aiofile import AIOFile, Reader, Writer


async def main():
    async with AIOFile("/tmp/hello.txt", 'w+') as afp:
        await afp.write("Hello ")
        await afp.write("world", offset=7)
        await afp.fsync()

        print(await afp.read())


loop = asyncio.get_event_loop()
loop.run_until_complete(main())

Write and read with helpers

import asyncio
from aiofile import AIOFile, Reader, Writer


async def main():
    async with AIOFile("/tmp/hello.txt", 'w+') as afp:
        writer = Writer(afp)
        reader = Reader(afp, chunk_size=8)

        await writer("Hello")
        await writer(" ")
        await writer("World")
        await afp.fsync()

        async for chunk in reader:
            print(chunk)


loop = asyncio.get_event_loop()
loop.run_until_complete(main())

Read file line by line

import asyncio
from aiofile import AIOFile, LineReader, Writer


async def main():
    async with AIOFile("/tmp/hello.txt", 'w+') as afp:
        writer = Writer(afp)

        await writer("Hello")
        await writer(" ")
        await writer("World")
        await writer("\n")
        await writer("\n")
        await writer("From async world")
        await afp.fsync()

        async for line in LineReader(afp):
            print(line)


loop = asyncio.get_event_loop()
loop.run_until_complete(main())

Reading and Writing for the unix pipe

import os
import asyncio
from aiofile import AIOFile, Reader, Writer


async def reader(fname):
    print('Start reader')
    async with AIOFile(fname, 'r') as afp:
        while True:
            # Maximum expected chunk size, must be passed.
            # Otherwise will be read zero bytes
            # (because unix pipe has zero size)
            data = await afp.read(4096)
            print(data)


async def writer(fname):
    print('Start writer')
    async with AIOFile(fname, 'w') as afp:
        while True:
            await asyncio.sleep(1)
            await afp.write('%06f' % loop.time())


async def main():
    fifo_name = "/tmp/test.fifo"

    if os.path.exists(fifo_name):
        os.remove(fifo_name)

    os.mkfifo(fifo_name)

    # Starting two readers and one writer, but only one reader
    # will be reading at the same time.
    await asyncio.gather(
        reader(fifo_name),
        reader(fifo_name),
        writer(fifo_name),
    )


loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)

try:
    loop.run_until_complete(main())
finally:
    # Shutting down and closing file descriptors after interrupt
    loop.run_until_complete(loop.shutdown_asyncgens())
    loop.close()
    print('Exited')

Read file line by line

import asyncio
from aiofile import AIOFile, LineReader, Writer


async def main():
    async with AIOFile("/tmp/hello.txt", 'w') as afp:
        writer = Writer(afp)

        for i in range(10):
            await writer("%d Hello World\n" % i)

        await writer("Tail-less string")


    async with AIOFile("/tmp/hello.txt", 'r') as afp:
        async for line in LineReader(afp):
            print(line[:-1])


loop = asyncio.get_event_loop()
loop.run_until_complete(main())

Async CSV Dict Reader

import asyncio
import io
from csv import DictReader

from aiofile import AIOFile, LineReader


class AsyncDictReader:
    def __init__(self, afp, **kwargs):
        self.buffer = io.BytesIO()
        self.file_reader = LineReader(
            afp, line_sep=kwargs.pop('line_sep', '\n'),
            chunk_size=kwargs.pop('chunk_size', 4096),
            offset=kwargs.pop('offset', 0),
        )
        self.reader = DictReader(
            io.TextIOWrapper(
                self.buffer,
                encoding=kwargs.pop('encoding', 'utf-8'),
                errors=kwargs.pop('errors', 'replace'),
            ), **kwargs,
        )

    async def __aiter__(self):
        header = await self.file_reader.readline()

        if header:
            self.buffer.write(header)

        return self

    async def __anext__(self):
        line = await self.file_reader.readline()

        if not line:
            raise StopAsyncIteration

        self.buffer.write(line)
        self.buffer.seek(0)

        try:
            result = next(self.reader)
        except StopIteration as e:
            raise StopAsyncIteration from e

        self.buffer.truncate(0)

        return result


async def main():
    async with AIOFile('sample.csv', 'rb') as afp:
        async for item in AsyncDictReader(afp, line_sep='\r'):
            print(item)


loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)


try:
    loop.run_until_complete(main())
finally:
    # Shutting down and closing file descriptors after interrupt
    loop.run_until_complete(loop.shutdown_asyncgens())
    loop.close()

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for mosquito from gravatar.com mosquito

Unverified details

These details have not been verified by PyPI
Project links
Meta
Classifiers

Release history Release notifications | RSS feed

3.9.0

3.8.8

3.8.7

3.8.6

3.8.5

3.8.4

3.8.3

3.8.2

3.8.1

3.8.0

3.7.4

3.7.3

3.7.2

3.7.1

3.7.0

3.6.0

3.5.1

3.5.0

3.4.0

3.3.3

3.3.1

3.3.0

3.2.1

3.2.0

3.1.1

3.1.0

3.0.0

2.0.4

2.0.3

2.0.2

2.0.1

2.0.0

1.5.2

1.5.1

This version

1.4.3

1.4.2

1.4.0

1.3.0

1.2.6

1.2.5

1.2.2

1.2.1

1.2.0

0.5.3

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

aiofile-1.4.3.tar.gz (77.5 kB view details)

Uploaded Source

Built Distributions

aiofile-1.4.3-cp37-cp37m-manylinux1_x86_64.whl (185.2 kB view details)

Uploaded CPython 3.7m

aiofile-1.4.3-cp36-cp36m-manylinux1_x86_64.whl (185.0 kB view details)

Uploaded CPython 3.6m

aiofile-1.4.3-cp36-cp36m-macosx_10_6_intel.whl (101.0 kB view details)

Uploaded CPython 3.6m macOS 10.6+ intel

aiofile-1.4.3-cp35-cp35m-manylinux1_x86_64.whl (185.7 kB view details)

Uploaded CPython 3.5m

aiofile-1.4.3-cp35-cp35m-macosx_10_6_intel.whl (100.2 kB view details)

Uploaded CPython 3.5m macOS 10.6+ intel

aiofile-1.4.3-cp34-cp34m-manylinux1_x86_64.whl (190.0 kB view details)

Uploaded CPython 3.4m

aiofile-1.4.3-cp34-cp34m-macosx_10_6_intel.whl (97.6 kB view details)

Uploaded CPython 3.4m macOS 10.6+ intel

File details

Details for the file aiofile-1.4.3.tar.gz.

File metadata

  • Download URL: aiofile-1.4.3.tar.gz
  • Upload date:
  • Size: 77.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for aiofile-1.4.3.tar.gz
Algorithm Hash digest
SHA256 d18d41842e9be7a63fa732de70acc6dde78defc688db5b1134e16136e419e26e
MD5 7fe677e6b9747fa045f5de3d26e1ba2f
BLAKE2b-256 38ec469c084dccd66216572289518ea1452bd07368f64b344165e0be97284278

See more details on using hashes here.

File details

Details for the file aiofile-1.4.3-cp37-cp37m-manylinux1_x86_64.whl.

File metadata

File hashes

Hashes for aiofile-1.4.3-cp37-cp37m-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 ef1dcb2900999d231c45fdd3626bedb1f2ec56b656a48c3ea6a854c3bcf9ebfa
MD5 882a5501421fd1e2e20ebc870274c610
BLAKE2b-256 1e0b148bd5e7bbd47135d0458b4ab0193f9fa21440f0dec0f08e1e9935d8ed66

See more details on using hashes here.

File details

Details for the file aiofile-1.4.3-cp36-cp36m-manylinux1_x86_64.whl.

File metadata

File hashes

Hashes for aiofile-1.4.3-cp36-cp36m-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 88d1d9a9a8f8eb45119faa98c1682a07628fa4cc1852a05e043b60770fcb9a9c
MD5 931bec7c0163c9b6a30a40ad60c4db29
BLAKE2b-256 6f401139e96c33e0b63809ac2df72999a0838f5380c7b518407a553cc397b3fa

See more details on using hashes here.

File details

Details for the file aiofile-1.4.3-cp36-cp36m-macosx_10_6_intel.whl.

File metadata

File hashes

Hashes for aiofile-1.4.3-cp36-cp36m-macosx_10_6_intel.whl
Algorithm Hash digest
SHA256 324007ad9fed27afd18840af7e2d1bd61f927ed58f7a47690b60190b4b5eec99
MD5 c63405d6a949841f0836d7f3fe5f2511
BLAKE2b-256 8860991e6720ce416d7680767f0d1ffa78387318f5f8e2494028ee418df30ed7

See more details on using hashes here.

File details

Details for the file aiofile-1.4.3-cp35-cp35m-manylinux1_x86_64.whl.

File metadata

File hashes

Hashes for aiofile-1.4.3-cp35-cp35m-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 93e5fa1b950036dd63e985de2367a779d73ed8370a18725db07c6e0e5152cba5
MD5 be1ec43cb9cfba4b8eb11a3dc667f173
BLAKE2b-256 183be6e9682a07c557ebc8c099012d2f547cceb72f066199b30f17d05b7d75e5

See more details on using hashes here.

File details

Details for the file aiofile-1.4.3-cp35-cp35m-macosx_10_6_intel.whl.

File metadata

File hashes

Hashes for aiofile-1.4.3-cp35-cp35m-macosx_10_6_intel.whl
Algorithm Hash digest
SHA256 aeb6ce34aac7c0b7af9014719f3a8bd84f5ab50a1c34a2598e9f1f805f988ebc
MD5 9813b98b37e6ac829c635f6ee9e4239d
BLAKE2b-256 047684c6dffc52161f19304cf9869e9637dae7ad91ca41078e08574e9a1e5c25

See more details on using hashes here.

File details

Details for the file aiofile-1.4.3-cp34-cp34m-manylinux1_x86_64.whl.

File metadata

File hashes

Hashes for aiofile-1.4.3-cp34-cp34m-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 c448780f52c88d313d7816421dc3dc3cc3fd6c4c9a459297e54bd08dc02ef1dd
MD5 af6f507a5da02e5f4ee3ca7f5e036805
BLAKE2b-256 6eda4d11299f647101453a0da0db5c60c2b45faaa352a3fb3f4b18f7cf6cfc7d

See more details on using hashes here.

File details

Details for the file aiofile-1.4.3-cp34-cp34m-macosx_10_6_intel.whl.

File metadata

File hashes

Hashes for aiofile-1.4.3-cp34-cp34m-macosx_10_6_intel.whl
Algorithm Hash digest
SHA256 39ca4d11226eb4186ee0c4e8f0cb15dcb0d8f4a54b4ed94649227c485cfc7282
MD5 479e1cd62db648d314be3cdf9870e838
BLAKE2b-256 a689894e9be4ad62fca6127a8e5e7ab2c662a9c880fe2a22ec3c55fd36e80772

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page