...
Project description
HTTPCore
A low-level async HTTP library.
Proposed functionality
- Support for streaming requests and responses. (Done)
- Support for connection pooling. (Done)
- gzip, deflate, and brotli decoding. (Done)
- SSL verification. (Done)
- Proxy support. (Not done)
- HTTP/2 support. (Not done)
- Support both async and sync operations. (Sync will be lightweight shim on top of async. Not done.)
Motivation
Some of the trickier remaining issues on requests-async such as request/response streaming, connection pooling, proxy support, would require a fully async variant of urllib3. I considered and started work on a straight port of urllib3-async, but having started to dive into it, my judgement is that a from-scratch implementation will be less overall work to achieve.
The intent is that this library could be the low-level implementation, that requests-async would then wrap up.
Credit
- Some inspiration from the design-work of
urllib3, but redone from scratch, and built as an async-first library. - Dependant on the absolutely excellent
h11package. - Uses the
certifipackage for the default SSL verification.
Usage
Making a request:
import httpcore
http = httpcore.ConnectionPool()
response = await http.request('GET', 'http://example.com')
assert response.status_code == 200
assert response.body == b'Hello, world'
Top-level API:
http = httpcore.ConnectionPool([ssl], [timeout], [limits])
response = await http.request(method, url, [headers], [body], [stream])
ConnectionPool as a context-manager:
async with httpcore.ConnectionPool([ssl], [timeout], [limits]) as http:
response = await http.request(method, url, [headers], [body], [stream])
Streaming responses:
http = httpcore.ConnectionPool()
response = await http.request(method, url, stream=True)
async for part in response.stream():
...
Raw data without gzip/deflate/brotli decompression applied:
http = httpcore.ConnectionPool()
response = await http.request(method, url, stream=True)
async for part in response.raw():
...
Building a Gateway Server
The level of abstraction fits in really well if you're just writing at the raw ASGI level. Eg. Here's an how an ASGI gateway server looks against the API, including streaming uploads and downloads...
import httpcore
class GatewayServer:
def __init__(self, base_url):
self.base_url = base_url
self.http = httpcore.ConnectionPool()
async def __call__(self, scope, receive, send):
assert scope['type'] == 'http'
path = scope['path']
query = scope['query_string']
method = scope['method']
headers = [
(k, v) for (k, v) in scope['headers']
if k not in (b'host', b'transfer-encoding')
]
url = self.base_url + path
if query:
url += '?' + query.decode()
initial_body, more_body = await self.initial_body(receive)
if more_body:
# Streaming request.
body = self.stream_body(receive, initial_body)
else:
# Standard request.
body = initial_body
response = await self.http.request(
method, url, headers=headers, body=body, stream=True
)
await send({
'type': 'http.response.start',
'status': response.status_code,
'headers': response.headers
})
data = b''
async for next_data in response.raw():
if data:
await send({
'type': 'http.response.body',
'body': data,
'more_body': True
})
data = next_data
await send({'type': 'http.response.body', 'body': data})
async def initial_body(self, receive):
"""
Pull the first body message off the 'receive' channel.
Allows us to determine if we should use a streaming request or not.
"""
message = await receive()
body = message.get('body', b'')
more_body = message.get('more_body', False)
return (body, more_body)
async def stream_body(self, receive, initial_body):
"""
Async iterator returning bytes for the request body.
"""
yield initial_body
while True:
message = await receive()
yield message.get('body', b'')
if not message.get('more_body', False):
break
app = GatewayServer('http://example.org')
Run with...
uvicorn example:app
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
File details
Details for the file httpcore-0.1.0.tar.gz.
File metadata
- Download URL: httpcore-0.1.0.tar.gz
- Upload date:
- Size: 12.0 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/39.0.1 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.7.1
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | f88d35d597baaed5f9022f01fff6ce42e1cfe7c9c2e5238d020b813ad532e389 |
|
| MD5 | 8016b08e9956f0546a0cdeeb12e41e34 |
|
| BLAKE2b-256 | fb6d29ce2c59b44689d0020f40c5f36f29c03a9067e708ec2031ca6529ab1d6e |
