Skip to main content

A full async docsplit previewer server based on Tornado and Redis

Project description

Introduction

We are working to find the best solution for our previewer media server.

The case is simple, we have PDF and we want to preview them (small, medium, large, header).

We first started with Insight which allows people to ask for sync and async previews generations from an url with caching.

The system is able to register engines so that we can manipulate documents on the flow.

The idea of Insight-Reloaded is to remove the sync mode and delegate it to Nginx with disk or S3 storage.

If the document preview is not ready, we want a 404 error.

We will get the document preview url with a callback.

The API

Simple example

curl -X GET "http://localhost:8888/?url=http://my_file_url.com/file.pdf&callback=http://requestb.in/12vsewg"
{"insight_reloaded": "Job added to queue.", "number_in_queue": 14}

curl -X GET http://localhost:8888/status
{"insight_reloaded": "There is 14 job in the queue.", "number_in_queue": 14}

curl -X GET http://localhost:8888/
{"version": "1.5", "insight_reloaded": "Bonjour", "name": "insight-reloaded"}

Multi-queues example

You must to configure REDIS_QUEUE_KEYS in your settings.py.

REDIS_QUEUE_KEYS = ['urgent', 'normal']
DEFAULT_REDIS_QUEUE_KEY = 'normal'

Then you can use:

curl -X GET "http://localhost:8888/urgent?url=http://my_file_url.com/file.pdf&callback=http://requestb.in/12vsewg"
{"insight_reloaded": "Job added to queue 'normal'.", "number_in_queue": 14}

curl -X GET http://localhost:8888/urgent/status
{"insight_reloaded": "There is 14 job in the 'urgent' queue.", "number_in_queue": 14}

curl -X GET http://localhost:8888/
{"version": "1.5", "insight_reloaded": "Bonjour", "name": "insight-reloaded"}

Set hash

By default, insight compute preview hash using url (sha1(url)). You can also send it within the request if you want, be sure that each preview has an uniq hash:

curl -X GET "http://localhost:8888/?url=http://my_file_url.com/file.pdf&callback=http://requestb.in/12vsewg?hash=123456789"
{"insight_reloaded": "Job added to queue.", "number_in_queue": 14}

Service architecture

https://raw.github.com/novapost/insight-reloaded/master/docs/_static/InsightReloaded.png

Server provisioning

You can find insight-reloaded chef cookbooks here : https://github.com/novapost/insight-installer

This will helps you install all requirements to run your insight-reloaded server.

Run the tests

make test

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

insight_reloaded-1.6.6.zip (39.9 kB view details)

Uploaded Source

File details

Details for the file insight_reloaded-1.6.6.zip.

File metadata

File hashes

Hashes for insight_reloaded-1.6.6.zip
Algorithm Hash digest
SHA256 66635a8e88cdc87acb0452caf2045ad8372143994cf6c101df511f9109f36e3c
MD5 caf8912f2096e404c7ceb637d57f5c93
BLAKE2b-256 ea3a765377b82e298ff41995bacc6e8275c4a04a474a81482829d5d533a72902

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page