A common WSGI stack
Project description
Talisker is a runtime for your wsgi app that aims to provide a common platform for your python services.
tl;dr
Simply run your wsgi app with talisker as if it was gunicorn.:
talisker app:wsgi -c config.py ...
Talisker will wrap your app in a some simple WSGI middleware, and configure logging to output structured logging like so:
logger = logging.getLogger('app')
logger.info('something happened', extra={'context': 'I haz it'})
will output:
2016-01-13 10:24:07.357Z INFO app "something happened" svc.context="I haz it" request_id=...
It also exposes some status endpoints you can use, go to the /_status/ url on your app to see them.
This all works out of the box by using the talisker runner instead of gunicorn’s, and there are many more features you can use too.
Elevator Pitch
Talisker is based on a number of standard python tools:
stdlib logging for logs
gunicorn for a wsgi runner
requests for http requests
statsd for metrics (and optionally, prometheus_client)
sentry for errors
werkzeug for thread locals and wsgi utilities
It also supports additionaly optional tools:
celery for async tasks
prometheus as an alternate metrics tool
It’s main job is to integrate and configure all the above in a single tool, for use in both dev and production, which provides a standard set of features out of the box:
drop-in replacement for gunicorn as a wsgi runner
standardised structured logging on top of python stdlib logging
request id tracing
standard set of status endpoints for your app
easier configuration via env vars
metrics for everything
deep sentry integration (WIP)
All the above are available by just using the talisker entry point script, rather than gunicorn.
In addition, with a small amount of effort, your app can benefit from additional features:
add structured logging tags to your application logs
simple deeper nagios checks - just implement a /_status/check url in your app
per-thread requests connection pool management
Additionally, talisker provides additional tools for integrating with your infrastructure:
grok filters for log parsing
rsyslog templates and config for log shipping (TODO)
Talisker is opinionated, and derived directly from the authors’ needs and as such not currently very configurable. However, PR’s are very welcome!
For more information, see The Documentation, which should be found at:
0.9.0 (2017-01-24)
The major feature in this release is support for sentry, which is integrated with wsgi, logging, and celery. Also supports opt-in integration with flask and django, see the relevant docs for more info.
Other changes
refactor of how logging contexts were implemented. More flexible and reliable. Note talisker.logs.extra_logging and talisker.logs.set_logging_context are now deprecated, you should use talisker.logs.logging_context and talisker.logs.logging_context.push, respectively, as covered in the updated logging docs.
improved celery logging, tasks logs now have task_id and task_name automatically added to their logs.
improved logging messages when parsing TALISKER_NETWORKS at startup
0.8.0 (2016-12-13)
prometheus: add optinal support for promethues_client
celery: request id automatically sent and logged, and support for 4.0
docs: initial ‘talisker contract’
statsd: better client initialisation
internal: refactoring of global variables, better /_status/ url dispatch
0.7.1 (2016-11-09)
remove use of future’s import hooks, as they mess with raven’s vendored imports
slight tweak to logfmt serialisation, and update docs to match
0.7.0 (2016-11-03)
Upgrading
This release includes a couple of minor backwards incompatible changes:
access logs now use the talisker format, rather than CLF. See the docs for more info. If you are using access logs already, then the easiest upgrade path is to output the access logs to stderr (access_logfile=”-“), and delete your old log files.
talisker no longer prefixes developer supplied tags with ‘svc.’. This should only matter if you’ve already set up dashboards or similar with the old prefixed name, and you will need to remove the prefix
Changes:
access logs now in logfmt rather than CLF
dummy statsd client is now useful in testing
logs are colored in development, to aid reading
the ‘svc’ prefix for tags has been removed
0.6.7 (2016-10-05)
actually include the encoding fix for check endpoint
0.6.6 (2016-10-05)
add celery metrics
fix issue with encoding in check endpoint when iterable
0.6.5 (2016-09-26)
make celery runner actually work, wrt logging
0.6.4 (2016-09-23)
fix encoding issue with X-Request-Id header (again!)
0.6.3 (2016-09-21)
fix setuptools entry points, which were typoed into oblivion.
0.6.2 (2016-09-21)
make gunicorn use proper statsd client
log some extra warnings if we try to configure gunicorn things that talisker overides.
better documented public api via __all__
first take on some celery helpers
some packaging improvements
0.6.1 (2016-09-12)
actually do remove old DEBUGLOG backups, as backupCount=0 does not remove any. Of course.
0.6.0 (2016-09-09)
Propagate gunicorn.error log, and remove its default handler.
This allows consistant logging, making the choice in all cases that your gunicorn logs go to the same stream as your other application log, making the choice in all cases that your gunicorn logs go to the same stream as your other application logs.
We issue a warning if the user tries to configure errorlog manually, as it won’t work as expected.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
