Python CloudWatch Logging
Project description
Watchtower is a log handler for Amazon Web Services CloudWatch Logs.
CloudWatch Logs is a log management service built into AWS. It is conceptually similar to services like Splunk and Loggly, but is more lightweight, cheaper, and tightly integrated with the rest of AWS.
Watchtower, in turn, is a lightweight adapter between the Python logging system and CloudWatch Logs. It uses the boto3 AWS SDK, and lets you plug your application logging directly into CloudWatch without the need to install a system-wide log collector. It aggregates logs into batches to avoid sending an API request per each log message, while guaranteeing a delivery deadline (60 seconds by default).
Installation
pip install watchtower
Synopsis
Install awscli and set your AWS credentials (run aws configure).
import watchtower, logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
logger.addHandler(watchtower.CloudWatchLogHandler())
logger.info("Hi")
logger.info(dict(foo="bar", details={}))After running the example, you can see the log output in your AWS console.
Example: Flask logging with Watchtower
import watchtower, flask, logging
logging.basicConfig(level=logging.INFO)
app = flask.Flask("loggable")
handler = watchtower.CloudWatchLogHandler()
app.logger.addHandler(handler)
logging.getLogger("werkzeug").addHandler(handler)
@app.route('/')
def hello_world():
return 'Hello World!'
if __name__ == '__main__':
app.run()(See also http://flask.pocoo.org/docs/errorhandling/.)
Examples: Querying CloudWatch logs
This section is not specific to Watchtower. It demonstrates the use of awscli and jq to read and search CloudWatch logs on the command line.
For the Flask example above, you can retrieve your application logs with the following two commands:
aws logs get-log-events --log-group-name watchtower --log-stream-name loggable | jq '.events[].message' aws logs get-log-events --log-group-name watchtower --log-stream-name werkzeug | jq '.events[].message'
CloudWatch Logs supports alerting and dashboards based on metric filters, which are pattern rules that extract information from your logs and feed it to alarms and dashboard graphs. The following example shows logging structured JSON data using Watchtower, setting up a metric filter to extract data from the log stream, a dashboard to visualize it, and an alarm that sends an email:
TODO
Links
Bugs
Please report bugs, issues, feature requests, etc. on GitHub.
License
Licensed under the terms of the Apache License, Version 2.0.
