Skip to main content

cosmolog: structured python logger

Project description

Cosmolog

Cosmolog provides structured log formatting for your application. It treats logs as event streams and will record each log event as a json object with a schema.

CosmologEvent Schema

A cosmolog event looks like the following:

{
    "version": 0,
    "stream_name": "foo_nginx",
    "origin": "foo-api1.com",
    "timestamp": "2016-09-02T16:34:12.019105Z",
    "format": "service {name} started",
    "level": 400,
    "payload": {"name": "foo"}
}

version     (int)    The version of the logging schema.
stream_name (string) The name that identifies the log stream.
origin      (string) FQDN of the host on which the stream originated.
timestamp   (string) UTC ISO8601 formatted datetime string.
format      (string) Optional Python Format String to format the payload to
                     make it human-readable. Please use format strings with
                     replacement fields that are delimited with "{}".
level       (int)    The log level.
payload     (dict)   A flat dictionary of key-value pairs where keys are
                     strings and values can be any scalar type.

Installation

pip install cosmolog

Quick Start

from cosmolog import setup_logging
from cosmolog import Cosmologger

setup_logging()

l = Cosmologger('foo')
l.info('Hello World')
l.info('Hello {person}', person='Dave')
l.info(value1=0.98, value2=4.0)

Human

human is a command line tool that formats machine readable logs for humans. It reads stdin and writes formatted lines to stdout.

$ echo '{"origin": "enterprise.starfleet.com", "stream_name": "telemetry", "format": "Measurement complete: gravity={gravity}", "timestamp": "2016-10-19T04:13:15.049920Z", "level": 400, "version": 0, "payload": {"gravity": 1.8}}' | human
Oct 19 04:13:15 enterprise.starfleet.com telemetry: [INFO] Measurement complete: gravity=1.8

Advanced Usage: setup_logging

setup_logging provides basic Python logging configuration with CosmologgerFormatter class as the default logging formatter. The default logging configuration looks like the following:

LOGGING_CONFIG = {
    'version': 1,
    'disable_existing_loggers': False,
    'formatters': {
        'cosmolog': {
            '()': CosmologgerFormatter,
            'origin': origin,
            'version': 0,
        },
    },
    'handlers': {
        'default': {
            'class': 'logging.StreamHandler',
            'formatter': 'cosmolog'
        },
    },
    'root': {
        'handlers': ['default'],
        'level': level,
    }
}

By default, origin is set to socket.getfqdn(). To set it yourself:

setup_logging(origin='my-fully-qualified-domain.com')

You can configure Cosmologger with a custom configuration dictionary, as you would with regular Python logging. Let's say you want to stream your log entries to a file in the CosmologEvent schema, as well as stderr in a human- readable format:

my_custom_config = {
    'version': 1,
    'disable_existing_loggers': False,
    'formatters': {
        'cosmolog': {
            '()': CosmologgerFormatter,
            'origin': origin,
            'version': 0,
        },
        'cosmolog-human': {
            '()': CosmologgerHumanFormatter,
            'origin': origin,
            'version': 0,
        },
    },
    'handlers': {
        'file_handler': {
            'class': 'logging.FileHandler',
            'formatter': 'cosmolog',
            'filename': '/path/to/my/file.log'
        },
        'stderr': {
            'class': 'logging.StreamHandler',
            'formatter': 'cosmolog-human',
	'color': True
        },
    },
    'root': {
        'handlers': ['file_handler', 'stderr'],
        'level': 'DEBUG',
    }
}

setup_logging(custom_config=my_custom_config)

By default, your log level will be set to INFO. You can set it to a different level:

setup_logging(level='DEBUG')

The table below shows the level names, and the numberic values they will be mapped to in the log output:

Level Numeric value
FATAL 100
ERROR 200
WARN 300
INFO 400
DEBUG 500
TRACE 600

Advanced usage: Exceptions

Normally, all keyword arguments are faithfully passed into the CosmologEvent payload field, except in the case of logging exceptions. Here, the conventions of the builtin logging library are followed. Calling Cosmologger.exception or any of the other logging methods with the keyword argument exc_info=1 will add traceback information to the payload's exc_text field. If there is no {exc_text} specified in the format string, then the format field will be appended with '\n{exc_text}'.

Development

pip install -e .[test]
tox

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

cosmolog-0.9.12.tar.gz (17.4 kB view details)

Uploaded Source

Built Distribution

cosmolog-0.9.12-py3-none-any.whl (14.3 kB view details)

Uploaded Python 3

File details

Details for the file cosmolog-0.9.12.tar.gz.

File metadata

  • Download URL: cosmolog-0.9.12.tar.gz
  • Upload date:
  • Size: 17.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.10.13

File hashes

Hashes for cosmolog-0.9.12.tar.gz
Algorithm Hash digest
SHA256 f6fcae8b2efc30e352e75e24a635d204d359acc886826c18d2ac377f217dc757
MD5 f45ff2772547268d0cf946019e0a44e5
BLAKE2b-256 f996f2a4fb9e1c3d91c7f43f795ebf0323f2c1c9f57777b66a67390d351cffd5

See more details on using hashes here.

File details

Details for the file cosmolog-0.9.12-py3-none-any.whl.

File metadata

  • Download URL: cosmolog-0.9.12-py3-none-any.whl
  • Upload date:
  • Size: 14.3 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.10.13

File hashes

Hashes for cosmolog-0.9.12-py3-none-any.whl
Algorithm Hash digest
SHA256 6f7e9fa8114e6015908db4fd10bfa58ecb015f8a8ff8c880986b9cbee3afa898
MD5 6d269f20da0aaba1e3227614c70c8828
BLAKE2b-256 cb2646f7df904f56273b1841423257f74e4ce411737d4838f282f9aaffde6b4e

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page