Skip to main content

Python logging handler for MongoDB

Project description

log4mongo-python is mongo database handler for python logging, part of log4mongo.org project. log4mongo-python is using pymongo driver - http://github.com/mongodb/mongo-python-driver

Requirements

  • python 3.6.2+

  • pymongo 4.0+

  • mongo database

For more information see debian_requirements.txt and requirements.txt files.

Configuration

Example handler python configuration:

import logging
from log4mongo.handlers import MongoHandler

logger = logging.getLogger('test')
logger.addHandler(MongoHandler(host='localhost'))
logger.warning('test')

Contextual information

It is possible to decorate you document with contextual information. There are tow approaches.

1.) approach

import logging
from log4mongo.handlers import MongoHandler

handler = MongoHandler(host='localhost')
logger = logging.getLogger('test')
logger.addHandler(handler)
logging.LoggerAdapter(logger, {'ip': '127.0.0.1'}).info('test')

2.) approach

import logging
from log4mongo.handlers import MongoHandler

handler = MongoHandler(host='localhost')
logger = logging.getLogger('test')
logger.addHandler(handler)
logger.info('test', extra={'ip': '127.0.0.1'})

As you can see, second approach is more straightforward and there is no need to use LoggerAdapter.

Capped collections

Capped collections are fixed-size collections that support high-throughput operations that insert, retrieve, and delete documents based on insertion order. Capped collections work in a way similar to circular buffers: once a collection fills its allocated space, it makes room for new documents by overwriting the oldest documents in the collection.

Before switching to capped collections, read this document please: http://docs.mongodb.org/manual/core/capped-collections/

This behaviour is disabled by default. You can enable this behaviour in constructor with capped=True:

import logging
from log4mongo.handlers import MongoHandler

handler = MongoHandler(host='localhost', capped=True)

Buffered handler

BufferedMongoHandler is a subclass of MongoHandler allowing to buffer log messages and write them all at once to the database. The goal is to avoid too many writes to the database, thus avoiding too frequent write-locks. Log message buffer flush happens when the buffer is full, when a critical log message is emitted, and also periodically. An early buffer flush can happen when a critical message is emitted. And in order to avoid messages to stay indefinitively in the buffer queue before appearing in database, a periodical flush happens every X seconds.

This periodical flush can also be deactivated with buffer_periodical_flush_timing=False, thus avoiding the timer thread to be created.

Buffer size is configurable, as well as the log level for early flush (default is logging.CRITICAL):

import logging
from log4mongo.handlers import BufferedMongoHandler

handler = BufferedMongoHandler(host='localhost',                          # All MongoHandler parameters are valid
                               capped=True,
                               buffer_size=100,                           # buffer size.
                               buffer_periodical_flush_timing=10.0,       # periodical flush every 10 seconds
                               buffer_early_flush_level=logging.CRITICAL) # early flush level

logger = logging.getLogger().addHandler(handler)

Test

** Tested on evnironment **

  • Ubuntu 18.04

  • python >=3.6.4

  • pymongo >=3.9.0

  • mongod - db version 4.0.13

  • pytest

Running tests

Before you run the test you must start mongo database. You will do so by this command:

$ mongod --dbpath /tmp/

To run the test run command:

$ python3 setup.py test

See vagrant file to quickly setup the test environment.

Original Author

Current Maintainer

Oz Nahum Tiram

References

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

log4mongo-1.8.1.tar.gz (16.0 kB view details)

Uploaded Source

Built Distribution

log4mongo-1.8.1-py3-none-any.whl (7.5 kB view details)

Uploaded Python 3

File details

Details for the file log4mongo-1.8.1.tar.gz.

File metadata

  • Download URL: log4mongo-1.8.1.tar.gz
  • Upload date:
  • Size: 16.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.12.3

File hashes

Hashes for log4mongo-1.8.1.tar.gz
Algorithm Hash digest
SHA256 235562b27b42fe5d560ce5ace0d09dddb2fbea34b8df214d725b4724b9620b0d
MD5 167b2d436e5a8ddf7438fb7eb5f10b10
BLAKE2b-256 9a208a3ea27dd3e5a2683744309fc976d11d5fabed23b708db7754d5233101de

See more details on using hashes here.

File details

Details for the file log4mongo-1.8.1-py3-none-any.whl.

File metadata

  • Download URL: log4mongo-1.8.1-py3-none-any.whl
  • Upload date:
  • Size: 7.5 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.12.3

File hashes

Hashes for log4mongo-1.8.1-py3-none-any.whl
Algorithm Hash digest
SHA256 10e5dc019771b0b4b4c731783014d5ae67bedc946189825a27c88131fda8e9ef
MD5 1559ab37fba81f6f0ae2dff003177957
BLAKE2b-256 3deb3376961aa2e7adcadc8c34d1dc13127bde0ddef3c8d065079a2e1cd71a10

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page