Python logging handler for MongoDB
Project description
log4mongo-python is mongo database handler for python logging, part of log4mongo.org project. log4mongo-python is using pymongo driver - http://github.com/mongodb/mongo-python-driver
Requirements
python 3.6.2+
pymongo 4.0+
mongo database
For more information see debian_requirements.txt and requirements.txt files.
Configuration
Example handler python configuration:
import logging
from log4mongo.handlers import MongoHandler
logger = logging.getLogger('test')
logger.addHandler(MongoHandler(host='localhost'))
logger.warning('test')
Contextual information
It is possible to decorate you document with contextual information. There are tow approaches.
1.) approach
import logging
from log4mongo.handlers import MongoHandler
handler = MongoHandler(host='localhost')
logger = logging.getLogger('test')
logger.addHandler(handler)
logging.LoggerAdapter(logger, {'ip': '127.0.0.1'}).info('test')
2.) approach
import logging
from log4mongo.handlers import MongoHandler
handler = MongoHandler(host='localhost')
logger = logging.getLogger('test')
logger.addHandler(handler)
logger.info('test', extra={'ip': '127.0.0.1'})
As you can see, second approach is more straightforward and there is no need to use LoggerAdapter.
Capped collections
Capped collections are fixed-size collections that support high-throughput operations that insert, retrieve, and delete documents based on insertion order. Capped collections work in a way similar to circular buffers: once a collection fills its allocated space, it makes room for new documents by overwriting the oldest documents in the collection.
Before switching to capped collections, read this document please: http://docs.mongodb.org/manual/core/capped-collections/
This behaviour is disabled by default. You can enable this behaviour in constructor with capped=True:
import logging
from log4mongo.handlers import MongoHandler
handler = MongoHandler(host='localhost', capped=True)
Buffered handler
BufferedMongoHandler is a subclass of MongoHandler allowing to buffer log messages and write them all at once to the database. The goal is to avoid too many writes to the database, thus avoiding too frequent write-locks. Log message buffer flush happens when the buffer is full, when a critical log message is emitted, and also periodically. An early buffer flush can happen when a critical message is emitted. And in order to avoid messages to stay indefinitively in the buffer queue before appearing in database, a periodical flush happens every X seconds.
This periodical flush can also be deactivated with buffer_periodical_flush_timing=False, thus avoiding the timer thread to be created.
Buffer size is configurable, as well as the log level for early flush (default is logging.CRITICAL):
import logging
from log4mongo.handlers import BufferedMongoHandler
handler = BufferedMongoHandler(host='localhost', # All MongoHandler parameters are valid
capped=True,
buffer_size=100, # buffer size.
buffer_periodical_flush_timing=10.0, # periodical flush every 10 seconds
buffer_early_flush_level=logging.CRITICAL) # early flush level
logger = logging.getLogger().addHandler(handler)
Test
** Tested on evnironment **
Ubuntu 18.04
python >=3.6.4
pymongo >=3.9.0
mongod - db version 4.0.13
pytest
Running tests
Before you run the test you must start mongo database. You will do so by this command:
$ mongod --dbpath /tmp/
To run the test run command:
$ python3 setup.py test
See vagrant file to quickly setup the test environment.
Current Maintainer
References
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file log4mongo-1.8.1.tar.gz
.
File metadata
- Download URL: log4mongo-1.8.1.tar.gz
- Upload date:
- Size: 16.0 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.1.1 CPython/3.12.3
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 235562b27b42fe5d560ce5ace0d09dddb2fbea34b8df214d725b4724b9620b0d |
|
MD5 | 167b2d436e5a8ddf7438fb7eb5f10b10 |
|
BLAKE2b-256 | 9a208a3ea27dd3e5a2683744309fc976d11d5fabed23b708db7754d5233101de |
File details
Details for the file log4mongo-1.8.1-py3-none-any.whl
.
File metadata
- Download URL: log4mongo-1.8.1-py3-none-any.whl
- Upload date:
- Size: 7.5 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.1.1 CPython/3.12.3
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 10e5dc019771b0b4b4c731783014d5ae67bedc946189825a27c88131fda8e9ef |
|
MD5 | 1559ab37fba81f6f0ae2dff003177957 |
|
BLAKE2b-256 | 3deb3376961aa2e7adcadc8c34d1dc13127bde0ddef3c8d065079a2e1cd71a10 |