Routines and classes supporting MongoDB environments
Project description
jaraco.mongodb
==============
`Documentation <https://pythonhosted.org/jaraco.mongodb>`_
Provides support for MongoDB environments.
sessions
--------
``jaraco.mongodb.sessions`` implements a CherryPy Sessions store backed by
MongoDB.
By default, the session store will handle sessions with any objects that can
be inserted into a MongoDB collection naturally.
To support richer objects, one may configure the codec to use ``jaraco.modb``.
oplog
-----
This package provides an ``oplog`` module, which is based on the
`mongooplog-alt <https://github.com/asivokon/mongooplog-alt/>`_ project,
which itself is a Python remake of `official mongooplog utility`_,
shipped with MongoDB starting from version 2.2.0. It reads oplog of a remote
server, and applies operations to the local server. This can be used to keep
independed replica set loosly synced in a sort of one way replication, and may
be useful in various backup and migration scenarios.
``oplog`` implements basic functionality of the official utility and
adds following features:
* tailable oplog reader: runs forever polling new oplog event which is extremly
useful for keeping two independent replica sets in almost real-time sync.
* option to sync only selected databases/collections.
* option to exclude one or more namespaces (i.e. dbs or collections) from
being synced.
* ability to "rename" dbs/collections on fly, i.e. destination namespaces can
differ from the original ones.
* works on mongodb 1.8.x, 2.0.x, and 2.2.x. Official utility only supports
version 2.2.x and higher.
* save last processed timestamp to file, resume from saved point later.
* at the time of writing (2.2.0), official ``mongooplog`` suffers from bug that
limits its usage with replica sets (https://jira.mongodb.org/browse/SERVER-6915)
.. _official mongooplog utility: http://docs.mongodb.org/manual/reference/mongooplog/
Invoke the command as a module script: ``python -m jaraco.mongodb.oplog``.
Command-line options
*******************
Options common to original ``mongooplog``::
--from <hostname><:port>
Hostname of the mongod server from which oplog operations are going to be
pulled.
--host <hostname><:port>, -h
Hostname of the mongod server to which oplog operations are going to be
applied. Default is "localhost"
--port <number>
Port of the mongod server to which oplog operations are going to be
applied, if not specified in ``--host``. Default is 27017.
-s SECONDS, --seconds SECONDS
seconds to go back. If not set, try read timestamp from --resume-file.
If the file not found, assume --seconds=86400 (24 hours)
Options specific to this implementation::
--to
An alias for ``--host``.
--follow, -f
Wait for new data in oplog. Makes the utility polling oplog forever (until
interrupted). New data is going to be applied immideately with at most one
second delay.
--exclude, -x
List of space separated namespaces which should be ignored. Can be in form
of ``dname`` or ``dbname.collection``.
--ns
Process only these namespaces, ignoring all others. Space separated list of
strings in form of ``dname`` or ``dbname.collection``.
--rename [ns_old=ns_new [ns_old=ns_new ...]]
Rename database(s) and/or collection(s). Operations on namespace ``ns_old``
from the source server will be applied to namespace ``ns_new`` on the
destination server.
--resume-file FILENAME
resume from timestamp read from this file and write last processed
timestamp back to this file (default is mongooplog.ts).
Pass empty string or 'none' to disable this feature.
Example usages
**************
Consider the following sample usage::
python -m jaraco.mongodb.oplog --from prod.example.com:28000 --to dev.example.com:28500 -f --exclude logdb data.transactions --seconds 600
This command is going to take operations from the last 10 minutes from prod,
and apply them to dev. Database ``logdb`` and collection ``transactions`` of
``data`` database will be omitted. After operations for the last minutes will
be applied, command will wait for new changes to come, keep running until
Ctrl+C or other termination signal recieved.
Testing
=======
|BuildStatus|_
.. |BuildStatus| image:: https://secure.travis-ci.org/jaraco/jaraco.mongodb.png
.. _BuildStatus: http://travis-ci.org/jaraco/jaraco.mongodb
Tests for ``oplog`` are written in javascript using test harness
which is used for testing MongoDB iteself. You can run the whole suite with::
mongo tests/suite.js
Note, that you will need existing writable ``/data/db`` dir.
Tests produce alot of output. Succesful execution ends with line like this::
ReplSetTest stopSet *** Shut down repl set - test worked ****
==============
`Documentation <https://pythonhosted.org/jaraco.mongodb>`_
Provides support for MongoDB environments.
sessions
--------
``jaraco.mongodb.sessions`` implements a CherryPy Sessions store backed by
MongoDB.
By default, the session store will handle sessions with any objects that can
be inserted into a MongoDB collection naturally.
To support richer objects, one may configure the codec to use ``jaraco.modb``.
oplog
-----
This package provides an ``oplog`` module, which is based on the
`mongooplog-alt <https://github.com/asivokon/mongooplog-alt/>`_ project,
which itself is a Python remake of `official mongooplog utility`_,
shipped with MongoDB starting from version 2.2.0. It reads oplog of a remote
server, and applies operations to the local server. This can be used to keep
independed replica set loosly synced in a sort of one way replication, and may
be useful in various backup and migration scenarios.
``oplog`` implements basic functionality of the official utility and
adds following features:
* tailable oplog reader: runs forever polling new oplog event which is extremly
useful for keeping two independent replica sets in almost real-time sync.
* option to sync only selected databases/collections.
* option to exclude one or more namespaces (i.e. dbs or collections) from
being synced.
* ability to "rename" dbs/collections on fly, i.e. destination namespaces can
differ from the original ones.
* works on mongodb 1.8.x, 2.0.x, and 2.2.x. Official utility only supports
version 2.2.x and higher.
* save last processed timestamp to file, resume from saved point later.
* at the time of writing (2.2.0), official ``mongooplog`` suffers from bug that
limits its usage with replica sets (https://jira.mongodb.org/browse/SERVER-6915)
.. _official mongooplog utility: http://docs.mongodb.org/manual/reference/mongooplog/
Invoke the command as a module script: ``python -m jaraco.mongodb.oplog``.
Command-line options
*******************
Options common to original ``mongooplog``::
--from <hostname><:port>
Hostname of the mongod server from which oplog operations are going to be
pulled.
--host <hostname><:port>, -h
Hostname of the mongod server to which oplog operations are going to be
applied. Default is "localhost"
--port <number>
Port of the mongod server to which oplog operations are going to be
applied, if not specified in ``--host``. Default is 27017.
-s SECONDS, --seconds SECONDS
seconds to go back. If not set, try read timestamp from --resume-file.
If the file not found, assume --seconds=86400 (24 hours)
Options specific to this implementation::
--to
An alias for ``--host``.
--follow, -f
Wait for new data in oplog. Makes the utility polling oplog forever (until
interrupted). New data is going to be applied immideately with at most one
second delay.
--exclude, -x
List of space separated namespaces which should be ignored. Can be in form
of ``dname`` or ``dbname.collection``.
--ns
Process only these namespaces, ignoring all others. Space separated list of
strings in form of ``dname`` or ``dbname.collection``.
--rename [ns_old=ns_new [ns_old=ns_new ...]]
Rename database(s) and/or collection(s). Operations on namespace ``ns_old``
from the source server will be applied to namespace ``ns_new`` on the
destination server.
--resume-file FILENAME
resume from timestamp read from this file and write last processed
timestamp back to this file (default is mongooplog.ts).
Pass empty string or 'none' to disable this feature.
Example usages
**************
Consider the following sample usage::
python -m jaraco.mongodb.oplog --from prod.example.com:28000 --to dev.example.com:28500 -f --exclude logdb data.transactions --seconds 600
This command is going to take operations from the last 10 minutes from prod,
and apply them to dev. Database ``logdb`` and collection ``transactions`` of
``data`` database will be omitted. After operations for the last minutes will
be applied, command will wait for new changes to come, keep running until
Ctrl+C or other termination signal recieved.
Testing
=======
|BuildStatus|_
.. |BuildStatus| image:: https://secure.travis-ci.org/jaraco/jaraco.mongodb.png
.. _BuildStatus: http://travis-ci.org/jaraco/jaraco.mongodb
Tests for ``oplog`` are written in javascript using test harness
which is used for testing MongoDB iteself. You can run the whole suite with::
mongo tests/suite.js
Note, that you will need existing writable ``/data/db`` dir.
Tests produce alot of output. Succesful execution ends with line like this::
ReplSetTest stopSet *** Shut down repl set - test worked ****
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
jaraco.mongodb-3.2.tar.gz
(16.9 kB
view hashes)
