Skip to main content

Elastic2 plugin for mongo-connector

Project description

====================
elastic2-doc-manager
====================

The mongo-connector project originated as a MongoDB mongo-labs
project and is now community-maintained under the custody of YouGov, Plc.

.. image:: https://travis-ci.org/yougov/elastic2-doc-manager.svg?branch=master
:alt: View build status
:target: https://travis-ci.org/yougov/elastic2-doc-manager

Getting Started
===============

This package is a document manager for
`mongo-connector <https://github.com/yougov/mongo-connector>`_ that
targets Elasticsearch versions 2.x and 5.x -- don't let the name fool you!
For information on running mongo-connector with Elasticsearch, please see the
`MongoConnector Usage with Elasticsearch
<https://github.com/yougov/mongo-connector/wiki/Usage%20with%20ElasticSearch>`_
wiki page.

Installation
============

The installation of the elastic2-doc-manager depends on which version of
Elasticsearch you are targeting.

Elasticsearch 1.x
-----------------

This is the document manager for Elasticsearch 2.x and 5.x. If you
want to target Elasticsearch 1.x, please install the
`elastic-doc-manager <https://github.com/yougov/elastic-doc-manager>`_.

Elasticsearch 2.x
-----------------

For use with an Elasticsearch 2.x server, install with
`pip <https://pypi-hypernode.com/pypi/pip>`__::

pip install 'elastic2-doc-manager[elastic2]'

Elasticsearch 5.x
-----------------

For use with an Elasticsearch 5.x server, install with::

pip install 'elastic2-doc-manager[elastic5]'

.. note:: Version 0.3.0 added support for Elasticsearch 5.x.


Amazon Elasticsearch Service
----------------------------

To use with Amazon Elasticsearch Service, you must install the required AWS
dependencies along with the version of Elasticsearch::

pip install 'elastic2-doc-manager[elastic2,aws]'


Development
-----------

You can also install the development version of elastic2-doc-manager
manually::

git clone https://github.com/yougov/elastic2-doc-manager.git
pip install -e './elastic2-doc-manager[elastic2]'

You may have to run ``pip`` with ``sudo``, depending on where you're
installing and what privileges you have.

.. note:: Please note that before mongo-connector version 2.2.2, the elastic
doc manager was packaged with mongo-connector and only supported
Elasticsearch 1.x.

Running the tests
-----------------
Requirements
~~~~~~~~~~~~

1. Copy of the Elastic 2.x Document Manager Github repository

The tests are not included in the package from PyPI and can only be acquired
by cloning this repository on Github::

git clone https://github.com/yougov/elastic2-doc-manager

2. Tox

Install `tox <https://pypi-hypernode.com/project/tox>`_.

2. Environment variables

There are a few influential environment variables that affect the tests. These are
defined in the tox.ini.

All the tests live in the `tests` directory.

Running tests on the command-line
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

While the tests take care of setting up and tearing down MongoDB clusters on
their own, make sure to start Elasticsearch before doing a full test run!

You can run all the tests with one command (this works in all supported Python versions)::

tox

Error messages
~~~~~~~~~~~~~~

Some of the tests are meant to generate lots of ``ERROR``-level log messages,
especially the rollback tests. mongo-connector logs exceptions it encounters
while iterating the cursor in the oplog, so we see these in the console output
while MongoDB clusters are being torn apart in the tests. As long as all the
tests pass with an `OK` message, all is well.


Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

elastic2-doc-manager-1.0.0.tar.gz (23.2 kB view details)

Uploaded Source

Built Distribution

elastic2_doc_manager-1.0.0-py2.py3-none-any.whl (14.7 kB view details)

Uploaded Python 2 Python 3

File details

Details for the file elastic2-doc-manager-1.0.0.tar.gz.

File metadata

  • Download URL: elastic2-doc-manager-1.0.0.tar.gz
  • Upload date:
  • Size: 23.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.12.1 pkginfo/1.4.2 requests/2.20.0 setuptools/40.5.0 requests-toolbelt/0.8.0 tqdm/4.28.1 CPython/3.6.3

File hashes

Hashes for elastic2-doc-manager-1.0.0.tar.gz
Algorithm Hash digest
SHA256 cd6e12f5b0b6ffe5bf9dc341f2fefc87fe5a8c090e7a175e1450a268484c4451
MD5 a8739fb7eca4f21ba39945f1e17033a8
BLAKE2b-256 5a0e85e7670d807efd34b296e3bb4e908f77908e618a6cb2500b1bf94fcf5cf0

See more details on using hashes here.

Provenance

File details

Details for the file elastic2_doc_manager-1.0.0-py2.py3-none-any.whl.

File metadata

  • Download URL: elastic2_doc_manager-1.0.0-py2.py3-none-any.whl
  • Upload date:
  • Size: 14.7 kB
  • Tags: Python 2, Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.12.1 pkginfo/1.4.2 requests/2.20.0 setuptools/40.5.0 requests-toolbelt/0.8.0 tqdm/4.28.1 CPython/3.6.3

File hashes

Hashes for elastic2_doc_manager-1.0.0-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 32024eec86637ffe5f48c88947e9f28fbe9c12d62d8dc6ef97ab1aa9c5412f65
MD5 283c07c3e110e0d715d6620c2186b212
BLAKE2b-256 09cf0a18c6a925c4c95ded1d40bc84f1d9e701ed4d19e68c21b984d4b33d65f5

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page