Skip to main content

Import and export metadata about mongodb databases and collections

Project description

Import/Export Mongo Schema
===================
Import and export mongodb schemas **without** copying all the data. It will extract the following meta data about a mongo database:

* Collections
* Indexes
* Cap sizes
* Schema validators

The primary use case is when you have developed an application that uses mongodb, and want to setup a new instance with
the appopriate database layout. You can then provide a `config.json` file with your application and have this script
setup the database for you without the need for extra `ensureIndex` calls!

This data will be stored in a `json` file for a database that looks something like this:

```json
{
"databases": {
"test": {
"location": {
"indexes": [
{
"name": "_id_",
"keys": [
[
"_id",
1
]
]
},
{
"name": "pos_2dsphere",
"2dsphereIndexVersion": 3,
"keys": [
[
"pos",
"2dsphere"
]
]
},
{
"name": "device_1_timestamp_1",
"keys": [
[
"device",
1
],
[
"timestamp",
1
]
]
}
],
"options": {}
},
"users": {
"indexes": [
{
"name": "_id_",
"keys": [
[
"_id",
1
]
]
},
{
"unique": true,
"name": "username_idx",
"keys": [
[
"username",
1
]
]
}
],
"options": {
"validator": {
"$jsonSchema": {
"bsonType": "object",
"required": [
"username",
"password",
"level"
],
"properties": {
"username": {
"bsonType": "string",
"description": "must be a string and is required"
},
"level": {
"bsonType": "string",
"enum": [
"user",
"admin",
"moderator"
],
"description": "must be a string"
},
"password": {
"bsonType": "string",
"description": "must be a bcrypt password",
"pattern": "^\\$2b\\$\\d{1,2}\\$[A-Za-z0-9\\.\\/]{53}$"
}
}
}
},
"validationLevel": "strict",
"validationAction": "error"
}
},
"capped": {
"indexes": [
{
"name": "_id_",
"keys": [
[
"_id",
1
]
]
},
{
"unique": true,
"name": "key_1",
"keys": [
[
"key",
1
]
]
}
],
"options": {
"capped": true,
"size": 64000,
"max": 5000,
"validator": {
"$jsonSchema": {
"bsonType": "object",
"description": "Simple key value store",
"required": [
"key",
"value"
],
"properties": {
"key": {
"bsonType": "string",
"maxLength": 64.0,
"description": "the key value"
},
"value": {
"bsonType": "string",
"description": "the associated value"
}
}
}
},
"validationLevel": "strict",
"validationAction": "error"
}
}
}
},
"exported": "2018-07-18T17:12:43.460992"
}
```

Installation
------------

```bash
pip install MongoSchemaImportExport
```

Usage
-----
Make sure the user you are using to import/export has the appropriate privileges, they'll probably need to have the
`root` role or `dbOwner` on the source and destination.
To export your data run:

```bash
mongo-schema-export.py --uri mongodb://user:password@database.host1.com:27017/admin --databases test2,testIgnore
```

To import your schema run as bellow. Use `--delete-col` to delete collections before creating them (**WARNING:** this
will delete your data, you cannot change
an existing collection into a capped one, although, you can set a validator after creation):

```bash
mongo-schema-import.py --uri mongodb://user:password@database.host2.com:27017/admin --databases db_1,db_2 --verbose --delete-col
```
You will get an output like this:

```
Skipping: testIgnore
Creating database: test2
Dropping collection location
Creating collection: location
Options {}
Creating index: {'name': '_id_', 'keys': [['_id', 1]]}
Creating index: {'name': 'pos_2dsphere', '2dsphereIndexVersion': 3, 'keys': [['pos', '2dsphere']]}
Creating index: {'name': 'device_1_timestamp_1', 'keys': [['device', 1], ['timestamp', 1]]}
Dropping collection users
Creating collection: users
Options {'validator': {'$jsonSchema': {'bsonType': 'object', 'required': ['username', 'password', 'level'], 'properties': {'username': {'bsonType': 'string', 'description': 'must be a string and is required'}, 'level': {'bsonType': 'string', 'enum': ['user', 'admin', 'moderator'], 'description': 'must be a string'}, 'password': {'bsonType': 'string', 'description': 'must be a bcrypt password', 'pattern': '^\\$2b\\$\\d{1,2}\\$[A-Za-z0-9\\.\\/]{53}$'}}}}, 'validationLevel': 'strict', 'validationAction': 'error'}
Creating index: {'name': '_id_', 'keys': [['_id', 1]]}
Creating index: {'unique': True, 'name': 'username_idx', 'keys': [['username', 1]]}
Dropping collection capped
Creating collection: capped
Options {'capped': True, 'size': 64000, 'max': 5000, 'validator': {'$jsonSchema': {'bsonType': 'object', 'description': 'Simple key value store', 'required': ['key', 'value'], 'properties': {'key': {'bsonType': 'string', 'maxLength': 64.0, 'description': 'the key value'}, 'value': {'bsonType': 'string', 'description': 'the associated value'}}}}, 'validationLevel': 'strict', 'validationAction': 'error'}
Creating index: {'name': '_id_', 'keys': [['_id', 1]]}
Creating index: {'unique': True, 'name': 'key_1', 'keys': [['key', 1]]}

```


If you get permission errors, make sure your user has the right roles to read and write databases and collections.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

MongoSchemaImportExport-0.2.1.tar.gz (5.2 kB view details)

Uploaded Source

Built Distribution

MongoSchemaImportExport-0.2.1-py3-none-any.whl (9.0 kB view details)

Uploaded Python 3

File details

Details for the file MongoSchemaImportExport-0.2.1.tar.gz.

File metadata

  • Download URL: MongoSchemaImportExport-0.2.1.tar.gz
  • Upload date:
  • Size: 5.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.12.1 pkginfo/1.4.2 requests/2.18.4 setuptools/40.0.0 requests-toolbelt/0.8.0 tqdm/4.23.0 CPython/3.6.4

File hashes

Hashes for MongoSchemaImportExport-0.2.1.tar.gz
Algorithm Hash digest
SHA256 01aaef2ea78a1f80e5005584e75e81300d54135f9571c75557bed631bd8f8e65
MD5 ef1e4772ed9c86612a16d76ee02bdf77
BLAKE2b-256 e6aa0ceb4e88b27cd8a7ebdb8acf0da99f1ff1bf218f8fd53ee74f523b3cf67b

See more details on using hashes here.

File details

Details for the file MongoSchemaImportExport-0.2.1-py3-none-any.whl.

File metadata

  • Download URL: MongoSchemaImportExport-0.2.1-py3-none-any.whl
  • Upload date:
  • Size: 9.0 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.12.1 pkginfo/1.4.2 requests/2.18.4 setuptools/40.0.0 requests-toolbelt/0.8.0 tqdm/4.23.0 CPython/3.6.4

File hashes

Hashes for MongoSchemaImportExport-0.2.1-py3-none-any.whl
Algorithm Hash digest
SHA256 0768db1950bc125f9c20cfe116342e2b72c845248bac8aa3de7d750ab767eb28
MD5 df927290b78cbfd047e529f4b44562e5
BLAKE2b-256 d6280ac4d1390039a2fb64c0deda2d851ffbcb2db8a5919a0800cfaa3853d4e1

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page