Skip to main content

A python library for accurate and scaleable data deduplication and entity-resolution

Project description

dedupe is a library that uses machine learning to perform de-duplication and entity resolution quickly on structured data. dedupe is the open source engine for dedupe.io

dedupe will help you:

  • remove duplicate entries from a spreadsheet of names and addresses

  • link a list with customer information to another with order history, even without unique customer id’s

  • take a database of campaign contributions and figure out which ones were made by the same person, even if the names were entered slightly differently for each record

dedupe takes in human training data and comes up with the best rules for your dataset to quickly and automatically find similar records, even with very large databases.

Important links:

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dedupe-1.6.17.tar.gz (48.7 kB view details)

Uploaded Source

Built Distributions

dedupe-1.6.17-cp36-cp36m-manylinux1_x86_64.whl (74.5 kB view details)

Uploaded CPython 3.6m

dedupe-1.6.17-cp36-cp36m-manylinux1_i686.whl (71.2 kB view details)

Uploaded CPython 3.6m

dedupe-1.6.17-cp36-cp36m-macosx_10_11_x86_64.whl (50.2 kB view details)

Uploaded CPython 3.6m macOS 10.11+ x86-64

dedupe-1.6.17-cp35-cp35m-manylinux1_x86_64.whl (74.3 kB view details)

Uploaded CPython 3.5m

dedupe-1.6.17-cp35-cp35m-manylinux1_i686.whl (71.0 kB view details)

Uploaded CPython 3.5m

dedupe-1.6.17-cp34-cp34m-win_amd64.whl (50.9 kB view details)

Uploaded CPython 3.4m Windows x86-64

dedupe-1.6.17-cp34-cp34m-win32.whl (50.2 kB view details)

Uploaded CPython 3.4m Windows x86

dedupe-1.6.17-cp34-cp34m-manylinux1_x86_64.whl (74.5 kB view details)

Uploaded CPython 3.4m

dedupe-1.6.17-cp34-cp34m-manylinux1_i686.whl (71.2 kB view details)

Uploaded CPython 3.4m

dedupe-1.6.17-cp27-cp27mu-manylinux1_x86_64.whl (72.2 kB view details)

Uploaded CPython 2.7mu

dedupe-1.6.17-cp27-cp27mu-manylinux1_i686.whl (69.5 kB view details)

Uploaded CPython 2.7mu

dedupe-1.6.17-cp27-cp27m-win_amd64.whl (51.0 kB view details)

Uploaded CPython 2.7m Windows x86-64

dedupe-1.6.17-cp27-cp27m-win32.whl (50.2 kB view details)

Uploaded CPython 2.7m Windows x86

dedupe-1.6.17-cp27-cp27m-manylinux1_x86_64.whl (72.1 kB view details)

Uploaded CPython 2.7m

dedupe-1.6.17-cp27-cp27m-manylinux1_i686.whl (69.5 kB view details)

Uploaded CPython 2.7m

dedupe-1.6.17-cp27-cp27m-macosx_10_11_x86_64.whl (49.8 kB view details)

Uploaded CPython 2.7m macOS 10.11+ x86-64

File details

Details for the file dedupe-1.6.17.tar.gz.

File metadata

  • Download URL: dedupe-1.6.17.tar.gz
  • Upload date:
  • Size: 48.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for dedupe-1.6.17.tar.gz
Algorithm Hash digest
SHA256 9fd51c942b9b4c302f22a7db1f151ced5140cd3c7b3f94b24ceceba9fd07b159
MD5 9210a0e03018985ae807ca31b58cbc54
BLAKE2b-256 0167a2c9c4b65dc57277c4b94ae0f84912910f0cdb02ec8b533295dd6e9cf19c

See more details on using hashes here.

File details

Details for the file dedupe-1.6.17-cp36-cp36m-manylinux1_x86_64.whl.

File metadata

File hashes

Hashes for dedupe-1.6.17-cp36-cp36m-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 ed6aab474edd2d52f8579bed13f104e2f5a834d7fe2acf0254da1dd01dd6ea9e
MD5 d1cd42e607834c090aa5647f09ac09d1
BLAKE2b-256 fa37900c851bfcee9109bf52debc6227c2cef14229a79eeecc3fe277de18b36a

See more details on using hashes here.

File details

Details for the file dedupe-1.6.17-cp36-cp36m-manylinux1_i686.whl.

File metadata

File hashes

Hashes for dedupe-1.6.17-cp36-cp36m-manylinux1_i686.whl
Algorithm Hash digest
SHA256 c90ebf8fe4491bc55bdfb077f17532cb6cad7676edad6371e3edb9808e3c2ab5
MD5 2f24edbccaa11bde6de9f5f263ccffcd
BLAKE2b-256 b41fdb254fc65f8f6b40602bec421a9ca90172b2e8a6067626a32d0907f7dad6

See more details on using hashes here.

File details

Details for the file dedupe-1.6.17-cp36-cp36m-macosx_10_11_x86_64.whl.

File metadata

File hashes

Hashes for dedupe-1.6.17-cp36-cp36m-macosx_10_11_x86_64.whl
Algorithm Hash digest
SHA256 8f82d8d4c150929f40627621e543572faa081d2fdf7e7e8605ebc097454c7805
MD5 c6152889c77b16427fb2fc86935f398c
BLAKE2b-256 7cedcf8e809c993688d99ae801a4c62bb261a4067fafff182a0e3c9c7272fb61

See more details on using hashes here.

File details

Details for the file dedupe-1.6.17-cp35-cp35m-manylinux1_x86_64.whl.

File metadata

File hashes

Hashes for dedupe-1.6.17-cp35-cp35m-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 5a6d9870ac48984b407dc3f579df3974d4e2c0dff1d0e074f51e42e08f332194
MD5 c76d4c30cf30b2f2fc93ba572213976f
BLAKE2b-256 4c43a5e295babdda375515b2aadd5adca822be8d21ca7716522af59f00a9e435

See more details on using hashes here.

File details

Details for the file dedupe-1.6.17-cp35-cp35m-manylinux1_i686.whl.

File metadata

File hashes

Hashes for dedupe-1.6.17-cp35-cp35m-manylinux1_i686.whl
Algorithm Hash digest
SHA256 c60974fe770ad38ff8e68dd0be8283ee87efd13f4b7f2d3d4a36e802df1d5905
MD5 4cc6f3f7337c26b5c1458d7d145ec7fc
BLAKE2b-256 48f5c5a8e11a0b0456da7f55632a439a3b212dd77b0faedead9b8a94d4e75fed

See more details on using hashes here.

File details

Details for the file dedupe-1.6.17-cp34-cp34m-win_amd64.whl.

File metadata

File hashes

Hashes for dedupe-1.6.17-cp34-cp34m-win_amd64.whl
Algorithm Hash digest
SHA256 11fa8469ef6b7b124f43916e32e7f7f03352ab98c263910a446f3d9f7062d71a
MD5 5193a847af45431ef2cb92948b50946d
BLAKE2b-256 293f53016e44e20f357a065cf4a65de782f659aeb8525392abfa12feca7f3e7f

See more details on using hashes here.

File details

Details for the file dedupe-1.6.17-cp34-cp34m-win32.whl.

File metadata

File hashes

Hashes for dedupe-1.6.17-cp34-cp34m-win32.whl
Algorithm Hash digest
SHA256 2cea4dad0511cd6ded8c05f6ee4a81c5de6c8836827183d390b0467731100e19
MD5 91edcc785fb751e6ae26c11777afe49b
BLAKE2b-256 66074c6779a4bdbddacfcf36fe97eb92bd9b298c0deaa992509b44d5393608f9

See more details on using hashes here.

File details

Details for the file dedupe-1.6.17-cp34-cp34m-manylinux1_x86_64.whl.

File metadata

File hashes

Hashes for dedupe-1.6.17-cp34-cp34m-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 5fef36cb0203551c46bab0a4e261315441105389d5b9ef950cae5f40b6263992
MD5 6ef29e8763decfff952e669f89d01f82
BLAKE2b-256 d04db8d7651467990c4d80b93d1f48828be915c47d94b5924b34b77c1c10765d

See more details on using hashes here.

File details

Details for the file dedupe-1.6.17-cp34-cp34m-manylinux1_i686.whl.

File metadata

File hashes

Hashes for dedupe-1.6.17-cp34-cp34m-manylinux1_i686.whl
Algorithm Hash digest
SHA256 05ae3aa8aa39240a1ecbf69f6987a5ed69eefa4368415014fdd849e6e9358635
MD5 083a8c24902cf51500bab62c0b6cadff
BLAKE2b-256 c765c24c62a665d2d261fb9da85a54bfb6494ae92ee3251fb19460183181a809

See more details on using hashes here.

File details

Details for the file dedupe-1.6.17-cp27-cp27mu-manylinux1_x86_64.whl.

File metadata

File hashes

Hashes for dedupe-1.6.17-cp27-cp27mu-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 0af946f683aaf7bab4b5a1113e136842b82d4e2e55a13f439c9c029f5bfb42e0
MD5 21535bf2037cce7e8508761056ac1ff4
BLAKE2b-256 a900fc847d2d94338e9475a8178180d179e075b5f039aa4c42daee7bb353de1e

See more details on using hashes here.

File details

Details for the file dedupe-1.6.17-cp27-cp27mu-manylinux1_i686.whl.

File metadata

File hashes

Hashes for dedupe-1.6.17-cp27-cp27mu-manylinux1_i686.whl
Algorithm Hash digest
SHA256 af08754850531b3fc002a94d34f732195ba0a7d81d0f0c4151ed836026278f60
MD5 d50d85d6d5c8ae5a1e81c111ef6cc447
BLAKE2b-256 6af4df1f666f8f99f8dc5c2561f7c911c5402117e96c3190580d2822dafcbe73

See more details on using hashes here.

File details

Details for the file dedupe-1.6.17-cp27-cp27m-win_amd64.whl.

File metadata

File hashes

Hashes for dedupe-1.6.17-cp27-cp27m-win_amd64.whl
Algorithm Hash digest
SHA256 5b2536687cca9433ae8f64141ff31dd54f88d226acefe466262e77e28c4e9806
MD5 541b699e50e076eea1ddc10eb7fe3bcc
BLAKE2b-256 cd1b5002e27945998d8c7a978d4d62d7e75c56769b6db436d733815253815cfa

See more details on using hashes here.

File details

Details for the file dedupe-1.6.17-cp27-cp27m-win32.whl.

File metadata

File hashes

Hashes for dedupe-1.6.17-cp27-cp27m-win32.whl
Algorithm Hash digest
SHA256 a14ec77205a5a91f49fec69bbb93cf8f29f15a01d56e86f4f07e277eb3c803cf
MD5 8adfbc40819c948aa4912e62a473ec75
BLAKE2b-256 359b8492306d20dbf945b3304ea017cda721d2da41b7870d5bc5dd3a60ba396d

See more details on using hashes here.

File details

Details for the file dedupe-1.6.17-cp27-cp27m-manylinux1_x86_64.whl.

File metadata

File hashes

Hashes for dedupe-1.6.17-cp27-cp27m-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 96827049a1d89e5fbd86ccf34148d79971925531103f9fc16b33fc432b33b719
MD5 5c1c4dabb206e6eea9cce56b9637cbab
BLAKE2b-256 0fb60ebc9d46296ef36eb3ca07ebd1cf27e77e25238e199b4c39e7e6de7f2883

See more details on using hashes here.

File details

Details for the file dedupe-1.6.17-cp27-cp27m-manylinux1_i686.whl.

File metadata

File hashes

Hashes for dedupe-1.6.17-cp27-cp27m-manylinux1_i686.whl
Algorithm Hash digest
SHA256 de7cecacf7fdc2a3211504339168db794c8064955ba321c52c37baa87350008c
MD5 b6885aed53961231a1c38bb561a7bcf8
BLAKE2b-256 706db4a5f2653e70fec1ad549ebd82272fc3883c1ac14ffc3df56c00e95dee4d

See more details on using hashes here.

File details

Details for the file dedupe-1.6.17-cp27-cp27m-macosx_10_11_x86_64.whl.

File metadata

File hashes

Hashes for dedupe-1.6.17-cp27-cp27m-macosx_10_11_x86_64.whl
Algorithm Hash digest
SHA256 10c433d4f84034af26adeb2675f49e4d8ccbea0df687d83e5f815a048d863491
MD5 4dcf653affdbcc214aa1cf6e43b114fb
BLAKE2b-256 c88ad6f0204230a2759f193284bb7d4ccead396fcad6fdc47d032dbc15ffed4e

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page