Skip to main content

A python library for accurate and scaleable data deduplication and entity-resolution

Project description

dedupe is a library that uses machine learning to perform de-duplication and entity resolution quickly on structured data. dedupe is the open source engine for dedupe.io

dedupe will help you:

  • remove duplicate entries from a spreadsheet of names and addresses

  • link a list with customer information to another with order history, even without unique customer id’s

  • take a database of campaign contributions and figure out which ones were made by the same person, even if the names were entered slightly differently for each record

dedupe takes in human training data and comes up with the best rules for your dataset to quickly and automatically find similar records, even with very large databases.

Important links:

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dedupe-1.6.9.tar.gz (48.3 kB view details)

Uploaded Source

Built Distributions

dedupe-1.6.9-cp36-cp36m-manylinux1_x86_64.whl (74.1 kB view details)

Uploaded CPython 3.6m

dedupe-1.6.9-cp36-cp36m-manylinux1_i686.whl (70.8 kB view details)

Uploaded CPython 3.6m

dedupe-1.6.9-cp36-cp36m-macosx_10_11_x86_64.whl (49.8 kB view details)

Uploaded CPython 3.6m macOS 10.11+ x86-64

dedupe-1.6.9-cp35-cp35m-manylinux1_x86_64.whl (73.9 kB view details)

Uploaded CPython 3.5m

dedupe-1.6.9-cp35-cp35m-manylinux1_i686.whl (70.6 kB view details)

Uploaded CPython 3.5m

dedupe-1.6.9-cp34-cp34m-win_amd64.whl (50.5 kB view details)

Uploaded CPython 3.4m Windows x86-64

dedupe-1.6.9-cp34-cp34m-win32.whl (49.8 kB view details)

Uploaded CPython 3.4m Windows x86

dedupe-1.6.9-cp34-cp34m-manylinux1_x86_64.whl (74.1 kB view details)

Uploaded CPython 3.4m

dedupe-1.6.9-cp34-cp34m-manylinux1_i686.whl (70.8 kB view details)

Uploaded CPython 3.4m

dedupe-1.6.9-cp27-cp27mu-manylinux1_x86_64.whl (71.8 kB view details)

Uploaded CPython 2.7mu

dedupe-1.6.9-cp27-cp27mu-manylinux1_i686.whl (69.1 kB view details)

Uploaded CPython 2.7mu

dedupe-1.6.9-cp27-cp27m-win_amd64.whl (50.6 kB view details)

Uploaded CPython 2.7m Windows x86-64

dedupe-1.6.9-cp27-cp27m-win32.whl (49.8 kB view details)

Uploaded CPython 2.7m Windows x86

dedupe-1.6.9-cp27-cp27m-manylinux1_x86_64.whl (71.7 kB view details)

Uploaded CPython 2.7m

dedupe-1.6.9-cp27-cp27m-manylinux1_i686.whl (69.1 kB view details)

Uploaded CPython 2.7m

dedupe-1.6.9-cp27-cp27m-macosx_10_11_x86_64.whl (49.4 kB view details)

Uploaded CPython 2.7m macOS 10.11+ x86-64

File details

Details for the file dedupe-1.6.9.tar.gz.

File metadata

  • Download URL: dedupe-1.6.9.tar.gz
  • Upload date:
  • Size: 48.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for dedupe-1.6.9.tar.gz
Algorithm Hash digest
SHA256 23ee323eaa3df37ddf3a02cee112d05b6e1f6cadb66d446888d8f6ee7d5396f1
MD5 6bc55f9e56c3c20e9ee1152352d79736
BLAKE2b-256 1c582c1510602974dbdad0cd3f61f85053bb5cccb9b8a3d01363b2ba09a01f4a

See more details on using hashes here.

File details

Details for the file dedupe-1.6.9-cp36-cp36m-manylinux1_x86_64.whl.

File metadata

File hashes

Hashes for dedupe-1.6.9-cp36-cp36m-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 6382c84106e000b9e94e7eb6f893dad8de3b8df9fcbaff6d79fe29cac1baa1b9
MD5 c3b44481834b8059d61e5ecbf414bb7b
BLAKE2b-256 b03a686dc0fb1565102f7f21d812d521a206cf4ee500110a83b4ab2f4a713f36

See more details on using hashes here.

File details

Details for the file dedupe-1.6.9-cp36-cp36m-manylinux1_i686.whl.

File metadata

File hashes

Hashes for dedupe-1.6.9-cp36-cp36m-manylinux1_i686.whl
Algorithm Hash digest
SHA256 d1e0865c76a39d7e23cce5a63f6c6795f875f8aa3170467f9e40797691b9b811
MD5 e3f2e6b7a1f565b312404917c8a164ca
BLAKE2b-256 1ca65e73056b006d68f247fd2f16e63fa265781625c11518fbeefddd7d7c6f20

See more details on using hashes here.

File details

Details for the file dedupe-1.6.9-cp36-cp36m-macosx_10_11_x86_64.whl.

File metadata

File hashes

Hashes for dedupe-1.6.9-cp36-cp36m-macosx_10_11_x86_64.whl
Algorithm Hash digest
SHA256 51cb31c411b7e290b53a9ee95727bb15122de22a35fb6cf17ae4e70349ae5034
MD5 105101550447d23cd0f055d4abe17853
BLAKE2b-256 6437e839cae7ec5344a3157c7d26f96f67e709c59e48d2b4f5a2718d1a8d0659

See more details on using hashes here.

File details

Details for the file dedupe-1.6.9-cp35-cp35m-manylinux1_x86_64.whl.

File metadata

File hashes

Hashes for dedupe-1.6.9-cp35-cp35m-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 20d9164d5901593a6a47135834a3be8d1218efc022620c2087dca060ed6504a7
MD5 bcbb312e6d6bd9d6749b6d472f69086e
BLAKE2b-256 d4baf3a2593eb7f6bc1edb98b9b5962fa5031a968631f69318589cbda716ee2d

See more details on using hashes here.

File details

Details for the file dedupe-1.6.9-cp35-cp35m-manylinux1_i686.whl.

File metadata

File hashes

Hashes for dedupe-1.6.9-cp35-cp35m-manylinux1_i686.whl
Algorithm Hash digest
SHA256 80080f83ab2c8fef2fba1446098c35933951c268aba457be08f743890c27ac11
MD5 e37ad6e9f9fb9c71909e6ef55a82251d
BLAKE2b-256 65870edb15fbeb6b2a699efadd6107a13a6110994bf6a47a4326505d6f94ee3b

See more details on using hashes here.

File details

Details for the file dedupe-1.6.9-cp34-cp34m-win_amd64.whl.

File metadata

File hashes

Hashes for dedupe-1.6.9-cp34-cp34m-win_amd64.whl
Algorithm Hash digest
SHA256 b876b4e92809f3993801f7d82752f3ad77e4ce8fa896b4066ff937b9dd8e25b3
MD5 6a35757ed465b61ceee9703b24e690ec
BLAKE2b-256 5268970c7b7278ace1240e30aa03cb9222fbd5604a45835afe21dd8b7a7675b2

See more details on using hashes here.

File details

Details for the file dedupe-1.6.9-cp34-cp34m-win32.whl.

File metadata

File hashes

Hashes for dedupe-1.6.9-cp34-cp34m-win32.whl
Algorithm Hash digest
SHA256 a1ab7252760b19b00530dad58d2c5c2c7a2d692ffa80e17975b0323a764fdb2f
MD5 1a3a0a8ab7c4e3f66f64279c6c5c5fc6
BLAKE2b-256 60af9e5a302f77f6a86befa6ebabf673196811d87c61414a0db7a1914b8bf3dd

See more details on using hashes here.

File details

Details for the file dedupe-1.6.9-cp34-cp34m-manylinux1_x86_64.whl.

File metadata

File hashes

Hashes for dedupe-1.6.9-cp34-cp34m-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 5f5344e564837e8352d3f35bd6cf41b69e9149a3af4b9a3ed189380f4f8b05a1
MD5 e751174ffe23dd2ef13142b5072ca051
BLAKE2b-256 5c305e209661b2f4eab618f37590545423eeca8a938ab0a2dec810894a8d872f

See more details on using hashes here.

File details

Details for the file dedupe-1.6.9-cp34-cp34m-manylinux1_i686.whl.

File metadata

File hashes

Hashes for dedupe-1.6.9-cp34-cp34m-manylinux1_i686.whl
Algorithm Hash digest
SHA256 1d0b95104a6bdf02f42d381e2b8f4137ab83b9acabc174e89489b2487daed8a2
MD5 c6203b5eda3cd6b9b8d802a87d854b6d
BLAKE2b-256 0c9da56480de27d53549181d3947fea6535f370b97f976275b2f63acac2e169c

See more details on using hashes here.

File details

Details for the file dedupe-1.6.9-cp27-cp27mu-manylinux1_x86_64.whl.

File metadata

File hashes

Hashes for dedupe-1.6.9-cp27-cp27mu-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 ab0f27b9af32ba779802db52a7a42cb8a6fa8211b0f0e038bbcb39f9dca4711e
MD5 4f8e3a325f2c197610ba8c75c71abb2e
BLAKE2b-256 a8bae3adf6314951eeeae9a95ced32b822e67e422b7658918b7511430fd4153c

See more details on using hashes here.

File details

Details for the file dedupe-1.6.9-cp27-cp27mu-manylinux1_i686.whl.

File metadata

File hashes

Hashes for dedupe-1.6.9-cp27-cp27mu-manylinux1_i686.whl
Algorithm Hash digest
SHA256 1dbbe4d0c6244c8b65ce9a242e5e1eeba4949a290d4011d3bc72f88119725ec9
MD5 57e732f56ed8f4f607ea458647347a2c
BLAKE2b-256 c1635061ba31969e09ee863130b0f2b0f4a82f9301c3801bb713861b0436f07e

See more details on using hashes here.

File details

Details for the file dedupe-1.6.9-cp27-cp27m-win_amd64.whl.

File metadata

File hashes

Hashes for dedupe-1.6.9-cp27-cp27m-win_amd64.whl
Algorithm Hash digest
SHA256 c7dd2139b03bdd3cf1179c36facfc0ffd953ed3a43ed681ab779b7d246c5d4fc
MD5 eddb586c5c73ce498dad664bfff0dff7
BLAKE2b-256 c8ec97eca8ff7ae0120260f7523a216a469b6dc1f319da10072989378eb75131

See more details on using hashes here.

File details

Details for the file dedupe-1.6.9-cp27-cp27m-win32.whl.

File metadata

File hashes

Hashes for dedupe-1.6.9-cp27-cp27m-win32.whl
Algorithm Hash digest
SHA256 c438f42282c175e1f0fd923ac3b6f5ef159cc688f57c167ed4bc8fb26c6a23bc
MD5 ec43f80008ba572ac99f508136f2819c
BLAKE2b-256 a12e00cf3d4ede6f58b7f74b87010957526bbb2d1a200f4a0b521275594400b6

See more details on using hashes here.

File details

Details for the file dedupe-1.6.9-cp27-cp27m-manylinux1_x86_64.whl.

File metadata

File hashes

Hashes for dedupe-1.6.9-cp27-cp27m-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 afc91153e76edc7d899defbc2147b89ef6ab7badef939d0ba7c7ceee54bf293f
MD5 80dbd4fa459e5205c65e22efe38d45b6
BLAKE2b-256 bcd6eeedcd34bd9466b47bf0274811b7b6479f6dc1c1d30d1e3727e660f4715c

See more details on using hashes here.

File details

Details for the file dedupe-1.6.9-cp27-cp27m-manylinux1_i686.whl.

File metadata

File hashes

Hashes for dedupe-1.6.9-cp27-cp27m-manylinux1_i686.whl
Algorithm Hash digest
SHA256 b37744f7ddd3d942f567cae02fdb13c3e229b061960943767b490e14bebdd792
MD5 a5005a388020902ae4bbf9a559468ac2
BLAKE2b-256 de1a0874674fc017cc1ea92bb34f890605de9cee7723a683b387d699d5fa042f

See more details on using hashes here.

File details

Details for the file dedupe-1.6.9-cp27-cp27m-macosx_10_11_x86_64.whl.

File metadata

File hashes

Hashes for dedupe-1.6.9-cp27-cp27m-macosx_10_11_x86_64.whl
Algorithm Hash digest
SHA256 7d15d3cad74e52a2b5ad4d0c08c96a796674e6a7d219b0f9f794c8b30c544edd
MD5 9da7ea1533f7a9bb2f4a729fd4d1fa72
BLAKE2b-256 028b4aeeead02e6fcc80b16598a6d9d8aa40158c422380d798dc19084276499c

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page