Skip to main content

A python library for accurate and scaleable data deduplication and entity-resolution

Project description

dedupe is a library that uses machine learning to perform de-duplication and entity resolution quickly on structured data. dedupe is the open source engine for dedupe.io

dedupe will help you:

  • remove duplicate entries from a spreadsheet of names and addresses

  • link a list with customer information to another with order history, even without unique customer id’s

  • take a database of campaign contributions and figure out which ones were made by the same person, even if the names were entered slightly differently for each record

dedupe takes in human training data and comes up with the best rules for your dataset to quickly and automatically find similar records, even with very large databases.

Important links:

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dedupe-1.6.10.tar.gz (48.6 kB view details)

Uploaded Source

Built Distributions

dedupe-1.6.10-cp36-cp36m-manylinux1_x86_64.whl (74.4 kB view details)

Uploaded CPython 3.6m

dedupe-1.6.10-cp36-cp36m-manylinux1_i686.whl (71.2 kB view details)

Uploaded CPython 3.6m

dedupe-1.6.10-cp36-cp36m-macosx_10_11_x86_64.whl (50.1 kB view details)

Uploaded CPython 3.6m macOS 10.11+ x86-64

dedupe-1.6.10-cp35-cp35m-manylinux1_x86_64.whl (74.2 kB view details)

Uploaded CPython 3.5m

dedupe-1.6.10-cp35-cp35m-manylinux1_i686.whl (70.9 kB view details)

Uploaded CPython 3.5m

dedupe-1.6.10-cp34-cp34m-win_amd64.whl (50.8 kB view details)

Uploaded CPython 3.4m Windows x86-64

dedupe-1.6.10-cp34-cp34m-win32.whl (50.1 kB view details)

Uploaded CPython 3.4m Windows x86

dedupe-1.6.10-cp34-cp34m-manylinux1_x86_64.whl (74.4 kB view details)

Uploaded CPython 3.4m

dedupe-1.6.10-cp34-cp34m-manylinux1_i686.whl (71.1 kB view details)

Uploaded CPython 3.4m

dedupe-1.6.10-cp27-cp27mu-manylinux1_x86_64.whl (72.1 kB view details)

Uploaded CPython 2.7mu

dedupe-1.6.10-cp27-cp27mu-manylinux1_i686.whl (69.4 kB view details)

Uploaded CPython 2.7mu

dedupe-1.6.10-cp27-cp27m-win_amd64.whl (50.9 kB view details)

Uploaded CPython 2.7m Windows x86-64

dedupe-1.6.10-cp27-cp27m-win32.whl (50.1 kB view details)

Uploaded CPython 2.7m Windows x86

dedupe-1.6.10-cp27-cp27m-manylinux1_x86_64.whl (72.1 kB view details)

Uploaded CPython 2.7m

dedupe-1.6.10-cp27-cp27m-manylinux1_i686.whl (69.4 kB view details)

Uploaded CPython 2.7m

dedupe-1.6.10-cp27-cp27m-macosx_10_11_x86_64.whl (49.7 kB view details)

Uploaded CPython 2.7m macOS 10.11+ x86-64

File details

Details for the file dedupe-1.6.10.tar.gz.

File metadata

  • Download URL: dedupe-1.6.10.tar.gz
  • Upload date:
  • Size: 48.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for dedupe-1.6.10.tar.gz
Algorithm Hash digest
SHA256 5ca4a63396b12a4e219e79281140eef21bd8b7d983f3da798a58a808a0248804
MD5 e070463b5dc38e0e0c02eac911379050
BLAKE2b-256 a9806936e0de1de8660ed617e934e3b69a76469232f3dc36f903a3dbca57097a

See more details on using hashes here.

File details

Details for the file dedupe-1.6.10-cp36-cp36m-manylinux1_x86_64.whl.

File metadata

File hashes

Hashes for dedupe-1.6.10-cp36-cp36m-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 696c88b5ab8b5bdaf7e09607f75ac75b757d875491c8e5cd96e2e33f34003a40
MD5 47bc42c98b5e539bfc1598960850b6de
BLAKE2b-256 340f7b90491703648eb94491988c66a3b3cdc71cb26f20046402e27579721663

See more details on using hashes here.

File details

Details for the file dedupe-1.6.10-cp36-cp36m-manylinux1_i686.whl.

File metadata

File hashes

Hashes for dedupe-1.6.10-cp36-cp36m-manylinux1_i686.whl
Algorithm Hash digest
SHA256 12dbc45553a820bf0f8f2245d801fcdfb5b8a48669748e1dadfa8a433ea19813
MD5 a048d16c6a94a7762613b77fe0b7e2bd
BLAKE2b-256 6fb2bbde24d837af4361d13e7d27a14ae6a55fdac4dcd62bf58fd96e477a13fe

See more details on using hashes here.

File details

Details for the file dedupe-1.6.10-cp36-cp36m-macosx_10_11_x86_64.whl.

File metadata

File hashes

Hashes for dedupe-1.6.10-cp36-cp36m-macosx_10_11_x86_64.whl
Algorithm Hash digest
SHA256 3f9c08ca1e9d403942de8b7ab1b2da9dda08d15fc2b2ffdddaf95ba3997b1d5d
MD5 b07ad42365c52c5187302822ef47c968
BLAKE2b-256 bc9c4e647b8b7dc0c7de180fbd80159c231e5098a63d3a8b8372433c2c7d3364

See more details on using hashes here.

File details

Details for the file dedupe-1.6.10-cp35-cp35m-manylinux1_x86_64.whl.

File metadata

File hashes

Hashes for dedupe-1.6.10-cp35-cp35m-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 c73ca83e660ca5e6fd3542a9983f1d9db95d3b70509577116bfa851fd1f85a95
MD5 9b11a5b4edf8f60916750fb984ee365a
BLAKE2b-256 191a8d240736686f59772dc0d93e100b9cdd2771b582554ad8fb5c4a070c71c8

See more details on using hashes here.

File details

Details for the file dedupe-1.6.10-cp35-cp35m-manylinux1_i686.whl.

File metadata

File hashes

Hashes for dedupe-1.6.10-cp35-cp35m-manylinux1_i686.whl
Algorithm Hash digest
SHA256 0e9ff789455fcdf0ef10144d719066ba240c64e7543347ec7f1e75c576942404
MD5 43e2c71d1ab129158472ee047212aa73
BLAKE2b-256 4345c3768771fecc8961c88501d707f8cac133a8ca29c71f9822132e7fcd32d6

See more details on using hashes here.

File details

Details for the file dedupe-1.6.10-cp34-cp34m-win_amd64.whl.

File metadata

File hashes

Hashes for dedupe-1.6.10-cp34-cp34m-win_amd64.whl
Algorithm Hash digest
SHA256 f9613eecd8db37613e2f58fbba27e101060140f424fe4a870d6d30dd64bc435a
MD5 58e07fd28456cf866d34f8e9ad6733b1
BLAKE2b-256 2b934f724251c7537d7c8158c7de2e22ab154ae1e56db509f0f3caef9a6040bb

See more details on using hashes here.

File details

Details for the file dedupe-1.6.10-cp34-cp34m-win32.whl.

File metadata

File hashes

Hashes for dedupe-1.6.10-cp34-cp34m-win32.whl
Algorithm Hash digest
SHA256 5ca5dd337dece4521d3641fd32a1895582002c6b022f78d8ac3665438524aa23
MD5 03b14bbe28bf464efa947165558fa2c9
BLAKE2b-256 e9fc79d0d74502bead2ea3f24a0dfbbaa3a4bd54bc672272a9d50855a31a6f88

See more details on using hashes here.

File details

Details for the file dedupe-1.6.10-cp34-cp34m-manylinux1_x86_64.whl.

File metadata

File hashes

Hashes for dedupe-1.6.10-cp34-cp34m-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 aa7af5c16c6621fdb3538589c1ff920fec1e5f8938d028077ccf2d400912fa29
MD5 7755a9fa881a63ecceea0e2b06fc552e
BLAKE2b-256 391d970e0bf704913120a418ab8dbc8b2693911739bb2d3d0a21017871ce6f60

See more details on using hashes here.

File details

Details for the file dedupe-1.6.10-cp34-cp34m-manylinux1_i686.whl.

File metadata

File hashes

Hashes for dedupe-1.6.10-cp34-cp34m-manylinux1_i686.whl
Algorithm Hash digest
SHA256 17c493d45574e0801640df8334af19b0ac0c0fba1d5897c56b6cdbe15f18bfb5
MD5 7fd6b350ce8d69b4a6267b7763ce70f5
BLAKE2b-256 3ed49b643c4534b9c44a33b236eecc0a18aef9f915ef0387bf589df9bfe233db

See more details on using hashes here.

File details

Details for the file dedupe-1.6.10-cp27-cp27mu-manylinux1_x86_64.whl.

File metadata

File hashes

Hashes for dedupe-1.6.10-cp27-cp27mu-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 efa162232cab0fd499bac3b6639fb59e4d7b173a34c532f225938046e28075d7
MD5 025b37ad9587c5186aa319b44ecc8dd8
BLAKE2b-256 47192b0c6b4f4012b5e8545ecfddfeed9a9b671497065b270db26b9a8baf674a

See more details on using hashes here.

File details

Details for the file dedupe-1.6.10-cp27-cp27mu-manylinux1_i686.whl.

File metadata

File hashes

Hashes for dedupe-1.6.10-cp27-cp27mu-manylinux1_i686.whl
Algorithm Hash digest
SHA256 c4f8e06a44dddf4221057f239cefe1a033d7fca912216f827132fa3cfa454eba
MD5 4f0ddcd39b45c67487d1d01d33054f8f
BLAKE2b-256 0b244c8a88d393d7916b6d9823b3726a36cd4722610687b3344aff2f64f88f56

See more details on using hashes here.

File details

Details for the file dedupe-1.6.10-cp27-cp27m-win_amd64.whl.

File metadata

File hashes

Hashes for dedupe-1.6.10-cp27-cp27m-win_amd64.whl
Algorithm Hash digest
SHA256 64e3391a641a7e1d6e24b55624f7a30903f5060725359843cb6d943a3ae928cd
MD5 08397ae5c6a305bc6c74c54abe05489e
BLAKE2b-256 144dbe791c539faa0d4c3fb5a57322ce72c7a8a267e9f5eb9d8cf40ab1414f4b

See more details on using hashes here.

File details

Details for the file dedupe-1.6.10-cp27-cp27m-win32.whl.

File metadata

File hashes

Hashes for dedupe-1.6.10-cp27-cp27m-win32.whl
Algorithm Hash digest
SHA256 48c7e0d002d0812ee78bc93f3374ec63fd3b13de76b2b57fb88e809118e5e692
MD5 89f2c71ad0080e4cd7ea102f8214bfa9
BLAKE2b-256 276e0d9dd40d5e7a3ee987449831c9d6ea4360fb9b3cc7ab70a05c0c1bd81783

See more details on using hashes here.

File details

Details for the file dedupe-1.6.10-cp27-cp27m-manylinux1_x86_64.whl.

File metadata

File hashes

Hashes for dedupe-1.6.10-cp27-cp27m-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 7f4962acc9141f67999d6ab72b61382bb928e872081784125f0acd24b9ff5e61
MD5 b26d284464e8586baa5d41c2b31df6b6
BLAKE2b-256 5b5fc64aa32ac900fee99fa8190c93ee6f91c8a2e108760caa230d3b723fb38a

See more details on using hashes here.

File details

Details for the file dedupe-1.6.10-cp27-cp27m-manylinux1_i686.whl.

File metadata

File hashes

Hashes for dedupe-1.6.10-cp27-cp27m-manylinux1_i686.whl
Algorithm Hash digest
SHA256 5ef7cfcbbd5e935caa286c8821e92a36d9571a14c767059babe92c06ae2ae5c8
MD5 75831deb8ce631860a9d599805465ad7
BLAKE2b-256 7cd03bb85ccdf1225452bb3105d93910b72ec3996a807fcb8d8a14307cee0c64

See more details on using hashes here.

File details

Details for the file dedupe-1.6.10-cp27-cp27m-macosx_10_11_x86_64.whl.

File metadata

File hashes

Hashes for dedupe-1.6.10-cp27-cp27m-macosx_10_11_x86_64.whl
Algorithm Hash digest
SHA256 28833269f245e9221a05421b4189e042e97a303aa73bbd401d652ab7265482a0
MD5 f3a04ae938f369995cdc0e89f2718b0f
BLAKE2b-256 4668922b959314da3ce35ee7b7829fe31bc214136ddaa16f12010e7963c2fbfd

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page