Skip to main content

A python library for accurate and scaleable data deduplication and entity-resolution

Project description

dedupe is a library that uses machine learning to perform de-duplication and entity resolution quickly on structured data. dedupe is the open source engine for dedupe.io

dedupe will help you:

  • remove duplicate entries from a spreadsheet of names and addresses

  • link a list with customer information to another with order history, even without unique customer id’s

  • take a database of campaign contributions and figure out which ones were made by the same person, even if the names were entered slightly differently for each record

dedupe takes in human training data and comes up with the best rules for your dataset to quickly and automatically find similar records, even with very large databases.

Important links:

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dedupe-1.8.0.dev1.tar.gz (75.5 kB view details)

Uploaded Source

Built Distributions

dedupe-1.8.0.dev1-cp36-cp36m-manylinux1_x86_64.whl (78.7 kB view details)

Uploaded CPython 3.6m

dedupe-1.8.0.dev1-cp36-cp36m-manylinux1_i686.whl (75.0 kB view details)

Uploaded CPython 3.6m

dedupe-1.8.0.dev1-cp36-cp36m-macosx_10_11_x86_64.whl (52.4 kB view details)

Uploaded CPython 3.6m macOS 10.11+ x86-64

dedupe-1.8.0.dev1-cp35-cp35m-manylinux1_x86_64.whl (78.5 kB view details)

Uploaded CPython 3.5m

dedupe-1.8.0.dev1-cp35-cp35m-manylinux1_i686.whl (74.8 kB view details)

Uploaded CPython 3.5m

dedupe-1.8.0.dev1-cp34-cp34m-win_amd64.whl (53.1 kB view details)

Uploaded CPython 3.4m Windows x86-64

dedupe-1.8.0.dev1-cp34-cp34m-win32.whl (52.4 kB view details)

Uploaded CPython 3.4m Windows x86

dedupe-1.8.0.dev1-cp34-cp34m-manylinux1_x86_64.whl (78.6 kB view details)

Uploaded CPython 3.4m

dedupe-1.8.0.dev1-cp34-cp34m-manylinux1_i686.whl (75.0 kB view details)

Uploaded CPython 3.4m

dedupe-1.8.0.dev1-cp27-cp27mu-manylinux1_x86_64.whl (75.8 kB view details)

Uploaded CPython 2.7mu

dedupe-1.8.0.dev1-cp27-cp27mu-manylinux1_i686.whl (72.9 kB view details)

Uploaded CPython 2.7mu

dedupe-1.8.0.dev1-cp27-cp27m-win_amd64.whl (53.0 kB view details)

Uploaded CPython 2.7m Windows x86-64

dedupe-1.8.0.dev1-cp27-cp27m-win32.whl (52.2 kB view details)

Uploaded CPython 2.7m Windows x86

dedupe-1.8.0.dev1-cp27-cp27m-manylinux1_x86_64.whl (75.8 kB view details)

Uploaded CPython 2.7m

dedupe-1.8.0.dev1-cp27-cp27m-manylinux1_i686.whl (72.9 kB view details)

Uploaded CPython 2.7m

dedupe-1.8.0.dev1-cp27-cp27m-macosx_10_11_x86_64.whl (51.8 kB view details)

Uploaded CPython 2.7m macOS 10.11+ x86-64

File details

Details for the file dedupe-1.8.0.dev1.tar.gz.

File metadata

  • Download URL: dedupe-1.8.0.dev1.tar.gz
  • Upload date:
  • Size: 75.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for dedupe-1.8.0.dev1.tar.gz
Algorithm Hash digest
SHA256 3a27a5fe6a1eb90ba0d0d17ee220fe0426e641df71c85fa50047bd5bc907f292
MD5 85a3eaca7d958844fd8d5f5361022104
BLAKE2b-256 239352a8c3e5b0be8bde662044dbbf0ffadb1880fd2573eb6998bb3292fac673

See more details on using hashes here.

File details

Details for the file dedupe-1.8.0.dev1-cp36-cp36m-manylinux1_x86_64.whl.

File metadata

File hashes

Hashes for dedupe-1.8.0.dev1-cp36-cp36m-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 d1bcd22c34aa6403d420655b1ef6f625232af9e9cfc2558520e0a677ceda28d8
MD5 eac35a91f51def2ba42e8f1361164a7d
BLAKE2b-256 e54203693d2724d693ba049df42aacaeb044a9018e52f6ef29af61517a84afc4

See more details on using hashes here.

File details

Details for the file dedupe-1.8.0.dev1-cp36-cp36m-manylinux1_i686.whl.

File metadata

File hashes

Hashes for dedupe-1.8.0.dev1-cp36-cp36m-manylinux1_i686.whl
Algorithm Hash digest
SHA256 f61af320f42b4e46fd6d0f4c8cea90b7dd7a9717f5176c2e42acdb093724d9b6
MD5 a103a2278eced397a638f3379d26e029
BLAKE2b-256 e8751d7d65f82a8a0d9a0d65d814ca8acfc8b41bde5b406e179e0738e018cc79

See more details on using hashes here.

File details

Details for the file dedupe-1.8.0.dev1-cp36-cp36m-macosx_10_11_x86_64.whl.

File metadata

File hashes

Hashes for dedupe-1.8.0.dev1-cp36-cp36m-macosx_10_11_x86_64.whl
Algorithm Hash digest
SHA256 c64f5508c6a52ec5f2e3eade542e4d6bb762ffbbbfd6cd83e860fb9f68b3d2f6
MD5 a30381e8aea6925a8dc29fb35026bfb2
BLAKE2b-256 8b06635d7f84a2b27813f706e3e2cd22baa338aad0bc1d5e65859f55f2e29bd8

See more details on using hashes here.

File details

Details for the file dedupe-1.8.0.dev1-cp35-cp35m-manylinux1_x86_64.whl.

File metadata

File hashes

Hashes for dedupe-1.8.0.dev1-cp35-cp35m-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 0a0d846256638dfc1a96438204571df5d54ec351dc922c5b5460ac8fa68b5087
MD5 3bb519780220a974e47c2533b3d48779
BLAKE2b-256 5b314e61bba9f14fa893b818198c1898e4dc5cc7d7178d922620451ef63e2e63

See more details on using hashes here.

File details

Details for the file dedupe-1.8.0.dev1-cp35-cp35m-manylinux1_i686.whl.

File metadata

File hashes

Hashes for dedupe-1.8.0.dev1-cp35-cp35m-manylinux1_i686.whl
Algorithm Hash digest
SHA256 03828cfdf819b4ebe65078e135051d842e05531d8a5debab14df4bb602613844
MD5 933267e9c010aa33575ff24e5647a135
BLAKE2b-256 29ce33ddac970fd1735dd766fbfa192ab2aad835ba7bc879a42901cc5f01b21f

See more details on using hashes here.

File details

Details for the file dedupe-1.8.0.dev1-cp34-cp34m-win_amd64.whl.

File metadata

File hashes

Hashes for dedupe-1.8.0.dev1-cp34-cp34m-win_amd64.whl
Algorithm Hash digest
SHA256 0d8f0fafbd011268896a19020a43f50274e2fe881576c4fa60199dc57be2c0f2
MD5 15cde58d4e4c600936dc6194d66dbff6
BLAKE2b-256 b62494e9c7b4529bafe6eb2a906746668ba5bfca77f26625a6c4fee43e39b7ea

See more details on using hashes here.

File details

Details for the file dedupe-1.8.0.dev1-cp34-cp34m-win32.whl.

File metadata

File hashes

Hashes for dedupe-1.8.0.dev1-cp34-cp34m-win32.whl
Algorithm Hash digest
SHA256 a6e8721fc2d6d3e684e154837b021a7f52bcc8a028e9715dbf4d7d29686dbf0a
MD5 df70af7f100d07e582b5e3efae64360e
BLAKE2b-256 c2be4ad54c29d128b7b1e2eee80fd79e781d480fade0252160009f8d99e8aa78

See more details on using hashes here.

File details

Details for the file dedupe-1.8.0.dev1-cp34-cp34m-manylinux1_x86_64.whl.

File metadata

File hashes

Hashes for dedupe-1.8.0.dev1-cp34-cp34m-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 6851a4952dd2631bdac7cc58c70a804a8bea2b2c2512b4962002dc1d8f9db87e
MD5 ac8927db3a4d7ea93f4bb4240ec0b31c
BLAKE2b-256 02d1311e3c23ca4e3f560fe0a4eff4183da2795f9066e40411961cbe275c5c73

See more details on using hashes here.

File details

Details for the file dedupe-1.8.0.dev1-cp34-cp34m-manylinux1_i686.whl.

File metadata

File hashes

Hashes for dedupe-1.8.0.dev1-cp34-cp34m-manylinux1_i686.whl
Algorithm Hash digest
SHA256 f0e45a6ac56514474829715e5856095ec1dd7ec77d02a508d1c5089689bbbe16
MD5 6f3fa5906495d54d224658ed7709ded3
BLAKE2b-256 70c3caa1af6a9d59b3749da1b6d07df19ff6d083a7cbb9c8c87461420e9dc10a

See more details on using hashes here.

File details

Details for the file dedupe-1.8.0.dev1-cp27-cp27mu-manylinux1_x86_64.whl.

File metadata

File hashes

Hashes for dedupe-1.8.0.dev1-cp27-cp27mu-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 295c6525de8b3f7451fcac3d3232e1e3c479d1bbb540c28c2dbb907bea30598c
MD5 4098d591c9be50702230c74544dd48bb
BLAKE2b-256 176d76e40a03d42f99037fcdcdd12b42beb6cc51893c445dda92e4a3204557ae

See more details on using hashes here.

File details

Details for the file dedupe-1.8.0.dev1-cp27-cp27mu-manylinux1_i686.whl.

File metadata

File hashes

Hashes for dedupe-1.8.0.dev1-cp27-cp27mu-manylinux1_i686.whl
Algorithm Hash digest
SHA256 a7b346184c93c78cde1a734655918dd8a5f49902892f77d0cafd58f1798e4a3b
MD5 57fdcd7c88814e21c561fa707b74630d
BLAKE2b-256 da7025fde9c835897c3e51d5bf529cc02f84125528c64d7801969e2181c0322b

See more details on using hashes here.

File details

Details for the file dedupe-1.8.0.dev1-cp27-cp27m-win_amd64.whl.

File metadata

File hashes

Hashes for dedupe-1.8.0.dev1-cp27-cp27m-win_amd64.whl
Algorithm Hash digest
SHA256 255efb05a3ef204dc26c25fbe46d8e32f9b3081312c903ec6d9b1206c3492306
MD5 7fd72969f8d4c0629b76156092cc1622
BLAKE2b-256 02328cd8137041329e14d6f691cf00b9f635c13b5a27bb0087f44dd5b1bac98a

See more details on using hashes here.

File details

Details for the file dedupe-1.8.0.dev1-cp27-cp27m-win32.whl.

File metadata

File hashes

Hashes for dedupe-1.8.0.dev1-cp27-cp27m-win32.whl
Algorithm Hash digest
SHA256 e3eefe303bdbaaa88f30ba74212b8f321548c87fdb5b5aba0982525f248abc97
MD5 f2077776a6623e6ae0ea4d7f3021193d
BLAKE2b-256 9ffc60805df9128c4fdf34830d94b8338f612ae1c2672e18017e2866f0761ecf

See more details on using hashes here.

File details

Details for the file dedupe-1.8.0.dev1-cp27-cp27m-manylinux1_x86_64.whl.

File metadata

File hashes

Hashes for dedupe-1.8.0.dev1-cp27-cp27m-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 d90f866330be990dfadf8e935b615c1f76e4af98b99bfa944043f12dc79fc641
MD5 f3d1626cdfe890cf0d2883d892a72c2c
BLAKE2b-256 89b509cc3150fe7de66f7705218a8b7df8c9a505fc13408f91b8190bf831b78d

See more details on using hashes here.

File details

Details for the file dedupe-1.8.0.dev1-cp27-cp27m-manylinux1_i686.whl.

File metadata

File hashes

Hashes for dedupe-1.8.0.dev1-cp27-cp27m-manylinux1_i686.whl
Algorithm Hash digest
SHA256 a2dff8b09d9c3cc985b815879bb9d76dd220873679966987683e96964fbc23f5
MD5 8f5ce3cf8d769048c9cf8584350d5225
BLAKE2b-256 2e385d8f8e3ab4e6cafaba95374ed1a40bf7993a89e523c1d65e5030016a7ad3

See more details on using hashes here.

File details

Details for the file dedupe-1.8.0.dev1-cp27-cp27m-macosx_10_11_x86_64.whl.

File metadata

File hashes

Hashes for dedupe-1.8.0.dev1-cp27-cp27m-macosx_10_11_x86_64.whl
Algorithm Hash digest
SHA256 52dd897ce4505c7512b7cb8ee60347fe11f62fe68d1159ac518d60193bf90477
MD5 0b22b7a955a5bc410bb5deed01a659b6
BLAKE2b-256 c6eb54c488096d1065f3ee016fe70f210677cf360c2a02a85eb4ce226875ebaa

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page