textdistance 3.0.0

Algorithms

Installation

pip install textdistance

pip install textdistance[common]

pip install textdistance[all]

pip install textdistance[Hamming]

pip install -e git+https://github.com/orsinium/textdistance.git#egg=textdistance

git clone https://github.com/orsinium/textdistance.git
pip install -e .[all]

Usage

All algorithms have 2 interfaces:

1. Class with algorithm-specific params for customizing.

2. Class instance with default params for quick and simple usage.

All algorithms have some common methods:

1. .distance(*sequences) – calculate distance between sequences.

2. .similarity(*sequences) – calculate similarity for sequences.

3. .maximum(*sequences) – maximum possible value for distance and similarity. For any sequence: distance + similarity == maximum.

4. .normalized_distance(*sequences) – normalized distance between sequences. The return value is a float between 0 and 1, where 0 means equal, and 1 totally different.

5. .normalized_similarity(*sequences) – normalized similarity for sequences. The return value is a float between 0 and 1, where 0 means totally different, and 1 equal.

Most common init arguments:

1. qval – q-value for split sequences into q-grams. Possible values:

   • 1 (default) – compare sequences by chars.

   • 2 or more – transform sequences to q-grams.

   • None – split sequences by words.

2. as_set – for token-based algorithms:

   • True – t and ttt is equal.

   • False (default) – t and ttt is different.

Example

For example, Hamming distance:

import textdistance

textdistance.hamming('test', 'text')
# 1

textdistance.hamming.distance('test', 'text')
# 1

textdistance.hamming.similarity('test', 'text')
# 3

textdistance.hamming.normalized_distance('test', 'text')
# 0.25

textdistance.hamming.normalized_similarity('test', 'text')
# 0.75

textdistance.Hamming(qval=2).distance('test', 'text')
# 2

Any other algorithms have same interface.

Side libraries

For main algorithms textdistance try to call known external libraries (fastest first) if available (installed in your system) and possible (this implementation can compare this sequences). Install textdistance with common extras for this feature.

You can disable this by passing external=False argument on init:

import textdistance
hamming = textdistance.Hamming(external=False)
hamming('text', 'testit')
# 3

Supported libraries:

1. abydos

2. Distance

3. jellyfish

4. py_stringmatching

5. pylev

6. python-Levenshtein

7. pyxDamerauLevenshtein

Benchmarks

For textdistance without extra requirements:

Total: 24 libs.

Textdistance use benchmark’s results for algorithm’s optimization and try call fastest external libs first (if possible).

If you want you can run benchmark manually on youre system:

pip install textdistance[all]
python3 -m textdistance.benchmark

Consequently textdistance show benchmarks results table for your system and save libraries priorities into libraries.json file in textdistance’s folder. This file will be used by textdistance for calling fastest algorithm implementation first.

Test

You can run tests via tox:

sudo pip3 install tox
tox