Simple Thai Wordcut in Python using Maximum Matching
Project description
# pythaiwordcut - Thai Wordcut in Python
A simple Thai wordcut written in Python, based on Maximum Matching algorithm by [S. Manabu](http://www.aclweb.org/anthology/E14-4016)
. Uses Lexitron (by [NECTEC](http://www.sansarn.com/lexto/license-lexitron.php)) dictionary as default
> Please note: This project is under development and should not be use in production , all function and interface are subject to change. If you have issue or suggestion please feel free to ask, contribution is also very welcome :)
## Installation
```
pip install pythaiwordcut
```
or
```
git clone https://github.com/zenyai/pythaiwordcut.git
python setup.py install
```
## Usage
```
import pythaiwordcut as pwt
pt = pwt.wordcut(removeRepeat=True, stopDictionary="<full path to txt file>", removeSpaces=True, minLength=1, stopNumber=False, removeNonCharacter=False, caseSensitive=True, ngram=(1, 2), negation=False)
print "|".join(pt.segment(u'ทดสอบการตัดคำ'))
```
* removeRepeat: remove intention insertion spelling error such as (สบายยยยยย)
* stopDictionary: remove word that exist in this specify text file (one word one line)
* removeSpaces: remove blank space
* minLength: minimum length of each word
* stopNumber: remove number if exist
* removeNonCharacter: remove character that are not Thai or English character
* caseSensitive: if set to false, will remove stop word without regarding the case
* ngram: Add word ngram from (1, 2)
* negation: If set to true, then it will add NOT_ to every word after negation word and space
A simple Thai wordcut written in Python, based on Maximum Matching algorithm by [S. Manabu](http://www.aclweb.org/anthology/E14-4016)
. Uses Lexitron (by [NECTEC](http://www.sansarn.com/lexto/license-lexitron.php)) dictionary as default
> Please note: This project is under development and should not be use in production , all function and interface are subject to change. If you have issue or suggestion please feel free to ask, contribution is also very welcome :)
## Installation
```
pip install pythaiwordcut
```
or
```
git clone https://github.com/zenyai/pythaiwordcut.git
python setup.py install
```
## Usage
```
import pythaiwordcut as pwt
pt = pwt.wordcut(removeRepeat=True, stopDictionary="<full path to txt file>", removeSpaces=True, minLength=1, stopNumber=False, removeNonCharacter=False, caseSensitive=True, ngram=(1, 2), negation=False)
print "|".join(pt.segment(u'ทดสอบการตัดคำ'))
```
* removeRepeat: remove intention insertion spelling error such as (สบายยยยยย)
* stopDictionary: remove word that exist in this specify text file (one word one line)
* removeSpaces: remove blank space
* minLength: minimum length of each word
* stopNumber: remove number if exist
* removeNonCharacter: remove character that are not Thai or English character
* caseSensitive: if set to false, will remove stop word without regarding the case
* ngram: Add word ngram from (1, 2)
* negation: If set to true, then it will add NOT_ to every word after negation word and space
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
pythaiwordcut-0.1.15.tar.gz
(218.9 kB
view details)
Built Distribution
File details
Details for the file pythaiwordcut-0.1.15.tar.gz
.
File metadata
- Download URL: pythaiwordcut-0.1.15.tar.gz
- Upload date:
- Size: 218.9 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.12.1 pkginfo/1.5.0.1 requests/2.21.0 setuptools/40.6.3 requests-toolbelt/0.8.0 tqdm/4.29.1 CPython/3.6.7
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | f79bc83a01dfefe59e9a6608ac2b7bf583b9d3ff1540ea5ed5139f1aafdc8ded |
|
MD5 | aa5fa2a0d459e347982d37806ef1ff30 |
|
BLAKE2b-256 | 83af08112ea47fb89254f93224a41df66c5d819621095db8fb1a766bd1b87a6c |
File details
Details for the file pythaiwordcut-0.1.15-py2-none-any.whl
.
File metadata
- Download URL: pythaiwordcut-0.1.15-py2-none-any.whl
- Upload date:
- Size: 227.7 kB
- Tags: Python 2
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.12.1 pkginfo/1.5.0.1 requests/2.21.0 setuptools/40.6.3 requests-toolbelt/0.8.0 tqdm/4.29.1 CPython/3.6.7
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8c459ae0cb6607966738dd3a9d626bcb601b4d83bd9c3689a2072a6997a738ac |
|
MD5 | ebeb0370a3be519b6223376813b33659 |
|
BLAKE2b-256 | 33cd309b0db7ea182aed59eadea9ceefb3c399dd25284ea13fc2330724ad0026 |