Simple Thai Wordcut in Python using Maximum Matching
Project description
# pythaiwordcut - Thai Wordcut in Python
A simple Thai wordcut written in Python, based on Maximum Matching algorithm by [S. Manabu](http://www.aclweb.org/anthology/E14-4016)
. Uses Lexitron (by [NECTEC](http://www.sansarn.com/lexto/license-lexitron.php)) dictionary as default
> Please note: This project is under development and should not be use in production , all function and interface are subject to change. If you have issue or suggestion please feel free to ask, contribution is also very welcome :)
## Installation
```
pip install pythaiwordcut
```
or
```
git clone https://github.com/zenyai/pythaiwordcut.git
python setup.py install
```
## Usage
```
import pythaiwordcut as pwt
pt = pwt.wordcut(removeRepeat=True, stopDictionary="<full path to txt file>", removeSpaces=True, minLength=1, stopNumber=False, removeNonCharacter=False, caseSensitive=True, ngram=(1, 2), negation=False)
print "|".join(pt.segment(u'ทดสอบการตัดคำ'))
```
* removeRepeat: remove intention insertion spelling error such as (สบายยยยยย)
* stopDictionary: remove word that exist in this specify text file (one word one line)
* removeSpaces: remove blank space
* minLength: minimum length of each word
* stopNumber: remove number if exist
* removeNonCharacter: remove character that are not Thai or English character
* caseSensitive: if set to false, will remove stop word without regarding the case
* ngram: Add word ngram from (1, 2)
* negation: If set to true, then it will add NOT_ to every word after negation word and space
A simple Thai wordcut written in Python, based on Maximum Matching algorithm by [S. Manabu](http://www.aclweb.org/anthology/E14-4016)
. Uses Lexitron (by [NECTEC](http://www.sansarn.com/lexto/license-lexitron.php)) dictionary as default
> Please note: This project is under development and should not be use in production , all function and interface are subject to change. If you have issue or suggestion please feel free to ask, contribution is also very welcome :)
## Installation
```
pip install pythaiwordcut
```
or
```
git clone https://github.com/zenyai/pythaiwordcut.git
python setup.py install
```
## Usage
```
import pythaiwordcut as pwt
pt = pwt.wordcut(removeRepeat=True, stopDictionary="<full path to txt file>", removeSpaces=True, minLength=1, stopNumber=False, removeNonCharacter=False, caseSensitive=True, ngram=(1, 2), negation=False)
print "|".join(pt.segment(u'ทดสอบการตัดคำ'))
```
* removeRepeat: remove intention insertion spelling error such as (สบายยยยยย)
* stopDictionary: remove word that exist in this specify text file (one word one line)
* removeSpaces: remove blank space
* minLength: minimum length of each word
* stopNumber: remove number if exist
* removeNonCharacter: remove character that are not Thai or English character
* caseSensitive: if set to false, will remove stop word without regarding the case
* ngram: Add word ngram from (1, 2)
* negation: If set to true, then it will add NOT_ to every word after negation word and space
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
pythaiwordcut-0.1.14.tar.gz
(219.0 kB
view details)
File details
Details for the file pythaiwordcut-0.1.14.tar.gz
.
File metadata
- Download URL: pythaiwordcut-0.1.14.tar.gz
- Upload date:
- Size: 219.0 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | b16cb35273b2a68ad3a65d227523191620d898c76a68e6491b5fdc8b409b577a |
|
MD5 | 31cf1efba08742c68a7f94c86628c88d |
|
BLAKE2b-256 | 8dff0adf6b11e449e160f84b19d65e3874954454f817df2674fd4f60c855dfdd |