Simple Thai Wordcut in Python using Maximum Matching
Project description
# pythaiwordcut - Thai Wordcut in Python
A simple Thai wordcut written in Python, based on Maximum Matching algorithm by [S. Manabu](http://www.aclweb.org/anthology/E14-4016)
. Uses Lexitron (by [NECTEC](http://www.sansarn.com/lexto/license-lexitron.php)) dictionary as default
> Please note: This project is under development and should not be use in production , all function and interface are subject to change. If you have issue or suggestion please feel free to ask, contribution is also very welcome :)
## Installation
```
pip install pythaiwordcut
```
or
```
git clone https://github.com/zenyai/pythaiwordcut.git
python setup.py install
```
## Usage
```
import pythaiwordcut as pwt
pt = pwt.wordcut(removeRepeat=True, stopDictionary="<full path to txt file>", removeSpaces=True, minLength=1, stopNumber=False, removeNonCharacter=False, caseSensitive=True, ngram=1)
print "|".join(pt.segment(u'ทดสอบการตัดคำ'))
```
* removeRepeat: remove intention insertion spelling error such as (สบายยยยยย)
* stopDictionary: remove word that exist in this specify text file (one word one line)
* removeSpaces: remove blank space
* minLength: minimum length of each word
* stopNumber: remove number if exist
* removeNonCharacter: remove character that are not Thai or English character
* caseSensitive: if set to false, will remove stop word without regarding the case
* ngram: Add word ngram from 1->x (no space between word, since Thai is the priority)
A simple Thai wordcut written in Python, based on Maximum Matching algorithm by [S. Manabu](http://www.aclweb.org/anthology/E14-4016)
. Uses Lexitron (by [NECTEC](http://www.sansarn.com/lexto/license-lexitron.php)) dictionary as default
> Please note: This project is under development and should not be use in production , all function and interface are subject to change. If you have issue or suggestion please feel free to ask, contribution is also very welcome :)
## Installation
```
pip install pythaiwordcut
```
or
```
git clone https://github.com/zenyai/pythaiwordcut.git
python setup.py install
```
## Usage
```
import pythaiwordcut as pwt
pt = pwt.wordcut(removeRepeat=True, stopDictionary="<full path to txt file>", removeSpaces=True, minLength=1, stopNumber=False, removeNonCharacter=False, caseSensitive=True, ngram=1)
print "|".join(pt.segment(u'ทดสอบการตัดคำ'))
```
* removeRepeat: remove intention insertion spelling error such as (สบายยยยยย)
* stopDictionary: remove word that exist in this specify text file (one word one line)
* removeSpaces: remove blank space
* minLength: minimum length of each word
* stopNumber: remove number if exist
* removeNonCharacter: remove character that are not Thai or English character
* caseSensitive: if set to false, will remove stop word without regarding the case
* ngram: Add word ngram from 1->x (no space between word, since Thai is the priority)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
pythaiwordcut-0.1.8.tar.gz
(218.6 kB
view details)
File details
Details for the file pythaiwordcut-0.1.8.tar.gz
.
File metadata
- Download URL: pythaiwordcut-0.1.8.tar.gz
- Upload date:
- Size: 218.6 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | b13858ce2ee308b1e23d921aeace75faacda150c705d70375dc23d7272790fb5 |
|
MD5 | 4a0c4b1ad0854644e77868a9e62e3c2c |
|
BLAKE2b-256 | d671bc97d4911a4127f22aa35711f0e680983c1852501de379658d265cce659f |