Simple Thai Wordcut in Python using Maximum Matching
Project description
# pythaiwordcut - Thai Wordcut in Python
A simple Thai wordcut written in Python, based on Maximum Matching algorithm by [S. Manabu](http://www.aclweb.org/anthology/E14-4016)
. Uses Lexitron (by [NECTEC](http://www.sansarn.com/lexto/license-lexitron.php)) dictionary as default
> Please note: This project is under development and should not be use in production , all function and interface are subject to change. If you have issue or suggestion please feel free to ask, contribution is also very welcome :)
## Installation
```
pip install pythaiwordcut
```
or
```
git clone https://github.com/zenyai/pythaiwordcut.git
python setup.py install
```
## Usage
```
import pythaiwordcut as pwt
pt = pwt.wordcut(removeRepeat=True, stopDictionary="<full path to txt file>", removeSpaces=True, minLength=1, stopNumber=False, removeNonCharacter=False, caseSensitive=True, ngram=(1, 2))
print "|".join(pt.segment(u'ทดสอบการตัดคำ'))
```
* removeRepeat: remove intention insertion spelling error such as (สบายยยยยย)
* stopDictionary: remove word that exist in this specify text file (one word one line)
* removeSpaces: remove blank space
* minLength: minimum length of each word
* stopNumber: remove number if exist
* removeNonCharacter: remove character that are not Thai or English character
* caseSensitive: if set to false, will remove stop word without regarding the case
* ngram: Add word ngram from (1, 2)
A simple Thai wordcut written in Python, based on Maximum Matching algorithm by [S. Manabu](http://www.aclweb.org/anthology/E14-4016)
. Uses Lexitron (by [NECTEC](http://www.sansarn.com/lexto/license-lexitron.php)) dictionary as default
> Please note: This project is under development and should not be use in production , all function and interface are subject to change. If you have issue or suggestion please feel free to ask, contribution is also very welcome :)
## Installation
```
pip install pythaiwordcut
```
or
```
git clone https://github.com/zenyai/pythaiwordcut.git
python setup.py install
```
## Usage
```
import pythaiwordcut as pwt
pt = pwt.wordcut(removeRepeat=True, stopDictionary="<full path to txt file>", removeSpaces=True, minLength=1, stopNumber=False, removeNonCharacter=False, caseSensitive=True, ngram=(1, 2))
print "|".join(pt.segment(u'ทดสอบการตัดคำ'))
```
* removeRepeat: remove intention insertion spelling error such as (สบายยยยยย)
* stopDictionary: remove word that exist in this specify text file (one word one line)
* removeSpaces: remove blank space
* minLength: minimum length of each word
* stopNumber: remove number if exist
* removeNonCharacter: remove character that are not Thai or English character
* caseSensitive: if set to false, will remove stop word without regarding the case
* ngram: Add word ngram from (1, 2)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
pythaiwordcut-0.1.9.tar.gz
(218.6 kB
view details)
File details
Details for the file pythaiwordcut-0.1.9.tar.gz
.
File metadata
- Download URL: pythaiwordcut-0.1.9.tar.gz
- Upload date:
- Size: 218.6 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 57b121f80c4f48250872ee0472ecd3ca6bc172de58ea7863adb8653fdbb0a31d |
|
MD5 | e6a9d07c26e8d74f065bf6818c6ca3c6 |
|
BLAKE2b-256 | e8a0f97cc7c533d6a053b65fc8555fa06940b8642744fa1e47fce26a5ca565e4 |