Python tool to extract sentences from po files and create language datasets for NLP machine learning
Project description
PO2Dataset
po2dataset is a python tool to extract sentences from po files and create language datasets for machine translation.
This command line tool is intended to create dataset packages suitable for Argos Train.
How to install
From pip
pip install po2dataset
Manual installation
Create a virtual environment using virtualenv
git clone https://github.com/urtzai/po2dataset.git
virtualenv po2dataset
cd po2dataset
source ./bin/activate
Quick start guide
Create Argos Train suitable dataset
po2dataset <path_to_po_file> --name <project_name> --source_code <source_lang_code> --target_code <target_lang_code> --ref "Some reference information of the project"
Where:
name
: The name of the projectsource_code
: Source language code (ISO 639)target_code
: Target language code (ISO 639)ref
: Some reference information of the project
Optional arguments:
format
: Extension name of the zip file (default argosdata)license
: License to add into the package (default CC0)
Support
Should you experience any issues do not hesistate to post an issue or contribute in this project pulling requests.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
po2dataset-0.2.0.tar.gz
(6.4 kB
view details)
Built Distribution
File details
Details for the file po2dataset-0.2.0.tar.gz
.
File metadata
- Download URL: po2dataset-0.2.0.tar.gz
- Upload date:
- Size: 6.4 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via: twine/5.1.1 CPython/3.12.7
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5386c280b7933108d7494b360e1e99660dcbfb68ef20962760a8a6bbc29a1be6 |
|
MD5 | f69a76f3f63389d6435e1fd04d5e11d3 |
|
BLAKE2b-256 | a9061ed1c3e50d984c3f7f437e77d449fa8fa0a30b7b1925c31e0b919e994f67 |
File details
Details for the file po2dataset-0.2.0-py3-none-any.whl
.
File metadata
- Download URL: po2dataset-0.2.0-py3-none-any.whl
- Upload date:
- Size: 5.9 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via: twine/5.1.1 CPython/3.12.7
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | d60012a5275fd64c829777c09690c547b3055189c84b34307d49729ff1b32d45 |
|
MD5 | 85ffe86e280216d0ab1b98e7eddd4f04 |
|
BLAKE2b-256 | a19bd6210db1cc3be7e1a00dda8f95d34ea49181bea462c4965daec3649d1c36 |