PO2Dataset

po2dataset is a python tool to extract sentences from po files and create language datasets for machine translation.

This command line tool is intended to create dataset packages suitable for Argos Train.

How to install

From pip

pip install po2dataset

Manual installation

Create a virtual environment using virtualenv

git clone https://github.com/itzune/po2dataset.git
virtualenv po2dataset
cd po2dataset
source ./bin/activate

Quick start guide

Create Argos Train suitable dataset

po2dataset path/to/yourfile.po --name "MyProject" --source_code en --target_code eu --ref "Some reference information of the project"

Where:

• name: The name of the project
• source_code: Source language code (ISO 639)
• target_code: Target language code (ISO 639)
• ref: Some reference information of the project

Optional arguments:

• format: Extension name of the zip file (default argosdata)
• license: License to add into the package (default CC0). Options are: CC0, CC-BY, CC-BY-SA

Usage Examples

Basic Dataset Creation

To create a dataset from a .po file for an English-Basque translation project, run:

po2dataset path/to/yourfile.po --name "MyProject" --source_code en --target_code eu --ref "Translation dataset for project X"

Specifying Format and License

For different file format and license, use:

po2dataset path/to/yourfile.po --name "MyProject" --source_code en --target_code eu --format "zip" --license "CC-BY"

These commands create language dataset packages, with zip file format and CC-BY licensing options.

Support

Should you experience any issues do not hesistate to post an issue or contribute in this project pulling requests.