A package for developing Open Agents
Project description
Mask-Predict
Download model
Description | Dataset | Model |
---|---|---|
MASK-PREDICT | [WMT14 English-German] | download (.tar.bz2) |
MASK-PREDICT | [WMT14 German-English] | download (.tar.bz2) |
MASK-PREDICT | [WMT16 English-Romanian] | download (.tar.bz2) |
MASK-PREDICT | [WMT16 Romanian-English] | download (.tar.bz2) |
MASK-PREDICT | [WMT17 English-Chinese] | download (.tar.bz2) |
MASK-PREDICT | [WMT17 Chinese-English] | download (.tar.bz2) |
Preprocess
text=PATH_YOUR_DATA
output_dir=PATH_YOUR_OUTPUT
src=source_language
tgt=target_language
model_path=PATH_TO_MASKPREDICT_MODEL_DIR
python preprocess.py --source-lang ${src} --target-lang ${tgt} --trainpref $text/train --validpref $text/valid --testpref $text/test --destdir ${output_dir}/data-bin --workers 60 --srcdict ${model_path}/maskPredict_${src}${tgt}/dict.${src}.txt --tgtdict ${model_path}/maskPredict${src}_${tgt}/dict.${tgt}.txt
Train
model_dir=PLACE_TO_SAVE_YOUR_MODEL
python train.py ${output_dir}/data-bin --arch bert_transformer_seq2seq --share-all-embeddings --criterion label_smoothed_length_cross_entropy --label-smoothing 0.1 --lr 5e-4 --warmup-init-lr 1e-7 --min-lr 1e-9 --lr-scheduler inverse_sqrt --warmup-updates 10000 --optimizer adam --adam-betas '(0.9, 0.999)' --adam-eps 1e-6 --task translation_self --max-tokens 8192 --weight-decay 0.01 --dropout 0.3 --encoder-layers 6 --encoder-embed-dim 512 --decoder-layers 6 --decoder-embed-dim 512 --fp16 --max-source-positions 10000 --max-target-positions 10000 --max-update 300000 --seed 0 --save-dir ${model_dir}
Evaluation
python generate_cmlm.py ${output_dir}/data-bin --path ${model_dir}/checkpoint_best_average.pt --task translation_self --remove-bpe --max-sentences 20 --decoding-iterations 10 --decoding-strategy mask_predict
License
MASK-PREDICT is CC-BY-NC 4.0. The license applies to the pre-trained models as well.
Citation
Please cite as:
@inproceedings{ghazvininejad2019MaskPredict,
title = {Mask-Predict: Parallel Decoding of Conditional Masked Language Models},
author = {Marjan Ghazvininejad, Omer Levy, Yinhan Liu, Luke Zettlemoyer},
booktitle = {Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing},
year = {2019},
}
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file openagents-0.1.0.tar.gz
.
File metadata
- Download URL: openagents-0.1.0.tar.gz
- Upload date:
- Size: 144.4 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.9.16
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 32f06e609d0c7497166e99775cc71a85a943e9e5ad9a109b8445b4e9741fcc10 |
|
MD5 | 3f7284c05fb8400c9127d4f9a3d829d9 |
|
BLAKE2b-256 | 6c4d7243424251b4c82af7f52417e881933b4f3d9d126d3789123dfa79b14dd2 |
Provenance
File details
Details for the file openagents-0.1.0-py3-none-any.whl
.
File metadata
- Download URL: openagents-0.1.0-py3-none-any.whl
- Upload date:
- Size: 211.9 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.9.16
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0bca15e9bb55559c36bf75c1c0ca8bf11c7a5f3b34a43ea725e75ab3c719bd92 |
|
MD5 | 31849771abf2354234cf4e8cd89166b5 |
|
BLAKE2b-256 | b4c28ac8fbbe9e3e439a6e09e42d1b4515b4c3c31dc95aef1d0b45a8be1c9a41 |