Skip to main content

A package for developing Open Agents

Project description

Mask-Predict

Download model

Description Dataset Model
MASK-PREDICT [WMT14 English-German] download (.tar.bz2)
MASK-PREDICT [WMT14 German-English] download (.tar.bz2)
MASK-PREDICT [WMT16 English-Romanian] download (.tar.bz2)
MASK-PREDICT [WMT16 Romanian-English] download (.tar.bz2)
MASK-PREDICT [WMT17 English-Chinese] download (.tar.bz2)
MASK-PREDICT [WMT17 Chinese-English] download (.tar.bz2)

Preprocess

text=PATH_YOUR_DATA

output_dir=PATH_YOUR_OUTPUT

src=source_language

tgt=target_language

model_path=PATH_TO_MASKPREDICT_MODEL_DIR

python preprocess.py --source-lang ${src} --target-lang ${tgt} --trainpref $text/train --validpref $text/valid --testpref $text/test --destdir ${output_dir}/data-bin --workers 60 --srcdict ${model_path}/maskPredict_${src}${tgt}/dict.${src}.txt --tgtdict ${model_path}/maskPredict${src}_${tgt}/dict.${tgt}.txt

Train

model_dir=PLACE_TO_SAVE_YOUR_MODEL

python train.py ${output_dir}/data-bin --arch bert_transformer_seq2seq --share-all-embeddings --criterion label_smoothed_length_cross_entropy --label-smoothing 0.1 --lr 5e-4 --warmup-init-lr 1e-7 --min-lr 1e-9 --lr-scheduler inverse_sqrt --warmup-updates 10000 --optimizer adam --adam-betas '(0.9, 0.999)' --adam-eps 1e-6 --task translation_self --max-tokens 8192 --weight-decay 0.01 --dropout 0.3 --encoder-layers 6 --encoder-embed-dim 512 --decoder-layers 6 --decoder-embed-dim 512 --fp16 --max-source-positions 10000 --max-target-positions 10000 --max-update 300000 --seed 0 --save-dir ${model_dir}

Evaluation

python generate_cmlm.py ${output_dir}/data-bin --path ${model_dir}/checkpoint_best_average.pt --task translation_self --remove-bpe --max-sentences 20 --decoding-iterations 10 --decoding-strategy mask_predict

License

MASK-PREDICT is CC-BY-NC 4.0. The license applies to the pre-trained models as well.

Citation

Please cite as:

@inproceedings{ghazvininejad2019MaskPredict,
  title = {Mask-Predict: Parallel Decoding of Conditional Masked Language Models},
  author = {Marjan Ghazvininejad, Omer Levy, Yinhan Liu, Luke Zettlemoyer},
  booktitle = {Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing},
  year = {2019},
}

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

openagents-0.1.0.tar.gz (144.4 kB view details)

Uploaded Source

Built Distribution

openagents-0.1.0-py3-none-any.whl (211.9 kB view details)

Uploaded Python 3

File details

Details for the file openagents-0.1.0.tar.gz.

File metadata

  • Download URL: openagents-0.1.0.tar.gz
  • Upload date:
  • Size: 144.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.9.16

File hashes

Hashes for openagents-0.1.0.tar.gz
Algorithm Hash digest
SHA256 32f06e609d0c7497166e99775cc71a85a943e9e5ad9a109b8445b4e9741fcc10
MD5 3f7284c05fb8400c9127d4f9a3d829d9
BLAKE2b-256 6c4d7243424251b4c82af7f52417e881933b4f3d9d126d3789123dfa79b14dd2

See more details on using hashes here.

Provenance

File details

Details for the file openagents-0.1.0-py3-none-any.whl.

File metadata

  • Download URL: openagents-0.1.0-py3-none-any.whl
  • Upload date:
  • Size: 211.9 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.9.16

File hashes

Hashes for openagents-0.1.0-py3-none-any.whl
Algorithm Hash digest
SHA256 0bca15e9bb55559c36bf75c1c0ca8bf11c7a5f3b34a43ea725e75ab3c719bd92
MD5 31849771abf2354234cf4e8cd89166b5
BLAKE2b-256 b4c28ac8fbbe9e3e439a6e09e42d1b4515b4c3c31dc95aef1d0b45a8be1c9a41

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page