Skip to main content

Facebook AI Research Sequence-to-Sequence Toolkit

Project description



MIT License Latest Release Build Status Documentation Status


Fairseq(-py) is a sequence modeling toolkit that allows researchers and developers to train custom models for translation, summarization, language modeling and other text generation tasks. We provide reference implementations of various sequence modeling papers:

List of implemented papers

What's New:

Previous updates

Features:

  • multi-GPU training on one machine or across multiple machines (data and model parallel)
  • fast generation on both CPU and GPU with multiple search algorithms implemented:
  • large mini-batch training even on a single GPU via delayed updates
  • mixed precision training (trains faster with less GPU memory on NVIDIA tensor cores)
  • extensible: easily register new models, criterions, tasks, optimizers and learning rate schedulers

We also provide pre-trained models for translation and language modeling with a convenient torch.hub interface:

en2de = torch.hub.load('pytorch/fairseq', 'transformer.wmt19.en-de.single_model')
en2de.translate('Hello world', beam=5)
# 'Hallo Welt'

See the PyTorch Hub tutorials for translation and RoBERTa for more examples.

Requirements and Installation

  • PyTorch version >= 1.5.0
  • Python version >= 3.6
  • For training new models, you'll also need an NVIDIA GPU and NCCL
  • To install fairseq and develop locally:
git clone https://github.com/pytorch/fairseq
cd fairseq
pip install --editable ./

# on MacOS:
# CFLAGS="-stdlib=libc++" pip install --editable ./
  • For faster training install NVIDIA's apex library:
git clone https://github.com/NVIDIA/apex
cd apex
pip install -v --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" \
  --global-option="--deprecated_fused_adam" --global-option="--xentropy" \
  --global-option="--fast_multihead_attn" ./
  • For large datasets install PyArrow: pip install pyarrow
  • If you use Docker make sure to increase the shared memory size either with --ipc=host or --shm-size as command line options to nvidia-docker run.

Getting Started

The full documentation contains instructions for getting started, training new models and extending fairseq with new model types and tasks.

Pre-trained models and examples

We provide pre-trained models and pre-processed, binarized test sets for several tasks listed below, as well as example training and evaluation commands.

We also have more detailed READMEs to reproduce results from specific papers:

Join the fairseq community

License

fairseq(-py) is MIT-licensed. The license applies to the pre-trained models as well.

Citation

Please cite as:

@inproceedings{ott2019fairseq,
  title = {fairseq: A Fast, Extensible Toolkit for Sequence Modeling},
  author = {Myle Ott and Sergey Edunov and Alexei Baevski and Angela Fan and Sam Gross and Nathan Ng and David Grangier and Michael Auli},
  booktitle = {Proceedings of NAACL-HLT 2019: Demonstrations},
  year = {2019},
}

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

fairseq-0.10.2.tar.gz (938.1 kB view details)

Uploaded Source

Built Distributions

fairseq-0.10.2-cp38-cp38-manylinux1_x86_64.whl (1.7 MB view details)

Uploaded CPython 3.8

fairseq-0.10.2-cp38-cp38-macosx_10_9_x86_64.whl (1.1 MB view details)

Uploaded CPython 3.8 macOS 10.9+ x86-64

fairseq-0.10.2-cp37-cp37m-manylinux1_x86_64.whl (1.7 MB view details)

Uploaded CPython 3.7m

fairseq-0.10.2-cp37-cp37m-macosx_10_9_x86_64.whl (1.1 MB view details)

Uploaded CPython 3.7m macOS 10.9+ x86-64

fairseq-0.10.2-cp36-cp36m-manylinux1_x86_64.whl (1.7 MB view details)

Uploaded CPython 3.6m

fairseq-0.10.2-cp36-cp36m-macosx_10_9_x86_64.whl (1.1 MB view details)

Uploaded CPython 3.6m macOS 10.9+ x86-64

File details

Details for the file fairseq-0.10.2.tar.gz.

File metadata

  • Download URL: fairseq-0.10.2.tar.gz
  • Upload date:
  • Size: 938.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.3.0 pkginfo/1.6.1 requests/2.25.1 setuptools/51.1.1 requests-toolbelt/0.9.1 tqdm/4.55.0 CPython/3.8.5

File hashes

Hashes for fairseq-0.10.2.tar.gz
Algorithm Hash digest
SHA256 45b90d8ccc3f5a4623a523fe2d0465f54413d03fd1ec9a9d7af0461148ca1a68
MD5 705821d37b4907db9409f7e1315c4e78
BLAKE2b-256 02591f466d82d64482e2809a167e0067979c23057b3752e668537d5ff24453c9

See more details on using hashes here.

Provenance

File details

Details for the file fairseq-0.10.2-cp38-cp38-manylinux1_x86_64.whl.

File metadata

  • Download URL: fairseq-0.10.2-cp38-cp38-manylinux1_x86_64.whl
  • Upload date:
  • Size: 1.7 MB
  • Tags: CPython 3.8
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.3.0 pkginfo/1.6.1 requests/2.25.1 setuptools/51.1.1 requests-toolbelt/0.9.1 tqdm/4.55.0 CPython/3.8.5

File hashes

Hashes for fairseq-0.10.2-cp38-cp38-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 e807a39a44e93d830e8372a05aa27315b0c18499748f3862f262b7e3fcda646d
MD5 9e44cb2a8d2b3ff7f4572e02ca588c0a
BLAKE2b-256 88151b09022f20f971eae992497bdde69dc11b3e8d586c95e93e43a6842979c5

See more details on using hashes here.

Provenance

File details

Details for the file fairseq-0.10.2-cp38-cp38-macosx_10_9_x86_64.whl.

File metadata

  • Download URL: fairseq-0.10.2-cp38-cp38-macosx_10_9_x86_64.whl
  • Upload date:
  • Size: 1.1 MB
  • Tags: CPython 3.8, macOS 10.9+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.3.0 pkginfo/1.6.1 requests/2.25.1 setuptools/51.1.1 requests-toolbelt/0.9.1 tqdm/4.55.0 CPython/3.8.5

File hashes

Hashes for fairseq-0.10.2-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 67fe259dbe0c4c17f65bc7c077b9cf1437eae3819022e84fc2e6e3bfe04746c2
MD5 e6ca19858c8894bd87ba773f01d1d2f7
BLAKE2b-256 1470ac2c867b1eecaf5b9b6096165646a41954f3d6f9c9f8ffb906e344a905fa

See more details on using hashes here.

Provenance

File details

Details for the file fairseq-0.10.2-cp37-cp37m-manylinux1_x86_64.whl.

File metadata

  • Download URL: fairseq-0.10.2-cp37-cp37m-manylinux1_x86_64.whl
  • Upload date:
  • Size: 1.7 MB
  • Tags: CPython 3.7m
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.3.0 pkginfo/1.6.1 requests/2.25.1 setuptools/51.1.1 requests-toolbelt/0.9.1 tqdm/4.55.0 CPython/3.8.5

File hashes

Hashes for fairseq-0.10.2-cp37-cp37m-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 b9c40a0c17003ef7b1bd971813f48e913bf8ee8f9517565b2e63c1ba3d3a0d23
MD5 2ff2255abe046b766f62fd4f25443b4f
BLAKE2b-256 15ab92c6efb05ffdfe16fbdc9e463229d9af8c3b74dc943ed4b4857a87b223c2

See more details on using hashes here.

Provenance

File details

Details for the file fairseq-0.10.2-cp37-cp37m-macosx_10_9_x86_64.whl.

File metadata

  • Download URL: fairseq-0.10.2-cp37-cp37m-macosx_10_9_x86_64.whl
  • Upload date:
  • Size: 1.1 MB
  • Tags: CPython 3.7m, macOS 10.9+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.3.0 pkginfo/1.6.1 requests/2.25.1 setuptools/51.1.1 requests-toolbelt/0.9.1 tqdm/4.55.0 CPython/3.8.5

File hashes

Hashes for fairseq-0.10.2-cp37-cp37m-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 8ecd29be6d769187d06b42b4d929fe0ad799a79614343465be1b461acfa0acf3
MD5 60c4f5888d39fe717c27935310156796
BLAKE2b-256 69659b7315be536fbb8e5233c062ba61746452f2628a86863ef36684482ec99f

See more details on using hashes here.

Provenance

File details

Details for the file fairseq-0.10.2-cp36-cp36m-manylinux1_x86_64.whl.

File metadata

  • Download URL: fairseq-0.10.2-cp36-cp36m-manylinux1_x86_64.whl
  • Upload date:
  • Size: 1.7 MB
  • Tags: CPython 3.6m
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.3.0 pkginfo/1.6.1 requests/2.25.1 setuptools/51.1.1 requests-toolbelt/0.9.1 tqdm/4.55.0 CPython/3.8.5

File hashes

Hashes for fairseq-0.10.2-cp36-cp36m-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 8d3fb301f182b0168cac93327638fa4c044658fac16eb39dea18a1b9b471fc40
MD5 329f48a04827957176aba89070a7390f
BLAKE2b-256 617b2c90e007d737f4a2b7cd5066ac3a3d88acb2ce765972a61c308914c95568

See more details on using hashes here.

Provenance

File details

Details for the file fairseq-0.10.2-cp36-cp36m-macosx_10_9_x86_64.whl.

File metadata

  • Download URL: fairseq-0.10.2-cp36-cp36m-macosx_10_9_x86_64.whl
  • Upload date:
  • Size: 1.1 MB
  • Tags: CPython 3.6m, macOS 10.9+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.3.0 pkginfo/1.6.1 requests/2.25.1 setuptools/51.1.1 requests-toolbelt/0.9.1 tqdm/4.55.0 CPython/3.8.5

File hashes

Hashes for fairseq-0.10.2-cp36-cp36m-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 638ab9b4973fb8443cab6a62a0ccb4b02a814eb6aafbcf490e6df47391236a8e
MD5 1453b57123df56531569b262cdd3743e
BLAKE2b-256 1536538c62c2815af17ae23e4346ce9fc491977e8913417d3289da7519863b41

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page