Skip to main content

Industry-strength Natural Language Processing extensions for Keras.

Project description

KerasNLP: Multi-framework NLP Models

Python contributions welcome

KerasNLP is a natural language processing library that works natively with TensorFlow, JAX, or PyTorch. KerasNLP provides a repository of pre-trained models and a collection of lower-level building blocks for language modeling. Built on Keras 3, models can be trained and serialized in any framework and re-used in another without costly migrations.

This library is an extension of the core Keras API; all high-level modules are Layers and Models that receive that same level of polish as core Keras. If you are familiar with Keras, congratulations! You already understand most of KerasNLP.

All models support JAX, TensorFlow, and PyTorch from a single model definition and can be fine-tuned on GPUs and TPUs out of the box. Models can be trained on individual accelerators with built-in PEFT techniques, or fine-tuned at scale with model and data parallel training. See our Getting Started guide to start learning our API. Browse our models on Kaggle. We welcome contributions.

Quick Links

For everyone

For contributors

Quickstart

Fine-tune BERT on IMDb movie reviews:

import os
os.environ["KERAS_BACKEND"] = "jax"  # Or "tensorflow" or "torch"!

import keras_nlp
import tensorflow_datasets as tfds

imdb_train, imdb_test = tfds.load(
    "imdb_reviews",
    split=["train", "test"],
    as_supervised=True,
    batch_size=16,
)
# Load a BERT model.
classifier = keras_nlp.models.Classifier.from_preset(
    "bert_base_en", 
    num_classes=2,
    activation="softmax",
)
# Fine-tune on IMDb movie reviews.
classifier.fit(imdb_train, validation_data=imdb_test)
# Predict two new examples.
classifier.predict(["What an amazing movie!", "A total waste of my time."])

Try it out in a colab. For more in depth guides and examples, visit keras.io/keras_nlp.

Installation

To install the latest KerasNLP release with Keras 3, simply run:

pip install --upgrade keras-nlp

To install the latest nightly changes for both KerasNLP and Keras, you can use our nightly package.

pip install --upgrade keras-nlp-nightly

Note that currently, installing KerasNLP will always pull in TensorFlow for use of the tf.data API for preprocessing. Even when pre-processing with tf.data, training can still happen on any backend.

Read Getting started with Keras for more information on installing Keras 3 and compatibility with different frameworks.

[!IMPORTANT] We recommend using KerasNLP with TensorFlow 2.16 or later, as TF 2.16 packages Keras 3 by default.

Configuring your backend

If you have Keras 3 installed in your environment (see installation above), you can use KerasNLP with any of JAX, TensorFlow and PyTorch. To do so, set the KERAS_BACKEND environment variable. For example:

export KERAS_BACKEND=jax

Or in Colab, with:

import os
os.environ["KERAS_BACKEND"] = "jax"

import keras_nlp

[!IMPORTANT] Make sure to set the KERAS_BACKEND before import any Keras libraries, it will be used to set up Keras when it is first imported.

Compatibility

We follow Semantic Versioning, and plan to provide backwards compatibility guarantees both for code and saved models built with our components. While we continue with pre-release 0.y.z development, we may break compatibility at any time and APIs should not be consider stable.

Disclaimer

KerasNLP provides access to pre-trained models via the keras_nlp.models API. These pre-trained models are provided on an "as is" basis, without warranties or conditions of any kind. The following underlying models are provided by third parties, and subject to separate licenses: BART, BLOOM, DeBERTa, DistilBERT, GPT-2, Llama, Mistral, OPT, RoBERTa, Whisper, and XLM-RoBERTa.

Citing KerasNLP

If KerasNLP helps your research, we appreciate your citations. Here is the BibTeX entry:

@misc{kerasnlp2022,
  title={KerasNLP},
  author={Watson, Matthew, and Qian, Chen, and Bischof, Jonathan and Chollet, 
  Fran\c{c}ois and others},
  year={2022},
  howpublished={\url{https://github.com/keras-team/keras-nlp}},
}

Acknowledgements

Thank you to all of our wonderful contributors!

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

keras_nlp_nightly-0.15.0.dev2024073103.tar.gz (333.8 kB view details)

Uploaded Source

Built Distribution

File details

Details for the file keras_nlp_nightly-0.15.0.dev2024073103.tar.gz.

File metadata

File hashes

Hashes for keras_nlp_nightly-0.15.0.dev2024073103.tar.gz
Algorithm Hash digest
SHA256 bb472889a7b9be972bed722e9dc1afaa709d4190f0ae1dbe0cdd8c51b736a276
MD5 173f7ea25455a99cb0c3c0465440a86f
BLAKE2b-256 beae9ef972d8a10b3c90d60e7d7f6051c84ffa7ffe413e67b094a876c0cfe8b3

See more details on using hashes here.

Provenance

File details

Details for the file keras_nlp_nightly-0.15.0.dev2024073103-py3-none-any.whl.

File metadata

File hashes

Hashes for keras_nlp_nightly-0.15.0.dev2024073103-py3-none-any.whl
Algorithm Hash digest
SHA256 60c56c0eeced2d8c57afb6a5ddd595b63d4a2711feea084d603bb3967d57483f
MD5 8871c551c55f7d62a4449f71b52bd692
BLAKE2b-256 857791b601f8d3406fcd9b34be6e365389f403abafe372e62a7a5eb8c310d671

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page