SpanMarker for Named Entity Recognition

SpanMarker is a framework for training powerful Named Entity Recognition models using familiar encoders such as BERT, RoBERTa and DeBERTa. Tightly implemented on top of the 🤗 Transformers library, SpanMarker can take advantage of its valuable functionality.

Based on the PL-Marker paper, SpanMarker breaks the mold through its accessibility and ease of use. Crucially, SpanMarker works out of the box with many common encoders such as bert-base-cased and roberta-large, and automatically works with datasets using the IOB, IOB2, BIOES, BILOU or no label annotation scheme.

Documentation

Feel free to have a look at the documentation.

Installation

You may install the span_marker Python module via pip like so:

pip install span_marker

Quick Start

Please have a look at our Getting Started notebook for details on how SpanMarker is commonly used. It explains the following snippet in more detail.

Colab	Kaggle	Gradient	Studio Lab

from datasets import load_dataset
from span_marker import SpanMarkerModel, Trainer
from transformers import TrainingArguments

def main():
    dataset = load_dataset("DFKI-SLT/few-nerd", "supervised")
    labels = dataset["train"].features["ner_tags"].feature.names

    model_name = "bert-base-cased"
    model = SpanMarkerModel.from_pretrained(model_name, labels=labels)

    args = TrainingArguments(
        output_dir="my_span_marker_model",
        learning_rate=5e-5,
        gradient_accumulation_steps=2,
        per_device_train_batch_size=4,
        per_device_eval_batch_size=4,
        num_train_epochs=1,
        save_strategy="steps",
        eval_steps=200,
        logging_steps=50,
        fp16=True,
        warmup_ratio=0.1,
        dataloader_num_workers=2,
    )

    trainer = Trainer(
        model=model,
        args=args,
        train_dataset=dataset["train"].select(range(8000)),
        eval_dataset=dataset["validation"].select(range(2000)),
    )

    trainer.train()
    trainer.save_model("my_span_marker_model/checkpoint-final")

    metrics = trainer.evaluate()
    print(metrics)

if __name__ == "__main__":
    main()

Pretrained Models

• tomaarsen/span-marker-bert-base-fewnerd-fine-super is a model that I have trained in 2 hours on the finegrained, supervised Few-NERD dataset. It reached a 0.7053 Test F1, competitive in the all-time Few-NERD leaderboard using bert-base. My training script resembles the one that you can see above.

  • Try the model out online using this 🤗 Space.

• tomaarsen/span-marker-roberta-large-fewnerd-fine-super was trained in 6 hours on the finegrained, supervised Few-NERD dataset using roberta-large. It reached a 0.7103 Test F1, very competitive in the all-time Few-NERD leaderboard.

Changelog

See CHANGELOG.md for news on all SpanMarker versions.

License

See LICENSE for the current license.