mT0

by BigScience Workshop

About the model:

Large open multilingual language model.

Model type:

Text

Model performance class:

Full

Link to the model:

https://huggingface.co/bigscience/mt0-xxl-p3

Base models:

mT5-XXL

End model:

mT0-XXL-P3

End model license:

Apache-2.0

About the organisation:

Research workshop on large multilingual models.

Link to the organisation:

https://huggingface.co/bigscience/

Model release date:

May 2023

Availability

Base Model Data

mC4, a subset of C4

https://www.tensorflow.org/datasets/catalog/c4#c4multilingual

End User Model Data

xP3

https://github.com/bigscience-workshop/xmtf?tab=readme-ov-file#data

Base Model Weights

https://huggingface.co/bigscience/mt0-xxl-mt

End User Model Weights

various variants available

https://huggingface.co/bigscience/mt0-large

Training Code

Training procedure described in a guide with some code, but no model-specific repository available

https://github.com/google-research/t5x/blob/main/docs/usage/finetune.md

Documentation

Code Documentation

Training procedure

https://arxiv.org/pdf/2010.11934

Hardware Architecture

Architecture: Same as mt5-xxl, also refer to the config.json file

Preprint

https://arxiv.org/abs/2211.01786

Paper

https://virtual2023.aclweb.org/paper_P283.html

Modelcard

https://huggingface.co/bigscience/mt0-xxl

Datasheet

https://huggingface.co/datasets/bigscience/xP3

Access

Licenses

model weights and finetuning dataset (xP3) and basemodel dataset (mC4) all under apache-2.0

Is this information not up to date?

Contribute here ->

Supported by the Centre for Language Studies and the Dutch Research Council. Website design & development © 2024 by BSTN. This version of the index generated 09 April 2026, website content last updated 11 March 2026.