UltraLM

by OpenBMB

About the model:

Model trained using high-quality feedback.

Model type:

Text

Model performance class:

Full

Link to the model:

https://huggingface.co/openbmb/UltraLM-13b-v2.0

Base models:

Llama-13B

End model:

UltraLM-13B-v2.0

End model license:

Meta Community License

About the organisation:

OpenBMB (Open Lab for Big Model Base), founded by TsinghuaNLP & ModelBest Inc (面壁智能), aims to build foundation models and systems towards AGI.

Link to the organisation:

https://www.openbmb.cn/

Model release date:

September 2023

Availability

Base Model Data

Based on LLaMA whose pretraining data has nowhere been disclosed or documented.

https://github.com/facebookresearch/llama/blob/main/MODEL_CARD.md#training-dataset

End User Model Data

UltraFeedback dataset made available along with model release

https://huggingface.co/datasets/openbmb/UltraFeedback

Base Model Weights

Based on LLaMA weights, which are not openly available though a leaked versions is in wide circulation.

End User Model Weights

Online materials appear to be in flux and several HuggingFace links generate 404 errors, hence partial

https://huggingface.co/openbmb/UltraLM-13b-v2.0

Training Code

Documentation

Code Documentation

Code only covers minimal examples; no documentation available.

Hardware Architecture

Architecture sketched in online materials.

https://arxiv.org/pdf/2310.01377

Preprint

Preprint describes creation of UltraFeedback dataset but also offers some detail on training and architecture of UltraRM models

https://arxiv.org/abs/2310.01377

Paper

Paper published in ICML.

https://dl.acm.org/doi/10.5555/3692070.3692454

Modelcard

Model card on HuggingFace available for 13b (Llama1) model, but not for newer releases, hence partial

https://huggingface.co/openbmb/UltraLM-13b

Datasheet

Datasheet available for RLHF portion, but not for Llama2-based pretraining data, hence partial

https://huggingface.co/datasets/openbmb/UltraFeedback

Access

Licenses

Usage requires signing Meta's bespoke 'community license', not an OSI recognised open license

https://huggingface.co/openbmb/UltraLM-13b

Is this information not up to date?

Contribute here ->

Supported by the Centre for Language Studies and the Dutch Research Council. Website design & development © 2024 by BSTN. This version of the index generated 09 April 2026, website content last updated 11 March 2026.