LongAlign

by Zhipu AI

About the model:

Recipe for long-context LLM alignment.

Model type:

Text

Model performance class:

Limited

Link to the model:

https://huggingface.co/zai-org/LongAlign-13B-64k

Base models:

Llama-2-13B

End model:

LongAlign-13B-64k

End model license:

Apache-2.0

About the organisation:

Zhipu AI, one of China's AI tigers.

Link to the organisation:

https://github.com/THUDM

Model release date:

January 2024

Availability

Base Model Data

Data nowhere disclosed or documented, and described only in the vaguest terms in a corporate preprint released by Meta.

https://ai.meta.com/research/publications/llama-2-open-foundation-and-fine-tuned-chat-models/

End User Model Data

some on HF

https://huggingface.co/datasets/zai-org/LongAlign-10k

Base Model Weights

Download only after requesting access; requires signing a consent form

https://ai.meta.com/resources/models-and-libraries/llama-downloads/

End User Model Weights

https://huggingface.co/zai-org/LongAlign-13B-64k

Training Code

Documentation

Code Documentation

Hardware Architecture

Preprint

https://arxiv.org/pdf/2401.18058

Paper

https://aclanthology.org/2024.findings-emnlp.74/

Modelcard

Datasheet

No datasheet found.

Access

Package

API and Meta Prompts

Licenses

weights under Apache 2

Is this information not up to date?

Contribute here ->

Supported by the Centre for Language Studies and the Dutch Research Council. Website design & development © 2024 by BSTN. This version of the index generated 10 March 2026, website content last updated 11 March 2026.