European Open Source AI Index

Database News Guides About Contribute

RedPajama

by Together Computer

About the model:

Open AI model developed as a collaboration between various open-source entities.

Model type:

Text

Model performance class:

Limited

Link to the model:

https://huggingface.co/togethercomputer/RedPajama-INCITE-7B-Chat

Base models:

RedPajama-INCITE-7B-Base

End model:

RedPajama-INCITE-7B-Chat

End model license:

Apache-2.0

About the organisation:

Together Computer, a cloud platform for generative AI.

Link to the organisation:

https://together.ai/

Model release date:

March 2023

Availability

Base Model Data

RedPajama-Data-1T made available on HuggingFace

https://huggingface.co/datasets/togethercomputer/RedPajama-Data-1T

End User Model Data

The model was trained on a large collection of diverse data, including Chain-of-Thought (CoT), Public Pool of Prompts (P3) dataset, Natural-Instructions (NI) dataset. Chat-tuning using Databricks-Dolly and OASST1.

https://huggingface.co/datasets/togethercomputer/RedPajama-Data-Instruct https://huggingface.co/datasets/databricks/databricks-dolly-15k https://huggingface.co/datasets/OpenAssistant/oasst1

Base Model Weights

Base is RedPajama-INCITE-7B-Base

https://huggingface.co/togethercomputer/RedPajama-INCITE-7B-Base

End User Model Weights

Instruction-tuned version made available in parallel with base version.

https://huggingface.co/togethercomputer/RedPajama-INCITE-7B-Chat

Training Code

Code for datasets made available in exemplary ways; code for training and tuning harder to find.

https://github.com/togethercomputer/redpajama.cpp/tree/master/examples/redpajama

Documentation

Code Documentation

Code for base LLM and instruction tuning datasets beautifully documented; code specifying training and fine-tuning sparsely documented.

https://github.com/togethercomputer/redpajama.cpp/tree/master/examples/redpajama

Hardware Architecture

Architecture detailed on model card, crucial parts appear to be forked from GPT-NeoX

https://together.ai/blog/redpajama

Preprint

No preprint found.

Paper

No peer-reviewed paper found.

Modelcard

Model card and readme provide details on datasets and training procedure.

https://huggingface.co/togethercomputer/RedPajama-INCITE-7B-Chat

Datasheet

Base data sheet includes links to data and recipes to create from scratch. Other datasets are well-documented.

https://huggingface.co/datasets/togethercomputer/RedPajama-Data-1T https://huggingface.co/datasets/togethercomputer/RedPajama-Data-Instruct https://huggingface.co/datasets/databricks/databricks-dolly-15k https://huggingface.co/datasets/OpenAssistant/oasst1

Access

Package

No separate package found.

API and Meta Prompts

Hosted inference API available through HuggingFace.

https://huggingface.co/togethercomputer/RedPajama-INCITE-7B-Instruct

Licenses

Models licensed under Apache 2.0, but note that the data itself is variably licensed and so imposes some limitations.

https://huggingface.co/togethercomputer/RedPajama-INCITE-7B-Instruct/blob/main/README.md

Is this information not up to date?

Contribute here ->

Supported by the Centre for Language Studies and the Dutch Research Council. Website design & development © 2024 by BSTN. This version of the index generated 10 March 2026, website content last updated 11 March 2026.