WizardLM

Based on LLaMA, which is claimed to be public but nowhere exactly documented.

The Evol-Instruct dataset contains 70k instruction-following sequences generated from Evol-Instruct

Based on LLaMA weights, which are not openly available though a leaked versions is in wide circulation.

Model weights offered as a delta to LLaMA

Reasonably useful source code repository but as this is based on Llama, the underlying source is not available.

Code is comprehensively documented and contains demos.

Architecture described in preprint and partly accessible in code repository

Preprint describes method for creating large amounts of LLM-based synthetic RLHF data and fine-tuning WizardLM based on it

Paper published in ICLR.

Model card is available, however contains links to pages talking about the model architecture, training, fine-tuning, and evaluation rather than containing them itself.

Dataset card is available, but provides no information about data collection and curation. The preprint outlines data collection (based on a 52K instruction dataset from Alpaca) and curation at a high level.

Restricted for academic research purposes only. Code and Model diff release under CC-BY-NC-4.0, software code under Apache 2.0