Fietje

Based on phi2 by Microsoft, for which pretraining data has not been disclosed. Phi2 documentation says only '250B tokens, combination of NLP synthetic data created by AOAI GPT-3.5 and filtered web data from Falcon RefinedWeb and SlimPajama, which was assessed by AOAI GPT-4.'.

Finetuning of Phi not fully documented; derivative uses a Wikipedia and CulturaX mix for Dutch.

Phi 2 model weights available through HuggingFace.

Source code for base model not available, but training for derivative well-documented in github repository.

Source code for base model not available, but training for derivative model documented in exemplary detail.

No data sheet available for base model

Model weights provided under MIT, but training data inherits unclarity and limitations from upstream models (Falcon RefinedWeb, SlimPajama).