Teuken

Dataset described as deriving from the CommonCrawl, but no filtered dataset provided. Either a filtered dataset or a fully reproducible and persistent data pipeline would be preferred here.

The Huggingface shows a table with all datasets used for the end model.

Available via Huggingface repository.

Available via Huggingface repository.

SBATCH script with training code available at fork of Megatron-LM. However, no easily visible and easily navigable repository containing the code used to train the model is available. Making the repository more easily visible would alleviate this.

README of containing training code is unchanged from base repo. More elaborate documentation would be warranted. A good example for a good documentation style would be the repository for the OLMo model: https://github.com/allenai/OLMo

Preprint shows architecture, providing details about design decisions and hyperparameters.

Three corresponding preprints, detailing the models, data, and evaluation.

Peer-reviewed paper published in ECAI. Other publications only available as preprints.

Detailed modelcard showing training details, data, technical specifications, and example usage.

No datasheet containing a detailed description of data collection and curation is found attached to a persistent version of the model data, as would be preferred here. A persistent version of the filtered data with attached the information in the data preprint at https://arxiv.org/abs/2410.08800 would be sufficient here.

Apache 2.0, an OSI-approved license.