Introducing Beetle!
I've been working on Beetle, an open-source suite enabling research on how models learn to help build safer, fairer AI, for over eight months of my PhD.
I'm very happy to share the efforts of this work. I hope the AI research community find the models and training infrastructure helpful. I also hope that the models are useful to applied AI researchers and cognitive scientists. I'd like to share more about the models and what we hope they can offer to the community!
I believe that we should stop treating AI models as black boxes. Studying how models develop can help us build a better scientific understanding of language models for safer, fairer AI.
Beetle is an open-source suite enabling research on how models learn to help build safer, fairer AI.
We release 330 monolingual and bilingual models across 21 languages with checkpoints. I’ll be presenting our new paper, 🪲Beetle🪲, at #EMNLP2026 in Budapest next week as an oral!
What can YOU do with Beetle? Whether you are interested in building small language models, AI interpretability, building better AI systems for more languages, or you research how humans learn language, the Beetle model suite and our open-source training process, data and analysis tools in our research suite allow you to ask cool questions! Beetle lets researchers study model representations and the emergence of new capabilities under different conditions across training regimes, across 100M, 2B and 24B tokens. Find out more: www.beetlelm.github.io .
We need more Open Models with Checkpoints trained on Diverse Data!
If you're an AI enthusiast, the Beetle models and framework are a good opportunity to get hands-on with open models and explore questions that go beyond benchmark scores. The Beetle Team believe that one overlook aspect of open-models are their training processes – see Don't Just "Fix it in Post" for a similar position!
We hope the Beetle models are useful specifically for:
AI researchers to investigate model internals: Explore questions about tokenisation, memorisation, representations, and interpretability to learn when capabilities emerge during training, test hypotheses about learning dynamics.
Multilingual NLP researchers: We release a suite of monolingual and bilingual models with checkpoints from Finnish and Basque to Hindi, Japanese and Korean. You can use them to learn how multilingual transfer happens! Bilingual models are models where we only expose models to two languages. Beetle’s bilingual models, trained on English and one of 21 other languages, are controlled settings to study multilingual pretraining and to understand crosslingual transfer and catastrophic forgetting.
Model builders: Train your own small language models, experiment with different data mixtures and language-exposure curricula, and see what happens when you change how a model learns. These might be the same people investigating "BabyLMs"" or training models in "Open AI's Parameter Golf"".
We need more Open Models with Checkpoints trained on Diverse Data!
We hope the models can also be used to compare AI models with Human Learning using human cognitive data.
Most of the world speaks 2+ languages. Can AI help us understand how we learn and use multiple languages? Beetle models five ways of becoming bilingual, inspired by human experience — from balanced bilingualism to learning a second language in the classroom. Our EMNLP paper compares Beetle’s bilingual learning dynamics with human cognitive data. Beetle predicts English reading times for second-language speakers better than both monolingual and larger, massively multilingual models.
We also hope the Beetle Models can be used to learn about the Science behind AI models. We hope students and people hoping to get into more technical AI research can use the Beetle models as hands-on exploration of the science behind language models! You could first begin by analysing our models and then learn how to train your own.
Have a research question? Find the right Beetle models with our website tool. Compare training strategies, track how capabilities emerge, study cross-lingual transfer, or test how well models predict human behaviour.
Read the paper here: https://aclanthology.org/2026.emnlp-main.2276/
Use the models: https://beetlelm.github.io