CodeSoft PRO
CodeSoft
AI & ML interests
Working on small language models
Recent Activity
new activity 18 minutes ago
CodeSoft/SLM-Arena:Can you add SupraLabs models please? updated a Space 21 minutes ago
CodeSoft/SLM-Arena repliedto their post about 5 hours ago
Sorbet Mini Experimental is out!
It's not the best model in the 5M parameter range, but it's going to be a really useful model to train on top of. This release was mostly to prove that the model actually works. I trained it on 150M tokens from TinyStories in about 12 minutes.
Does anyone have any tips on training models in this size range? I want to make the full Sorbet Mini release as good as it can be.