anlp-a2-part1-moe_top2

Decoder-only transformer trained from scratch to translate Vietnamese and Japanese to English (belumind/en-vi-ja-curated-500k-triplets). FFN variant: moe_top2.

• total params: 17,020,800 | active params per token: 13,481,856
• trained on 39,003,133 tokens
• W&B run: https://wandb.ai/dunkronit-iiit-hyderabad/anlp-a2-part1/runs/moe_top2-8210c110

Input format: <vi> source <en> or <ja> source <en>; the model continues with English and <eos>. Load with src.part1.model.Transformer.from_pretrained("DunkRonit/anlp-a2-part1-moe_top2") from the assignment repo.