cajie
/

Soft_MoA

Mixture of Experts

Model card Files Files and versions

arxiv: https://arxiv.org/abs/2506.05928

Downloads last month: -

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for cajie/Soft_MoA

Base model

meta-llama/Llama-3.1-8B

Finetuned

meta-llama/Llama-3.1-8B-Instruct

Adapter

(1424)

this model

Paper for cajie/Soft_MoA

MoA: Heterogeneous Mixture of Adapters for Parameter-Efficient Fine-Tuning of Large Language Models

Paper • 2506.05928 • Published Jun 6, 2025 • 4