DELLA-Merging: Reducing Interference in Model Merging through Magnitude-Based Sampling
Paper • 2406.11617 • Published • 10
This is a merge of pre-trained language models created using mergekit.
This model was merged using the DELLA merge method using yamatazen/Qwen3-V-Science-14B as a base.
The following models were included in the merge:
The following YAML configuration was used to produce this model:
merge_method: della
dtype: bfloat16
out_dtype: bfloat16
base_model: yamatazen/Qwen3-V-Science-14B
models:
- model: soob3123/GrayLine-Qwen3-14B
parameters:
density: 0.5
weight: 0.7
- model: ValiantLabs/Qwen3-14B-Esper3
parameters:
density: 0.5
weight: 0.3
parameters:
epsilon: 0.1
lambda: 1.0