Card: bge-m3 embedding reference, leaderboard link
Browse files
README.md
CHANGED
|
@@ -138,6 +138,9 @@ script (`code/`). MIRACL: 60 queries per language with 100 candidates each. Wiki
|
|
| 138 |
| [Qwen/Qwen3-Reranker-4B](https://huggingface.co/Qwen/Qwen3-Reranker-4B) (4B) | Apache-2.0 | 0.833 | 0.971 | 0.822 | 0.875 | multilingual LLM reranker; **our teacher** |
|
| 139 |
|
| 140 |
- The table shows the released checkpoint. Means over two training seeds: small MIRACL .730 / Wikipedia .959 / other .783 / mean .824; base .760 / .964 / .796 / .840.
|
|
|
|
|
|
|
|
|
|
| 141 |
- Models ahead of this one: MIRACL: bge-reranker-v2-m3, Qwen3-Reranker-0.6B, Qwen3-Reranker-4B; Wikipedia: Qwen3-Reranker-4B;
|
| 142 |
other: bge-reranker-v2-m3, Qwen3-Reranker-0.6B, Qwen3-Reranker-4B. bge-reranker-v2-m3 was trained on MIRACL's training set; we were not.
|
| 143 |
|
|
|
|
| 138 |
| [Qwen/Qwen3-Reranker-4B](https://huggingface.co/Qwen/Qwen3-Reranker-4B) (4B) | Apache-2.0 | 0.833 | 0.971 | 0.822 | 0.875 | multilingual LLM reranker; **our teacher** |
|
| 139 |
|
| 140 |
- The table shows the released checkpoint. Means over two training seeds: small MIRACL .730 / Wikipedia .959 / other .783 / mean .824; base .760 / .964 / .796 / .840.
|
| 141 |
+
- For reference, ranking the same candidates by [bge-m3](https://huggingface.co/BAAI/bge-m3) dense-embedding similarity alone
|
| 142 |
+
scores 0.797 / 0.927 / 0.788 (mean 0.837), so on this test a reranker adds most on top of a weaker first-stage
|
| 143 |
+
retriever (BM25 or a small embedding model). Full comparison: [leaderboard](https://huggingface.co/spaces/Horizon-Labs/multilingual-retrieval-leaderboard).
|
| 144 |
- Models ahead of this one: MIRACL: bge-reranker-v2-m3, Qwen3-Reranker-0.6B, Qwen3-Reranker-4B; Wikipedia: Qwen3-Reranker-4B;
|
| 145 |
other: bge-reranker-v2-m3, Qwen3-Reranker-0.6B, Qwen3-Reranker-4B. bge-reranker-v2-m3 was trained on MIRACL's training set; we were not.
|
| 146 |
|