model-index:
- name: no_model_name_available
results:
- dataset:
config: default
name: MTEB ArguAna (default)
revision: c22ab2a51041ffd869aaddef7af8d8215647e41a
split: test
type: mteb/arguana
metrics:
- type: main_score
value: 61.696
- type: map_at_1
value: 37.553
- type: map_at_10
value: 53.488
- type: map_at_100
value: 54.174
- type: map_at_1000
value: 54.178000000000004
- type: map_at_20
value: 54.045
- type: map_at_3
value: 49.348
- type: map_at_5
value: 51.900999999999996
- type: mrr_at_1
value: 38.26458036984353
- type: mrr_at_10
value: 53.738371830477185
- type: mrr_at_100
value: 54.43126719288253
- type: mrr_at_1000
value: 54.43524103084228
- type: mrr_at_20
value: 54.30184605147953
- type: mrr_at_3
value: 49.58511142721674
- type: mrr_at_5
value: 52.145566619250936
- type: nauc_map_at_1000_diff1
value: 11.221506369409738
- type: nauc_map_at_1000_max
value: -8.445508307916437
- type: nauc_map_at_1000_std
value: -12.381267290593208
- type: nauc_map_at_100_diff1
value: 11.227790100675787
- type: nauc_map_at_100_max
value: -8.437531922545098
- type: nauc_map_at_100_std
value: -12.373422661586755
- type: nauc_map_at_10_diff1
value: 11.072793219141905
- type: nauc_map_at_10_max
value: -8.312828632293854
- type: nauc_map_at_10_std
value: -12.423506115905337
- type: nauc_map_at_1_diff1
value: 15.220354810399739
- type: nauc_map_at_1_max
value: -11.005854137112008
- type: nauc_map_at_1_std
value: -12.88033375281974
- type: nauc_map_at_20_diff1
value: 11.17399810551959
- type: nauc_map_at_20_max
value: -8.309704396078265
- type: nauc_map_at_20_std
value: -12.31628938741066
- type: nauc_map_at_3_diff1
value: 11.13415033339711
- type: nauc_map_at_3_max
value: -8.681943533315684
- type: nauc_map_at_3_std
value: -12.89456603475663
- type: nauc_map_at_5_diff1
value: 10.91995276847765
- type: nauc_map_at_5_max
value: -8.31368106487608
- type: nauc_map_at_5_std
value: -12.747425008501697
- type: nauc_mrr_at_1000_diff1
value: 9.170026021742572
- type: nauc_mrr_at_1000_max
value: -9.17534528359451
- type: nauc_mrr_at_1000_std
value: -12.16851969333307
- type: nauc_mrr_at_100_diff1
value: 9.176564177369368
- type: nauc_mrr_at_100_max
value: -9.167297045700144
- type: nauc_mrr_at_100_std
value: -12.16072906797416
- type: nauc_mrr_at_10_diff1
value: 9.028156479189121
- type: nauc_mrr_at_10_max
value: -9.062303453873763
- type: nauc_mrr_at_10_std
value: -12.2500914002082
- type: nauc_mrr_at_1_diff1
value: 13.191668992010946
- type: nauc_mrr_at_1_max
value: -10.925013045327509
- type: nauc_mrr_at_1_std
value: -12.232380523762979
- type: nauc_mrr_at_20_diff1
value: 9.13214969584694
- type: nauc_mrr_at_20_max
value: -9.036476679079245
- type: nauc_mrr_at_20_std
value: -12.104714451417257
- type: nauc_mrr_at_3_diff1
value: 9.151473180722105
- type: nauc_mrr_at_3_max
value: -9.532829106604618
- type: nauc_mrr_at_3_std
value: -12.836115074338034
- type: nauc_mrr_at_5_diff1
value: 8.953917200120168
- type: nauc_mrr_at_5_max
value: -9.042911382928725
- type: nauc_mrr_at_5_std
value: -12.530298454542493
- type: nauc_ndcg_at_1000_diff1
value: 10.863407729775824
- type: nauc_ndcg_at_1000_max
value: -7.868989312884095
- type: nauc_ndcg_at_1000_std
value: -11.88614215866286
- type: nauc_ndcg_at_100_diff1
value: 10.999299247670388
- type: nauc_ndcg_at_100_max
value: -7.6862652464192704
- type: nauc_ndcg_at_100_std
value: -11.651296278498132
- type: nauc_ndcg_at_10_diff1
value: 10.074409692068278
- type: nauc_ndcg_at_10_max
value: -6.989935197224304
- type: nauc_ndcg_at_10_std
value: -11.715897777643933
- type: nauc_ndcg_at_1_diff1
value: 15.220354810399739
- type: nauc_ndcg_at_1_max
value: -11.005854137112008
- type: nauc_ndcg_at_1_std
value: -12.88033375281974
- type: nauc_ndcg_at_20_diff1
value: 10.597196019116632
- type: nauc_ndcg_at_20_max
value: -6.8065432063496285
- type: nauc_ndcg_at_20_std
value: -11.188969740275711
- type: nauc_ndcg_at_3_diff1
value: 10.122390462459538
- type: nauc_ndcg_at_3_max
value: -7.97194437583794
- type: nauc_ndcg_at_3_std
value: -13.06215406827939
- type: nauc_ndcg_at_5_diff1
value: 9.590778318105452
- type: nauc_ndcg_at_5_max
value: -7.261562098697961
- type: nauc_ndcg_at_5_std
value: -12.812872325188716
- type: nauc_precision_at_1000_diff1
value: 25.196544697410012
- type: nauc_precision_at_1000_max
value: 27.158141822834548
- type: nauc_precision_at_1000_std
value: 64.38851178902432
- type: nauc_precision_at_100_diff1
value: 37.9019837153977
- type: nauc_precision_at_100_max
value: 34.736605360292934
- type: nauc_precision_at_100_std
value: 57.335032116509325
- type: nauc_precision_at_10_diff1
value: 3.971994041181149
- type: nauc_precision_at_10_max
value: 3.168483093976094
- type: nauc_precision_at_10_std
value: -5.186686622285615
- type: nauc_precision_at_1_diff1
value: 15.220354810399739
- type: nauc_precision_at_1_max
value: -11.005854137112008
- type: nauc_precision_at_1_std
value: -12.88033375281974
- type: nauc_precision_at_20_diff1
value: 6.539599130365263
- type: nauc_precision_at_20_max
value: 22.235006087284845
- type: nauc_precision_at_20_std
value: 14.537186156798331
- type: nauc_precision_at_3_diff1
value: 6.910775130099809
- type: nauc_precision_at_3_max
value: -5.6507220516993355
- type: nauc_precision_at_3_std
value: -13.676386233280786
- type: nauc_precision_at_5_diff1
value: 4.075402230127663
- type: nauc_precision_at_5_max
value: -2.8089334099369507
- type: nauc_precision_at_5_std
value: -13.130074247585222
- type: nauc_recall_at_1000_diff1
value: 25.196544697407973
- type: nauc_recall_at_1000_max
value: 27.158141822829503
- type: nauc_recall_at_1000_std
value: 64.38851178902257
- type: nauc_recall_at_100_diff1
value: 37.901983715398075
- type: nauc_recall_at_100_max
value: 34.736605360292096
- type: nauc_recall_at_100_std
value: 57.335032116506376
- type: nauc_recall_at_10_diff1
value: 3.971994041181333
- type: nauc_recall_at_10_max
value: 3.1684830939761177
- type: nauc_recall_at_10_std
value: -5.186686622285636
- type: nauc_recall_at_1_diff1
value: 15.220354810399739
- type: nauc_recall_at_1_max
value: -11.005854137112008
- type: nauc_recall_at_1_std
value: -12.88033375281974
- type: nauc_recall_at_20_diff1
value: 6.539599130365463
- type: nauc_recall_at_20_max
value: 22.23500608728498
- type: nauc_recall_at_20_std
value: 14.537186156798532
- type: nauc_recall_at_3_diff1
value: 6.9107751300998554
- type: nauc_recall_at_3_max
value: -5.650722051699346
- type: nauc_recall_at_3_std
value: -13.676386233280741
- type: nauc_recall_at_5_diff1
value: 4.075402230127604
- type: nauc_recall_at_5_max
value: -2.808933409936906
- type: nauc_recall_at_5_std
value: -13.130074247585242
- type: ndcg_at_1
value: 37.553
- type: ndcg_at_10
value: 61.696
- type: ndcg_at_100
value: 64.424
- type: ndcg_at_1000
value: 64.511
- type: ndcg_at_20
value: 63.644999999999996
- type: ndcg_at_3
value: 53.323
- type: ndcg_at_5
value: 57.908
- type: precision_at_1
value: 37.553
- type: precision_at_10
value: 8.755
- type: precision_at_100
value: 0.9900000000000001
- type: precision_at_1000
value: 0.1
- type: precision_at_20
value: 4.755
- type: precision_at_3
value: 21.622
- type: precision_at_5
value: 15.192
- type: recall_at_1
value: 37.553
- type: recall_at_10
value: 87.553
- type: recall_at_100
value: 99.004
- type: recall_at_1000
value: 99.644
- type: recall_at_20
value: 95.092
- type: recall_at_3
value: 64.865
- type: recall_at_5
value: 75.96000000000001
task:
type: Retrieval
- dataset:
config: default
name: MTEB CQADupstackAndroidRetrieval (default)
revision: f46a197baaae43b4f621051089b82a364682dfeb
split: test
type: mteb/cqadupstack-android
metrics:
- type: main_score
value: 51.489
- type: map_at_1
value: 33.039
- type: map_at_10
value: 44.812000000000005
- type: map_at_100
value: 46.160000000000004
- type: map_at_1000
value: 46.288000000000004
- type: map_at_20
value: 45.539
- type: map_at_3
value: 40.577000000000005
- type: map_at_5
value: 43.053999999999995
- type: mrr_at_1
value: 40.2002861230329
- type: mrr_at_10
value: 50.3305175193587
- type: mrr_at_100
value: 50.92206549712921
- type: mrr_at_1000
value: 50.965142722486554
- type: mrr_at_20
value: 50.66083966009553
- type: mrr_at_3
value: 47.04339532665711
- type: mrr_at_5
value: 49.03195040534092
- type: nauc_map_at_1000_diff1
value: 52.30986710726043
- type: nauc_map_at_1000_max
value: 40.42440330617406
- type: nauc_map_at_1000_std
value: -2.2644436193903976
- type: nauc_map_at_100_diff1
value: 52.311090092873805
- type: nauc_map_at_100_max
value: 40.45151398317936
- type: nauc_map_at_100_std
value: -2.26978021290327
- type: nauc_map_at_10_diff1
value: 52.18228715152593
- type: nauc_map_at_10_max
value: 40.01076168476963
- type: nauc_map_at_10_std
value: -2.89277969549037
- type: nauc_map_at_1_diff1
value: 58.064811599875554
- type: nauc_map_at_1_max
value: 34.67580228928722
- type: nauc_map_at_1_std
value: -5.67621959001428
- type: nauc_map_at_20_diff1
value: 52.325765916530955
- type: nauc_map_at_20_max
value: 40.28951762467691
- type: nauc_map_at_20_std
value: -2.6749268623880877
- type: nauc_map_at_3_diff1
value: 53.45968147096472
- type: nauc_map_at_3_max
value: 40.60355246079136
- type: nauc_map_at_3_std
value: -2.8827965782367544
- type: nauc_map_at_5_diff1
value: 52.45184910505131
- type: nauc_map_at_5_max
value: 39.75737390251331
- type: nauc_map_at_5_std
value: -3.3718123778308935
- type: nauc_mrr_at_1000_diff1
value: 51.12471768511012
- type: nauc_mrr_at_1000_max
value: 40.981308953185454
- type: nauc_mrr_at_1000_std
value: -2.399500753535585
- type: nauc_mrr_at_100_diff1
value: 51.12168575001752
- type: nauc_mrr_at_100_max
value: 40.9760480542777
- type: nauc_mrr_at_100_std
value: -2.4144273528725475
- type: nauc_mrr_at_10_diff1
value: 51.04018381064975
- type: nauc_mrr_at_10_max
value: 41.003292260169374
- type: nauc_mrr_at_10_std
value: -2.4171421212966964
- type: nauc_mrr_at_1_diff1
value: 54.77850105034208
- type: nauc_mrr_at_1_max
value: 38.617853930056306
- type: nauc_mrr_at_1_std
value: -5.863261741458299
- type: nauc_mrr_at_20_diff1
value: 51.139734313275845
- type: nauc_mrr_at_20_max
value: 40.92389172446841
- type: nauc_mrr_at_20_std
value: -2.5035795007431654
- type: nauc_mrr_at_3_diff1
value: 51.51287925552885
- type: nauc_mrr_at_3_max
value: 42.225260775926074
- type: nauc_mrr_at_3_std
value: -2.306428628618144
- type: nauc_mrr_at_5_diff1
value: 51.10080635226991
- type: nauc_mrr_at_5_max
value: 41.0340227344158
- type: nauc_mrr_at_5_std
value: -2.801571352627902
- type: nauc_ndcg_at_1000_diff1
value: 50.76197955317768
- type: nauc_ndcg_at_1000_max
value: 41.21817915071362
- type: nauc_ndcg_at_1000_std
value: -0.391325532632662
- type: nauc_ndcg_at_100_diff1
value: 50.52419287283379
- type: nauc_ndcg_at_100_max
value: 41.552522119164784
- type: nauc_ndcg_at_100_std
value: -0.036032562878298896
- type: nauc_ndcg_at_10_diff1
value: 49.530566482368876
- type: nauc_ndcg_at_10_max
value: 40.18006026960266
- type: nauc_ndcg_at_10_std
value: -1.8844244877781948
- type: nauc_ndcg_at_1_diff1
value: 54.77850105034208
- type: nauc_ndcg_at_1_max
value: 38.617853930056306
- type: nauc_ndcg_at_1_std
value: -5.863261741458299
- type: nauc_ndcg_at_20_diff1
value: 50.34642284246142
- type: nauc_ndcg_at_20_max
value: 40.481739804267264
- type: nauc_ndcg_at_20_std
value: -1.8870976456072137
- type: nauc_ndcg_at_3_diff1
value: 51.24991022836175
- type: nauc_ndcg_at_3_max
value: 42.05741870860054
- type: nauc_ndcg_at_3_std
value: -1.4174269333857321
- type: nauc_ndcg_at_5_diff1
value: 49.84841673075104
- type: nauc_ndcg_at_5_max
value: 39.9343344658457
- type: nauc_ndcg_at_5_std
value: -2.6419914312584534
- type: nauc_precision_at_1000_diff1
value: -17.86241424441575
- type: nauc_precision_at_1000_max
value: -13.104253008586179
- type: nauc_precision_at_1000_std
value: 0.48249434561668
- type: nauc_precision_at_100_diff1
value: -9.578960904334071
- type: nauc_precision_at_100_max
value: 2.704957537551818
- type: nauc_precision_at_100_std
value: 8.835834433273401
- type: nauc_precision_at_10_diff1
value: 8.196823901166631
- type: nauc_precision_at_10_max
value: 21.763201013538325
- type: nauc_precision_at_10_std
value: 5.766876192845922
- type: nauc_precision_at_1_diff1
value: 54.77850105034208
- type: nauc_precision_at_1_max
value: 38.617853930056306
- type: nauc_precision_at_1_std
value: -5.863261741458299
- type: nauc_precision_at_20_diff1
value: 3.182481517444498
- type: nauc_precision_at_20_max
value: 16.282394914543726
- type: nauc_precision_at_20_std
value: 6.446099543369652
- type: nauc_precision_at_3_diff1
value: 29.403358161131415
- type: nauc_precision_at_3_max
value: 37.66880010058852
- type: nauc_precision_at_3_std
value: 2.212573449639095
- type: nauc_precision_at_5_diff1
value: 17.672198484206056
- type: nauc_precision_at_5_max
value: 27.984636982251065
- type: nauc_precision_at_5_std
value: 2.5950490230843757
- type: nauc_recall_at_1000_diff1
value: 31.680457531090667
- type: nauc_recall_at_1000_max
value: 55.4324783421187
- type: nauc_recall_at_1000_std
value: 44.31810562317212
- type: nauc_recall_at_100_diff1
value: 38.660696650615215
- type: nauc_recall_at_100_max
value: 45.55377816438656
- type: nauc_recall_at_100_std
value: 16.919492724613395
- type: nauc_recall_at_10_diff1
value: 39.020006246915415
- type: nauc_recall_at_10_max
value: 35.588135955395586
- type: nauc_recall_at_10_std
value: -0.26969523665978906
- type: nauc_recall_at_1_diff1
value: 58.064811599875554
- type: nauc_recall_at_1_max
value: 34.67580228928722
- type: nauc_recall_at_1_std
value: -5.67621959001428
- type: nauc_recall_at_20_diff1
value: 40.231921335233146
- type: nauc_recall_at_20_max
value: 35.54503756146707
- type: nauc_recall_at_20_std
value: 0.21383444630797618
- type: nauc_recall_at_3_diff1
value: 47.49873276616812
- type: nauc_recall_at_3_max
value: 42.28383907021981
- type: nauc_recall_at_3_std
value: -0.11739469718977918
- type: nauc_recall_at_5_diff1
value: 42.767952699205054
- type: nauc_recall_at_5_max
value: 36.78024856479531
- type: nauc_recall_at_5_std
value: -3.043075366192802
- type: ndcg_at_1
value: 40.2
- type: ndcg_at_10
value: 51.489
- type: ndcg_at_100
value: 56.216
- type: ndcg_at_1000
value: 58.143
- type: ndcg_at_20
value: 53.208999999999996
- type: ndcg_at_3
value: 45.194
- type: ndcg_at_5
value: 48.382999999999996
- type: precision_at_1
value: 40.2
- type: precision_at_10
value: 9.957
- type: precision_at_100
value: 1.536
- type: precision_at_1000
value: 0.201
- type: precision_at_20
value: 5.7509999999999994
- type: precision_at_3
value: 21.507
- type: precision_at_5
value: 16.023
- type: recall_at_1
value: 33.039
- type: recall_at_10
value: 64.732
- type: recall_at_100
value: 84.451
- type: recall_at_1000
value: 96.711
- type: recall_at_20
value: 70.865
- type: recall_at_3
value: 47.087
- type: recall_at_5
value: 55.568
task:
type: Retrieval
- dataset:
config: default
name: MTEB CQADupstackEnglishRetrieval (default)
revision: ad9991cb51e31e31e430383c75ffb2885547b5f0
split: test
type: mteb/cqadupstack-english
metrics:
- type: main_score
value: 49.659
- type: map_at_1
value: 33.756
- type: map_at_10
value: 44.073
- type: map_at_100
value: 45.373999999999995
- type: map_at_1000
value: 45.504
- type: map_at_20
value: 44.778
- type: map_at_3
value: 41.063
- type: map_at_5
value: 42.941
- type: mrr_at_1
value: 42.038216560509554
- type: mrr_at_10
value: 50.46292083712469
- type: mrr_at_100
value: 51.069095106531805
- type: mrr_at_1000
value: 51.10835326287548
- type: mrr_at_20
value: 50.82019914166541
- type: mrr_at_3
value: 48.25902335456478
- type: mrr_at_5
value: 49.76857749469221
- type: nauc_map_at_1000_diff1
value: 55.80498981672627
- type: nauc_map_at_1000_max
value: 38.26386983200567
- type: nauc_map_at_1000_std
value: -1.722609258110279
- type: nauc_map_at_100_diff1
value: 55.83624507721622
- type: nauc_map_at_100_max
value: 38.18012639419995
- type: nauc_map_at_100_std
value: -1.7922103705396886
- type: nauc_map_at_10_diff1
value: 56.292929981488285
- type: nauc_map_at_10_max
value: 37.01004902094887
- type: nauc_map_at_10_std
value: -3.623023317097144
- type: nauc_map_at_1_diff1
value: 62.46120648089736
- type: nauc_map_at_1_max
value: 32.472573151270595
- type: nauc_map_at_1_std
value: -8.824972585344513
- type: nauc_map_at_20_diff1
value: 56.009618411365835
- type: nauc_map_at_20_max
value: 37.67533628911946
- type: nauc_map_at_20_std
value: -2.572896738144715
- type: nauc_map_at_3_diff1
value: 57.22563904357422
- type: nauc_map_at_3_max
value: 35.016546423861975
- type: nauc_map_at_3_std
value: -6.273815642770271
- type: nauc_map_at_5_diff1
value: 56.51143838048147
- type: nauc_map_at_5_max
value: 36.1162041187686
- type: nauc_map_at_5_std
value: -4.575391814766582
- type: nauc_mrr_at_1000_diff1
value: 53.70506684733427
- type: nauc_mrr_at_1000_max
value: 40.745286648191474
- type: nauc_mrr_at_1000_std
value: 2.3630512587923165
- type: nauc_mrr_at_100_diff1
value: 53.69847816314084
- type: nauc_mrr_at_100_max
value: 40.747562256523416
- type: nauc_mrr_at_100_std
value: 2.376752783372874
- type: nauc_mrr_at_10_diff1
value: 53.72107663200513
- type: nauc_mrr_at_10_max
value: 40.61487500593867
- type: nauc_mrr_at_10_std
value: 1.9707059480789084
- type: nauc_mrr_at_1_diff1
value: 58.72865682486209
- type: nauc_mrr_at_1_max
value: 40.328775719145284
- type: nauc_mrr_at_1_std
value: -0.2516595660358684
- type: nauc_mrr_at_20_diff1
value: 53.67880861456903
- type: nauc_mrr_at_20_max
value: 40.75598871965413
- type: nauc_mrr_at_20_std
value: 2.2872649237129306
- type: nauc_mrr_at_3_diff1
value: 54.657333806417974
- type: nauc_mrr_at_3_max
value: 40.6467539132959
- type: nauc_mrr_at_3_std
value: 1.4674092535243768
- type: nauc_mrr_at_5_diff1
value: 53.945372935894106
- type: nauc_mrr_at_5_max
value: 40.557412208903074
- type: nauc_mrr_at_5_std
value: 1.821724562681988
- type: nauc_ndcg_at_1000_diff1
value: 52.44022578439438
- type: nauc_ndcg_at_1000_max
value: 40.654320935718694
- type: nauc_ndcg_at_1000_std
value: 3.84043663003601
- type: nauc_ndcg_at_100_diff1
value: 52.637626908931026
- type: nauc_ndcg_at_100_max
value: 40.586151202025825
- type: nauc_ndcg_at_100_std
value: 4.089145971553043
- type: nauc_ndcg_at_10_diff1
value: 53.48911939902461
- type: nauc_ndcg_at_10_max
value: 39.126921730430745
- type: nauc_ndcg_at_10_std
value: 0.2318751744695264
- type: nauc_ndcg_at_1_diff1
value: 58.72865682486209
- type: nauc_ndcg_at_1_max
value: 40.328775719145284
- type: nauc_ndcg_at_1_std
value: -0.2516595660358684
- type: nauc_ndcg_at_20_diff1
value: 53.11164095668294
- type: nauc_ndcg_at_20_max
value: 39.97676538781469
- type: nauc_ndcg_at_20_std
value: 2.0794579358557437
- type: nauc_ndcg_at_3_diff1
value: 54.34226761829514
- type: nauc_ndcg_at_3_max
value: 38.48277846169928
- type: nauc_ndcg_at_3_std
value: -1.1217742879848902
- type: nauc_ndcg_at_5_diff1
value: 53.68406928784684
- type: nauc_ndcg_at_5_max
value: 38.75518301179718
- type: nauc_ndcg_at_5_std
value: -0.307785644206231
- type: nauc_precision_at_1000_diff1
value: -19.190831985104566
- type: nauc_precision_at_1000_max
value: 18.016093547405134
- type: nauc_precision_at_1000_std
value: 27.781204433490963
- type: nauc_precision_at_100_diff1
value: -11.915770144248645
- type: nauc_precision_at_100_max
value: 27.771010445059602
- type: nauc_precision_at_100_std
value: 35.169646360589
- type: nauc_precision_at_10_diff1
value: 9.239652324270908
- type: nauc_precision_at_10_max
value: 36.73730897574167
- type: nauc_precision_at_10_std
value: 24.79922189801306
- type: nauc_precision_at_1_diff1
value: 58.72865682486209
- type: nauc_precision_at_1_max
value: 40.328775719145284
- type: nauc_precision_at_1_std
value: -0.2516595660358684
- type: nauc_precision_at_20_diff1
value: 0.14061478328628754
- type: nauc_precision_at_20_max
value: 34.49978969743403
- type: nauc_precision_at_20_std
value: 31.6277564397416
- type: nauc_precision_at_3_diff1
value: 28.84038400301922
- type: nauc_precision_at_3_max
value: 38.15587739264783
- type: nauc_precision_at_3_std
value: 12.110129612812704
- type: nauc_precision_at_5_diff1
value: 18.79985707638106
- type: nauc_precision_at_5_max
value: 37.35261001794856
- type: nauc_precision_at_5_std
value: 18.60235034567574
- type: nauc_recall_at_1000_diff1
value: 30.29744347177445
- type: nauc_recall_at_1000_max
value: 47.62958612849435
- type: nauc_recall_at_1000_std
value: 38.82407061060325
- type: nauc_recall_at_100_diff1
value: 38.478750845798785
- type: nauc_recall_at_100_max
value: 43.25134937529998
- type: nauc_recall_at_100_std
value: 26.0880625914293
- type: nauc_recall_at_10_diff1
value: 45.530522541288995
- type: nauc_recall_at_10_max
value: 34.998803742796
- type: nauc_recall_at_10_std
value: 1.7649084503982129
- type: nauc_recall_at_1_diff1
value: 62.46120648089736
- type: nauc_recall_at_1_max
value: 32.472573151270595
- type: nauc_recall_at_1_std
value: -8.824972585344513
- type: nauc_recall_at_20_diff1
value: 42.86658344001697
- type: nauc_recall_at_20_max
value: 38.595507867414334
- type: nauc_recall_at_20_std
value: 9.547570815114458
- type: nauc_recall_at_3_diff1
value: 50.9059806899254
- type: nauc_recall_at_3_max
value: 32.92800577019243
- type: nauc_recall_at_3_std
value: -4.588379884785415
- type: nauc_recall_at_5_diff1
value: 47.696357938908726
- type: nauc_recall_at_5_max
value: 33.68283781068374
- type: nauc_recall_at_5_std
value: -1.108362904106836
- type: ndcg_at_1
value: 42.038
- type: ndcg_at_10
value: 49.659
- type: ndcg_at_100
value: 54.054
- type: ndcg_at_1000
value: 55.991
- type: ndcg_at_20
value: 51.371
- type: ndcg_at_3
value: 45.692
- type: ndcg_at_5
value: 47.937999999999995
- type: precision_at_1
value: 42.038
- type: precision_at_10
value: 9.229
- type: precision_at_100
value: 1.478
- type: precision_at_1000
value: 0.194
- type: precision_at_20
value: 5.439
- type: precision_at_3
value: 21.996
- type: precision_at_5
value: 15.693999999999999
- type: recall_at_1
value: 33.756
- type: recall_at_10
value: 58.605
- type: recall_at_100
value: 77.171
- type: recall_at_1000
value: 89.42099999999999
- type: recall_at_20
value: 64.842
- type: recall_at_3
value: 46.778999999999996
- type: recall_at_5
value: 52.988
task:
type: Retrieval
- dataset:
config: default
name: MTEB CQADupstackGamingRetrieval (default)
revision: 4885aa143210c98657558c04aaf3dc47cfb54340
split: test
type: mteb/cqadupstack-gaming
metrics:
- type: main_score
value: 60.62499999999999
- type: map_at_1
value: 42.49
- type: map_at_10
value: 54.962999999999994
- type: map_at_100
value: 55.95
- type: map_at_1000
value: 55.998000000000005
- type: map_at_20
value: 55.550999999999995
- type: map_at_3
value: 51.662
- type: map_at_5
value: 53.632000000000005
- type: mrr_at_1
value: 48.589341692789965
- type: mrr_at_10
value: 58.36843807533468
- type: mrr_at_100
value: 59.01175824407762
- type: mrr_at_1000
value: 59.036262879032385
- type: mrr_at_20
value: 58.785682610602365
- type: mrr_at_3
value: 56.09195402298857
- type: mrr_at_5
value: 57.50574712643688
- type: nauc_map_at_1000_diff1
value: 55.0442169239736
- type: nauc_map_at_1000_max
value: 39.57492659519619
- type: nauc_map_at_1000_std
value: -2.4114105449912597
- type: nauc_map_at_100_diff1
value: 55.03298002109766
- type: nauc_map_at_100_max
value: 39.562531879243565
- type: nauc_map_at_100_std
value: -2.3989624011090585
- type: nauc_map_at_10_diff1
value: 55.07059523683087
- type: nauc_map_at_10_max
value: 38.984066662814534
- type: nauc_map_at_10_std
value: -3.248495670183591
- type: nauc_map_at_1_diff1
value: 58.05099902669696
- type: nauc_map_at_1_max
value: 32.78859871251381
- type: nauc_map_at_1_std
value: -6.8362274411770185
- type: nauc_map_at_20_diff1
value: 55.07867137235747
- type: nauc_map_at_20_max
value: 39.338513039269415
- type: nauc_map_at_20_std
value: -2.7901965641663193
- type: nauc_map_at_3_diff1
value: 55.725343313377515
- type: nauc_map_at_3_max
value: 36.70623611414168
- type: nauc_map_at_3_std
value: -5.38922981002907
- type: nauc_map_at_5_diff1
value: 55.34860098665244
- type: nauc_map_at_5_max
value: 38.155527574954334
- type: nauc_map_at_5_std
value: -4.410988667504543
- type: nauc_mrr_at_1000_diff1
value: 54.68704728069744
- type: nauc_mrr_at_1000_max
value: 41.248484810806154
- type: nauc_mrr_at_1000_std
value: -1.1504232296913572
- type: nauc_mrr_at_100_diff1
value: 54.68294395653655
- type: nauc_mrr_at_100_max
value: 41.26462380655141
- type: nauc_mrr_at_100_std
value: -1.1243771660425617
- type: nauc_mrr_at_10_diff1
value: 54.551492987545046
- type: nauc_mrr_at_10_max
value: 41.1848884498371
- type: nauc_mrr_at_10_std
value: -1.3450812747262126
- type: nauc_mrr_at_1_diff1
value: 57.61745956951423
- type: nauc_mrr_at_1_max
value: 39.07829716661263
- type: nauc_mrr_at_1_std
value: -4.06903035046345
- type: nauc_mrr_at_20_diff1
value: 54.6333737758253
- type: nauc_mrr_at_20_max
value: 41.28437228917431
- type: nauc_mrr_at_20_std
value: -1.1953941701515725
- type: nauc_mrr_at_3_diff1
value: 54.94583102618593
- type: nauc_mrr_at_3_max
value: 40.59781328371594
- type: nauc_mrr_at_3_std
value: -1.8602727105690913
- type: nauc_mrr_at_5_diff1
value: 54.558425898850295
- type: nauc_mrr_at_5_max
value: 41.04233761275661
- type: nauc_mrr_at_5_std
value: -1.6442836291230383
- type: nauc_ndcg_at_1000_diff1
value: 53.999245967336094
- type: nauc_ndcg_at_1000_max
value: 42.043310183678116
- type: nauc_ndcg_at_1000_std
value: 0.6376304564130958
- type: nauc_ndcg_at_100_diff1
value: 53.8085409629583
- type: nauc_ndcg_at_100_max
value: 42.451986891379406
- type: nauc_ndcg_at_100_std
value: 1.4673130808247419
- type: nauc_ndcg_at_10_diff1
value: 53.61810206266415
- type: nauc_ndcg_at_10_max
value: 41.44833461873951
- type: nauc_ndcg_at_10_std
value: -0.795550206706578
- type: nauc_ndcg_at_1_diff1
value: 57.61745956951423
- type: nauc_ndcg_at_1_max
value: 39.07829716661263
- type: nauc_ndcg_at_1_std
value: -4.06903035046345
- type: nauc_ndcg_at_20_diff1
value: 53.81663067148986
- type: nauc_ndcg_at_20_max
value: 41.979487734160195
- type: nauc_ndcg_at_20_std
value: 0.11420532465314373
- type: nauc_ndcg_at_3_diff1
value: 54.44647321686452
- type: nauc_ndcg_at_3_max
value: 38.72907885888278
- type: nauc_ndcg_at_3_std
value: -3.0842255604104434
- type: nauc_ndcg_at_5_diff1
value: 53.96422312169755
- type: nauc_ndcg_at_5_max
value: 40.509155065647704
- type: nauc_ndcg_at_5_std
value: -2.38803561566978
- type: nauc_precision_at_1000_diff1
value: -14.986600071659906
- type: nauc_precision_at_1000_max
value: 21.967375999858852
- type: nauc_precision_at_1000_std
value: 25.427090814224133
- type: nauc_precision_at_100_diff1
value: -10.041859048892281
- type: nauc_precision_at_100_max
value: 28.096955065789665
- type: nauc_precision_at_100_std
value: 29.9226812514703
- type: nauc_precision_at_10_diff1
value: 9.879046672723295
- type: nauc_precision_at_10_max
value: 35.2125175415903
- type: nauc_precision_at_10_std
value: 18.049405226428185
- type: nauc_precision_at_1_diff1
value: 57.61745956951423
- type: nauc_precision_at_1_max
value: 39.07829716661263
- type: nauc_precision_at_1_std
value: -4.06903035046345
- type: nauc_precision_at_20_diff1
value: 2.090317659883479
- type: nauc_precision_at_20_max
value: 33.733594712420285
- type: nauc_precision_at_20_std
value: 24.20651359460664
- type: nauc_precision_at_3_diff1
value: 31.657897405365404
- type: nauc_precision_at_3_max
value: 37.3176839095402
- type: nauc_precision_at_3_std
value: 4.7585612155449954
- type: nauc_precision_at_5_diff1
value: 21.504317148610664
- type: nauc_precision_at_5_max
value: 36.98672018829801
- type: nauc_precision_at_5_std
value: 9.48802133563749
- type: nauc_recall_at_1000_diff1
value: 38.34514263143671
- type: nauc_recall_at_1000_max
value: 65.26851560996016
- type: nauc_recall_at_1000_std
value: 50.141313534646144
- type: nauc_recall_at_100_diff1
value: 42.199342404417116
- type: nauc_recall_at_100_max
value: 57.552837181588025
- type: nauc_recall_at_100_std
value: 34.207126382770056
- type: nauc_recall_at_10_diff1
value: 46.72328715834629
- type: nauc_recall_at_10_max
value: 43.601938956571736
- type: nauc_recall_at_10_std
value: 3.9959931946981873
- type: nauc_recall_at_1_diff1
value: 58.05099902669696
- type: nauc_recall_at_1_max
value: 32.78859871251381
- type: nauc_recall_at_1_std
value: -6.8362274411770185
- type: nauc_recall_at_20_diff1
value: 46.23265528982643
- type: nauc_recall_at_20_max
value: 48.02567065402622
- type: nauc_recall_at_20_std
value: 10.922350598954127
- type: nauc_recall_at_3_diff1
value: 51.39812148128243
- type: nauc_recall_at_3_max
value: 36.73161178544267
- type: nauc_recall_at_3_std
value: -4.14759512791178
- type: nauc_recall_at_5_diff1
value: 48.93788534189359
- type: nauc_recall_at_5_max
value: 40.693530247790264
- type: nauc_recall_at_5_std
value: -1.4447466756303786
- type: ndcg_at_1
value: 48.589
- type: ndcg_at_10
value: 60.62499999999999
- type: ndcg_at_100
value: 64.482
- type: ndcg_at_1000
value: 65.443
- type: ndcg_at_20
value: 62.367
- type: ndcg_at_3
value: 55.324
- type: ndcg_at_5
value: 58.108000000000004
- type: precision_at_1
value: 48.589
- type: precision_at_10
value: 9.618
- type: precision_at_100
value: 1.2449999999999999
- type: precision_at_1000
value: 0.136
- type: precision_at_20
value: 5.339
- type: precision_at_3
value: 24.514
- type: precision_at_5
value: 16.814999999999998
- type: recall_at_1
value: 42.49
- type: recall_at_10
value: 73.817
- type: recall_at_100
value: 90.35499999999999
- type: recall_at_1000
value: 97.158
- type: recall_at_20
value: 80.345
- type: recall_at_3
value: 59.714
- type: recall_at_5
value: 66.545
task:
type: Retrieval
- dataset:
config: default
name: MTEB CQADupstackGisRetrieval (default)
revision: 5003b3064772da1887988e05400cf3806fe491f2
split: test
type: mteb/cqadupstack-gis
metrics:
- type: main_score
value: 41.657
- type: map_at_1
value: 27.546
- type: map_at_10
value: 36.571999999999996
- type: map_at_100
value: 37.617
- type: map_at_1000
value: 37.697
- type: map_at_20
value: 37.181
- type: map_at_3
value: 33.894000000000005
- type: map_at_5
value: 35.38
- type: mrr_at_1
value: 29.491525423728817
- type: mrr_at_10
value: 38.582324455205786
- type: mrr_at_100
value: 39.47140776816037
- type: mrr_at_1000
value: 39.53215817683643
- type: mrr_at_20
value: 39.09768103301972
- type: mrr_at_3
value: 36.04519774011299
- type: mrr_at_5
value: 37.51977401129942
- type: nauc_map_at_1000_diff1
value: 46.64371556448634
- type: nauc_map_at_1000_max
value: 32.7685783979018
- type: nauc_map_at_1000_std
value: 0.252907073291861
- type: nauc_map_at_100_diff1
value: 46.641497461920125
- type: nauc_map_at_100_max
value: 32.76050187436589
- type: nauc_map_at_100_std
value: 0.25060431368990005
- type: nauc_map_at_10_diff1
value: 46.73238378557523
- type: nauc_map_at_10_max
value: 32.52292111951917
- type: nauc_map_at_10_std
value: -0.08943369954712965
- type: nauc_map_at_1_diff1
value: 52.365074294185774
- type: nauc_map_at_1_max
value: 31.479079738443833
- type: nauc_map_at_1_std
value: -3.5343210117553236
- type: nauc_map_at_20_diff1
value: 46.586893769506986
- type: nauc_map_at_20_max
value: 32.69375129068728
- type: nauc_map_at_20_std
value: 0.053444924369365696
- type: nauc_map_at_3_diff1
value: 47.99938279731178
- type: nauc_map_at_3_max
value: 32.17359387179817
- type: nauc_map_at_3_std
value: -1.093878031386867
- type: nauc_map_at_5_diff1
value: 47.16122715188318
- type: nauc_map_at_5_max
value: 32.45926972751325
- type: nauc_map_at_5_std
value: -0.6741064995053677
- type: nauc_mrr_at_1000_diff1
value: 45.30445137492832
- type: nauc_mrr_at_1000_max
value: 33.574100643632335
- type: nauc_mrr_at_1000_std
value: 1.4554396923762167
- type: nauc_mrr_at_100_diff1
value: 45.3021240311195
- type: nauc_mrr_at_100_max
value: 33.57865938948554
- type: nauc_mrr_at_100_std
value: 1.4700086747711112
- type: nauc_mrr_at_10_diff1
value: 45.33679560393517
- type: nauc_mrr_at_10_max
value: 33.613141004163104
- type: nauc_mrr_at_10_std
value: 1.4618756140600853
- type: nauc_mrr_at_1_diff1
value: 50.76920173801882
- type: nauc_mrr_at_1_max
value: 33.15485424863947
- type: nauc_mrr_at_1_std
value: -1.9770225985333183
- type: nauc_mrr_at_20_diff1
value: 45.26278617551855
- type: nauc_mrr_at_20_max
value: 33.54090454353577
- type: nauc_mrr_at_20_std
value: 1.315954535389389
- type: nauc_mrr_at_3_diff1
value: 46.69543535444215
- type: nauc_mrr_at_3_max
value: 33.310967131967075
- type: nauc_mrr_at_3_std
value: 0.4331816772474548
- type: nauc_mrr_at_5_diff1
value: 45.854532532996764
- type: nauc_mrr_at_5_max
value: 33.543171491597015
- type: nauc_mrr_at_5_std
value: 1.126443288383718
- type: nauc_ndcg_at_1000_diff1
value: 44.04108026398546
- type: nauc_ndcg_at_1000_max
value: 33.85625934828308
- type: nauc_ndcg_at_1000_std
value: 3.4377340403939796
- type: nauc_ndcg_at_100_diff1
value: 43.744185935147904
- type: nauc_ndcg_at_100_max
value: 33.572431122976006
- type: nauc_ndcg_at_100_std
value: 3.6852532675904555
- type: nauc_ndcg_at_10_diff1
value: 43.959538473699595
- type: nauc_ndcg_at_10_max
value: 33.073472647769215
- type: nauc_ndcg_at_10_std
value: 2.1090188319460537
- type: nauc_ndcg_at_1_diff1
value: 50.76920173801882
- type: nauc_ndcg_at_1_max
value: 33.15485424863947
- type: nauc_ndcg_at_1_std
value: -1.9770225985333183
- type: nauc_ndcg_at_20_diff1
value: 43.411334966477845
- type: nauc_ndcg_at_20_max
value: 33.293926947365264
- type: nauc_ndcg_at_20_std
value: 2.220080028577616
- type: nauc_ndcg_at_3_diff1
value: 46.523293896100185
- type: nauc_ndcg_at_3_max
value: 32.7038574353448
- type: nauc_ndcg_at_3_std
value: 0.1193014435621674
- type: nauc_ndcg_at_5_diff1
value: 45.14841006493745
- type: nauc_ndcg_at_5_max
value: 33.12175384441183
- type: nauc_ndcg_at_5_std
value: 1.0948075278546274
- type: nauc_precision_at_1000_diff1
value: -2.6467112220265907
- type: nauc_precision_at_1000_max
value: 18.287539005114457
- type: nauc_precision_at_1000_std
value: 16.038219314209957
- type: nauc_precision_at_100_diff1
value: 12.552020819249044
- type: nauc_precision_at_100_max
value: 26.351962576748278
- type: nauc_precision_at_100_std
value: 18.01891312596408
- type: nauc_precision_at_10_diff1
value: 29.157382675022458
- type: nauc_precision_at_10_max
value: 34.98181169997513
- type: nauc_precision_at_10_std
value: 11.08592118988915
- type: nauc_precision_at_1_diff1
value: 50.76920173801882
- type: nauc_precision_at_1_max
value: 33.15485424863947
- type: nauc_precision_at_1_std
value: -1.9770225985333183
- type: nauc_precision_at_20_diff1
value: 22.77749317612865
- type: nauc_precision_at_20_max
value: 33.77830866103203
- type: nauc_precision_at_20_std
value: 12.293728764793507
- type: nauc_precision_at_3_diff1
value: 39.56004686919892
- type: nauc_precision_at_3_max
value: 35.35459717391885
- type: nauc_precision_at_3_std
value: 4.646578221523918
- type: nauc_precision_at_5_diff1
value: 34.503008179786114
- type: nauc_precision_at_5_max
value: 35.75252363539026
- type: nauc_precision_at_5_std
value: 7.166979765009376
- type: nauc_recall_at_1000_diff1
value: 21.221212523313234
- type: nauc_recall_at_1000_max
value: 44.13351833139584
- type: nauc_recall_at_1000_std
value: 40.087640819909765
- type: nauc_recall_at_100_diff1
value: 29.389636542285626
- type: nauc_recall_at_100_max
value: 33.59631288629025
- type: nauc_recall_at_100_std
value: 21.206059374954965
- type: nauc_recall_at_10_diff1
value: 34.959430857464405
- type: nauc_recall_at_10_max
value: 32.09717004724124
- type: nauc_recall_at_10_std
value: 6.752586127466409
- type: nauc_recall_at_1_diff1
value: 52.365074294185774
- type: nauc_recall_at_1_max
value: 31.479079738443833
- type: nauc_recall_at_1_std
value: -3.5343210117553236
- type: nauc_recall_at_20_diff1
value: 31.39296498241759
- type: nauc_recall_at_20_max
value: 32.15885772797268
- type: nauc_recall_at_20_std
value: 7.159400260126331
- type: nauc_recall_at_3_diff1
value: 42.79168863924967
- type: nauc_recall_at_3_max
value: 31.762155645661604
- type: nauc_recall_at_3_std
value: 2.196757471031164
- type: nauc_recall_at_5_diff1
value: 39.5255822009034
- type: nauc_recall_at_5_max
value: 32.94403778206411
- type: nauc_recall_at_5_std
value: 4.303804658498762
- type: ndcg_at_1
value: 29.492
- type: ndcg_at_10
value: 41.657
- type: ndcg_at_100
value: 46.573
- type: ndcg_at_1000
value: 48.541000000000004
- type: ndcg_at_20
value: 43.689
- type: ndcg_at_3
value: 36.477
- type: ndcg_at_5
value: 38.984
- type: precision_at_1
value: 29.492
- type: precision_at_10
value: 6.315999999999999
- type: precision_at_100
value: 0.922
- type: precision_at_1000
value: 0.11199999999999999
- type: precision_at_20
value: 3.644
- type: precision_at_3
value: 15.292
- type: precision_at_5
value: 10.667
- type: recall_at_1
value: 27.546
- type: recall_at_10
value: 55.359
- type: recall_at_100
value: 77.46
- type: recall_at_1000
value: 92.205
- type: recall_at_20
value: 62.946000000000005
- type: recall_at_3
value: 41.501
- type: recall_at_5
value: 47.402
task:
type: Retrieval
- dataset:
config: default
name: MTEB CQADupstackMathematicaRetrieval (default)
revision: 90fceea13679c63fe563ded68f3b6f06e50061de
split: test
type: mteb/cqadupstack-mathematica
metrics:
- type: main_score
value: 32.094
- type: map_at_1
value: 18.114
- type: map_at_10
value: 26.418999999999997
- type: map_at_100
value: 27.571
- type: map_at_1000
value: 27.685
- type: map_at_20
value: 27.017000000000003
- type: map_at_3
value: 23.185
- type: map_at_5
value: 25.145
- type: mrr_at_1
value: 22.388059701492537
- type: mrr_at_10
value: 31.26041419884701
- type: mrr_at_100
value: 32.18086832352193
- type: mrr_at_1000
value: 32.239701638052914
- type: mrr_at_20
value: 31.79589213295487
- type: mrr_at_3
value: 28.171641791044777
- type: mrr_at_5
value: 30.09328358208954
- type: nauc_map_at_1000_diff1
value: 36.547104211161574
- type: nauc_map_at_1000_max
value: 26.232937210016445
- type: nauc_map_at_1000_std
value: 3.7298712392657354
- type: nauc_map_at_100_diff1
value: 36.537505852341276
- type: nauc_map_at_100_max
value: 26.246840141152063
- type: nauc_map_at_100_std
value: 3.7320331192887792
- type: nauc_map_at_10_diff1
value: 36.63285267150619
- type: nauc_map_at_10_max
value: 25.754810692097703
- type: nauc_map_at_10_std
value: 3.316281417571231
- type: nauc_map_at_1_diff1
value: 42.76051570018438
- type: nauc_map_at_1_max
value: 27.280053995003268
- type: nauc_map_at_1_std
value: 0.9988567525974422
- type: nauc_map_at_20_diff1
value: 36.487646930680604
- type: nauc_map_at_20_max
value: 25.995111909458537
- type: nauc_map_at_20_std
value: 3.5667106982185914
- type: nauc_map_at_3_diff1
value: 37.59295352703098
- type: nauc_map_at_3_max
value: 25.60161931151291
- type: nauc_map_at_3_std
value: 3.083713373459857
- type: nauc_map_at_5_diff1
value: 36.788439621346306
- type: nauc_map_at_5_max
value: 25.632331755255706
- type: nauc_map_at_5_std
value: 3.1498001343472954
- type: nauc_mrr_at_1000_diff1
value: 37.24491529620153
- type: nauc_mrr_at_1000_max
value: 28.025849316370135
- type: nauc_mrr_at_1000_std
value: 2.497991021482928
- type: nauc_mrr_at_100_diff1
value: 37.23089604190849
- type: nauc_mrr_at_100_max
value: 28.02854068654799
- type: nauc_mrr_at_100_std
value: 2.4918045940748628
- type: nauc_mrr_at_10_diff1
value: 37.29298050888863
- type: nauc_mrr_at_10_max
value: 27.76885624231521
- type: nauc_mrr_at_10_std
value: 2.2485319842764295
- type: nauc_mrr_at_1_diff1
value: 43.90681636625015
- type: nauc_mrr_at_1_max
value: 28.932581562766497
- type: nauc_mrr_at_1_std
value: -0.8697956494444896
- type: nauc_mrr_at_20_diff1
value: 37.22358560158573
- type: nauc_mrr_at_20_max
value: 27.916137779918017
- type: nauc_mrr_at_20_std
value: 2.455121655564212
- type: nauc_mrr_at_3_diff1
value: 38.54010072668797
- type: nauc_mrr_at_3_max
value: 27.98776700061011
- type: nauc_mrr_at_3_std
value: 2.2699056062431398
- type: nauc_mrr_at_5_diff1
value: 37.549148552294184
- type: nauc_mrr_at_5_max
value: 27.979221111926183
- type: nauc_mrr_at_5_std
value: 2.4684701997216383
- type: nauc_ndcg_at_1000_diff1
value: 34.435617340922406
- type: nauc_ndcg_at_1000_max
value: 27.628967284947358
- type: nauc_ndcg_at_1000_std
value: 5.911838589273425
- type: nauc_ndcg_at_100_diff1
value: 34.2780900043709
- type: nauc_ndcg_at_100_max
value: 27.64832881094927
- type: nauc_ndcg_at_100_std
value: 5.9731923704150764
- type: nauc_ndcg_at_10_diff1
value: 34.63367473270069
- type: nauc_ndcg_at_10_max
value: 25.933512573194566
- type: nauc_ndcg_at_10_std
value: 4.148707647963745
- type: nauc_ndcg_at_1_diff1
value: 43.90681636625015
- type: nauc_ndcg_at_1_max
value: 28.932581562766497
- type: nauc_ndcg_at_1_std
value: -0.8697956494444896
- type: nauc_ndcg_at_20_diff1
value: 34.22188383179007
- type: nauc_ndcg_at_20_max
value: 26.518139867033575
- type: nauc_ndcg_at_20_std
value: 4.866776710170176
- type: nauc_ndcg_at_3_diff1
value: 36.77565391921702
- type: nauc_ndcg_at_3_max
value: 25.990116566248034
- type: nauc_ndcg_at_3_std
value: 3.6154482154832026
- type: nauc_ndcg_at_5_diff1
value: 35.05810918990998
- type: nauc_ndcg_at_5_max
value: 25.926492701845728
- type: nauc_ndcg_at_5_std
value: 4.008846035455256
- type: nauc_precision_at_1000_diff1
value: -0.8816544062816003
- type: nauc_precision_at_1000_max
value: 2.659968534023147
- type: nauc_precision_at_1000_std
value: -0.23898006639991531
- type: nauc_precision_at_100_diff1
value: 9.252337889441803
- type: nauc_precision_at_100_max
value: 15.7473263197486
- type: nauc_precision_at_100_std
value: 7.041410200070057
- type: nauc_precision_at_10_diff1
value: 23.46738965136796
- type: nauc_precision_at_10_max
value: 21.705577114654027
- type: nauc_precision_at_10_std
value: 5.0699670250199125
- type: nauc_precision_at_1_diff1
value: 43.90681636625015
- type: nauc_precision_at_1_max
value: 28.932581562766497
- type: nauc_precision_at_1_std
value: -0.8697956494444896
- type: nauc_precision_at_20_diff1
value: 19.018007524361167
- type: nauc_precision_at_20_max
value: 21.792753400253282
- type: nauc_precision_at_20_std
value: 6.018387201774473
- type: nauc_precision_at_3_diff1
value: 31.95395112895907
- type: nauc_precision_at_3_max
value: 24.63594035058901
- type: nauc_precision_at_3_std
value: 4.397483988354452
- type: nauc_precision_at_5_diff1
value: 27.209400705378783
- type: nauc_precision_at_5_max
value: 22.786488526839538
- type: nauc_precision_at_5_std
value: 5.421275827056416
- type: nauc_recall_at_1000_diff1
value: 13.757448646708099
- type: nauc_recall_at_1000_max
value: 37.61282402281419
- type: nauc_recall_at_1000_std
value: 31.243111625707936
- type: nauc_recall_at_100_diff1
value: 22.918274040700464
- type: nauc_recall_at_100_max
value: 29.405783838243728
- type: nauc_recall_at_100_std
value: 16.510682564081595
- type: nauc_recall_at_10_diff1
value: 26.688230397831575
- type: nauc_recall_at_10_max
value: 22.638607254188507
- type: nauc_recall_at_10_std
value: 6.320166943996537
- type: nauc_recall_at_1_diff1
value: 42.76051570018438
- type: nauc_recall_at_1_max
value: 27.280053995003268
- type: nauc_recall_at_1_std
value: 0.9988567525974422
- type: nauc_recall_at_20_diff1
value: 25.089842677170015
- type: nauc_recall_at_20_max
value: 24.055904631207994
- type: nauc_recall_at_20_std
value: 8.741260530294397
- type: nauc_recall_at_3_diff1
value: 32.11825375449307
- type: nauc_recall_at_3_max
value: 23.660049290130708
- type: nauc_recall_at_3_std
value: 6.059610998940883
- type: nauc_recall_at_5_diff1
value: 28.261264641031325
- type: nauc_recall_at_5_max
value: 23.297121813128527
- type: nauc_recall_at_5_std
value: 6.31559443948309
- type: ndcg_at_1
value: 22.387999999999998
- type: ndcg_at_10
value: 32.094
- type: ndcg_at_100
value: 37.676
- type: ndcg_at_1000
value: 40.338
- type: ndcg_at_20
value: 34.123999999999995
- type: ndcg_at_3
value: 26.232
- type: ndcg_at_5
value: 29.298999999999996
- type: precision_at_1
value: 22.387999999999998
- type: precision_at_10
value: 6.069999999999999
- type: precision_at_100
value: 1.016
- type: precision_at_1000
value: 0.13699999999999998
- type: precision_at_20
value: 3.588
- type: precision_at_3
value: 12.645000000000001
- type: precision_at_5
value: 9.677
- type: recall_at_1
value: 18.114
- type: recall_at_10
value: 44.812000000000005
- type: recall_at_100
value: 69.37100000000001
- type: recall_at_1000
value: 88.465
- type: recall_at_20
value: 52.17699999999999
- type: recall_at_3
value: 28.84
- type: recall_at_5
value: 36.524
task:
type: Retrieval
- dataset:
config: default
name: MTEB CQADupstackPhysicsRetrieval (default)
revision: 79531abbd1fb92d06c6d6315a0cbbbf5bb247ea4
split: test
type: mteb/cqadupstack-physics
metrics:
- type: main_score
value: 47.392
- type: map_at_1
value: 31.184
- type: map_at_10
value: 41.388999999999996
- type: map_at_100
value: 42.717
- type: map_at_1000
value: 42.824
- type: map_at_20
value: 42.173
- type: map_at_3
value: 38.04
- type: map_at_5
value: 39.893
- type: mrr_at_1
value: 37.921077959576515
- type: mrr_at_10
value: 47.24326657194797
- type: mrr_at_100
value: 48.09337498756284
- type: mrr_at_1000
value: 48.13701720898694
- type: mrr_at_20
value: 47.78987774517749
- type: mrr_at_3
value: 44.73853063843437
- type: mrr_at_5
value: 46.12929098492135
- type: nauc_map_at_1000_diff1
value: 53.832770651928385
- type: nauc_map_at_1000_max
value: 36.05214174181181
- type: nauc_map_at_1000_std
value: -0.26823733386553206
- type: nauc_map_at_100_diff1
value: 53.8465517808178
- type: nauc_map_at_100_max
value: 36.02504193265796
- type: nauc_map_at_100_std
value: -0.2970919560590155
- type: nauc_map_at_10_diff1
value: 54.13788168392246
- type: nauc_map_at_10_max
value: 35.56538539209599
- type: nauc_map_at_10_std
value: -1.1471061359311105
- type: nauc_map_at_1_diff1
value: 59.8215952283572
- type: nauc_map_at_1_max
value: 33.12636725249456
- type: nauc_map_at_1_std
value: -5.8855081788994426
- type: nauc_map_at_20_diff1
value: 53.82330666305302
- type: nauc_map_at_20_max
value: 35.756597739016556
- type: nauc_map_at_20_std
value: -0.8189341923940018
- type: nauc_map_at_3_diff1
value: 54.60209193467526
- type: nauc_map_at_3_max
value: 35.12485641266856
- type: nauc_map_at_3_std
value: -2.374660245248787
- type: nauc_map_at_5_diff1
value: 54.42904648965896
- type: nauc_map_at_5_max
value: 35.64840413541652
- type: nauc_map_at_5_std
value: -1.2551357786908561
- type: nauc_mrr_at_1000_diff1
value: 52.792235640928034
- type: nauc_mrr_at_1000_max
value: 37.01592108981346
- type: nauc_mrr_at_1000_std
value: 1.1840423852937316
- type: nauc_mrr_at_100_diff1
value: 52.78102857164482
- type: nauc_mrr_at_100_max
value: 37.02689347051387
- type: nauc_mrr_at_100_std
value: 1.204280143100286
- type: nauc_mrr_at_10_diff1
value: 52.881983064911786
- type: nauc_mrr_at_10_max
value: 36.77776807029599
- type: nauc_mrr_at_10_std
value: 0.7855901246582556
- type: nauc_mrr_at_1_diff1
value: 57.33640696039994
- type: nauc_mrr_at_1_max
value: 36.5204924145658
- type: nauc_mrr_at_1_std
value: -1.4711855723217255
- type: nauc_mrr_at_20_diff1
value: 52.68909138930995
- type: nauc_mrr_at_20_max
value: 36.87667798023464
- type: nauc_mrr_at_20_std
value: 0.9666329262219553
- type: nauc_mrr_at_3_diff1
value: 53.174709647944226
- type: nauc_mrr_at_3_max
value: 37.30731859468477
- type: nauc_mrr_at_3_std
value: 0.5983585812920581
- type: nauc_mrr_at_5_diff1
value: 52.98696707218856
- type: nauc_mrr_at_5_max
value: 37.24200659304458
- type: nauc_mrr_at_5_std
value: 0.7873226001470035
- type: nauc_ndcg_at_1000_diff1
value: 51.77336066774053
- type: nauc_ndcg_at_1000_max
value: 37.33484716494564
- type: nauc_ndcg_at_1000_std
value: 3.4897213816902797
- type: nauc_ndcg_at_100_diff1
value: 51.517855623678145
- type: nauc_ndcg_at_100_max
value: 37.28753758759313
- type: nauc_ndcg_at_100_std
value: 3.7916614425014505
- type: nauc_ndcg_at_10_diff1
value: 52.19312040537212
- type: nauc_ndcg_at_10_max
value: 35.454043426004375
- type: nauc_ndcg_at_10_std
value: 0.11261377272176304
- type: nauc_ndcg_at_1_diff1
value: 57.33640696039994
- type: nauc_ndcg_at_1_max
value: 36.5204924145658
- type: nauc_ndcg_at_1_std
value: -1.4711855723217255
- type: nauc_ndcg_at_20_diff1
value: 51.180003517470475
- type: nauc_ndcg_at_20_max
value: 35.808657815300954
- type: nauc_ndcg_at_20_std
value: 0.9181626595803211
- type: nauc_ndcg_at_3_diff1
value: 52.15499168076373
- type: nauc_ndcg_at_3_max
value: 36.298204760804936
- type: nauc_ndcg_at_3_std
value: -0.5611421248036095
- type: nauc_ndcg_at_5_diff1
value: 52.39994328034564
- type: nauc_ndcg_at_5_max
value: 36.330303995608936
- type: nauc_ndcg_at_5_std
value: 0.37444767396372436
- type: nauc_precision_at_1000_diff1
value: -20.404122745051737
- type: nauc_precision_at_1000_max
value: 5.312457203159795
- type: nauc_precision_at_1000_std
value: 17.213242606819847
- type: nauc_precision_at_100_diff1
value: -8.42790566562044
- type: nauc_precision_at_100_max
value: 16.84086690052803
- type: nauc_precision_at_100_std
value: 24.019700615464902
- type: nauc_precision_at_10_diff1
value: 15.81766147983048
- type: nauc_precision_at_10_max
value: 27.484691185221283
- type: nauc_precision_at_10_std
value: 14.56489367081778
- type: nauc_precision_at_1_diff1
value: 57.33640696039994
- type: nauc_precision_at_1_max
value: 36.5204924145658
- type: nauc_precision_at_1_std
value: -1.4711855723217255
- type: nauc_precision_at_20_diff1
value: 5.879061729055104
- type: nauc_precision_at_20_max
value: 23.653566476735964
- type: nauc_precision_at_20_std
value: 16.503931318384556
- type: nauc_precision_at_3_diff1
value: 33.45155737947474
- type: nauc_precision_at_3_max
value: 34.7568767770609
- type: nauc_precision_at_3_std
value: 7.67006740712776
- type: nauc_precision_at_5_diff1
value: 25.110442490351748
- type: nauc_precision_at_5_max
value: 32.29114656055464
- type: nauc_precision_at_5_std
value: 12.155988425672343
- type: nauc_recall_at_1000_diff1
value: 35.532706398200745
- type: nauc_recall_at_1000_max
value: 43.19385344871575
- type: nauc_recall_at_1000_std
value: 48.491467893524295
- type: nauc_recall_at_100_diff1
value: 36.58816403449305
- type: nauc_recall_at_100_max
value: 38.638485536057296
- type: nauc_recall_at_100_std
value: 27.11603908950955
- type: nauc_recall_at_10_diff1
value: 44.24689368792456
- type: nauc_recall_at_10_max
value: 30.05978573450996
- type: nauc_recall_at_10_std
value: 0.5954869714785576
- type: nauc_recall_at_1_diff1
value: 59.8215952283572
- type: nauc_recall_at_1_max
value: 33.12636725249456
- type: nauc_recall_at_1_std
value: -5.8855081788994426
- type: nauc_recall_at_20_diff1
value: 38.47479293773967
- type: nauc_recall_at_20_max
value: 30.075626342498296
- type: nauc_recall_at_20_std
value: 3.995253031595806
- type: nauc_recall_at_3_diff1
value: 48.27791887502807
- type: nauc_recall_at_3_max
value: 33.70850285999274
- type: nauc_recall_at_3_std
value: -1.2815912111364185
- type: nauc_recall_at_5_diff1
value: 47.33675444851813
- type: nauc_recall_at_5_max
value: 33.59000233724411
- type: nauc_recall_at_5_std
value: 0.6724142806879441
- type: ndcg_at_1
value: 37.921
- type: ndcg_at_10
value: 47.392
- type: ndcg_at_100
value: 52.798
- type: ndcg_at_1000
value: 54.684
- type: ndcg_at_20
value: 49.704
- type: ndcg_at_3
value: 42.241
- type: ndcg_at_5
value: 44.631
- type: precision_at_1
value: 37.921
- type: precision_at_10
value: 8.518
- type: precision_at_100
value: 1.3010000000000002
- type: precision_at_1000
value: 0.165
- type: precision_at_20
value: 5.005
- type: precision_at_3
value: 19.763
- type: precision_at_5
value: 14.052000000000001
- type: recall_at_1
value: 31.184
- type: recall_at_10
value: 59.265
- type: recall_at_100
value: 81.902
- type: recall_at_1000
value: 93.86
- type: recall_at_20
value: 67.393
- type: recall_at_3
value: 44.687
- type: recall_at_5
value: 50.985
task:
type: Retrieval
- dataset:
config: default
name: MTEB CQADupstackProgrammersRetrieval (default)
revision: 6184bc1440d2dbc7612be22b50686b8826d22b32
split: test
type: mteb/cqadupstack-programmers
metrics:
- type: main_score
value: 43.702000000000005
- type: map_at_1
value: 27.122
- type: map_at_10
value: 37.582
- type: map_at_100
value: 38.914
- type: map_at_1000
value: 39.019999999999996
- type: map_at_20
value: 38.321
- type: map_at_3
value: 34.528
- type: map_at_5
value: 36.09
- type: mrr_at_1
value: 34.01826484018265
- type: mrr_at_10
value: 43.263798289483205
- type: mrr_at_100
value: 44.1380956121081
- type: mrr_at_1000
value: 44.185697833855556
- type: mrr_at_20
value: 43.793797549721766
- type: mrr_at_3
value: 40.84855403348552
- type: mrr_at_5
value: 42.075722983257194
- type: nauc_map_at_1000_diff1
value: 48.54809280945058
- type: nauc_map_at_1000_max
value: 40.776528333405224
- type: nauc_map_at_1000_std
value: 6.661907895750691
- type: nauc_map_at_100_diff1
value: 48.53220732259419
- type: nauc_map_at_100_max
value: 40.777485555716844
- type: nauc_map_at_100_std
value: 6.667373289049959
- type: nauc_map_at_10_diff1
value: 48.696272225102724
- type: nauc_map_at_10_max
value: 40.16855196737864
- type: nauc_map_at_10_std
value: 5.699631954994352
- type: nauc_map_at_1_diff1
value: 55.02702675976629
- type: nauc_map_at_1_max
value: 36.26449598791867
- type: nauc_map_at_1_std
value: 0.732425634333215
- type: nauc_map_at_20_diff1
value: 48.33225604820497
- type: nauc_map_at_20_max
value: 40.439321766092654
- type: nauc_map_at_20_std
value: 6.188882132784543
- type: nauc_map_at_3_diff1
value: 48.849531596809356
- type: nauc_map_at_3_max
value: 38.9860445253163
- type: nauc_map_at_3_std
value: 4.144400337604119
- type: nauc_map_at_5_diff1
value: 49.18884784858743
- type: nauc_map_at_5_max
value: 39.655783829375565
- type: nauc_map_at_5_std
value: 4.817315072102742
- type: nauc_mrr_at_1000_diff1
value: 48.00305317802568
- type: nauc_mrr_at_1000_max
value: 44.23613484147676
- type: nauc_mrr_at_1000_std
value: 10.408238644010625
- type: nauc_mrr_at_100_diff1
value: 47.99496049306132
- type: nauc_mrr_at_100_max
value: 44.246498083388914
- type: nauc_mrr_at_100_std
value: 10.445977920073455
- type: nauc_mrr_at_10_diff1
value: 47.859228287955766
- type: nauc_mrr_at_10_max
value: 44.151674776960654
- type: nauc_mrr_at_10_std
value: 10.100723523960575
- type: nauc_mrr_at_1_diff1
value: 53.97500150046671
- type: nauc_mrr_at_1_max
value: 43.203684587322286
- type: nauc_mrr_at_1_std
value: 7.710806763535423
- type: nauc_mrr_at_20_diff1
value: 47.78264352952027
- type: nauc_mrr_at_20_max
value: 44.15886914310631
- type: nauc_mrr_at_20_std
value: 10.310916847695328
- type: nauc_mrr_at_3_diff1
value: 48.29090871013734
- type: nauc_mrr_at_3_max
value: 44.13906017331203
- type: nauc_mrr_at_3_std
value: 9.559567097555584
- type: nauc_mrr_at_5_diff1
value: 47.97269828112085
- type: nauc_mrr_at_5_max
value: 43.91612661455393
- type: nauc_mrr_at_5_std
value: 9.57617449812722
- type: nauc_ndcg_at_1000_diff1
value: 46.82941305510076
- type: nauc_ndcg_at_1000_max
value: 42.94251110430697
- type: nauc_ndcg_at_1000_std
value: 10.837155455040696
- type: nauc_ndcg_at_100_diff1
value: 46.55370804024034
- type: nauc_ndcg_at_100_max
value: 43.15479803311034
- type: nauc_ndcg_at_100_std
value: 11.64220068240346
- type: nauc_ndcg_at_10_diff1
value: 46.35684130144189
- type: nauc_ndcg_at_10_max
value: 41.59070439480394
- type: nauc_ndcg_at_10_std
value: 8.178046869165756
- type: nauc_ndcg_at_1_diff1
value: 53.97500150046671
- type: nauc_ndcg_at_1_max
value: 43.203684587322286
- type: nauc_ndcg_at_1_std
value: 7.710806763535423
- type: nauc_ndcg_at_20_diff1
value: 45.40626340072992
- type: nauc_ndcg_at_20_max
value: 42.09649444554957
- type: nauc_ndcg_at_20_std
value: 9.605463104740371
- type: nauc_ndcg_at_3_diff1
value: 46.39869019714142
- type: nauc_ndcg_at_3_max
value: 41.377156858610526
- type: nauc_ndcg_at_3_std
value: 7.0800717378487965
- type: nauc_ndcg_at_5_diff1
value: 47.128631288457996
- type: nauc_ndcg_at_5_max
value: 41.231380730849395
- type: nauc_ndcg_at_5_std
value: 6.870252693122028
- type: nauc_precision_at_1000_diff1
value: -12.683546963943378
- type: nauc_precision_at_1000_max
value: 2.168906725125969
- type: nauc_precision_at_1000_std
value: 12.048485072481888
- type: nauc_precision_at_100_diff1
value: 0.16871272463759232
- type: nauc_precision_at_100_max
value: 20.552672729747563
- type: nauc_precision_at_100_std
value: 23.488859046633483
- type: nauc_precision_at_10_diff1
value: 18.31525727659663
- type: nauc_precision_at_10_max
value: 35.98290060366415
- type: nauc_precision_at_10_std
value: 19.45559264624959
- type: nauc_precision_at_1_diff1
value: 53.97500150046671
- type: nauc_precision_at_1_max
value: 43.203684587322286
- type: nauc_precision_at_1_std
value: 7.710806763535423
- type: nauc_precision_at_20_diff1
value: 9.816958556392452
- type: nauc_precision_at_20_max
value: 32.44269638911288
- type: nauc_precision_at_20_std
value: 22.7695387792997
- type: nauc_precision_at_3_diff1
value: 30.831240534396088
- type: nauc_precision_at_3_max
value: 42.59739567911967
- type: nauc_precision_at_3_std
value: 15.434501692108938
- type: nauc_precision_at_5_diff1
value: 26.98589990414524
- type: nauc_precision_at_5_max
value: 39.18613048668369
- type: nauc_precision_at_5_std
value: 16.636453687202735
- type: nauc_recall_at_1000_diff1
value: 30.87302088559141
- type: nauc_recall_at_1000_max
value: 52.78156738930873
- type: nauc_recall_at_1000_std
value: 50.4397156770463
- type: nauc_recall_at_100_diff1
value: 34.931429900655544
- type: nauc_recall_at_100_max
value: 45.57822499214305
- type: nauc_recall_at_100_std
value: 33.67446897012978
- type: nauc_recall_at_10_diff1
value: 37.5976175865455
- type: nauc_recall_at_10_max
value: 38.5556407722093
- type: nauc_recall_at_10_std
value: 10.550126985646322
- type: nauc_recall_at_1_diff1
value: 55.02702675976629
- type: nauc_recall_at_1_max
value: 36.26449598791867
- type: nauc_recall_at_1_std
value: 0.732425634333215
- type: nauc_recall_at_20_diff1
value: 32.70441950285509
- type: nauc_recall_at_20_max
value: 39.69700118046791
- type: nauc_recall_at_20_std
value: 15.937760155492564
- type: nauc_recall_at_3_diff1
value: 40.94665128274178
- type: nauc_recall_at_3_max
value: 37.52094929329511
- type: nauc_recall_at_3_std
value: 5.2786717659493725
- type: nauc_recall_at_5_diff1
value: 41.33714477726403
- type: nauc_recall_at_5_max
value: 38.29879971662555
- type: nauc_recall_at_5_std
value: 6.268258432785931
- type: ndcg_at_1
value: 34.018
- type: ndcg_at_10
value: 43.702000000000005
- type: ndcg_at_100
value: 49.309
- type: ndcg_at_1000
value: 51.391
- type: ndcg_at_20
value: 45.897
- type: ndcg_at_3
value: 38.842999999999996
- type: ndcg_at_5
value: 40.72
- type: precision_at_1
value: 34.018
- type: precision_at_10
value: 8.082
- type: precision_at_100
value: 1.2590000000000001
- type: precision_at_1000
value: 0.16
- type: precision_at_20
value: 4.743
- type: precision_at_3
value: 18.798000000000002
- type: precision_at_5
value: 13.128
- type: recall_at_1
value: 27.122
- type: recall_at_10
value: 56.220000000000006
- type: recall_at_100
value: 80.328
- type: recall_at_1000
value: 94.28999999999999
- type: recall_at_20
value: 64.041
- type: recall_at_3
value: 42.257
- type: recall_at_5
value: 47.499
task:
type: Retrieval
- dataset:
config: default
name: MTEB CQADupstackRetrieval (default)
revision: CQADupstackRetrieval_is_a_combined_dataset
split: test
type: CQADupstackRetrieval_is_a_combined_dataset
metrics:
- type: main_score
value: 43.05741666666666
- type: ndcg_at_10
value: 43.05741666666666
task:
type: Retrieval
- dataset:
config: default
name: MTEB CQADupstackStatsRetrieval (default)
revision: 65ac3a16b8e91f9cee4c9828cc7c335575432a2a
split: test
type: mteb/cqadupstack-stats
metrics:
- type: main_score
value: 37.451
- type: map_at_1
value: 26.613999999999997
- type: map_at_10
value: 33.693
- type: map_at_100
value: 34.676
- type: map_at_1000
value: 34.78
- type: map_at_20
value: 34.271
- type: map_at_3
value: 31.95
- type: map_at_5
value: 33.001000000000005
- type: mrr_at_1
value: 29.601226993865033
- type: mrr_at_10
value: 36.36959538416591
- type: mrr_at_100
value: 37.210798693231375
- type: mrr_at_1000
value: 37.27823979110381
- type: mrr_at_20
value: 36.85899834554723
- type: mrr_at_3
value: 34.637014314928436
- type: mrr_at_5
value: 35.66462167689161
- type: nauc_map_at_1000_diff1
value: 49.37410791347397
- type: nauc_map_at_1000_max
value: 44.71830636526569
- type: nauc_map_at_1000_std
value: 6.33933824946907
- type: nauc_map_at_100_diff1
value: 49.340714029723124
- type: nauc_map_at_100_max
value: 44.67514800282011
- type: nauc_map_at_100_std
value: 6.316791709089924
- type: nauc_map_at_10_diff1
value: 49.58237553466596
- type: nauc_map_at_10_max
value: 44.405433944466694
- type: nauc_map_at_10_std
value: 5.420633435918038
- type: nauc_map_at_1_diff1
value: 56.857540982769386
- type: nauc_map_at_1_max
value: 45.23834250721237
- type: nauc_map_at_1_std
value: 1.802945533830376
- type: nauc_map_at_20_diff1
value: 49.32562290762172
- type: nauc_map_at_20_max
value: 44.545352441428896
- type: nauc_map_at_20_std
value: 6.021456769240318
- type: nauc_map_at_3_diff1
value: 51.09322731096909
- type: nauc_map_at_3_max
value: 44.79300881629309
- type: nauc_map_at_3_std
value: 4.011630043646612
- type: nauc_map_at_5_diff1
value: 49.97975659382682
- type: nauc_map_at_5_max
value: 44.22306761891805
- type: nauc_map_at_5_std
value: 5.003553888563371
- type: nauc_mrr_at_1000_diff1
value: 49.32837991155049
- type: nauc_mrr_at_1000_max
value: 45.93427489688434
- type: nauc_mrr_at_1000_std
value: 8.332345145460296
- type: nauc_mrr_at_100_diff1
value: 49.286286999726336
- type: nauc_mrr_at_100_max
value: 45.91561099870164
- type: nauc_mrr_at_100_std
value: 8.32831666531875
- type: nauc_mrr_at_10_diff1
value: 49.46015725591609
- type: nauc_mrr_at_10_max
value: 45.81782601823628
- type: nauc_mrr_at_10_std
value: 7.789698556495055
- type: nauc_mrr_at_1_diff1
value: 56.0872059675405
- type: nauc_mrr_at_1_max
value: 47.386480338399714
- type: nauc_mrr_at_1_std
value: 5.586277541713057
- type: nauc_mrr_at_20_diff1
value: 49.28417059076057
- type: nauc_mrr_at_20_max
value: 45.99725997530277
- type: nauc_mrr_at_20_std
value: 8.23052817007787
- type: nauc_mrr_at_3_diff1
value: 50.86538908994447
- type: nauc_mrr_at_3_max
value: 46.30931682738229
- type: nauc_mrr_at_3_std
value: 6.711465064236412
- type: nauc_mrr_at_5_diff1
value: 49.79097166622943
- type: nauc_mrr_at_5_max
value: 45.64379284105549
- type: nauc_mrr_at_5_std
value: 7.549376197213498
- type: nauc_ndcg_at_1000_diff1
value: 46.655241847522234
- type: nauc_ndcg_at_1000_max
value: 45.54057445564807
- type: nauc_ndcg_at_1000_std
value: 10.592785454436834
- type: nauc_ndcg_at_100_diff1
value: 45.97315906249984
- type: nauc_ndcg_at_100_max
value: 45.103821828576706
- type: nauc_ndcg_at_100_std
value: 10.823822034321271
- type: nauc_ndcg_at_10_diff1
value: 46.67685938503584
- type: nauc_ndcg_at_10_max
value: 43.95449931036324
- type: nauc_ndcg_at_10_std
value: 7.184463169195402
- type: nauc_ndcg_at_1_diff1
value: 56.0872059675405
- type: nauc_ndcg_at_1_max
value: 47.386480338399714
- type: nauc_ndcg_at_1_std
value: 5.586277541713057
- type: nauc_ndcg_at_20_diff1
value: 45.82228282738618
- type: nauc_ndcg_at_20_max
value: 44.43719295005832
- type: nauc_ndcg_at_20_std
value: 8.95789892457707
- type: nauc_ndcg_at_3_diff1
value: 49.37120917171752
- type: nauc_ndcg_at_3_max
value: 45.0413771949304
- type: nauc_ndcg_at_3_std
value: 5.179406852505315
- type: nauc_ndcg_at_5_diff1
value: 47.583182746063
- type: nauc_ndcg_at_5_max
value: 43.72295298313421
- type: nauc_ndcg_at_5_std
value: 6.5138412650415285
- type: nauc_precision_at_1000_diff1
value: 2.4617272692758774
- type: nauc_precision_at_1000_max
value: 22.936450605128723
- type: nauc_precision_at_1000_std
value: 19.21953983352782
- type: nauc_precision_at_100_diff1
value: 14.091495244420054
- type: nauc_precision_at_100_max
value: 36.22553109048817
- type: nauc_precision_at_100_std
value: 27.712752655919605
- type: nauc_precision_at_10_diff1
value: 25.69073847737471
- type: nauc_precision_at_10_max
value: 38.58233566502399
- type: nauc_precision_at_10_std
value: 15.934219646184086
- type: nauc_precision_at_1_diff1
value: 56.0872059675405
- type: nauc_precision_at_1_max
value: 47.386480338399714
- type: nauc_precision_at_1_std
value: 5.586277541713057
- type: nauc_precision_at_20_diff1
value: 20.21306431284034
- type: nauc_precision_at_20_max
value: 38.872513079451124
- type: nauc_precision_at_20_std
value: 22.150084369107148
- type: nauc_precision_at_3_diff1
value: 38.96362161150934
- type: nauc_precision_at_3_max
value: 43.842391385540274
- type: nauc_precision_at_3_std
value: 10.767834731788952
- type: nauc_precision_at_5_diff1
value: 32.690374841846115
- type: nauc_precision_at_5_max
value: 40.248174393941426
- type: nauc_precision_at_5_std
value: 15.139497820050835
- type: nauc_recall_at_1000_diff1
value: 29.9064034258407
- type: nauc_recall_at_1000_max
value: 45.385182668917615
- type: nauc_recall_at_1000_std
value: 35.31089190812142
- type: nauc_recall_at_100_diff1
value: 30.93677471473096
- type: nauc_recall_at_100_max
value: 42.49191296305999
- type: nauc_recall_at_100_std
value: 28.396218854730165
- type: nauc_recall_at_10_diff1
value: 37.65584211536675
- type: nauc_recall_at_10_max
value: 39.88064430621519
- type: nauc_recall_at_10_std
value: 9.578129253821126
- type: nauc_recall_at_1_diff1
value: 56.857540982769386
- type: nauc_recall_at_1_max
value: 45.23834250721237
- type: nauc_recall_at_1_std
value: 1.802945533830376
- type: nauc_recall_at_20_diff1
value: 33.296862029858254
- type: nauc_recall_at_20_max
value: 40.626394722604864
- type: nauc_recall_at_20_std
value: 15.646897719879163
- type: nauc_recall_at_3_diff1
value: 44.976755997237994
- type: nauc_recall_at_3_max
value: 42.3703322359472
- type: nauc_recall_at_3_std
value: 4.515124030573786
- type: nauc_recall_at_5_diff1
value: 40.24017796620052
- type: nauc_recall_at_5_max
value: 39.25075943429505
- type: nauc_recall_at_5_std
value: 7.8771910498658375
- type: ndcg_at_1
value: 29.601
- type: ndcg_at_10
value: 37.451
- type: ndcg_at_100
value: 42.099
- type: ndcg_at_1000
value: 44.493
- type: ndcg_at_20
value: 39.353
- type: ndcg_at_3
value: 34.266999999999996
- type: ndcg_at_5
value: 35.903
- type: precision_at_1
value: 29.601
- type: precision_at_10
value: 5.644
- type: precision_at_100
value: 0.857
- type: precision_at_1000
value: 0.11399999999999999
- type: precision_at_20
value: 3.29
- type: precision_at_3
value: 14.571000000000002
- type: precision_at_5
value: 9.908
- type: recall_at_1
value: 26.613999999999997
- type: recall_at_10
value: 46.047
- type: recall_at_100
value: 66.99199999999999
- type: recall_at_1000
value: 84.248
- type: recall_at_20
value: 53.236000000000004
- type: recall_at_3
value: 37.322
- type: recall_at_5
value: 41.589
task:
type: Retrieval
- dataset:
config: default
name: MTEB CQADupstackTexRetrieval (default)
revision: 46989137a86843e03a6195de44b09deda022eec7
split: test
type: mteb/cqadupstack-tex
metrics:
- type: main_score
value: 31.654
- type: map_at_1
value: 18.969
- type: map_at_10
value: 26.809
- type: map_at_100
value: 27.912
- type: map_at_1000
value: 28.035
- type: map_at_20
value: 27.371000000000002
- type: map_at_3
value: 24.486
- type: map_at_5
value: 25.654
- type: mrr_at_1
value: 22.883688919476946
- type: mrr_at_10
value: 30.661876358710526
- type: mrr_at_100
value: 31.573962917203158
- type: mrr_at_1000
value: 31.647366121679493
- type: mrr_at_20
value: 31.165428342513707
- type: mrr_at_3
value: 28.561596696490067
- type: mrr_at_5
value: 29.619752236751623
- type: nauc_map_at_1000_diff1
value: 42.37078386806742
- type: nauc_map_at_1000_max
value: 34.37145780392427
- type: nauc_map_at_1000_std
value: 2.068593126222689
- type: nauc_map_at_100_diff1
value: 42.345728925007506
- type: nauc_map_at_100_max
value: 34.34243452283795
- type: nauc_map_at_100_std
value: 2.065635415209545
- type: nauc_map_at_10_diff1
value: 42.794154300371254
- type: nauc_map_at_10_max
value: 34.21031260645627
- type: nauc_map_at_10_std
value: 1.529499265583685
- type: nauc_map_at_1_diff1
value: 50.618976883020984
- type: nauc_map_at_1_max
value: 33.44366905061848
- type: nauc_map_at_1_std
value: -0.2561553942461052
- type: nauc_map_at_20_diff1
value: 42.41518581158486
- type: nauc_map_at_20_max
value: 34.289849695696326
- type: nauc_map_at_20_std
value: 1.8427804797569582
- type: nauc_map_at_3_diff1
value: 44.10575895830247
- type: nauc_map_at_3_max
value: 34.496483048848724
- type: nauc_map_at_3_std
value: 0.8619761795903047
- type: nauc_map_at_5_diff1
value: 43.05223688723017
- type: nauc_map_at_5_max
value: 34.275490257564755
- type: nauc_map_at_5_std
value: 1.3357491095068115
- type: nauc_mrr_at_1000_diff1
value: 42.272024628061544
- type: nauc_mrr_at_1000_max
value: 35.702218886367085
- type: nauc_mrr_at_1000_std
value: 1.7919072868566186
- type: nauc_mrr_at_100_diff1
value: 42.2596894211699
- type: nauc_mrr_at_100_max
value: 35.69994903774682
- type: nauc_mrr_at_100_std
value: 1.7975713541531313
- type: nauc_mrr_at_10_diff1
value: 42.448163550272184
- type: nauc_mrr_at_10_max
value: 35.70237351804026
- type: nauc_mrr_at_10_std
value: 1.4122330103505247
- type: nauc_mrr_at_1_diff1
value: 49.485098062394485
- type: nauc_mrr_at_1_max
value: 35.74934353822275
- type: nauc_mrr_at_1_std
value: -0.6837069933233858
- type: nauc_mrr_at_20_diff1
value: 42.24078922524194
- type: nauc_mrr_at_20_max
value: 35.67506965997057
- type: nauc_mrr_at_20_std
value: 1.6826209417239275
- type: nauc_mrr_at_3_diff1
value: 43.40494299428709
- type: nauc_mrr_at_3_max
value: 36.20963468809214
- type: nauc_mrr_at_3_std
value: 0.7755723577443603
- type: nauc_mrr_at_5_diff1
value: 42.710330788610676
- type: nauc_mrr_at_5_max
value: 35.864570505319406
- type: nauc_mrr_at_5_std
value: 1.203946392444734
- type: nauc_ndcg_at_1000_diff1
value: 39.25825010007144
- type: nauc_ndcg_at_1000_max
value: 34.89017754877883
- type: nauc_ndcg_at_1000_std
value: 5.055264214158204
- type: nauc_ndcg_at_100_diff1
value: 38.853039089202404
- type: nauc_ndcg_at_100_max
value: 34.60799256623427
- type: nauc_ndcg_at_100_std
value: 5.09207024590202
- type: nauc_ndcg_at_10_diff1
value: 40.10143662798981
- type: nauc_ndcg_at_10_max
value: 34.19377736655788
- type: nauc_ndcg_at_10_std
value: 2.43022975944463
- type: nauc_ndcg_at_1_diff1
value: 49.485098062394485
- type: nauc_ndcg_at_1_max
value: 35.74934353822275
- type: nauc_ndcg_at_1_std
value: -0.6837069933233858
- type: nauc_ndcg_at_20_diff1
value: 39.080326954381775
- type: nauc_ndcg_at_20_max
value: 34.3351142863056
- type: nauc_ndcg_at_20_std
value: 3.521683030178308
- type: nauc_ndcg_at_3_diff1
value: 42.11543283456072
- type: nauc_ndcg_at_3_max
value: 35.192239360740004
- type: nauc_ndcg_at_3_std
value: 1.0692497979314295
- type: nauc_ndcg_at_5_diff1
value: 40.71206429908097
- type: nauc_ndcg_at_5_max
value: 34.5730228994852
- type: nauc_ndcg_at_5_std
value: 1.9772979663263806
- type: nauc_precision_at_1000_diff1
value: 3.5354535617208387
- type: nauc_precision_at_1000_max
value: 16.581113567699536
- type: nauc_precision_at_1000_std
value: 3.482270045420498
- type: nauc_precision_at_100_diff1
value: 9.572723289429502
- type: nauc_precision_at_100_max
value: 23.17116722759213
- type: nauc_precision_at_100_std
value: 8.860449930114486
- type: nauc_precision_at_10_diff1
value: 23.648840455312765
- type: nauc_precision_at_10_max
value: 30.278265843942837
- type: nauc_precision_at_10_std
value: 4.259105398374112
- type: nauc_precision_at_1_diff1
value: 49.485098062394485
- type: nauc_precision_at_1_max
value: 35.74934353822275
- type: nauc_precision_at_1_std
value: -0.6837069933233858
- type: nauc_precision_at_20_diff1
value: 17.48088981374241
- type: nauc_precision_at_20_max
value: 28.748893195730002
- type: nauc_precision_at_20_std
value: 6.561803745450412
- type: nauc_precision_at_3_diff1
value: 33.862032336074236
- type: nauc_precision_at_3_max
value: 35.79041119640643
- type: nauc_precision_at_3_std
value: 1.5937410892962152
- type: nauc_precision_at_5_diff1
value: 28.495710316153282
- type: nauc_precision_at_5_max
value: 33.427051980650184
- type: nauc_precision_at_5_std
value: 3.072561335447542
- type: nauc_recall_at_1000_diff1
value: 17.955316782859114
- type: nauc_recall_at_1000_max
value: 32.70702390514625
- type: nauc_recall_at_1000_std
value: 30.962165982690966
- type: nauc_recall_at_100_diff1
value: 24.194564580798453
- type: nauc_recall_at_100_max
value: 30.848715334135292
- type: nauc_recall_at_100_std
value: 17.751928934112463
- type: nauc_recall_at_10_diff1
value: 31.380996533162627
- type: nauc_recall_at_10_max
value: 30.43006883317946
- type: nauc_recall_at_10_std
value: 4.737791026720057
- type: nauc_recall_at_1_diff1
value: 50.618976883020984
- type: nauc_recall_at_1_max
value: 33.44366905061848
- type: nauc_recall_at_1_std
value: -0.2561553942461052
- type: nauc_recall_at_20_diff1
value: 27.543465285281833
- type: nauc_recall_at_20_max
value: 30.33419844248669
- type: nauc_recall_at_20_std
value: 8.32732288800037
- type: nauc_recall_at_3_diff1
value: 36.87479044768415
- type: nauc_recall_at_3_max
value: 33.27030983464594
- type: nauc_recall_at_3_std
value: 2.5793253070723625
- type: nauc_recall_at_5_diff1
value: 33.81298144138381
- type: nauc_recall_at_5_max
value: 32.02226031454496
- type: nauc_recall_at_5_std
value: 4.15867733492116
- type: ndcg_at_1
value: 22.884
- type: ndcg_at_10
value: 31.654
- type: ndcg_at_100
value: 36.909
- type: ndcg_at_1000
value: 39.748
- type: ndcg_at_20
value: 33.501
- type: ndcg_at_3
value: 27.511999999999997
- type: ndcg_at_5
value: 29.15
- type: precision_at_1
value: 22.884
- type: precision_at_10
value: 5.75
- type: precision_at_100
value: 0.991
- type: precision_at_1000
value: 0.14100000000000001
- type: precision_at_20
value: 3.438
- type: precision_at_3
value: 13.019
- type: precision_at_5
value: 9.195
- type: recall_at_1
value: 18.969
- type: recall_at_10
value: 42.327999999999996
- type: recall_at_100
value: 65.948
- type: recall_at_1000
value: 86.188
- type: recall_at_20
value: 49.152
- type: recall_at_3
value: 30.531000000000002
- type: recall_at_5
value: 34.775
task:
type: Retrieval
- dataset:
config: default
name: MTEB CQADupstackUnixRetrieval (default)
revision: 6c6430d3a6d36f8d2a829195bc5dc94d7e063e53
split: test
type: mteb/cqadupstack-unix
metrics:
- type: main_score
value: 43.3
- type: map_at_1
value: 28.841
- type: map_at_10
value: 38.034
- type: map_at_100
value: 39.238
- type: map_at_1000
value: 39.339
- type: map_at_20
value: 38.747
- type: map_at_3
value: 35.154
- type: map_at_5
value: 36.868
- type: mrr_at_1
value: 33.48880597014926
- type: mrr_at_10
value: 41.8903621772092
- type: mrr_at_100
value: 42.786281409284065
- type: mrr_at_1000
value: 42.836128521501664
- type: mrr_at_20
value: 42.44864446695604
- type: mrr_at_3
value: 39.36567164179104
- type: mrr_at_5
value: 40.88152985074623
- type: nauc_map_at_1000_diff1
value: 49.56159269351109
- type: nauc_map_at_1000_max
value: 43.419336352406
- type: nauc_map_at_1000_std
value: 1.2942357797191233
- type: nauc_map_at_100_diff1
value: 49.558107692839585
- type: nauc_map_at_100_max
value: 43.4108332375891
- type: nauc_map_at_100_std
value: 1.3081266310873843
- type: nauc_map_at_10_diff1
value: 49.80343449350858
- type: nauc_map_at_10_max
value: 43.15977018476029
- type: nauc_map_at_10_std
value: 0.8559971247760423
- type: nauc_map_at_1_diff1
value: 55.931076742985006
- type: nauc_map_at_1_max
value: 42.992320097398476
- type: nauc_map_at_1_std
value: -3.8362554822192627
- type: nauc_map_at_20_diff1
value: 49.632288848046365
- type: nauc_map_at_20_max
value: 43.3329280720487
- type: nauc_map_at_20_std
value: 1.1202828446838298
- type: nauc_map_at_3_diff1
value: 51.4868308340231
- type: nauc_map_at_3_max
value: 42.78769120741557
- type: nauc_map_at_3_std
value: -0.14398700922347876
- type: nauc_map_at_5_diff1
value: 50.29013429695699
- type: nauc_map_at_5_max
value: 42.856929797702506
- type: nauc_map_at_5_std
value: 0.4044555857303451
- type: nauc_mrr_at_1000_diff1
value: 48.59839196969955
- type: nauc_mrr_at_1000_max
value: 45.13238631682302
- type: nauc_mrr_at_1000_std
value: 1.7892755871367219
- type: nauc_mrr_at_100_diff1
value: 48.57126728231675
- type: nauc_mrr_at_100_max
value: 45.11947312378173
- type: nauc_mrr_at_100_std
value: 1.8039540672282843
- type: nauc_mrr_at_10_diff1
value: 48.47209224236492
- type: nauc_mrr_at_10_max
value: 44.99787083543906
- type: nauc_mrr_at_10_std
value: 1.4846086161409817
- type: nauc_mrr_at_1_diff1
value: 53.56968475977211
- type: nauc_mrr_at_1_max
value: 45.803534193599546
- type: nauc_mrr_at_1_std
value: -2.2181858062254145
- type: nauc_mrr_at_20_diff1
value: 48.57206460549483
- type: nauc_mrr_at_20_max
value: 45.07493477039947
- type: nauc_mrr_at_20_std
value: 1.693563490147581
- type: nauc_mrr_at_3_diff1
value: 49.740096383562076
- type: nauc_mrr_at_3_max
value: 45.42335868582593
- type: nauc_mrr_at_3_std
value: 0.9005835368626827
- type: nauc_mrr_at_5_diff1
value: 48.77683841696394
- type: nauc_mrr_at_5_max
value: 45.041633084239955
- type: nauc_mrr_at_5_std
value: 1.2903653442881702
- type: nauc_ndcg_at_1000_diff1
value: 46.73746959577089
- type: nauc_ndcg_at_1000_max
value: 44.258247648662795
- type: nauc_ndcg_at_1000_std
value: 4.154272793605619
- type: nauc_ndcg_at_100_diff1
value: 46.649174419902515
- type: nauc_ndcg_at_100_max
value: 44.251685454090754
- type: nauc_ndcg_at_100_std
value: 4.857119852669744
- type: nauc_ndcg_at_10_diff1
value: 47.15858525124804
- type: nauc_ndcg_at_10_max
value: 43.3858676772148
- type: nauc_ndcg_at_10_std
value: 2.700265540191218
- type: nauc_ndcg_at_1_diff1
value: 53.56968475977211
- type: nauc_ndcg_at_1_max
value: 45.803534193599546
- type: nauc_ndcg_at_1_std
value: -2.2181858062254145
- type: nauc_ndcg_at_20_diff1
value: 46.864934030796114
- type: nauc_ndcg_at_20_max
value: 43.682943174485914
- type: nauc_ndcg_at_20_std
value: 3.641120998700425
- type: nauc_ndcg_at_3_diff1
value: 49.597275186605124
- type: nauc_ndcg_at_3_max
value: 43.64798244009618
- type: nauc_ndcg_at_3_std
value: 1.256767705733725
- type: nauc_ndcg_at_5_diff1
value: 48.06101229681208
- type: nauc_ndcg_at_5_max
value: 43.0336705102281
- type: nauc_ndcg_at_5_std
value: 1.891806190702824
- type: nauc_precision_at_1000_diff1
value: -22.271945917482924
- type: nauc_precision_at_1000_max
value: -1.3615705776103613
- type: nauc_precision_at_1000_std
value: 2.6492562797064156
- type: nauc_precision_at_100_diff1
value: -7.230420637284722
- type: nauc_precision_at_100_max
value: 15.631818700346638
- type: nauc_precision_at_100_std
value: 12.05797323666178
- type: nauc_precision_at_10_diff1
value: 18.03973109968359
- type: nauc_precision_at_10_max
value: 33.64979451444693
- type: nauc_precision_at_10_std
value: 8.22230274561303
- type: nauc_precision_at_1_diff1
value: 53.56968475977211
- type: nauc_precision_at_1_max
value: 45.803534193599546
- type: nauc_precision_at_1_std
value: -2.2181858062254145
- type: nauc_precision_at_20_diff1
value: 8.575420211614727
- type: nauc_precision_at_20_max
value: 27.241489377360057
- type: nauc_precision_at_20_std
value: 11.040868656896368
- type: nauc_precision_at_3_diff1
value: 35.872211735265445
- type: nauc_precision_at_3_max
value: 40.39546303556062
- type: nauc_precision_at_3_std
value: 5.340493961821114
- type: nauc_precision_at_5_diff1
value: 27.482651262158498
- type: nauc_precision_at_5_max
value: 37.179977084686065
- type: nauc_precision_at_5_std
value: 6.445383767427032
- type: nauc_recall_at_1000_diff1
value: 22.90674388340293
- type: nauc_recall_at_1000_max
value: 48.618421794734665
- type: nauc_recall_at_1000_std
value: 37.066202126053085
- type: nauc_recall_at_100_diff1
value: 33.80092096185138
- type: nauc_recall_at_100_max
value: 43.86182876134091
- type: nauc_recall_at_100_std
value: 24.419734546074615
- type: nauc_recall_at_10_diff1
value: 38.91156254301526
- type: nauc_recall_at_10_max
value: 39.96637702386504
- type: nauc_recall_at_10_std
value: 7.619397668800581
- type: nauc_recall_at_1_diff1
value: 55.931076742985006
- type: nauc_recall_at_1_max
value: 42.992320097398476
- type: nauc_recall_at_1_std
value: -3.8362554822192627
- type: nauc_recall_at_20_diff1
value: 37.23708873841602
- type: nauc_recall_at_20_max
value: 40.39366037786138
- type: nauc_recall_at_20_std
value: 11.486382734050729
- type: nauc_recall_at_3_diff1
value: 46.55627480537957
- type: nauc_recall_at_3_max
value: 41.00557157142305
- type: nauc_recall_at_3_std
value: 3.5249163267069754
- type: nauc_recall_at_5_diff1
value: 42.18876270817466
- type: nauc_recall_at_5_max
value: 39.83406182096331
- type: nauc_recall_at_5_std
value: 5.414038646991049
- type: ndcg_at_1
value: 33.489000000000004
- type: ndcg_at_10
value: 43.3
- type: ndcg_at_100
value: 48.585
- type: ndcg_at_1000
value: 50.698
- type: ndcg_at_20
value: 45.562000000000005
- type: ndcg_at_3
value: 38.175
- type: ndcg_at_5
value: 40.760999999999996
- type: precision_at_1
value: 33.489000000000004
- type: precision_at_10
value: 7.1739999999999995
- type: precision_at_100
value: 1.105
- type: precision_at_1000
value: 0.13899999999999998
- type: precision_at_20
value: 4.234999999999999
- type: precision_at_3
value: 17.04
- type: precision_at_5
value: 12.052
- type: recall_at_1
value: 28.841
- type: recall_at_10
value: 55.347
- type: recall_at_100
value: 77.838
- type: recall_at_1000
value: 92.414
- type: recall_at_20
value: 63.343
- type: recall_at_3
value: 41.660000000000004
- type: recall_at_5
value: 48.041
task:
type: Retrieval
- dataset:
config: default
name: MTEB CQADupstackWebmastersRetrieval (default)
revision: 160c094312a0e1facb97e55eeddb698c0abe3571
split: test
type: mteb/cqadupstack-webmasters
metrics:
- type: main_score
value: 42.387
- type: map_at_1
value: 27.453
- type: map_at_10
value: 36.381
- type: map_at_100
value: 37.973
- type: map_at_1000
value: 38.195
- type: map_at_20
value: 37.212
- type: map_at_3
value: 33.021
- type: map_at_5
value: 34.921
- type: mrr_at_1
value: 32.608695652173914
- type: mrr_at_10
value: 40.989867620302384
- type: mrr_at_100
value: 41.94072291491216
- type: mrr_at_1000
value: 41.98613401421269
- type: mrr_at_20
value: 41.55934364742902
- type: mrr_at_3
value: 38.274044795783944
- type: mrr_at_5
value: 39.736495388669304
- type: nauc_map_at_1000_diff1
value: 52.068635517217146
- type: nauc_map_at_1000_max
value: 34.70466717596325
- type: nauc_map_at_1000_std
value: 8.564252372637736
- type: nauc_map_at_100_diff1
value: 51.82585568111554
- type: nauc_map_at_100_max
value: 34.82883414051153
- type: nauc_map_at_100_std
value: 8.489340935245284
- type: nauc_map_at_10_diff1
value: 52.20523670068483
- type: nauc_map_at_10_max
value: 34.47013712034222
- type: nauc_map_at_10_std
value: 7.317056278370855
- type: nauc_map_at_1_diff1
value: 57.26002072948526
- type: nauc_map_at_1_max
value: 33.992920262304196
- type: nauc_map_at_1_std
value: 2.85708312047863
- type: nauc_map_at_20_diff1
value: 51.91495706831645
- type: nauc_map_at_20_max
value: 34.70789481651518
- type: nauc_map_at_20_std
value: 7.700842499936993
- type: nauc_map_at_3_diff1
value: 52.87950808047478
- type: nauc_map_at_3_max
value: 34.24108231259805
- type: nauc_map_at_3_std
value: 6.1217951739715915
- type: nauc_map_at_5_diff1
value: 52.064983140167996
- type: nauc_map_at_5_max
value: 34.5142202774603
- type: nauc_map_at_5_std
value: 7.698580879272761
- type: nauc_mrr_at_1000_diff1
value: 51.37901015568489
- type: nauc_mrr_at_1000_max
value: 34.93888305124791
- type: nauc_mrr_at_1000_std
value: 9.396573371198718
- type: nauc_mrr_at_100_diff1
value: 51.36342086076231
- type: nauc_mrr_at_100_max
value: 34.92869241502653
- type: nauc_mrr_at_100_std
value: 9.411034792254869
- type: nauc_mrr_at_10_diff1
value: 51.50874768578031
- type: nauc_mrr_at_10_max
value: 34.87929769160853
- type: nauc_mrr_at_10_std
value: 8.962923737022022
- type: nauc_mrr_at_1_diff1
value: 56.18053114178021
- type: nauc_mrr_at_1_max
value: 36.10096515340783
- type: nauc_mrr_at_1_std
value: 5.764327995710113
- type: nauc_mrr_at_20_diff1
value: 51.323303376004425
- type: nauc_mrr_at_20_max
value: 34.90401963870763
- type: nauc_mrr_at_20_std
value: 9.123385008963087
- type: nauc_mrr_at_3_diff1
value: 51.41750772667899
- type: nauc_mrr_at_3_max
value: 34.194724327364156
- type: nauc_mrr_at_3_std
value: 8.47755021610427
- type: nauc_mrr_at_5_diff1
value: 51.17838125129989
- type: nauc_mrr_at_5_max
value: 34.83854353084797
- type: nauc_mrr_at_5_std
value: 9.90813617252476
- type: nauc_ndcg_at_1000_diff1
value: 50.12742074119453
- type: nauc_ndcg_at_1000_max
value: 35.55751986886828
- type: nauc_ndcg_at_1000_std
value: 11.771760137767643
- type: nauc_ndcg_at_100_diff1
value: 48.788141047586194
- type: nauc_ndcg_at_100_max
value: 34.86794041942259
- type: nauc_ndcg_at_100_std
value: 11.961059836221457
- type: nauc_ndcg_at_10_diff1
value: 50.61505051581248
- type: nauc_ndcg_at_10_max
value: 34.00830131568619
- type: nauc_ndcg_at_10_std
value: 9.106607296489782
- type: nauc_ndcg_at_1_diff1
value: 56.18053114178021
- type: nauc_ndcg_at_1_max
value: 36.10096515340783
- type: nauc_ndcg_at_1_std
value: 5.764327995710113
- type: nauc_ndcg_at_20_diff1
value: 49.84654455929144
- type: nauc_ndcg_at_20_max
value: 34.4467716270897
- type: nauc_ndcg_at_20_std
value: 9.538489315626752
- type: nauc_ndcg_at_3_diff1
value: 50.63005106573348
- type: nauc_ndcg_at_3_max
value: 33.85867832031728
- type: nauc_ndcg_at_3_std
value: 8.17428250068972
- type: nauc_ndcg_at_5_diff1
value: 49.96013415746643
- type: nauc_ndcg_at_5_max
value: 34.502255185100054
- type: nauc_ndcg_at_5_std
value: 10.8726719718032
- type: nauc_precision_at_1000_diff1
value: 11.538506412200755
- type: nauc_precision_at_1000_max
value: -10.27094203158407
- type: nauc_precision_at_1000_std
value: 8.253706747664506
- type: nauc_precision_at_100_diff1
value: 10.274499589292818
- type: nauc_precision_at_100_max
value: 4.4464873223864
- type: nauc_precision_at_100_std
value: 18.15979211460786
- type: nauc_precision_at_10_diff1
value: 21.080068548327347
- type: nauc_precision_at_10_max
value: 20.678194765395684
- type: nauc_precision_at_10_std
value: 14.969268386190768
- type: nauc_precision_at_1_diff1
value: 56.18053114178021
- type: nauc_precision_at_1_max
value: 36.10096515340783
- type: nauc_precision_at_1_std
value: 5.764327995710113
- type: nauc_precision_at_20_diff1
value: 13.16668502783464
- type: nauc_precision_at_20_max
value: 16.210494253246534
- type: nauc_precision_at_20_std
value: 16.170849126443827
- type: nauc_precision_at_3_diff1
value: 34.634736242337006
- type: nauc_precision_at_3_max
value: 27.79912762487075
- type: nauc_precision_at_3_std
value: 11.56645534528478
- type: nauc_precision_at_5_diff1
value: 26.5364287176931
- type: nauc_precision_at_5_max
value: 25.792037842955466
- type: nauc_precision_at_5_std
value: 16.880419851911416
- type: nauc_recall_at_1000_diff1
value: 28.098675584638883
- type: nauc_recall_at_1000_max
value: 55.58850042879164
- type: nauc_recall_at_1000_std
value: 59.45492072811959
- type: nauc_recall_at_100_diff1
value: 26.37132408196824
- type: nauc_recall_at_100_max
value: 28.6914597857662
- type: nauc_recall_at_100_std
value: 30.16956640881801
- type: nauc_recall_at_10_diff1
value: 43.77304791334009
- type: nauc_recall_at_10_max
value: 30.424605236232026
- type: nauc_recall_at_10_std
value: 9.051646627698911
- type: nauc_recall_at_1_diff1
value: 57.26002072948526
- type: nauc_recall_at_1_max
value: 33.992920262304196
- type: nauc_recall_at_1_std
value: 2.85708312047863
- type: nauc_recall_at_20_diff1
value: 39.75031838508006
- type: nauc_recall_at_20_max
value: 31.14158866813173
- type: nauc_recall_at_20_std
value: 12.041755220788037
- type: nauc_recall_at_3_diff1
value: 46.232619145437745
- type: nauc_recall_at_3_max
value: 31.58015164282607
- type: nauc_recall_at_3_std
value: 8.728672179092152
- type: nauc_recall_at_5_diff1
value: 43.337420889005806
- type: nauc_recall_at_5_max
value: 32.28988058513371
- type: nauc_recall_at_5_std
value: 13.670606574574675
- type: ndcg_at_1
value: 32.609
- type: ndcg_at_10
value: 42.387
- type: ndcg_at_100
value: 48.192
- type: ndcg_at_1000
value: 50.480000000000004
- type: ndcg_at_20
value: 44.588
- type: ndcg_at_3
value: 37.080999999999996
- type: ndcg_at_5
value: 39.641
- type: precision_at_1
value: 32.609
- type: precision_at_10
value: 8.202
- type: precision_at_100
value: 1.5709999999999997
- type: precision_at_1000
value: 0.242
- type: precision_at_20
value: 5.148
- type: precision_at_3
value: 17.26
- type: precision_at_5
value: 12.806000000000001
- type: recall_at_1
value: 27.453
- type: recall_at_10
value: 53.642
- type: recall_at_100
value: 80.142
- type: recall_at_1000
value: 94.045
- type: recall_at_20
value: 62.138000000000005
- type: recall_at_3
value: 38.896
- type: recall_at_5
value: 45.498
task:
type: Retrieval
- dataset:
config: default
name: MTEB CQADupstackWordpressRetrieval (default)
revision: 4ffe81d471b1924886b33c7567bfb200e9eec5c4
split: test
type: mteb/cqadupstack-wordpress
metrics:
- type: main_score
value: 35.278999999999996
- type: map_at_1
value: 22.927
- type: map_at_10
value: 30.563000000000002
- type: map_at_100
value: 31.552999999999997
- type: map_at_1000
value: 31.651
- type: map_at_20
value: 31.04
- type: map_at_3
value: 27.962999999999997
- type: map_at_5
value: 29.389
- type: mrr_at_1
value: 24.58410351201479
- type: mrr_at_10
value: 32.5369685767098
- type: mrr_at_100
value: 33.397775856693876
- type: mrr_at_1000
value: 33.46464817817391
- type: mrr_at_20
value: 32.95268685553276
- type: mrr_at_3
value: 30.16019716574246
- type: mrr_at_5
value: 31.472581638940238
- type: nauc_map_at_1000_diff1
value: 47.29695789090712
- type: nauc_map_at_1000_max
value: 32.28518369534312
- type: nauc_map_at_1000_std
value: -2.5126907903497275
- type: nauc_map_at_100_diff1
value: 47.23959114706962
- type: nauc_map_at_100_max
value: 32.27344155453334
- type: nauc_map_at_100_std
value: -2.5533625745048263
- type: nauc_map_at_10_diff1
value: 47.37396108245503
- type: nauc_map_at_10_max
value: 32.126004121683216
- type: nauc_map_at_10_std
value: -3.453898841871271
- type: nauc_map_at_1_diff1
value: 55.682159271913456
- type: nauc_map_at_1_max
value: 30.66889816568006
- type: nauc_map_at_1_std
value: -7.004520095092218
- type: nauc_map_at_20_diff1
value: 47.389725912182136
- type: nauc_map_at_20_max
value: 32.259511388238174
- type: nauc_map_at_20_std
value: -2.8457788997344213
- type: nauc_map_at_3_diff1
value: 48.98298294521397
- type: nauc_map_at_3_max
value: 32.53058789285204
- type: nauc_map_at_3_std
value: -3.1342896986184097
- type: nauc_map_at_5_diff1
value: 48.298584104750674
- type: nauc_map_at_5_max
value: 32.407236632127095
- type: nauc_map_at_5_std
value: -3.2089294997536495
- type: nauc_mrr_at_1000_diff1
value: 47.30210193406738
- type: nauc_mrr_at_1000_max
value: 33.401728880120324
- type: nauc_mrr_at_1000_std
value: -1.154137716273939
- type: nauc_mrr_at_100_diff1
value: 47.23675453288927
- type: nauc_mrr_at_100_max
value: 33.395265508702124
- type: nauc_mrr_at_100_std
value: -1.1709832550651733
- type: nauc_mrr_at_10_diff1
value: 47.27878223121543
- type: nauc_mrr_at_10_max
value: 33.340206772572856
- type: nauc_mrr_at_10_std
value: -1.763755096880497
- type: nauc_mrr_at_1_diff1
value: 56.71642453837117
- type: nauc_mrr_at_1_max
value: 32.57395754414265
- type: nauc_mrr_at_1_std
value: -6.410603991729164
- type: nauc_mrr_at_20_diff1
value: 47.29854351132387
- type: nauc_mrr_at_20_max
value: 33.41217871789473
- type: nauc_mrr_at_20_std
value: -1.3457234524473638
- type: nauc_mrr_at_3_diff1
value: 49.37951376805829
- type: nauc_mrr_at_3_max
value: 34.2485630104482
- type: nauc_mrr_at_3_std
value: -1.3000132041170331
- type: nauc_mrr_at_5_diff1
value: 48.294954276400354
- type: nauc_mrr_at_5_max
value: 33.325321505802705
- type: nauc_mrr_at_5_std
value: -1.4542576597515255
- type: nauc_ndcg_at_1000_diff1
value: 43.553592194630035
- type: nauc_ndcg_at_1000_max
value: 32.57827057061685
- type: nauc_ndcg_at_1000_std
value: 1.8408475554303905
- type: nauc_ndcg_at_100_diff1
value: 42.17840889367743
- type: nauc_ndcg_at_100_max
value: 31.816803664903016
- type: nauc_ndcg_at_100_std
value: 1.2078284672957822
- type: nauc_ndcg_at_10_diff1
value: 43.31249469655541
- type: nauc_ndcg_at_10_max
value: 31.680660418442848
- type: nauc_ndcg_at_10_std
value: -2.571767770985386
- type: nauc_ndcg_at_1_diff1
value: 56.71642453837117
- type: nauc_ndcg_at_1_max
value: 32.57395754414265
- type: nauc_ndcg_at_1_std
value: -6.410603991729164
- type: nauc_ndcg_at_20_diff1
value: 43.217307154367866
- type: nauc_ndcg_at_20_max
value: 31.87062240427328
- type: nauc_ndcg_at_20_std
value: -0.5571171459253301
- type: nauc_ndcg_at_3_diff1
value: 47.08070710904174
- type: nauc_ndcg_at_3_max
value: 33.28022740280366
- type: nauc_ndcg_at_3_std
value: -1.494480342132224
- type: nauc_ndcg_at_5_diff1
value: 45.58676699653949
- type: nauc_ndcg_at_5_max
value: 32.332950617032516
- type: nauc_ndcg_at_5_std
value: -1.8728327940722693
- type: nauc_precision_at_1000_diff1
value: -7.538068423020358
- type: nauc_precision_at_1000_max
value: -1.7482377513247687
- type: nauc_precision_at_1000_std
value: 15.392390427048127
- type: nauc_precision_at_100_diff1
value: 4.578967237145689
- type: nauc_precision_at_100_max
value: 20.413498463420208
- type: nauc_precision_at_100_std
value: 21.948374431632555
- type: nauc_precision_at_10_diff1
value: 25.369687526079133
- type: nauc_precision_at_10_max
value: 29.026062761609634
- type: nauc_precision_at_10_std
value: 2.98198964045951
- type: nauc_precision_at_1_diff1
value: 56.71642453837117
- type: nauc_precision_at_1_max
value: 32.57395754414265
- type: nauc_precision_at_1_std
value: -6.410603991729164
- type: nauc_precision_at_20_diff1
value: 22.283416541821058
- type: nauc_precision_at_20_max
value: 27.425436939803216
- type: nauc_precision_at_20_std
value: 12.5008954948979
- type: nauc_precision_at_3_diff1
value: 39.582854814224284
- type: nauc_precision_at_3_max
value: 34.65501605346027
- type: nauc_precision_at_3_std
value: 3.1189467500344685
- type: nauc_precision_at_5_diff1
value: 34.87308543367999
- type: nauc_precision_at_5_max
value: 32.3395405130254
- type: nauc_precision_at_5_std
value: 2.2174849755453696
- type: nauc_recall_at_1000_diff1
value: 18.961781191263412
- type: nauc_recall_at_1000_max
value: 38.56189893918596
- type: nauc_recall_at_1000_std
value: 48.070723669295404
- type: nauc_recall_at_100_diff1
value: 18.499768281597692
- type: nauc_recall_at_100_max
value: 24.556122444473104
- type: nauc_recall_at_100_std
value: 16.90279820495104
- type: nauc_recall_at_10_diff1
value: 30.85581046640506
- type: nauc_recall_at_10_max
value: 27.861667996688798
- type: nauc_recall_at_10_std
value: -2.508305700409507
- type: nauc_recall_at_1_diff1
value: 55.682159271913456
- type: nauc_recall_at_1_max
value: 30.66889816568006
- type: nauc_recall_at_1_std
value: -7.004520095092218
- type: nauc_recall_at_20_diff1
value: 29.576541753550256
- type: nauc_recall_at_20_max
value: 27.753263778078967
- type: nauc_recall_at_20_std
value: 4.512246059861677
- type: nauc_recall_at_3_diff1
value: 41.31393053559837
- type: nauc_recall_at_3_max
value: 32.61483699324787
- type: nauc_recall_at_3_std
value: 0.4749359651904873
- type: nauc_recall_at_5_diff1
value: 37.709930638516084
- type: nauc_recall_at_5_max
value: 30.562381066906735
- type: nauc_recall_at_5_std
value: 0.12942001695231264
- type: ndcg_at_1
value: 24.584
- type: ndcg_at_10
value: 35.278999999999996
- type: ndcg_at_100
value: 40.346
- type: ndcg_at_1000
value: 42.763
- type: ndcg_at_20
value: 36.827
- type: ndcg_at_3
value: 30.175
- type: ndcg_at_5
value: 32.592999999999996
- type: precision_at_1
value: 24.584
- type: precision_at_10
value: 5.545
- type: precision_at_100
value: 0.876
- type: precision_at_1000
value: 0.11900000000000001
- type: precision_at_20
value: 3.152
- type: precision_at_3
value: 12.692999999999998
- type: precision_at_5
value: 9.131
- type: recall_at_1
value: 22.927
- type: recall_at_10
value: 47.887
- type: recall_at_100
value: 71.622
- type: recall_at_1000
value: 89.562
- type: recall_at_20
value: 53.55799999999999
- type: recall_at_3
value: 34.122
- type: recall_at_5
value: 39.911
task:
type: Retrieval
- dataset:
config: default
name: MTEB ClimateFEVER (default)
revision: 47f2ac6acb640fc46020b02a5b59fdda04d39380
split: test
type: mteb/climate-fever
metrics:
- type: main_score
value: 34.403
- type: map_at_1
value: 14.979999999999999
- type: map_at_10
value: 25.247000000000003
- type: map_at_100
value: 27.359
- type: map_at_1000
value: 27.541
- type: map_at_20
value: 26.451
- type: map_at_3
value: 21.351
- type: map_at_5
value: 23.418
- type: mrr_at_1
value: 33.48534201954397
- type: mrr_at_10
value: 45.994519414714894
- type: mrr_at_100
value: 46.83876242563609
- type: mrr_at_1000
value: 46.86734584208047
- type: mrr_at_20
value: 46.554675709164414
- type: mrr_at_3
value: 43.13789359391967
- type: mrr_at_5
value: 44.958740499457164
- type: nauc_map_at_1000_diff1
value: 24.542868591864057
- type: nauc_map_at_1000_max
value: 35.17062530519406
- type: nauc_map_at_1000_std
value: 17.59095330905714
- type: nauc_map_at_100_diff1
value: 24.54188790277331
- type: nauc_map_at_100_max
value: 35.14563117030601
- type: nauc_map_at_100_std
value: 17.53061487401564
- type: nauc_map_at_10_diff1
value: 24.307528834481303
- type: nauc_map_at_10_max
value: 34.19666824907002
- type: nauc_map_at_10_std
value: 15.809045770659601
- type: nauc_map_at_1_diff1
value: 31.137087129986767
- type: nauc_map_at_1_max
value: 30.743143219710895
- type: nauc_map_at_1_std
value: 9.269238234693052
- type: nauc_map_at_20_diff1
value: 24.37307987537168
- type: nauc_map_at_20_max
value: 34.816188656881444
- type: nauc_map_at_20_std
value: 16.96006173956885
- type: nauc_map_at_3_diff1
value: 24.40373599590744
- type: nauc_map_at_3_max
value: 31.428402086020434
- type: nauc_map_at_3_std
value: 11.510843615987866
- type: nauc_map_at_5_diff1
value: 24.445835117925792
- type: nauc_map_at_5_max
value: 33.2187974026072
- type: nauc_map_at_5_std
value: 14.407916582300887
- type: nauc_mrr_at_1000_diff1
value: 25.586480699800468
- type: nauc_mrr_at_1000_max
value: 32.91547098008033
- type: nauc_mrr_at_1000_std
value: 18.98828976122537
- type: nauc_mrr_at_100_diff1
value: 25.58686333508224
- type: nauc_mrr_at_100_max
value: 32.921529030276446
- type: nauc_mrr_at_100_std
value: 19.0030031564948
- type: nauc_mrr_at_10_diff1
value: 25.628991155936333
- type: nauc_mrr_at_10_max
value: 32.985081917104935
- type: nauc_mrr_at_10_std
value: 18.83885513960198
- type: nauc_mrr_at_1_diff1
value: 28.93673684578716
- type: nauc_mrr_at_1_max
value: 28.64745715409404
- type: nauc_mrr_at_1_std
value: 13.863643876558179
- type: nauc_mrr_at_20_diff1
value: 25.49368963472655
- type: nauc_mrr_at_20_max
value: 32.99074141657497
- type: nauc_mrr_at_20_std
value: 19.118893763525108
- type: nauc_mrr_at_3_diff1
value: 24.74979519660734
- type: nauc_mrr_at_3_max
value: 31.18346592120374
- type: nauc_mrr_at_3_std
value: 17.293916415010717
- type: nauc_mrr_at_5_diff1
value: 25.28131818602093
- type: nauc_mrr_at_5_max
value: 32.8017964540695
- type: nauc_mrr_at_5_std
value: 18.555047862215567
- type: nauc_ndcg_at_1000_diff1
value: 24.01037436102054
- type: nauc_ndcg_at_1000_max
value: 38.29644657817331
- type: nauc_ndcg_at_1000_std
value: 24.638778854093943
- type: nauc_ndcg_at_100_diff1
value: 23.882486615814315
- type: nauc_ndcg_at_100_max
value: 37.92277276436577
- type: nauc_ndcg_at_100_std
value: 24.169078377315063
- type: nauc_ndcg_at_10_diff1
value: 23.3573409673849
- type: nauc_ndcg_at_10_max
value: 35.72106349972407
- type: nauc_ndcg_at_10_std
value: 19.650154262568567
- type: nauc_ndcg_at_1_diff1
value: 28.93673684578716
- type: nauc_ndcg_at_1_max
value: 28.64745715409404
- type: nauc_ndcg_at_1_std
value: 13.863643876558179
- type: nauc_ndcg_at_20_diff1
value: 23.266046860547384
- type: nauc_ndcg_at_20_max
value: 36.81425896021297
- type: nauc_ndcg_at_20_std
value: 22.081396808708813
- type: nauc_ndcg_at_3_diff1
value: 22.637164244020997
- type: nauc_ndcg_at_3_max
value: 31.02842640810694
- type: nauc_ndcg_at_3_std
value: 14.215242033737482
- type: nauc_ndcg_at_5_diff1
value: 23.382160877142226
- type: nauc_ndcg_at_5_max
value: 34.50791441078589
- type: nauc_ndcg_at_5_std
value: 17.668301876879035
- type: nauc_precision_at_1000_diff1
value: -1.0805315448138373
- type: nauc_precision_at_1000_max
value: 11.905658299097883
- type: nauc_precision_at_1000_std
value: 24.219725240972696
- type: nauc_precision_at_100_diff1
value: 5.443379527828862
- type: nauc_precision_at_100_max
value: 22.475166562881686
- type: nauc_precision_at_100_std
value: 30.513578022558836
- type: nauc_precision_at_10_diff1
value: 10.939925455527892
- type: nauc_precision_at_10_max
value: 28.262686645352296
- type: nauc_precision_at_10_std
value: 25.539072940208847
- type: nauc_precision_at_1_diff1
value: 28.93673684578716
- type: nauc_precision_at_1_max
value: 28.64745715409404
- type: nauc_precision_at_1_std
value: 13.863643876558179
- type: nauc_precision_at_20_diff1
value: 8.77330841908142
- type: nauc_precision_at_20_max
value: 27.23086563249434
- type: nauc_precision_at_20_std
value: 28.96826995725968
- type: nauc_precision_at_3_diff1
value: 14.330928530870892
- type: nauc_precision_at_3_max
value: 28.36721121797032
- type: nauc_precision_at_3_std
value: 18.29234384665724
- type: nauc_precision_at_5_diff1
value: 13.483807784533445
- type: nauc_precision_at_5_max
value: 30.607108190727843
- type: nauc_precision_at_5_std
value: 24.649762580353933
- type: nauc_recall_at_1000_diff1
value: 15.147591049801376
- type: nauc_recall_at_1000_max
value: 44.69788852785355
- type: nauc_recall_at_1000_std
value: 45.03338261203177
- type: nauc_recall_at_100_diff1
value: 16.198695661693638
- type: nauc_recall_at_100_max
value: 37.94910180965323
- type: nauc_recall_at_100_std
value: 33.68808458543408
- type: nauc_recall_at_10_diff1
value: 17.25892260883711
- type: nauc_recall_at_10_max
value: 34.82455542076449
- type: nauc_recall_at_10_std
value: 21.559999882903846
- type: nauc_recall_at_1_diff1
value: 31.137087129986767
- type: nauc_recall_at_1_max
value: 30.743143219710895
- type: nauc_recall_at_1_std
value: 9.269238234693052
- type: nauc_recall_at_20_diff1
value: 15.853627315180349
- type: nauc_recall_at_20_max
value: 35.4402828692619
- type: nauc_recall_at_20_std
value: 26.303726125092197
- type: nauc_recall_at_3_diff1
value: 18.612092821372865
- type: nauc_recall_at_3_max
value: 30.195467018936846
- type: nauc_recall_at_3_std
value: 12.653864970927122
- type: nauc_recall_at_5_diff1
value: 17.968083464057948
- type: nauc_recall_at_5_max
value: 33.66376192344074
- type: nauc_recall_at_5_std
value: 18.31679422658197
- type: ndcg_at_1
value: 33.485
- type: ndcg_at_10
value: 34.403
- type: ndcg_at_100
value: 42.016
- type: ndcg_at_1000
value: 45.18
- type: ndcg_at_20
value: 37.611
- type: ndcg_at_3
value: 29.154999999999998
- type: ndcg_at_5
value: 30.989
- type: precision_at_1
value: 33.485
- type: precision_at_10
value: 10.593
- type: precision_at_100
value: 1.863
- type: precision_at_1000
value: 0.246
- type: precision_at_20
value: 6.658
- type: precision_at_3
value: 21.823999999999998
- type: precision_at_5
value: 16.482
- type: recall_at_1
value: 14.979999999999999
- type: recall_at_10
value: 39.641
- type: recall_at_100
value: 65.556
- type: recall_at_1000
value: 83.202
- type: recall_at_20
value: 48.701
- type: recall_at_3
value: 26.448
- type: recall_at_5
value: 32.309
task:
type: Retrieval
- dataset:
config: default
name: MTEB DBPedia (default)
revision: c0f706b76e590d620bd6618b3ca8efdd34e2d659
split: dev
type: mteb/dbpedia
metrics:
- type: main_score
value: 50.646
- type: map_at_1
value: 11.03
- type: map_at_10
value: 23.942
- type: map_at_100
value: 33.595000000000006
- type: map_at_1000
value: 35.406
- type: map_at_20
value: 27.947
- type: map_at_3
value: 16.21
- type: map_at_5
value: 19.158
- type: mrr_at_1
value: 77.61194029850746
- type: mrr_at_10
value: 83.80597014925374
- type: mrr_at_100
value: 83.94018280689576
- type: mrr_at_1000
value: 83.95212310540323
- type: mrr_at_20
value: 83.90547263681594
- type: mrr_at_3
value: 82.83582089552239
- type: mrr_at_5
value: 83.50746268656717
- type: nauc_map_at_1000_diff1
value: 46.749813822095994
- type: nauc_map_at_1000_max
value: 13.82268550007058
- type: nauc_map_at_1000_std
value: -0.41938982386833096
- type: nauc_map_at_100_diff1
value: 47.693546390121114
- type: nauc_map_at_100_max
value: 12.65050079951075
- type: nauc_map_at_100_std
value: -4.780014876773591
- type: nauc_map_at_10_diff1
value: 61.4710131705618
- type: nauc_map_at_10_max
value: 3.6540180261579187
- type: nauc_map_at_10_std
value: -28.94512699926511
- type: nauc_map_at_1_diff1
value: 78.97150551563644
- type: nauc_map_at_1_max
value: -6.130504498165464
- type: nauc_map_at_1_std
value: -40.805677041559186
- type: nauc_map_at_20_diff1
value: 55.21575059361192
- type: nauc_map_at_20_max
value: 6.2670759673565595
- type: nauc_map_at_20_std
value: -20.716907815975507
- type: nauc_map_at_3_diff1
value: 69.45845841968335
- type: nauc_map_at_3_max
value: -0.4053208701402535
- type: nauc_map_at_3_std
value: -38.392413410451695
- type: nauc_map_at_5_diff1
value: 65.25982068137156
- type: nauc_map_at_5_max
value: -0.23378047675269126
- type: nauc_map_at_5_std
value: -36.77011968366825
- type: nauc_mrr_at_1000_diff1
value: 44.16607496878154
- type: nauc_mrr_at_1000_max
value: 53.26474383081633
- type: nauc_mrr_at_1000_std
value: 44.64310496263336
- type: nauc_mrr_at_100_diff1
value: 44.11702190018642
- type: nauc_mrr_at_100_max
value: 53.27990989378437
- type: nauc_mrr_at_100_std
value: 44.66515255627964
- type: nauc_mrr_at_10_diff1
value: 44.400885532001745
- type: nauc_mrr_at_10_max
value: 53.6953402346631
- type: nauc_mrr_at_10_std
value: 44.97435307981567
- type: nauc_mrr_at_1_diff1
value: 46.27833298543312
- type: nauc_mrr_at_1_max
value: 42.95527694566197
- type: nauc_mrr_at_1_std
value: 38.34819072902029
- type: nauc_mrr_at_20_diff1
value: 44.21597352730565
- type: nauc_mrr_at_20_max
value: 53.38806174125155
- type: nauc_mrr_at_20_std
value: 44.79324661531812
- type: nauc_mrr_at_3_diff1
value: 43.66224705824827
- type: nauc_mrr_at_3_max
value: 53.23180791903105
- type: nauc_mrr_at_3_std
value: 44.43349044702786
- type: nauc_mrr_at_5_diff1
value: 43.528095394202474
- type: nauc_mrr_at_5_max
value: 54.03440152679073
- type: nauc_mrr_at_5_std
value: 44.79621802809768
- type: nauc_ndcg_at_1000_diff1
value: 43.48246353308846
- type: nauc_ndcg_at_1000_max
value: 31.816579886863018
- type: nauc_ndcg_at_1000_std
value: 24.158694249746574
- type: nauc_ndcg_at_100_diff1
value: 44.625167084995056
- type: nauc_ndcg_at_100_max
value: 26.395105829067983
- type: nauc_ndcg_at_100_std
value: 10.363304253958725
- type: nauc_ndcg_at_10_diff1
value: 45.86908731084752
- type: nauc_ndcg_at_10_max
value: 22.48203483834841
- type: nauc_ndcg_at_10_std
value: 6.2909997839301
- type: nauc_ndcg_at_1_diff1
value: 43.19904519276495
- type: nauc_ndcg_at_1_max
value: 39.28539458094619
- type: nauc_ndcg_at_1_std
value: 18.055622171581145
- type: nauc_ndcg_at_20_diff1
value: 46.49447236881872
- type: nauc_ndcg_at_20_max
value: 19.266991376566384
- type: nauc_ndcg_at_20_std
value: -2.421816962951546
- type: nauc_ndcg_at_3_diff1
value: 34.7260679671906
- type: nauc_ndcg_at_3_max
value: 31.13081490912525
- type: nauc_ndcg_at_3_std
value: 14.246240436306778
- type: nauc_ndcg_at_5_diff1
value: 38.26289083847805
- type: nauc_ndcg_at_5_max
value: 29.211908812453085
- type: nauc_ndcg_at_5_std
value: 11.889600649177105
- type: nauc_precision_at_1000_diff1
value: -30.748469800880645
- type: nauc_precision_at_1000_max
value: 19.417480125029428
- type: nauc_precision_at_1000_std
value: 56.70019782766074
- type: nauc_precision_at_100_diff1
value: -29.865813267831903
- type: nauc_precision_at_100_max
value: 23.12384878368056
- type: nauc_precision_at_100_std
value: 61.055054568941316
- type: nauc_precision_at_10_diff1
value: -21.009692231038027
- type: nauc_precision_at_10_max
value: 17.17005256478476
- type: nauc_precision_at_10_std
value: 46.84418408982082
- type: nauc_precision_at_1_diff1
value: 46.27833298543312
- type: nauc_precision_at_1_max
value: 42.95527694566197
- type: nauc_precision_at_1_std
value: 38.34819072902029
- type: nauc_precision_at_20_diff1
value: -28.241621476008145
- type: nauc_precision_at_20_max
value: 18.10002854156234
- type: nauc_precision_at_20_std
value: 49.6982351381192
- type: nauc_precision_at_3_diff1
value: -5.554611273606884
- type: nauc_precision_at_3_max
value: 36.00283130191982
- type: nauc_precision_at_3_std
value: 38.519667863269774
- type: nauc_precision_at_5_diff1
value: -19.733878565053846
- type: nauc_precision_at_5_max
value: 27.345271376901664
- type: nauc_precision_at_5_std
value: 42.2458596894888
- type: nauc_recall_at_1000_diff1
value: 30.821754146835833
- type: nauc_recall_at_1000_max
value: 20.332467117173373
- type: nauc_recall_at_1000_std
value: 34.14411348957753
- type: nauc_recall_at_100_diff1
value: 32.521295266192126
- type: nauc_recall_at_100_max
value: 18.108998962525813
- type: nauc_recall_at_100_std
value: 8.298216408897026
- type: nauc_recall_at_10_diff1
value: 54.54697313615533
- type: nauc_recall_at_10_max
value: -0.5557227949531418
- type: nauc_recall_at_10_std
value: -32.19440068210786
- type: nauc_recall_at_1_diff1
value: 78.97150551563644
- type: nauc_recall_at_1_max
value: -6.130504498165464
- type: nauc_recall_at_1_std
value: -40.805677041559186
- type: nauc_recall_at_20_diff1
value: 43.554781653824655
- type: nauc_recall_at_20_max
value: 1.4844544566658058
- type: nauc_recall_at_20_std
value: -25.71843978958663
- type: nauc_recall_at_3_diff1
value: 64.05074398888104
- type: nauc_recall_at_3_max
value: -0.9952472862393249
- type: nauc_recall_at_3_std
value: -40.44878148344615
- type: nauc_recall_at_5_diff1
value: 56.494223475808646
- type: nauc_recall_at_5_max
value: -2.1994046915880654
- type: nauc_recall_at_5_std
value: -39.86929189958492
- type: ndcg_at_1
value: 66.418
- type: ndcg_at_10
value: 50.646
- type: ndcg_at_100
value: 55.263
- type: ndcg_at_1000
value: 62.814
- type: ndcg_at_20
value: 49.552
- type: ndcg_at_3
value: 55.86
- type: ndcg_at_5
value: 52.897000000000006
- type: precision_at_1
value: 77.61200000000001
- type: precision_at_10
value: 37.761
- type: precision_at_100
value: 11.299
- type: precision_at_1000
value: 1.763
- type: precision_at_20
value: 28.731
- type: precision_at_3
value: 55.224
- type: precision_at_5
value: 47.760999999999996
- type: recall_at_1
value: 11.03
- type: recall_at_10
value: 31.252000000000002
- type: recall_at_100
value: 61.281
- type: recall_at_1000
value: 85.97
- type: recall_at_20
value: 41.092
- type: recall_at_3
value: 17.519000000000002
- type: recall_at_5
value: 22.18
task:
type: Retrieval
- dataset:
config: default
name: MTEB DBPedia (default)
revision: c0f706b76e590d620bd6618b3ca8efdd34e2d659
split: test
type: mteb/dbpedia
metrics:
- type: main_score
value: 44.463
- type: map_at_1
value: 9.788
- type: map_at_10
value: 21.609
- type: map_at_100
value: 30.814000000000004
- type: map_at_1000
value: 32.608
- type: map_at_20
value: 25.108999999999998
- type: map_at_3
value: 15.428
- type: map_at_5
value: 18.218
- type: mrr_at_1
value: 71.5
- type: mrr_at_10
value: 79.72668650793649
- type: mrr_at_100
value: 79.8883838490951
- type: mrr_at_1000
value: 79.89775721404739
- type: mrr_at_20
value: 79.82832120365015
- type: mrr_at_3
value: 78.79166666666666
- type: mrr_at_5
value: 79.40416666666665
- type: nauc_map_at_1000_diff1
value: 17.375031680280276
- type: nauc_map_at_1000_max
value: 24.851054504402914
- type: nauc_map_at_1000_std
value: 33.46293695471569
- type: nauc_map_at_100_diff1
value: 18.464324442263845
- type: nauc_map_at_100_max
value: 23.281690335566214
- type: nauc_map_at_100_std
value: 30.907383732999882
- type: nauc_map_at_10_diff1
value: 23.45077945244867
- type: nauc_map_at_10_max
value: 11.261334133386738
- type: nauc_map_at_10_std
value: 3.353783320134396
- type: nauc_map_at_1_diff1
value: 36.954746976863724
- type: nauc_map_at_1_max
value: -1.5297540745901643
- type: nauc_map_at_1_std
value: -16.679994927326756
- type: nauc_map_at_20_diff1
value: 21.776451852488343
- type: nauc_map_at_20_max
value: 16.074917961719752
- type: nauc_map_at_20_std
value: 13.058740140886712
- type: nauc_map_at_3_diff1
value: 25.689622022919046
- type: nauc_map_at_3_max
value: 0.5671357606826857
- type: nauc_map_at_3_std
value: -12.620721627341041
- type: nauc_map_at_5_diff1
value: 24.757257406644026
- type: nauc_map_at_5_max
value: 4.916904740159734
- type: nauc_map_at_5_std
value: -6.822930006128332
- type: nauc_mrr_at_1000_diff1
value: 46.559865560836386
- type: nauc_mrr_at_1000_max
value: 61.468428945905195
- type: nauc_mrr_at_1000_std
value: 44.03756629629259
- type: nauc_mrr_at_100_diff1
value: 46.56406362108066
- type: nauc_mrr_at_100_max
value: 61.46588836286131
- type: nauc_mrr_at_100_std
value: 44.02567173370796
- type: nauc_mrr_at_10_diff1
value: 46.78241494765344
- type: nauc_mrr_at_10_max
value: 61.62762334662772
- type: nauc_mrr_at_10_std
value: 43.99095446343537
- type: nauc_mrr_at_1_diff1
value: 45.576194549676195
- type: nauc_mrr_at_1_max
value: 57.72896906344298
- type: nauc_mrr_at_1_std
value: 40.748173981706906
- type: nauc_mrr_at_20_diff1
value: 46.56366719674904
- type: nauc_mrr_at_20_max
value: 61.53689285695657
- type: nauc_mrr_at_20_std
value: 44.05763327168646
- type: nauc_mrr_at_3_diff1
value: 46.75914809101897
- type: nauc_mrr_at_3_max
value: 61.85477011236961
- type: nauc_mrr_at_3_std
value: 45.048472367494604
- type: nauc_mrr_at_5_diff1
value: 46.6606478337335
- type: nauc_mrr_at_5_max
value: 61.664619760561266
- type: nauc_mrr_at_5_std
value: 44.28977107829588
- type: nauc_ndcg_at_1000_diff1
value: 19.86113026731972
- type: nauc_ndcg_at_1000_max
value: 40.89908227748882
- type: nauc_ndcg_at_1000_std
value: 47.99980597239699
- type: nauc_ndcg_at_100_diff1
value: 21.75146816571304
- type: nauc_ndcg_at_100_max
value: 35.05332254979754
- type: nauc_ndcg_at_100_std
value: 41.068841850422466
- type: nauc_ndcg_at_10_diff1
value: 21.570966857693907
- type: nauc_ndcg_at_10_max
value: 36.41902652959281
- type: nauc_ndcg_at_10_std
value: 36.08975047168605
- type: nauc_ndcg_at_1_diff1
value: 38.33163583796035
- type: nauc_ndcg_at_1_max
value: 44.26114500733438
- type: nauc_ndcg_at_1_std
value: 30.498097323497504
- type: nauc_ndcg_at_20_diff1
value: 21.960547673928733
- type: nauc_ndcg_at_20_max
value: 32.43831433494772
- type: nauc_ndcg_at_20_std
value: 32.79775752841865
- type: nauc_ndcg_at_3_diff1
value: 21.647250785375835
- type: nauc_ndcg_at_3_max
value: 36.362322526847706
- type: nauc_ndcg_at_3_std
value: 31.05235987942825
- type: nauc_ndcg_at_5_diff1
value: 21.271795742482368
- type: nauc_ndcg_at_5_max
value: 36.801753871534984
- type: nauc_ndcg_at_5_std
value: 34.51605931502119
- type: nauc_precision_at_1000_diff1
value: -20.270386845002413
- type: nauc_precision_at_1000_max
value: 2.871816895421538
- type: nauc_precision_at_1000_std
value: 15.01493673822994
- type: nauc_precision_at_100_diff1
value: -11.51914278765532
- type: nauc_precision_at_100_max
value: 28.241727791913412
- type: nauc_precision_at_100_std
value: 51.127572793111185
- type: nauc_precision_at_10_diff1
value: -5.380490059126128
- type: nauc_precision_at_10_max
value: 41.41789863727103
- type: nauc_precision_at_10_std
value: 56.13359935176473
- type: nauc_precision_at_1_diff1
value: 45.576194549676195
- type: nauc_precision_at_1_max
value: 57.72896906344298
- type: nauc_precision_at_1_std
value: 40.748173981706906
- type: nauc_precision_at_20_diff1
value: -7.909933493324983
- type: nauc_precision_at_20_max
value: 36.25734653049458
- type: nauc_precision_at_20_std
value: 55.801562166432994
- type: nauc_precision_at_3_diff1
value: 2.53267910566852
- type: nauc_precision_at_3_max
value: 38.532604738141046
- type: nauc_precision_at_3_std
value: 41.805860471025284
- type: nauc_precision_at_5_diff1
value: -2.0148529473816996
- type: nauc_precision_at_5_max
value: 40.644695403335
- type: nauc_precision_at_5_std
value: 49.375213376859065
- type: nauc_recall_at_1000_diff1
value: 8.339460249746237
- type: nauc_recall_at_1000_max
value: 34.634401102850354
- type: nauc_recall_at_1000_std
value: 53.21629431602282
- type: nauc_recall_at_100_diff1
value: 15.440680329005167
- type: nauc_recall_at_100_max
value: 26.287973712231683
- type: nauc_recall_at_100_std
value: 38.310999052091496
- type: nauc_recall_at_10_diff1
value: 21.420724634571435
- type: nauc_recall_at_10_max
value: 8.13694929974024
- type: nauc_recall_at_10_std
value: 0.5716587790246369
- type: nauc_recall_at_1_diff1
value: 36.954746976863724
- type: nauc_recall_at_1_max
value: -1.5297540745901643
- type: nauc_recall_at_1_std
value: -16.679994927326756
- type: nauc_recall_at_20_diff1
value: 18.4645978129915
- type: nauc_recall_at_20_max
value: 12.450382815835878
- type: nauc_recall_at_20_std
value: 9.632484163697542
- type: nauc_recall_at_3_diff1
value: 22.416124328204436
- type: nauc_recall_at_3_max
value: -3.218774980928332
- type: nauc_recall_at_3_std
value: -15.129273257126528
- type: nauc_recall_at_5_diff1
value: 22.557287106783814
- type: nauc_recall_at_5_max
value: 1.9967140457545238
- type: nauc_recall_at_5_std
value: -9.291925484883594
- type: ndcg_at_1
value: 58.62500000000001
- type: ndcg_at_10
value: 44.463
- type: ndcg_at_100
value: 49.278
- type: ndcg_at_1000
value: 56.843999999999994
- type: ndcg_at_20
value: 43.842
- type: ndcg_at_3
value: 48.689
- type: ndcg_at_5
value: 46.589000000000006
- type: precision_at_1
value: 71.5
- type: precision_at_10
value: 35.375
- type: precision_at_100
value: 11.403
- type: precision_at_1000
value: 2.242
- type: precision_at_20
value: 26.8
- type: precision_at_3
value: 51.917
- type: precision_at_5
value: 45.35
- type: recall_at_1
value: 9.788
- type: recall_at_10
value: 27.123
- type: recall_at_100
value: 55.48800000000001
- type: recall_at_1000
value: 79.113
- type: recall_at_20
value: 34.858
- type: recall_at_3
value: 16.958000000000002
- type: recall_at_5
value: 21.065
task:
type: Retrieval
- dataset:
config: default
name: MTEB FEVER (default)
revision: bea83ef9e8fb933d90a2f1d5515737465d613e12
split: dev
type: mteb/fever
metrics:
- type: main_score
value: 88.64999999999999
- type: map_at_1
value: 77.255
- type: map_at_10
value: 85.169
- type: map_at_100
value: 85.351
- type: map_at_1000
value: 85.36399999999999
- type: map_at_20
value: 85.26
- type: map_at_3
value: 84.381
- type: map_at_5
value: 84.878
- type: mrr_at_1
value: 83.78337833783378
- type: mrr_at_10
value: 90.34676681953897
- type: mrr_at_100
value: 90.38006152933879
- type: mrr_at_1000
value: 90.38105582794016
- type: mrr_at_20
value: 90.37167986475033
- type: mrr_at_3
value: 89.90899089908976
- type: mrr_at_5
value: 90.18801880188
- type: nauc_map_at_1000_diff1
value: 48.12511552508137
- type: nauc_map_at_1000_max
value: 13.568949269597658
- type: nauc_map_at_1000_std
value: -10.600969004079342
- type: nauc_map_at_100_diff1
value: 48.08934266008355
- type: nauc_map_at_100_max
value: 13.55396415695952
- type: nauc_map_at_100_std
value: -10.60212004059611
- type: nauc_map_at_10_diff1
value: 47.76536099671083
- type: nauc_map_at_10_max
value: 13.182370215559983
- type: nauc_map_at_10_std
value: -11.021531318070458
- type: nauc_map_at_1_diff1
value: 54.11679553553126
- type: nauc_map_at_1_max
value: 10.265575560156355
- type: nauc_map_at_1_std
value: -12.111083723276545
- type: nauc_map_at_20_diff1
value: 47.928752432985995
- type: nauc_map_at_20_max
value: 13.34623561791217
- type: nauc_map_at_20_std
value: -10.786785714400214
- type: nauc_map_at_3_diff1
value: 47.75273975443192
- type: nauc_map_at_3_max
value: 12.84892003333417
- type: nauc_map_at_3_std
value: -11.890757494905994
- type: nauc_map_at_5_diff1
value: 47.85999138041634
- type: nauc_map_at_5_max
value: 13.172930696542377
- type: nauc_map_at_5_std
value: -11.345868689049436
- type: nauc_mrr_at_1000_diff1
value: 71.43372441669159
- type: nauc_mrr_at_1000_max
value: 19.136227402838323
- type: nauc_mrr_at_1000_std
value: -20.710966351452793
- type: nauc_mrr_at_100_diff1
value: 71.43230899468782
- type: nauc_mrr_at_100_max
value: 19.14383336738965
- type: nauc_mrr_at_100_std
value: -20.71267993067476
- type: nauc_mrr_at_10_diff1
value: 71.4046256049301
- type: nauc_mrr_at_10_max
value: 19.215343144932866
- type: nauc_mrr_at_10_std
value: -20.838256012075515
- type: nauc_mrr_at_1_diff1
value: 71.81923147611779
- type: nauc_mrr_at_1_max
value: 15.506002297290431
- type: nauc_mrr_at_1_std
value: -18.15154459388298
- type: nauc_mrr_at_20_diff1
value: 71.42390021857346
- type: nauc_mrr_at_20_max
value: 19.16305870191451
- type: nauc_mrr_at_20_std
value: -20.742501642157677
- type: nauc_mrr_at_3_diff1
value: 71.35337406127367
- type: nauc_mrr_at_3_max
value: 20.152809849939516
- type: nauc_mrr_at_3_std
value: -21.210653055555845
- type: nauc_mrr_at_5_diff1
value: 71.50522468550169
- type: nauc_mrr_at_5_max
value: 19.723141393606312
- type: nauc_mrr_at_5_std
value: -20.86309506863733
- type: nauc_ndcg_at_1000_diff1
value: 50.39361652769403
- type: nauc_ndcg_at_1000_max
value: 16.175262322285946
- type: nauc_ndcg_at_1000_std
value: -10.122220891108254
- type: nauc_ndcg_at_100_diff1
value: 49.42758790272144
- type: nauc_ndcg_at_100_max
value: 16.000371562715237
- type: nauc_ndcg_at_100_std
value: -9.787189605632676
- type: nauc_ndcg_at_10_diff1
value: 48.055595169234834
- type: nauc_ndcg_at_10_max
value: 14.858325412282063
- type: nauc_ndcg_at_10_std
value: -11.106819781239894
- type: nauc_ndcg_at_1_diff1
value: 71.81923147611779
- type: nauc_ndcg_at_1_max
value: 15.506002297290431
- type: nauc_ndcg_at_1_std
value: -18.15154459388298
- type: nauc_ndcg_at_20_diff1
value: 48.465948689948355
- type: nauc_ndcg_at_20_max
value: 15.144792819318653
- type: nauc_ndcg_at_20_std
value: -10.493628648624705
- type: nauc_ndcg_at_3_diff1
value: 49.201048638097625
- type: nauc_ndcg_at_3_max
value: 15.548158608009535
- type: nauc_ndcg_at_3_std
value: -12.788537369942476
- type: nauc_ndcg_at_5_diff1
value: 48.656302018553994
- type: nauc_ndcg_at_5_max
value: 15.3777052187059
- type: nauc_ndcg_at_5_std
value: -11.761890570339622
- type: nauc_precision_at_1000_diff1
value: -7.587207859844254
- type: nauc_precision_at_1000_max
value: 10.034775019298706
- type: nauc_precision_at_1000_std
value: 10.771826580061331
- type: nauc_precision_at_100_diff1
value: -9.095610720732157
- type: nauc_precision_at_100_max
value: 13.304886901425087
- type: nauc_precision_at_100_std
value: 14.430000759554892
- type: nauc_precision_at_10_diff1
value: -7.766474069124403
- type: nauc_precision_at_10_max
value: 13.577880904242502
- type: nauc_precision_at_10_std
value: 5.601324437475049
- type: nauc_precision_at_1_diff1
value: 71.81923147611779
- type: nauc_precision_at_1_max
value: 15.506002297290431
- type: nauc_precision_at_1_std
value: -18.15154459388298
- type: nauc_precision_at_20_diff1
value: -8.711478204989803
- type: nauc_precision_at_20_max
value: 13.48344061715417
- type: nauc_precision_at_20_std
value: 10.629078920538515
- type: nauc_precision_at_3_diff1
value: 13.344156270804328
- type: nauc_precision_at_3_max
value: 18.283772246612788
- type: nauc_precision_at_3_std
value: -9.559293717760333
- type: nauc_precision_at_5_diff1
value: 2.335535732969584
- type: nauc_precision_at_5_max
value: 17.10399710293788
- type: nauc_precision_at_5_std
value: -1.111952947962031
- type: nauc_recall_at_1000_diff1
value: 5.8550778002058514
- type: nauc_recall_at_1000_max
value: 32.08063901915127
- type: nauc_recall_at_1000_std
value: 37.064521060746756
- type: nauc_recall_at_100_diff1
value: 4.3827059811871525
- type: nauc_recall_at_100_max
value: 23.036293742557053
- type: nauc_recall_at_100_std
value: 25.602427153026575
- type: nauc_recall_at_10_diff1
value: 11.166053941654045
- type: nauc_recall_at_10_max
value: 13.65241332067668
- type: nauc_recall_at_10_std
value: 3.484618971368651
- type: nauc_recall_at_1_diff1
value: 54.11679553553126
- type: nauc_recall_at_1_max
value: 10.265575560156355
- type: nauc_recall_at_1_std
value: -12.111083723276545
- type: nauc_recall_at_20_diff1
value: 8.71815473825628
- type: nauc_recall_at_20_max
value: 14.86256574978527
- type: nauc_recall_at_20_std
value: 9.660589150398796
- type: nauc_recall_at_3_diff1
value: 26.78083231166246
- type: nauc_recall_at_3_max
value: 16.004083622989636
- type: nauc_recall_at_3_std
value: -7.759307330933892
- type: nauc_recall_at_5_diff1
value: 21.33813171345775
- type: nauc_recall_at_5_max
value: 16.40061446525957
- type: nauc_recall_at_5_std
value: -2.7389917651976914
- type: ndcg_at_1
value: 83.783
- type: ndcg_at_10
value: 88.64999999999999
- type: ndcg_at_100
value: 89.25699999999999
- type: ndcg_at_1000
value: 89.49000000000001
- type: ndcg_at_20
value: 88.874
- type: ndcg_at_3
value: 87.57000000000001
- type: ndcg_at_5
value: 88.13600000000001
- type: precision_at_1
value: 83.783
- type: precision_at_10
value: 10.514999999999999
- type: precision_at_100
value: 1.111
- type: precision_at_1000
value: 0.11499999999999999
- type: precision_at_20
value: 5.341
- type: precision_at_3
value: 33.393
- type: precision_at_5
value: 20.519000000000002
- type: recall_at_1
value: 77.255
- type: recall_at_10
value: 94.28
- type: recall_at_100
value: 96.598
- type: recall_at_1000
value: 98.012
- type: recall_at_20
value: 95.018
- type: recall_at_3
value: 91.08000000000001
- type: recall_at_5
value: 92.708
task:
type: Retrieval
- dataset:
config: default
name: MTEB FEVER (default)
revision: bea83ef9e8fb933d90a2f1d5515737465d613e12
split: test
type: mteb/fever
metrics:
- type: main_score
value: 87.435
- type: map_at_1
value: 74.981
- type: map_at_10
value: 83.653
- type: map_at_100
value: 83.86099999999999
- type: map_at_1000
value: 83.87400000000001
- type: map_at_20
value: 83.771
- type: map_at_3
value: 82.674
- type: map_at_5
value: 83.287
- type: mrr_at_1
value: 81.05310531053105
- type: mrr_at_10
value: 88.52425718762343
- type: mrr_at_100
value: 88.57207103103651
- type: mrr_at_1000
value: 88.57296718224823
- type: mrr_at_20
value: 88.55991399827933
- type: mrr_at_3
value: 87.92629262926283
- type: mrr_at_5
value: 88.3530853085307
- type: nauc_map_at_1000_diff1
value: 46.14240498162393
- type: nauc_map_at_1000_max
value: 6.1846219189540195
- type: nauc_map_at_1000_std
value: -14.48212291262028
- type: nauc_map_at_100_diff1
value: 46.108096799513106
- type: nauc_map_at_100_max
value: 6.173512887966413
- type: nauc_map_at_100_std
value: -14.464833239817674
- type: nauc_map_at_10_diff1
value: 45.83687033921735
- type: nauc_map_at_10_max
value: 5.9061650252110525
- type: nauc_map_at_10_std
value: -14.678508784636241
- type: nauc_map_at_1_diff1
value: 50.61046862885979
- type: nauc_map_at_1_max
value: 2.425955740916627
- type: nauc_map_at_1_std
value: -16.328891990762585
- type: nauc_map_at_20_diff1
value: 45.97546559851971
- type: nauc_map_at_20_max
value: 6.0662775855496776
- type: nauc_map_at_20_std
value: -14.49963601234895
- type: nauc_map_at_3_diff1
value: 46.1202533097099
- type: nauc_map_at_3_max
value: 5.616814189395492
- type: nauc_map_at_3_std
value: -15.978283808772531
- type: nauc_map_at_5_diff1
value: 45.599136626253866
- type: nauc_map_at_5_max
value: 5.7843333783824775
- type: nauc_map_at_5_std
value: -15.309824576718336
- type: nauc_mrr_at_1000_diff1
value: 62.30030029453455
- type: nauc_mrr_at_1000_max
value: 6.356344776268262
- type: nauc_mrr_at_1000_std
value: -24.730184755892026
- type: nauc_mrr_at_100_diff1
value: 62.29832882893961
- type: nauc_mrr_at_100_max
value: 6.36350427889769
- type: nauc_mrr_at_100_std
value: -24.72049425785792
- type: nauc_mrr_at_10_diff1
value: 62.256874535633436
- type: nauc_mrr_at_10_max
value: 6.365853830220074
- type: nauc_mrr_at_10_std
value: -24.877671128830876
- type: nauc_mrr_at_1_diff1
value: 63.473487705718426
- type: nauc_mrr_at_1_max
value: 3.8567763282224803
- type: nauc_mrr_at_1_std
value: -23.25636348392456
- type: nauc_mrr_at_20_diff1
value: 62.303829622689
- type: nauc_mrr_at_20_max
value: 6.425728128294657
- type: nauc_mrr_at_20_std
value: -24.700586855661893
- type: nauc_mrr_at_3_diff1
value: 62.13552742024581
- type: nauc_mrr_at_3_max
value: 6.430964417533111
- type: nauc_mrr_at_3_std
value: -25.852406375576
- type: nauc_mrr_at_5_diff1
value: 62.060923711273965
- type: nauc_mrr_at_5_max
value: 6.322071082109083
- type: nauc_mrr_at_5_std
value: -25.6218764204596
- type: nauc_ndcg_at_1000_diff1
value: 47.81098089339254
- type: nauc_ndcg_at_1000_max
value: 7.970335565370969
- type: nauc_ndcg_at_1000_std
value: -13.954592108035785
- type: nauc_ndcg_at_100_diff1
value: 47.02098088114501
- type: nauc_ndcg_at_100_max
value: 8.050188640792946
- type: nauc_ndcg_at_100_std
value: -13.26180999246677
- type: nauc_ndcg_at_10_diff1
value: 45.717367513521225
- type: nauc_ndcg_at_10_max
value: 7.12252438038365
- type: nauc_ndcg_at_10_std
value: -14.125413198369552
- type: nauc_ndcg_at_1_diff1
value: 63.473487705718426
- type: nauc_ndcg_at_1_max
value: 3.8567763282224803
- type: nauc_ndcg_at_1_std
value: -23.25636348392456
- type: nauc_ndcg_at_20_diff1
value: 46.13337017312619
- type: nauc_ndcg_at_20_max
value: 7.620589557775606
- type: nauc_ndcg_at_20_std
value: -13.336897266394992
- type: nauc_ndcg_at_3_diff1
value: 47.36662761546531
- type: nauc_ndcg_at_3_max
value: 7.168741804362151
- type: nauc_ndcg_at_3_std
value: -17.634918228466123
- type: nauc_ndcg_at_5_diff1
value: 45.62932198258297
- type: nauc_ndcg_at_5_max
value: 6.996099116999571
- type: nauc_ndcg_at_5_std
value: -16.1735603835562
- type: nauc_precision_at_1000_diff1
value: -4.608190037353962
- type: nauc_precision_at_1000_max
value: 10.010479306018528
- type: nauc_precision_at_1000_std
value: 4.313257227180172
- type: nauc_precision_at_100_diff1
value: -6.6893029843110146
- type: nauc_precision_at_100_max
value: 12.413145442192345
- type: nauc_precision_at_100_std
value: 9.10695142014202
- type: nauc_precision_at_10_diff1
value: -5.092747483622398
- type: nauc_precision_at_10_max
value: 13.276263457275084
- type: nauc_precision_at_10_std
value: 5.687441173372954
- type: nauc_precision_at_1_diff1
value: 63.473487705718426
- type: nauc_precision_at_1_max
value: 3.8567763282224803
- type: nauc_precision_at_1_std
value: -23.25636348392456
- type: nauc_precision_at_20_diff1
value: -7.572611499118658
- type: nauc_precision_at_20_max
value: 13.865424005937452
- type: nauc_precision_at_20_std
value: 9.656961531290484
- type: nauc_precision_at_3_diff1
value: 20.95424640418592
- type: nauc_precision_at_3_max
value: 15.165327791145916
- type: nauc_precision_at_3_std
value: -13.928428150348598
- type: nauc_precision_at_5_diff1
value: 4.551646626065323
- type: nauc_precision_at_5_max
value: 15.356949460723632
- type: nauc_precision_at_5_std
value: -4.920978124087124
- type: nauc_recall_at_1000_diff1
value: 0.7642613575735239
- type: nauc_recall_at_1000_max
value: 36.15279773872626
- type: nauc_recall_at_1000_std
value: 50.6159041911611
- type: nauc_recall_at_100_diff1
value: 4.3525325750852915
- type: nauc_recall_at_100_max
value: 28.586607700542714
- type: nauc_recall_at_100_std
value: 39.43604834805734
- type: nauc_recall_at_10_diff1
value: 12.926832543562524
- type: nauc_recall_at_10_max
value: 13.171467441816146
- type: nauc_recall_at_10_std
value: 8.93368714277416
- type: nauc_recall_at_1_diff1
value: 50.61046862885979
- type: nauc_recall_at_1_max
value: 2.425955740916627
- type: nauc_recall_at_1_std
value: -16.328891990762585
- type: nauc_recall_at_20_diff1
value: 9.529885126882741
- type: nauc_recall_at_20_max
value: 18.260862028666438
- type: nauc_recall_at_20_std
value: 20.87997909359809
- type: nauc_recall_at_3_diff1
value: 29.327875346479388
- type: nauc_recall_at_3_max
value: 9.683655931257292
- type: nauc_recall_at_3_std
value: -12.017770715000532
- type: nauc_recall_at_5_diff1
value: 19.8873084969436
- type: nauc_recall_at_5_max
value: 10.63514554861997
- type: nauc_recall_at_5_std
value: -6.100038797712149
- type: ndcg_at_1
value: 81.053
- type: ndcg_at_10
value: 87.435
- type: ndcg_at_100
value: 88.142
- type: ndcg_at_1000
value: 88.36699999999999
- type: ndcg_at_20
value: 87.739
- type: ndcg_at_3
value: 85.978
- type: ndcg_at_5
value: 86.788
- type: precision_at_1
value: 81.053
- type: precision_at_10
value: 10.437000000000001
- type: precision_at_100
value: 1.103
- type: precision_at_1000
value: 0.11399999999999999
- type: precision_at_20
value: 5.314
- type: precision_at_3
value: 32.873000000000005
- type: precision_at_5
value: 20.279
- type: recall_at_1
value: 74.981
- type: recall_at_10
value: 94.242
- type: recall_at_100
value: 96.968
- type: recall_at_1000
value: 98.33800000000001
- type: recall_at_20
value: 95.275
- type: recall_at_3
value: 90.21000000000001
- type: recall_at_5
value: 92.364
task:
type: Retrieval
- dataset:
config: default
name: MTEB FEVER (default)
revision: bea83ef9e8fb933d90a2f1d5515737465d613e12
split: train
type: mteb/fever
metrics:
- type: main_score
value: 86.56099999999999
- type: map_at_1
value: 72.21300000000001
- type: map_at_10
value: 82.22699999999999
- type: map_at_100
value: 82.492
- type: map_at_1000
value: 82.509
- type: map_at_20
value: 82.384
- type: map_at_3
value: 81.012
- type: map_at_5
value: 81.827
- type: mrr_at_1
value: 80.48174118932701
- type: mrr_at_10
value: 88.44588090828918
- type: mrr_at_100
value: 88.4788741387046
- type: mrr_at_1000
value: 88.4790755480462
- type: mrr_at_20
value: 88.47304662364945
- type: mrr_at_3
value: 87.82108490422577
- type: mrr_at_5
value: 88.29622378046524
- type: nauc_map_at_1000_diff1
value: 37.32923541003442
- type: nauc_map_at_1000_max
value: -5.359095440076856
- type: nauc_map_at_1000_std
value: -22.442879064254956
- type: nauc_map_at_100_diff1
value: 37.29004026756419
- type: nauc_map_at_100_max
value: -5.361870410697577
- type: nauc_map_at_100_std
value: -22.41628823533935
- type: nauc_map_at_10_diff1
value: 36.861769872580815
- type: nauc_map_at_10_max
value: -5.567242961864721
- type: nauc_map_at_10_std
value: -22.412744554284842
- type: nauc_map_at_1_diff1
value: 42.9747708308183
- type: nauc_map_at_1_max
value: -7.249290294686461
- type: nauc_map_at_1_std
value: -23.10192038507521
- type: nauc_map_at_20_diff1
value: 37.07248863698633
- type: nauc_map_at_20_max
value: -5.441593629521792
- type: nauc_map_at_20_std
value: -22.36943344269823
- type: nauc_map_at_3_diff1
value: 36.74111398203012
- type: nauc_map_at_3_max
value: -6.452157513325058
- type: nauc_map_at_3_std
value: -23.93890877957013
- type: nauc_map_at_5_diff1
value: 36.625141407824735
- type: nauc_map_at_5_max
value: -5.864765833683385
- type: nauc_map_at_5_std
value: -22.773631881202967
- type: nauc_mrr_at_1000_diff1
value: 57.72385177985896
- type: nauc_mrr_at_1000_max
value: -8.132359498658397
- type: nauc_mrr_at_1000_std
value: -39.40225550515249
- type: nauc_mrr_at_100_diff1
value: 57.72408745167823
- type: nauc_mrr_at_100_max
value: -8.131150898290278
- type: nauc_mrr_at_100_std
value: -39.4020380770972
- type: nauc_mrr_at_10_diff1
value: 57.71763338551041
- type: nauc_mrr_at_10_max
value: -8.131006257564982
- type: nauc_mrr_at_10_std
value: -39.508932554524996
- type: nauc_mrr_at_1_diff1
value: 58.84813890985453
- type: nauc_mrr_at_1_max
value: -7.148657120271318
- type: nauc_mrr_at_1_std
value: -35.10095845964509
- type: nauc_mrr_at_20_diff1
value: 57.72281173423275
- type: nauc_mrr_at_20_max
value: -8.121468354197358
- type: nauc_mrr_at_20_std
value: -39.41212536881133
- type: nauc_mrr_at_3_diff1
value: 57.30929380153681
- type: nauc_mrr_at_3_max
value: -8.656817243234048
- type: nauc_mrr_at_3_std
value: -40.982621907605306
- type: nauc_mrr_at_5_diff1
value: 57.64357740137117
- type: nauc_mrr_at_5_max
value: -8.217447601430397
- type: nauc_mrr_at_5_std
value: -40.004654935245156
- type: nauc_ndcg_at_1000_diff1
value: 39.456836969012684
- type: nauc_ndcg_at_1000_max
value: -4.554039345654236
- type: nauc_ndcg_at_1000_std
value: -23.554219088125553
- type: nauc_ndcg_at_100_diff1
value: 38.57601564904512
- type: nauc_ndcg_at_100_max
value: -4.4180771506140175
- type: nauc_ndcg_at_100_std
value: -22.77318548595954
- type: nauc_ndcg_at_10_diff1
value: 36.95399053988775
- type: nauc_ndcg_at_10_max
value: -4.962155509183936
- type: nauc_ndcg_at_10_std
value: -22.67027102247201
- type: nauc_ndcg_at_1_diff1
value: 58.84813890985453
- type: nauc_ndcg_at_1_max
value: -7.148657120271318
- type: nauc_ndcg_at_1_std
value: -35.10095845964509
- type: nauc_ndcg_at_20_diff1
value: 37.47120121711935
- type: nauc_ndcg_at_20_max
value: -4.634556758700681
- type: nauc_ndcg_at_20_std
value: -22.414472983942492
- type: nauc_ndcg_at_3_diff1
value: 38.413347606529335
- type: nauc_ndcg_at_3_max
value: -6.14356883481873
- type: nauc_ndcg_at_3_std
value: -26.88170783033852
- type: nauc_ndcg_at_5_diff1
value: 37.005727276106775
- type: nauc_ndcg_at_5_max
value: -5.4137285022218435
- type: nauc_ndcg_at_5_std
value: -23.999236816373756
- type: nauc_precision_at_1000_diff1
value: -1.1635551306722967
- type: nauc_precision_at_1000_max
value: 10.896413712860566
- type: nauc_precision_at_1000_std
value: 2.3246874191515157
- type: nauc_precision_at_100_diff1
value: -2.727489712611685
- type: nauc_precision_at_100_max
value: 12.249725894620063
- type: nauc_precision_at_100_std
value: 4.609079555342843
- type: nauc_precision_at_10_diff1
value: -4.581131252588009
- type: nauc_precision_at_10_max
value: 10.143024794307918
- type: nauc_precision_at_10_std
value: 0.2701051149847926
- type: nauc_precision_at_1_diff1
value: 58.84813890985453
- type: nauc_precision_at_1_max
value: -7.148657120271318
- type: nauc_precision_at_1_std
value: -35.10095845964509
- type: nauc_precision_at_20_diff1
value: -4.403750400342156
- type: nauc_precision_at_20_max
value: 11.452041496155529
- type: nauc_precision_at_20_std
value: 3.216020187205585
- type: nauc_precision_at_3_diff1
value: 9.787754895986547
- type: nauc_precision_at_3_max
value: 1.8743526876348817
- type: nauc_precision_at_3_std
value: -22.800504315751173
- type: nauc_precision_at_5_diff1
value: -0.9174754849135418
- type: nauc_precision_at_5_max
value: 7.147940717069395
- type: nauc_precision_at_5_std
value: -8.059722729524335
- type: nauc_recall_at_1000_diff1
value: -12.945950337667655
- type: nauc_recall_at_1000_max
value: 24.06090741784359
- type: nauc_recall_at_1000_std
value: 40.15437535710061
- type: nauc_recall_at_100_diff1
value: -7.462722039743709
- type: nauc_recall_at_100_max
value: 16.082546490413804
- type: nauc_recall_at_100_std
value: 28.634606163725863
- type: nauc_recall_at_10_diff1
value: 1.7339686372342604
- type: nauc_recall_at_10_max
value: 5.097055736811297
- type: nauc_recall_at_10_std
value: 7.011828597941062
- type: nauc_recall_at_1_diff1
value: 42.9747708308183
- type: nauc_recall_at_1_max
value: -7.249290294686461
- type: nauc_recall_at_1_std
value: -23.10192038507521
- type: nauc_recall_at_20_diff1
value: -2.3238818761975795
- type: nauc_recall_at_20_max
value: 9.07708790464009
- type: nauc_recall_at_20_std
value: 15.568994701873468
- type: nauc_recall_at_3_diff1
value: 17.145249528698066
- type: nauc_recall_at_3_max
value: -4.5872505802077335
- type: nauc_recall_at_3_std
value: -17.921913670095183
- type: nauc_recall_at_5_diff1
value: 8.88215070605085
- type: nauc_recall_at_5_max
value: 0.3400651614243859
- type: nauc_recall_at_5_std
value: -5.632858563012691
- type: ndcg_at_1
value: 80.482
- type: ndcg_at_10
value: 86.56099999999999
- type: ndcg_at_100
value: 87.368
- type: ndcg_at_1000
value: 87.63499999999999
- type: ndcg_at_20
value: 86.92800000000001
- type: ndcg_at_3
value: 84.955
- type: ndcg_at_5
value: 85.919
- type: precision_at_1
value: 80.482
- type: precision_at_10
value: 10.84
- type: precision_at_100
value: 1.166
- type: precision_at_1000
value: 0.122
- type: precision_at_20
value: 5.563
- type: precision_at_3
value: 33.641
- type: precision_at_5
value: 20.964
- type: recall_at_1
value: 72.21300000000001
- type: recall_at_10
value: 93.795
- type: recall_at_100
value: 96.674
- type: recall_at_1000
value: 98.205
- type: recall_at_20
value: 94.948
- type: recall_at_3
value: 89.188
- type: recall_at_5
value: 91.89699999999999
task:
type: Retrieval
- dataset:
config: default
name: MTEB FiQA2018 (default)
revision: 27a168819829fe9bcd655c2df245fb19452e8e06
split: dev
type: mteb/fiqa
metrics:
- type: main_score
value: 44.874
- type: map_at_1
value: 25.554
- type: map_at_10
value: 37.157000000000004
- type: map_at_100
value: 38.943
- type: map_at_1000
value: 39.102
- type: map_at_20
value: 38.173
- type: map_at_3
value: 32.729
- type: map_at_5
value: 35.061
- type: mrr_at_1
value: 45.800000000000004
- type: mrr_at_10
value: 53.25738095238094
- type: mrr_at_100
value: 53.98819286210455
- type: mrr_at_1000
value: 54.030027403873724
- type: mrr_at_20
value: 53.74143904599013
- type: mrr_at_3
value: 51.000000000000014
- type: mrr_at_5
value: 52.30999999999999
- type: nauc_map_at_1000_diff1
value: 46.763509151363166
- type: nauc_map_at_1000_max
value: 28.768062016261155
- type: nauc_map_at_1000_std
value: 3.0129566502781753
- type: nauc_map_at_100_diff1
value: 46.75356281280302
- type: nauc_map_at_100_max
value: 28.697109688680534
- type: nauc_map_at_100_std
value: 2.98086242745498
- type: nauc_map_at_10_diff1
value: 46.90381011086511
- type: nauc_map_at_10_max
value: 27.4527629039281
- type: nauc_map_at_10_std
value: 1.1291745476435884
- type: nauc_map_at_1_diff1
value: 48.33032759955959
- type: nauc_map_at_1_max
value: 22.391437508015226
- type: nauc_map_at_1_std
value: -2.7345411216863766
- type: nauc_map_at_20_diff1
value: 46.62357425324139
- type: nauc_map_at_20_max
value: 28.137639704531686
- type: nauc_map_at_20_std
value: 2.175364829675028
- type: nauc_map_at_3_diff1
value: 47.09395643913574
- type: nauc_map_at_3_max
value: 25.522334862376482
- type: nauc_map_at_3_std
value: -1.7499764295586555
- type: nauc_map_at_5_diff1
value: 47.088687390770126
- type: nauc_map_at_5_max
value: 26.621894937003532
- type: nauc_map_at_5_std
value: -0.11347437212377846
- type: nauc_mrr_at_1000_diff1
value: 54.169580650511676
- type: nauc_mrr_at_1000_max
value: 42.589982827419334
- type: nauc_mrr_at_1000_std
value: 9.270447132748673
- type: nauc_mrr_at_100_diff1
value: 54.159671790628195
- type: nauc_mrr_at_100_max
value: 42.579289414249835
- type: nauc_mrr_at_100_std
value: 9.27315009020055
- type: nauc_mrr_at_10_diff1
value: 54.16721338694943
- type: nauc_mrr_at_10_max
value: 42.45505658459557
- type: nauc_mrr_at_10_std
value: 8.798155032599674
- type: nauc_mrr_at_1_diff1
value: 56.20829207953295
- type: nauc_mrr_at_1_max
value: 42.91690849928963
- type: nauc_mrr_at_1_std
value: 8.890861484401473
- type: nauc_mrr_at_20_diff1
value: 54.097509580707104
- type: nauc_mrr_at_20_max
value: 42.49047013868743
- type: nauc_mrr_at_20_std
value: 9.243287912038888
- type: nauc_mrr_at_3_diff1
value: 54.688451695195326
- type: nauc_mrr_at_3_max
value: 42.55241180331927
- type: nauc_mrr_at_3_std
value: 8.445158230066568
- type: nauc_mrr_at_5_diff1
value: 54.17733930320079
- type: nauc_mrr_at_5_max
value: 42.25518269900131
- type: nauc_mrr_at_5_std
value: 8.826463219453128
- type: nauc_ndcg_at_1000_diff1
value: 48.366830534050976
- type: nauc_ndcg_at_1000_max
value: 35.109898990740696
- type: nauc_ndcg_at_1000_std
value: 9.221531654043496
- type: nauc_ndcg_at_100_diff1
value: 48.07081030067787
- type: nauc_ndcg_at_100_max
value: 34.35730435839074
- type: nauc_ndcg_at_100_std
value: 9.592149120459222
- type: nauc_ndcg_at_10_diff1
value: 48.03168151591368
- type: nauc_ndcg_at_10_max
value: 31.31245092846216
- type: nauc_ndcg_at_10_std
value: 4.774236737326802
- type: nauc_ndcg_at_1_diff1
value: 56.20829207953295
- type: nauc_ndcg_at_1_max
value: 42.91690849928963
- type: nauc_ndcg_at_1_std
value: 8.890861484401473
- type: nauc_ndcg_at_20_diff1
value: 47.38090326186517
- type: nauc_ndcg_at_20_max
value: 32.19386385519328
- type: nauc_ndcg_at_20_std
value: 6.810139964510753
- type: nauc_ndcg_at_3_diff1
value: 47.6059376703799
- type: nauc_ndcg_at_3_max
value: 32.67819124349061
- type: nauc_ndcg_at_3_std
value: 3.6427274540943086
- type: nauc_ndcg_at_5_diff1
value: 47.729951651884925
- type: nauc_ndcg_at_5_max
value: 30.867372762701855
- type: nauc_ndcg_at_5_std
value: 3.8274061216487394
- type: nauc_precision_at_1000_diff1
value: -2.353765753893674
- type: nauc_precision_at_1000_max
value: 25.59608017932468
- type: nauc_precision_at_1000_std
value: 22.80181908619341
- type: nauc_precision_at_100_diff1
value: 4.615247028604855
- type: nauc_precision_at_100_max
value: 30.25611102396034
- type: nauc_precision_at_100_std
value: 28.233201922921896
- type: nauc_precision_at_10_diff1
value: 18.422388904203025
- type: nauc_precision_at_10_max
value: 30.89324316202028
- type: nauc_precision_at_10_std
value: 18.343630631993715
- type: nauc_precision_at_1_diff1
value: 56.20829207953295
- type: nauc_precision_at_1_max
value: 42.91690849928963
- type: nauc_precision_at_1_std
value: 8.890861484401473
- type: nauc_precision_at_20_diff1
value: 11.116959256700724
- type: nauc_precision_at_20_max
value: 30.484646790281882
- type: nauc_precision_at_20_std
value: 24.468092883172346
- type: nauc_precision_at_3_diff1
value: 34.10536344989137
- type: nauc_precision_at_3_max
value: 37.24921411319198
- type: nauc_precision_at_3_std
value: 10.598700100739142
- type: nauc_precision_at_5_diff1
value: 25.972678366343626
- type: nauc_precision_at_5_max
value: 33.782187544075285
- type: nauc_precision_at_5_std
value: 14.72781628417725
- type: nauc_recall_at_1000_diff1
value: 36.55627458374837
- type: nauc_recall_at_1000_max
value: 38.95489760216261
- type: nauc_recall_at_1000_std
value: 38.40584391418862
- type: nauc_recall_at_100_diff1
value: 36.04232284300105
- type: nauc_recall_at_100_max
value: 30.101566035272025
- type: nauc_recall_at_100_std
value: 25.726155244628004
- type: nauc_recall_at_10_diff1
value: 41.01172833164805
- type: nauc_recall_at_10_max
value: 23.976999088516028
- type: nauc_recall_at_10_std
value: 5.1785916214973815
- type: nauc_recall_at_1_diff1
value: 48.33032759955959
- type: nauc_recall_at_1_max
value: 22.391437508015226
- type: nauc_recall_at_1_std
value: -2.7345411216863766
- type: nauc_recall_at_20_diff1
value: 36.12608470753002
- type: nauc_recall_at_20_max
value: 24.27805595843604
- type: nauc_recall_at_20_std
value: 10.90998386167403
- type: nauc_recall_at_3_diff1
value: 44.4111049052806
- type: nauc_recall_at_3_max
value: 22.890760809978094
- type: nauc_recall_at_3_std
value: -2.2637204931966606
- type: nauc_recall_at_5_diff1
value: 41.877649098288735
- type: nauc_recall_at_5_max
value: 22.343121393202157
- type: nauc_recall_at_5_std
value: 1.2989742291659185
- type: ndcg_at_1
value: 45.800000000000004
- type: ndcg_at_10
value: 44.874
- type: ndcg_at_100
value: 51.064
- type: ndcg_at_1000
value: 53.734
- type: ndcg_at_20
value: 47.439
- type: ndcg_at_3
value: 40.952
- type: ndcg_at_5
value: 42.092
- type: precision_at_1
value: 45.800000000000004
- type: precision_at_10
value: 11.48
- type: precision_at_100
value: 1.804
- type: precision_at_1000
value: 0.22499999999999998
- type: precision_at_20
value: 6.84
- type: precision_at_3
value: 25.067
- type: precision_at_5
value: 18.32
- type: recall_at_1
value: 25.554
- type: recall_at_10
value: 51.474
- type: recall_at_100
value: 74.735
- type: recall_at_1000
value: 90.914
- type: recall_at_20
value: 59.736999999999995
- type: recall_at_3
value: 36.582
- type: recall_at_5
value: 42.917
task:
type: Retrieval
- dataset:
config: default
name: MTEB FiQA2018 (default)
revision: 27a168819829fe9bcd655c2df245fb19452e8e06
split: test
type: mteb/fiqa
metrics:
- type: main_score
value: 42.672
- type: map_at_1
value: 20.810000000000002
- type: map_at_10
value: 34.816
- type: map_at_100
value: 36.559000000000005
- type: map_at_1000
value: 36.739
- type: map_at_20
value: 35.793
- type: map_at_3
value: 30.709999999999997
- type: map_at_5
value: 32.903
- type: mrr_at_1
value: 40.74074074074074
- type: mrr_at_10
value: 50.31225504605135
- type: mrr_at_100
value: 51.10974962073793
- type: mrr_at_1000
value: 51.145458083558594
- type: mrr_at_20
value: 50.835867941005674
- type: mrr_at_3
value: 48.17386831275719
- type: mrr_at_5
value: 49.36213991769546
- type: nauc_map_at_1000_diff1
value: 42.6898312513211
- type: nauc_map_at_1000_max
value: 26.281671731871985
- type: nauc_map_at_1000_std
value: 0.7620867046583552
- type: nauc_map_at_100_diff1
value: 42.57169669036131
- type: nauc_map_at_100_max
value: 26.145640161493173
- type: nauc_map_at_100_std
value: 0.7367651475823449
- type: nauc_map_at_10_diff1
value: 42.31258901668185
- type: nauc_map_at_10_max
value: 24.77533473513109
- type: nauc_map_at_10_std
value: -0.7497212250554994
- type: nauc_map_at_1_diff1
value: 48.41859846141981
- type: nauc_map_at_1_max
value: 18.10254250440143
- type: nauc_map_at_1_std
value: -5.915838853200118
- type: nauc_map_at_20_diff1
value: 42.34856289190343
- type: nauc_map_at_20_max
value: 25.69107238299756
- type: nauc_map_at_20_std
value: 0.27588502783188223
- type: nauc_map_at_3_diff1
value: 44.04496251583725
- type: nauc_map_at_3_max
value: 22.64325180927134
- type: nauc_map_at_3_std
value: -2.204734451292917
- type: nauc_map_at_5_diff1
value: 43.40971038290712
- type: nauc_map_at_5_max
value: 23.97382467102329
- type: nauc_map_at_5_std
value: -1.1419214343255244
- type: nauc_mrr_at_1000_diff1
value: 53.07479963154328
- type: nauc_mrr_at_1000_max
value: 35.08039964920313
- type: nauc_mrr_at_1000_std
value: 2.395230067270119
- type: nauc_mrr_at_100_diff1
value: 53.05361788036026
- type: nauc_mrr_at_100_max
value: 35.09788748290212
- type: nauc_mrr_at_100_std
value: 2.418854060474198
- type: nauc_mrr_at_10_diff1
value: 52.72364127433612
- type: nauc_mrr_at_10_max
value: 34.776193049704396
- type: nauc_mrr_at_10_std
value: 2.003219818180847
- type: nauc_mrr_at_1_diff1
value: 56.825212094166545
- type: nauc_mrr_at_1_max
value: 32.907554590039815
- type: nauc_mrr_at_1_std
value: -1.364505805796406
- type: nauc_mrr_at_20_diff1
value: 52.99965015232407
- type: nauc_mrr_at_20_max
value: 34.968116095845254
- type: nauc_mrr_at_20_std
value: 2.3485485371198633
- type: nauc_mrr_at_3_diff1
value: 53.8968569125276
- type: nauc_mrr_at_3_max
value: 35.711600898114334
- type: nauc_mrr_at_3_std
value: 1.9859332685100484
- type: nauc_mrr_at_5_diff1
value: 53.582871272036016
- type: nauc_mrr_at_5_max
value: 34.93684472388838
- type: nauc_mrr_at_5_std
value: 2.3347159901923065
- type: nauc_ndcg_at_1000_diff1
value: 45.36787254964793
- type: nauc_ndcg_at_1000_max
value: 32.2681824363134
- type: nauc_ndcg_at_1000_std
value: 5.815945066818261
- type: nauc_ndcg_at_100_diff1
value: 44.08061590784506
- type: nauc_ndcg_at_100_max
value: 31.279868971364078
- type: nauc_ndcg_at_100_std
value: 6.33649305085534
- type: nauc_ndcg_at_10_diff1
value: 42.631906063381905
- type: nauc_ndcg_at_10_max
value: 27.516462342453263
- type: nauc_ndcg_at_10_std
value: 1.3896722992639912
- type: nauc_ndcg_at_1_diff1
value: 56.825212094166545
- type: nauc_ndcg_at_1_max
value: 32.907554590039815
- type: nauc_ndcg_at_1_std
value: -1.364505805796406
- type: nauc_ndcg_at_20_diff1
value: 43.316329717168955
- type: nauc_ndcg_at_20_max
value: 29.206746664257416
- type: nauc_ndcg_at_20_std
value: 3.7239674271850105
- type: nauc_ndcg_at_3_diff1
value: 44.80183033874905
- type: nauc_ndcg_at_3_max
value: 29.742746669981457
- type: nauc_ndcg_at_3_std
value: 1.227113841715857
- type: nauc_ndcg_at_5_diff1
value: 44.3701381872213
- type: nauc_ndcg_at_5_max
value: 28.065056973490492
- type: nauc_ndcg_at_5_std
value: 1.6689171525025643
- type: nauc_precision_at_1000_diff1
value: 2.3046803465908003
- type: nauc_precision_at_1000_max
value: 27.380473322815103
- type: nauc_precision_at_1000_std
value: 15.356878362150866
- type: nauc_precision_at_100_diff1
value: 7.03975347205041
- type: nauc_precision_at_100_max
value: 30.721605506174026
- type: nauc_precision_at_100_std
value: 19.547035843159
- type: nauc_precision_at_10_diff1
value: 16.402742978688057
- type: nauc_precision_at_10_max
value: 30.777182999989606
- type: nauc_precision_at_10_std
value: 9.164809596132375
- type: nauc_precision_at_1_diff1
value: 56.825212094166545
- type: nauc_precision_at_1_max
value: 32.907554590039815
- type: nauc_precision_at_1_std
value: -1.364505805796406
- type: nauc_precision_at_20_diff1
value: 13.139000469807192
- type: nauc_precision_at_20_max
value: 31.990538682090136
- type: nauc_precision_at_20_std
value: 14.393413944725811
- type: nauc_precision_at_3_diff1
value: 31.552710282041623
- type: nauc_precision_at_3_max
value: 33.51647517755997
- type: nauc_precision_at_3_std
value: 6.043833018841229
- type: nauc_precision_at_5_diff1
value: 25.367103130969593
- type: nauc_precision_at_5_max
value: 32.34107578507879
- type: nauc_precision_at_5_std
value: 8.706580223485966
- type: nauc_recall_at_1000_diff1
value: 36.63865503751282
- type: nauc_recall_at_1000_max
value: 43.61857713478981
- type: nauc_recall_at_1000_std
value: 41.20611452183899
- type: nauc_recall_at_100_diff1
value: 28.359239562981813
- type: nauc_recall_at_100_max
value: 31.2488585833301
- type: nauc_recall_at_100_std
value: 26.418183922479415
- type: nauc_recall_at_10_diff1
value: 29.03981058334878
- type: nauc_recall_at_10_max
value: 19.851141320829726
- type: nauc_recall_at_10_std
value: 2.9077823815816113
- type: nauc_recall_at_1_diff1
value: 48.41859846141981
- type: nauc_recall_at_1_max
value: 18.10254250440143
- type: nauc_recall_at_1_std
value: -5.915838853200118
- type: nauc_recall_at_20_diff1
value: 30.619719883102785
- type: nauc_recall_at_20_max
value: 24.55326818191543
- type: nauc_recall_at_20_std
value: 10.31945754397688
- type: nauc_recall_at_3_diff1
value: 37.03852287877488
- type: nauc_recall_at_3_max
value: 21.07728820212504
- type: nauc_recall_at_3_std
value: 0.5323752155647694
- type: nauc_recall_at_5_diff1
value: 35.71309469862529
- type: nauc_recall_at_5_max
value: 20.859225353060467
- type: nauc_recall_at_5_std
value: 2.9376290211854745
- type: ndcg_at_1
value: 40.741
- type: ndcg_at_10
value: 42.672
- type: ndcg_at_100
value: 49.056
- type: ndcg_at_1000
value: 51.989
- type: ndcg_at_20
value: 45.131
- type: ndcg_at_3
value: 39.472
- type: ndcg_at_5
value: 40.209
- type: precision_at_1
value: 40.741
- type: precision_at_10
value: 11.821
- type: precision_at_100
value: 1.847
- type: precision_at_1000
value: 0.23800000000000002
- type: precision_at_20
value: 6.9750000000000005
- type: precision_at_3
value: 26.749000000000002
- type: precision_at_5
value: 19.29
- type: recall_at_1
value: 20.810000000000002
- type: recall_at_10
value: 49.657000000000004
- type: recall_at_100
value: 73.547
- type: recall_at_1000
value: 90.971
- type: recall_at_20
value: 57.174
- type: recall_at_3
value: 36.401
- type: recall_at_5
value: 41.811
task:
type: Retrieval
- dataset:
config: default
name: MTEB FiQA2018 (default)
revision: 27a168819829fe9bcd655c2df245fb19452e8e06
split: train
type: mteb/fiqa
metrics:
- type: main_score
value: 43.324
- type: map_at_1
value: 22.042
- type: map_at_10
value: 35.506
- type: map_at_100
value: 37.330000000000005
- type: map_at_1000
value: 37.497
- type: map_at_20
value: 36.569
- type: map_at_3
value: 31.136000000000003
- type: map_at_5
value: 33.458
- type: mrr_at_1
value: 41.21818181818182
- type: mrr_at_10
value: 50.212178932178986
- type: mrr_at_100
value: 51.01730710363213
- type: mrr_at_1000
value: 51.0568603275777
- type: mrr_at_20
value: 50.727859227741014
- type: mrr_at_3
value: 47.77272727272733
- type: mrr_at_5
value: 49.16454545454549
- type: nauc_map_at_1000_diff1
value: 44.04272057810526
- type: nauc_map_at_1000_max
value: 25.591745036031522
- type: nauc_map_at_1000_std
value: -0.30445228328536716
- type: nauc_map_at_100_diff1
value: 44.01251041718803
- type: nauc_map_at_100_max
value: 25.50774810569885
- type: nauc_map_at_100_std
value: -0.3536644300946971
- type: nauc_map_at_10_diff1
value: 43.94792483556109
- type: nauc_map_at_10_max
value: 24.38124776391208
- type: nauc_map_at_10_std
value: -1.9008227013643013
- type: nauc_map_at_1_diff1
value: 47.5823442411594
- type: nauc_map_at_1_max
value: 16.098120209560246
- type: nauc_map_at_1_std
value: -6.474949554469241
- type: nauc_map_at_20_diff1
value: 43.904430971837215
- type: nauc_map_at_20_max
value: 25.02666309647016
- type: nauc_map_at_20_std
value: -0.9839666098211898
- type: nauc_map_at_3_diff1
value: 44.09023141566954
- type: nauc_map_at_3_max
value: 20.806911976005
- type: nauc_map_at_3_std
value: -4.717530470050773
- type: nauc_map_at_5_diff1
value: 44.136425375442464
- type: nauc_map_at_5_max
value: 22.8086611395014
- type: nauc_map_at_5_std
value: -3.4400450523751203
- type: nauc_mrr_at_1000_diff1
value: 50.07537524680542
- type: nauc_mrr_at_1000_max
value: 34.0564490817672
- type: nauc_mrr_at_1000_std
value: 1.7381240132846465
- type: nauc_mrr_at_100_diff1
value: 50.05816984827742
- type: nauc_mrr_at_100_max
value: 34.06001922775578
- type: nauc_mrr_at_100_std
value: 1.7565289963657589
- type: nauc_mrr_at_10_diff1
value: 50.03282241193434
- type: nauc_mrr_at_10_max
value: 34.06608751975744
- type: nauc_mrr_at_10_std
value: 1.63053669388884
- type: nauc_mrr_at_1_diff1
value: 53.903269711540794
- type: nauc_mrr_at_1_max
value: 33.07894487586781
- type: nauc_mrr_at_1_std
value: -1.5109127877147128
- type: nauc_mrr_at_20_diff1
value: 50.02068203379615
- type: nauc_mrr_at_20_max
value: 34.0442009191646
- type: nauc_mrr_at_20_std
value: 1.7370035901665568
- type: nauc_mrr_at_3_diff1
value: 50.512929925179925
- type: nauc_mrr_at_3_max
value: 33.68124055866945
- type: nauc_mrr_at_3_std
value: 0.6233671460050563
- type: nauc_mrr_at_5_diff1
value: 50.23877045325512
- type: nauc_mrr_at_5_max
value: 33.88189018188559
- type: nauc_mrr_at_5_std
value: 1.2285475999913853
- type: nauc_ndcg_at_1000_diff1
value: 44.5394614509097
- type: nauc_ndcg_at_1000_max
value: 30.757622799766203
- type: nauc_ndcg_at_1000_std
value: 4.7849677289040065
- type: nauc_ndcg_at_100_diff1
value: 43.972129444027566
- type: nauc_ndcg_at_100_max
value: 30.068116358576237
- type: nauc_ndcg_at_100_std
value: 4.98164700822003
- type: nauc_ndcg_at_10_diff1
value: 43.90316145848242
- type: nauc_ndcg_at_10_max
value: 27.942560497889126
- type: nauc_ndcg_at_10_std
value: 1.1639226955676085
- type: nauc_ndcg_at_1_diff1
value: 53.903269711540794
- type: nauc_ndcg_at_1_max
value: 33.07894487586781
- type: nauc_ndcg_at_1_std
value: -1.5109127877147128
- type: nauc_ndcg_at_20_diff1
value: 43.58418727337814
- type: nauc_ndcg_at_20_max
value: 28.49134809683963
- type: nauc_ndcg_at_20_std
value: 2.620919689677387
- type: nauc_ndcg_at_3_diff1
value: 44.619591071294174
- type: nauc_ndcg_at_3_max
value: 27.946260272394362
- type: nauc_ndcg_at_3_std
value: -0.5889494356050139
- type: nauc_ndcg_at_5_diff1
value: 44.373428421412875
- type: nauc_ndcg_at_5_max
value: 27.293622020469773
- type: nauc_ndcg_at_5_std
value: -0.2590215673685209
- type: nauc_precision_at_1000_diff1
value: 1.3555023302594147
- type: nauc_precision_at_1000_max
value: 26.715905651256104
- type: nauc_precision_at_1000_std
value: 20.552543709293232
- type: nauc_precision_at_100_diff1
value: 9.26195632526192
- type: nauc_precision_at_100_max
value: 32.764862859177754
- type: nauc_precision_at_100_std
value: 24.31720182442109
- type: nauc_precision_at_10_diff1
value: 21.491871495983993
- type: nauc_precision_at_10_max
value: 35.68968329387387
- type: nauc_precision_at_10_std
value: 14.395349431573997
- type: nauc_precision_at_1_diff1
value: 53.903269711540794
- type: nauc_precision_at_1_max
value: 33.07894487586781
- type: nauc_precision_at_1_std
value: -1.5109127877147128
- type: nauc_precision_at_20_diff1
value: 17.01838154349841
- type: nauc_precision_at_20_max
value: 35.026739540738795
- type: nauc_precision_at_20_std
value: 18.99756134043926
- type: nauc_precision_at_3_diff1
value: 32.5276923569714
- type: nauc_precision_at_3_max
value: 34.17028514167449
- type: nauc_precision_at_3_std
value: 4.701014289049224
- type: nauc_precision_at_5_diff1
value: 27.420289540859933
- type: nauc_precision_at_5_max
value: 35.090178846683756
- type: nauc_precision_at_5_std
value: 8.77127816260347
- type: nauc_recall_at_1000_diff1
value: 22.201742466449073
- type: nauc_recall_at_1000_max
value: 32.12548787064195
- type: nauc_recall_at_1000_std
value: 38.35298316557644
- type: nauc_recall_at_100_diff1
value: 27.007968582225523
- type: nauc_recall_at_100_max
value: 25.840258216813677
- type: nauc_recall_at_100_std
value: 21.309907791035144
- type: nauc_recall_at_10_diff1
value: 33.234435091698934
- type: nauc_recall_at_10_max
value: 22.561655534725276
- type: nauc_recall_at_10_std
value: 3.939638319426343
- type: nauc_recall_at_1_diff1
value: 47.5823442411594
- type: nauc_recall_at_1_max
value: 16.098120209560246
- type: nauc_recall_at_1_std
value: -6.474949554469241
- type: nauc_recall_at_20_diff1
value: 30.204301573132426
- type: nauc_recall_at_20_max
value: 22.50029208267097
- type: nauc_recall_at_20_std
value: 8.404433974064252
- type: nauc_recall_at_3_diff1
value: 37.77539231241753
- type: nauc_recall_at_3_max
value: 18.589687477983226
- type: nauc_recall_at_3_std
value: -3.145538286527593
- type: nauc_recall_at_5_diff1
value: 36.355054318958864
- type: nauc_recall_at_5_max
value: 20.47231279946122
- type: nauc_recall_at_5_std
value: -0.5883469550726023
- type: ndcg_at_1
value: 41.217999999999996
- type: ndcg_at_10
value: 43.324
- type: ndcg_at_100
value: 49.723
- type: ndcg_at_1000
value: 52.516
- type: ndcg_at_20
value: 45.967999999999996
- type: ndcg_at_3
value: 39.145
- type: ndcg_at_5
value: 40.483999999999995
- type: precision_at_1
value: 41.217999999999996
- type: precision_at_10
value: 11.664
- type: precision_at_100
value: 1.8270000000000002
- type: precision_at_1000
value: 0.232
- type: precision_at_20
value: 6.938
- type: precision_at_3
value: 25.496999999999996
- type: precision_at_5
value: 18.695
- type: recall_at_1
value: 22.042
- type: recall_at_10
value: 50.902
- type: recall_at_100
value: 74.74499999999999
- type: recall_at_1000
value: 91.57499999999999
- type: recall_at_20
value: 59.274
- type: recall_at_3
value: 36.215
- type: recall_at_5
value: 42.329
task:
type: Retrieval
- dataset:
config: default
name: MTEB HotpotQA (default)
revision: ab518f4d6fcca38d87c25209f94beba119d02014
split: dev
type: mteb/hotpotqa
metrics:
- type: main_score
value: 74.606
- type: map_at_1
value: 41.876000000000005
- type: map_at_10
value: 66.672
- type: map_at_100
value: 67.465
- type: map_at_1000
value: 67.52
- type: map_at_20
value: 67.162
- type: map_at_3
value: 63.44800000000001
- type: map_at_5
value: 65.491
- type: mrr_at_1
value: 83.75252432531668
- type: mrr_at_10
value: 88.5608723019224
- type: mrr_at_100
value: 88.68140252907958
- type: mrr_at_1000
value: 88.68453994013916
- type: mrr_at_20
value: 88.64372295908223
- type: mrr_at_3
value: 87.94443424515023
- type: mrr_at_5
value: 88.34924423229907
- type: nauc_map_at_1000_diff1
value: 9.822804401470737
- type: nauc_map_at_1000_max
value: 21.284133910316573
- type: nauc_map_at_1000_std
value: 16.162782727741114
- type: nauc_map_at_100_diff1
value: 9.788823095018454
- type: nauc_map_at_100_max
value: 21.279974906891518
- type: nauc_map_at_100_std
value: 16.19517987356137
- type: nauc_map_at_10_diff1
value: 9.669105278943347
- type: nauc_map_at_10_max
value: 20.976589835480443
- type: nauc_map_at_10_std
value: 15.740306189226885
- type: nauc_map_at_1_diff1
value: 70.1396495728672
- type: nauc_map_at_1_max
value: 46.290715660397034
- type: nauc_map_at_1_std
value: 2.383104054024225
- type: nauc_map_at_20_diff1
value: 9.67707467752077
- type: nauc_map_at_20_max
value: 21.13694646197173
- type: nauc_map_at_20_std
value: 16.077958081472104
- type: nauc_map_at_3_diff1
value: 9.93668046738331
- type: nauc_map_at_3_max
value: 20.138161755692852
- type: nauc_map_at_3_std
value: 12.195091775371187
- type: nauc_map_at_5_diff1
value: 9.673559969563692
- type: nauc_map_at_5_max
value: 20.52996974574843
- type: nauc_map_at_5_std
value: 14.344355689102406
- type: nauc_mrr_at_1000_diff1
value: 68.85537454860305
- type: nauc_mrr_at_1000_max
value: 48.66840943741123
- type: nauc_mrr_at_1000_std
value: 4.051382056479967
- type: nauc_mrr_at_100_diff1
value: 68.8544247452304
- type: nauc_mrr_at_100_max
value: 48.67420027786352
- type: nauc_mrr_at_100_std
value: 4.058034407401366
- type: nauc_mrr_at_10_diff1
value: 68.80039953887602
- type: nauc_mrr_at_10_max
value: 48.745351213021436
- type: nauc_mrr_at_10_std
value: 4.0948899045476965
- type: nauc_mrr_at_1_diff1
value: 70.1396495728672
- type: nauc_mrr_at_1_max
value: 46.290715660397034
- type: nauc_mrr_at_1_std
value: 2.383104054024225
- type: nauc_mrr_at_20_diff1
value: 68.83885181909754
- type: nauc_mrr_at_20_max
value: 48.693992176558055
- type: nauc_mrr_at_20_std
value: 4.081312314533584
- type: nauc_mrr_at_3_diff1
value: 68.60282007975034
- type: nauc_mrr_at_3_max
value: 48.78697037829542
- type: nauc_mrr_at_3_std
value: 3.401395836458664
- type: nauc_mrr_at_5_diff1
value: 68.66717032389758
- type: nauc_mrr_at_5_max
value: 48.74267981962927
- type: nauc_mrr_at_5_std
value: 3.9699315633740433
- type: nauc_ndcg_at_1000_diff1
value: 16.258648845029352
- type: nauc_ndcg_at_1000_max
value: 25.71076816350421
- type: nauc_ndcg_at_1000_std
value: 18.649106972833795
- type: nauc_ndcg_at_100_diff1
value: 15.14846889041812
- type: nauc_ndcg_at_100_max
value: 25.447364086163617
- type: nauc_ndcg_at_100_std
value: 19.434767405756897
- type: nauc_ndcg_at_10_diff1
value: 14.507220566488918
- type: nauc_ndcg_at_10_max
value: 24.376958992402177
- type: nauc_ndcg_at_10_std
value: 17.71975721219274
- type: nauc_ndcg_at_1_diff1
value: 70.1396495728672
- type: nauc_ndcg_at_1_max
value: 46.290715660397034
- type: nauc_ndcg_at_1_std
value: 2.383104054024225
- type: nauc_ndcg_at_20_diff1
value: 14.452470025035346
- type: nauc_ndcg_at_20_max
value: 24.730767716543014
- type: nauc_ndcg_at_20_std
value: 18.734318742569204
- type: nauc_ndcg_at_3_diff1
value: 15.432645820135981
- type: nauc_ndcg_at_3_max
value: 23.368066154380312
- type: nauc_ndcg_at_3_std
value: 11.874693788587596
- type: nauc_ndcg_at_5_diff1
value: 14.735306913463978
- type: nauc_ndcg_at_5_max
value: 23.705955789940493
- type: nauc_ndcg_at_5_std
value: 14.968624064682515
- type: nauc_precision_at_1000_diff1
value: -7.010818534432094
- type: nauc_precision_at_1000_max
value: 22.274170870851535
- type: nauc_precision_at_1000_std
value: 49.61485384634157
- type: nauc_precision_at_100_diff1
value: -6.7537040300001445
- type: nauc_precision_at_100_max
value: 20.72198971693192
- type: nauc_precision_at_100_std
value: 41.64088208023007
- type: nauc_precision_at_10_diff1
value: -1.7564554764573033
- type: nauc_precision_at_10_max
value: 18.513593922519217
- type: nauc_precision_at_10_std
value: 27.6910022428843
- type: nauc_precision_at_1_diff1
value: 70.1396495728672
- type: nauc_precision_at_1_max
value: 46.290715660397034
- type: nauc_precision_at_1_std
value: 2.383104054024225
- type: nauc_precision_at_20_diff1
value: -4.13241926820056
- type: nauc_precision_at_20_max
value: 18.894821060251417
- type: nauc_precision_at_20_std
value: 32.55210012738072
- type: nauc_precision_at_3_diff1
value: 3.2656032494325755
- type: nauc_precision_at_3_max
value: 18.24280155368556
- type: nauc_precision_at_3_std
value: 14.536719930577574
- type: nauc_precision_at_5_diff1
value: 0.8189990381009662
- type: nauc_precision_at_5_max
value: 17.993957619568864
- type: nauc_precision_at_5_std
value: 20.23613307485089
- type: nauc_recall_at_1000_diff1
value: -7.010818534432101
- type: nauc_recall_at_1000_max
value: 22.27417087085169
- type: nauc_recall_at_1000_std
value: 49.61485384634159
- type: nauc_recall_at_100_diff1
value: -6.753704030000011
- type: nauc_recall_at_100_max
value: 20.72198971693201
- type: nauc_recall_at_100_std
value: 41.64088208023008
- type: nauc_recall_at_10_diff1
value: -1.7564554764575586
- type: nauc_recall_at_10_max
value: 18.513593922519007
- type: nauc_recall_at_10_std
value: 27.69100224288406
- type: nauc_recall_at_1_diff1
value: 70.1396495728672
- type: nauc_recall_at_1_max
value: 46.290715660397034
- type: nauc_recall_at_1_std
value: 2.383104054024225
- type: nauc_recall_at_20_diff1
value: -4.132419268200746
- type: nauc_recall_at_20_max
value: 18.89482106025146
- type: nauc_recall_at_20_std
value: 32.55210012738058
- type: nauc_recall_at_3_diff1
value: 3.2656032494325924
- type: nauc_recall_at_3_max
value: 18.242801553685524
- type: nauc_recall_at_3_std
value: 14.53671993057761
- type: nauc_recall_at_5_diff1
value: 0.8189990381009014
- type: nauc_recall_at_5_max
value: 17.99395761956878
- type: nauc_recall_at_5_std
value: 20.236133074850805
- type: ndcg_at_1
value: 83.753
- type: ndcg_at_10
value: 74.606
- type: ndcg_at_100
value: 77.226
- type: ndcg_at_1000
value: 78.25
- type: ndcg_at_20
value: 75.78399999999999
- type: ndcg_at_3
value: 70.23700000000001
- type: ndcg_at_5
value: 72.72
- type: precision_at_1
value: 83.753
- type: precision_at_10
value: 15.427
- type: precision_at_100
value: 1.746
- type: precision_at_1000
value: 0.188
- type: precision_at_20
value: 8.092
- type: precision_at_3
value: 45.003
- type: precision_at_5
value: 28.970000000000002
- type: recall_at_1
value: 41.876000000000005
- type: recall_at_10
value: 77.134
- type: recall_at_100
value: 87.287
- type: recall_at_1000
value: 94.06099999999999
- type: recall_at_20
value: 80.916
- type: recall_at_3
value: 67.50500000000001
- type: recall_at_5
value: 72.425
task:
type: Retrieval
- dataset:
config: default
name: MTEB HotpotQA (default)
revision: ab518f4d6fcca38d87c25209f94beba119d02014
split: test
type: mteb/hotpotqa
metrics:
- type: main_score
value: 72.743
- type: map_at_1
value: 40.817
- type: map_at_10
value: 64.547
- type: map_at_100
value: 65.387
- type: map_at_1000
value: 65.44200000000001
- type: map_at_20
value: 65.03
- type: map_at_3
value: 61.23800000000001
- type: map_at_5
value: 63.305
- type: mrr_at_1
value: 81.63403106009454
- type: mrr_at_10
value: 86.90412955639134
- type: mrr_at_100
value: 87.04035069527386
- type: mrr_at_1000
value: 87.04487544894396
- type: mrr_at_20
value: 86.99122288282743
- type: mrr_at_3
value: 86.09498086878217
- type: mrr_at_5
value: 86.62570335358967
- type: nauc_map_at_1000_diff1
value: 10.381785418973225
- type: nauc_map_at_1000_max
value: 12.453266004493218
- type: nauc_map_at_1000_std
value: 9.351715274159906
- type: nauc_map_at_100_diff1
value: 10.347079650282726
- type: nauc_map_at_100_max
value: 12.438896318714795
- type: nauc_map_at_100_std
value: 9.379180828331103
- type: nauc_map_at_10_diff1
value: 10.141861202002811
- type: nauc_map_at_10_max
value: 12.097344032839436
- type: nauc_map_at_10_std
value: 8.831170396649975
- type: nauc_map_at_1_diff1
value: 66.502696606421
- type: nauc_map_at_1_max
value: 41.53031869889469
- type: nauc_map_at_1_std
value: -2.191672414299107
- type: nauc_map_at_20_diff1
value: 10.200694027304676
- type: nauc_map_at_20_max
value: 12.325683532818287
- type: nauc_map_at_20_std
value: 9.256220681393376
- type: nauc_map_at_3_diff1
value: 11.152534532572977
- type: nauc_map_at_3_max
value: 11.642745995749776
- type: nauc_map_at_3_std
value: 5.783821765761948
- type: nauc_map_at_5_diff1
value: 10.553374392069955
- type: nauc_map_at_5_max
value: 11.91290472912577
- type: nauc_map_at_5_std
value: 7.8322669846556225
- type: nauc_mrr_at_1000_diff1
value: 65.40812639602618
- type: nauc_mrr_at_1000_max
value: 43.8239039668551
- type: nauc_mrr_at_1000_std
value: -0.6428313796838251
- type: nauc_mrr_at_100_diff1
value: 65.40710609125459
- type: nauc_mrr_at_100_max
value: 43.83101205316724
- type: nauc_mrr_at_100_std
value: -0.6320502212235148
- type: nauc_mrr_at_10_diff1
value: 65.37243388394984
- type: nauc_mrr_at_10_max
value: 43.838670823575015
- type: nauc_mrr_at_10_std
value: -0.7784902122287074
- type: nauc_mrr_at_1_diff1
value: 66.502696606421
- type: nauc_mrr_at_1_max
value: 41.53031869889469
- type: nauc_mrr_at_1_std
value: -2.191672414299107
- type: nauc_mrr_at_20_diff1
value: 65.41608063050629
- type: nauc_mrr_at_20_max
value: 43.86617917223792
- type: nauc_mrr_at_20_std
value: -0.6009900804260204
- type: nauc_mrr_at_3_diff1
value: 65.14249488502335
- type: nauc_mrr_at_3_max
value: 43.95646843986658
- type: nauc_mrr_at_3_std
value: -0.8639291753276465
- type: nauc_mrr_at_5_diff1
value: 65.23387534166781
- type: nauc_mrr_at_5_max
value: 43.87784137078954
- type: nauc_mrr_at_5_std
value: -0.7793219824899724
- type: nauc_ndcg_at_1000_diff1
value: 16.35901408985652
- type: nauc_ndcg_at_1000_max
value: 17.521894806732156
- type: nauc_ndcg_at_1000_std
value: 12.017065592246968
- type: nauc_ndcg_at_100_diff1
value: 15.444833883381555
- type: nauc_ndcg_at_100_max
value: 17.12256451438098
- type: nauc_ndcg_at_100_std
value: 12.670587278800008
- type: nauc_ndcg_at_10_diff1
value: 14.486540021757053
- type: nauc_ndcg_at_10_max
value: 15.759466428223327
- type: nauc_ndcg_at_10_std
value: 10.36878152864418
- type: nauc_ndcg_at_1_diff1
value: 66.502696606421
- type: nauc_ndcg_at_1_max
value: 41.53031869889469
- type: nauc_ndcg_at_1_std
value: -2.191672414299107
- type: nauc_ndcg_at_20_diff1
value: 14.555779596109888
- type: nauc_ndcg_at_20_max
value: 16.35789269994937
- type: nauc_ndcg_at_20_std
value: 11.767787325358539
- type: nauc_ndcg_at_3_diff1
value: 16.771175692043837
- type: nauc_ndcg_at_3_max
value: 15.645248784503222
- type: nauc_ndcg_at_3_std
value: 5.7135509689853485
- type: nauc_ndcg_at_5_diff1
value: 15.503391424947639
- type: nauc_ndcg_at_5_max
value: 15.640657048320755
- type: nauc_ndcg_at_5_std
value: 8.450953821666825
- type: nauc_precision_at_1000_diff1
value: -12.369633125411433
- type: nauc_precision_at_1000_max
value: 12.755704941719651
- type: nauc_precision_at_1000_std
value: 45.00521568435039
- type: nauc_precision_at_100_diff1
value: -8.115137486501764
- type: nauc_precision_at_100_max
value: 10.646650995276286
- type: nauc_precision_at_100_std
value: 35.56308384338935
- type: nauc_precision_at_10_diff1
value: -3.7182289209127846
- type: nauc_precision_at_10_max
value: 7.7681979942398325
- type: nauc_precision_at_10_std
value: 18.92467245117897
- type: nauc_precision_at_1_diff1
value: 66.502696606421
- type: nauc_precision_at_1_max
value: 41.53031869889469
- type: nauc_precision_at_1_std
value: -2.191672414299107
- type: nauc_precision_at_20_diff1
value: -5.80639194394875
- type: nauc_precision_at_20_max
value: 8.609412843852052
- type: nauc_precision_at_20_std
value: 24.542537739429736
- type: nauc_precision_at_3_diff1
value: 4.488921802513501
- type: nauc_precision_at_3_max
value: 9.457927616694809
- type: nauc_precision_at_3_std
value: 8.375789445805138
- type: nauc_precision_at_5_diff1
value: 0.8178210962283003
- type: nauc_precision_at_5_max
value: 8.578978882201966
- type: nauc_precision_at_5_std
value: 13.561073970796885
- type: nauc_recall_at_1000_diff1
value: -12.36963312541116
- type: nauc_recall_at_1000_max
value: 12.755704941720039
- type: nauc_recall_at_1000_std
value: 45.005215684350574
- type: nauc_recall_at_100_diff1
value: -8.115137486501792
- type: nauc_recall_at_100_max
value: 10.646650995276245
- type: nauc_recall_at_100_std
value: 35.56308384338932
- type: nauc_recall_at_10_diff1
value: -3.7182289209128556
- type: nauc_recall_at_10_max
value: 7.768197994239799
- type: nauc_recall_at_10_std
value: 18.924672451178868
- type: nauc_recall_at_1_diff1
value: 66.502696606421
- type: nauc_recall_at_1_max
value: 41.53031869889469
- type: nauc_recall_at_1_std
value: -2.191672414299107
- type: nauc_recall_at_20_diff1
value: -5.8063919439487
- type: nauc_recall_at_20_max
value: 8.609412843852034
- type: nauc_recall_at_20_std
value: 24.542537739429847
- type: nauc_recall_at_3_diff1
value: 4.488921802513477
- type: nauc_recall_at_3_max
value: 9.457927616694777
- type: nauc_recall_at_3_std
value: 8.375789445805076
- type: nauc_recall_at_5_diff1
value: 0.817821096228331
- type: nauc_recall_at_5_max
value: 8.578978882202021
- type: nauc_recall_at_5_std
value: 13.56107397079683
- type: ndcg_at_1
value: 81.634
- type: ndcg_at_10
value: 72.743
- type: ndcg_at_100
value: 75.588
- type: ndcg_at_1000
value: 76.619
- type: ndcg_at_20
value: 73.906
- type: ndcg_at_3
value: 68.144
- type: ndcg_at_5
value: 70.711
- type: precision_at_1
value: 81.634
- type: precision_at_10
value: 15.110999999999999
- type: precision_at_100
value: 1.733
- type: precision_at_1000
value: 0.187
- type: precision_at_20
value: 7.928
- type: precision_at_3
value: 43.601
- type: precision_at_5
value: 28.186
- type: recall_at_1
value: 40.817
- type: recall_at_10
value: 75.557
- type: recall_at_100
value: 86.658
- type: recall_at_1000
value: 93.45
- type: recall_at_20
value: 79.28399999999999
- type: recall_at_3
value: 65.402
- type: recall_at_5
value: 70.466
task:
type: Retrieval
- dataset:
config: default
name: MTEB HotpotQA (default)
revision: ab518f4d6fcca38d87c25209f94beba119d02014
split: train
type: mteb/hotpotqa
metrics:
- type: main_score
value: 74.984
- type: map_at_1
value: 41.778999999999996
- type: map_at_10
value: 67.155
- type: map_at_100
value: 67.935
- type: map_at_1000
value: 67.987
- type: map_at_20
value: 67.625
- type: map_at_3
value: 63.846000000000004
- type: map_at_5
value: 65.962
- type: mrr_at_1
value: 83.55882352941177
- type: mrr_at_10
value: 88.33148552754291
- type: mrr_at_100
value: 88.45375789871105
- type: mrr_at_1000
value: 88.45763444191044
- type: mrr_at_20
value: 88.41352000428692
- type: mrr_at_3
value: 87.62666666666722
- type: mrr_at_5
value: 88.0801372549014
- type: nauc_map_at_1000_diff1
value: 10.87028656767292
- type: nauc_map_at_1000_max
value: 21.959290365120474
- type: nauc_map_at_1000_std
value: 15.44253415703075
- type: nauc_map_at_100_diff1
value: 10.838707271702939
- type: nauc_map_at_100_max
value: 21.95225291440458
- type: nauc_map_at_100_std
value: 15.469603007866311
- type: nauc_map_at_10_diff1
value: 10.610078190006284
- type: nauc_map_at_10_max
value: 21.686366674533605
- type: nauc_map_at_10_std
value: 14.921686758475653
- type: nauc_map_at_1_diff1
value: 69.80051370616638
- type: nauc_map_at_1_max
value: 46.884849189037844
- type: nauc_map_at_1_std
value: 3.4091886153700757
- type: nauc_map_at_20_diff1
value: 10.710717537313734
- type: nauc_map_at_20_max
value: 21.85879686180976
- type: nauc_map_at_20_std
value: 15.326344729518574
- type: nauc_map_at_3_diff1
value: 10.803492432001267
- type: nauc_map_at_3_max
value: 20.63654060659673
- type: nauc_map_at_3_std
value: 11.944023589119327
- type: nauc_map_at_5_diff1
value: 10.57350656967518
- type: nauc_map_at_5_max
value: 21.291050020056982
- type: nauc_map_at_5_std
value: 13.800853126281998
- type: nauc_mrr_at_1000_diff1
value: 68.91980008306939
- type: nauc_mrr_at_1000_max
value: 49.65818032977955
- type: nauc_mrr_at_1000_std
value: 6.255983503716273
- type: nauc_mrr_at_100_diff1
value: 68.920186133261
- type: nauc_mrr_at_100_max
value: 49.665349536327476
- type: nauc_mrr_at_100_std
value: 6.268878976883824
- type: nauc_mrr_at_10_diff1
value: 68.90388592892133
- type: nauc_mrr_at_10_max
value: 49.75965153367356
- type: nauc_mrr_at_10_std
value: 6.323690127864786
- type: nauc_mrr_at_1_diff1
value: 69.80051370616638
- type: nauc_mrr_at_1_max
value: 46.884849189037844
- type: nauc_mrr_at_1_std
value: 3.4091886153700757
- type: nauc_mrr_at_20_diff1
value: 68.90840597476175
- type: nauc_mrr_at_20_max
value: 49.70708679309621
- type: nauc_mrr_at_20_std
value: 6.315838513066106
- type: nauc_mrr_at_3_diff1
value: 68.64487191246296
- type: nauc_mrr_at_3_max
value: 49.66054010297392
- type: nauc_mrr_at_3_std
value: 5.717815934236954
- type: nauc_mrr_at_5_diff1
value: 68.83808477318325
- type: nauc_mrr_at_5_max
value: 49.75665999065203
- type: nauc_mrr_at_5_std
value: 6.089553843222202
- type: nauc_ndcg_at_1000_diff1
value: 17.564475589595123
- type: nauc_ndcg_at_1000_max
value: 26.660863179852495
- type: nauc_ndcg_at_1000_std
value: 18.220931823905893
- type: nauc_ndcg_at_100_diff1
value: 16.529629272288755
- type: nauc_ndcg_at_100_max
value: 26.362485651725976
- type: nauc_ndcg_at_100_std
value: 18.971583822121282
- type: nauc_ndcg_at_10_diff1
value: 15.554259291253217
- type: nauc_ndcg_at_10_max
value: 25.381172317904287
- type: nauc_ndcg_at_10_std
value: 16.911232099888434
- type: nauc_ndcg_at_1_diff1
value: 69.80051370616638
- type: nauc_ndcg_at_1_max
value: 46.884849189037844
- type: nauc_ndcg_at_1_std
value: 3.4091886153700757
- type: nauc_ndcg_at_20_diff1
value: 15.736796729809722
- type: nauc_ndcg_at_20_max
value: 25.816809250153007
- type: nauc_ndcg_at_20_std
value: 18.139020284042775
- type: nauc_ndcg_at_3_diff1
value: 16.39817980647894
- type: nauc_ndcg_at_3_max
value: 24.08287517626068
- type: nauc_ndcg_at_3_std
value: 12.06964501311477
- type: nauc_ndcg_at_5_diff1
value: 15.710994580525616
- type: nauc_ndcg_at_5_max
value: 24.75375703967725
- type: nauc_ndcg_at_5_std
value: 14.679841901035124
- type: nauc_precision_at_1000_diff1
value: -4.923050807549391
- type: nauc_precision_at_1000_max
value: 24.18328387479535
- type: nauc_precision_at_1000_std
value: 49.366523942899114
- type: nauc_precision_at_100_diff1
value: -4.651029899613218
- type: nauc_precision_at_100_max
value: 21.85221246397684
- type: nauc_precision_at_100_std
value: 41.45886054781206
- type: nauc_precision_at_10_diff1
value: -0.8792415343727439
- type: nauc_precision_at_10_max
value: 19.674496356854974
- type: nauc_precision_at_10_std
value: 25.454503031141645
- type: nauc_precision_at_1_diff1
value: 69.80051370616638
- type: nauc_precision_at_1_max
value: 46.884849189037844
- type: nauc_precision_at_1_std
value: 3.4091886153700757
- type: nauc_precision_at_20_diff1
value: -2.5121829554417165
- type: nauc_precision_at_20_max
value: 20.496150982288288
- type: nauc_precision_at_20_std
value: 31.12738775907587
- type: nauc_precision_at_3_diff1
value: 4.26503145939561
- type: nauc_precision_at_3_max
value: 18.9838377951004
- type: nauc_precision_at_3_std
value: 14.640822578265919
- type: nauc_precision_at_5_diff1
value: 1.6281319096948286
- type: nauc_precision_at_5_max
value: 19.300746238487978
- type: nauc_precision_at_5_std
value: 19.48177844713054
- type: nauc_recall_at_1000_diff1
value: -4.9230508075487895
- type: nauc_recall_at_1000_max
value: 24.183283874795666
- type: nauc_recall_at_1000_std
value: 49.366523942899825
- type: nauc_recall_at_100_diff1
value: -4.651029899613319
- type: nauc_recall_at_100_max
value: 21.85221246397671
- type: nauc_recall_at_100_std
value: 41.45886054781193
- type: nauc_recall_at_10_diff1
value: -0.879241534372706
- type: nauc_recall_at_10_max
value: 19.674496356854977
- type: nauc_recall_at_10_std
value: 25.454503031141655
- type: nauc_recall_at_1_diff1
value: 69.80051370616638
- type: nauc_recall_at_1_max
value: 46.884849189037844
- type: nauc_recall_at_1_std
value: 3.4091886153700757
- type: nauc_recall_at_20_diff1
value: -2.5121829554419732
- type: nauc_recall_at_20_max
value: 20.496150982288167
- type: nauc_recall_at_20_std
value: 31.1273877590757
- type: nauc_recall_at_3_diff1
value: 4.265031459395502
- type: nauc_recall_at_3_max
value: 18.98383779510037
- type: nauc_recall_at_3_std
value: 14.640822578265938
- type: nauc_recall_at_5_diff1
value: 1.628131909694958
- type: nauc_recall_at_5_max
value: 19.300746238487964
- type: nauc_recall_at_5_std
value: 19.481778447130544
- type: ndcg_at_1
value: 83.55900000000001
- type: ndcg_at_10
value: 74.984
- type: ndcg_at_100
value: 77.582
- type: ndcg_at_1000
value: 78.55499999999999
- type: ndcg_at_20
value: 76.12100000000001
- type: ndcg_at_3
value: 70.46900000000001
- type: ndcg_at_5
value: 73.045
- type: precision_at_1
value: 83.55900000000001
- type: precision_at_10
value: 15.565000000000001
- type: precision_at_100
value: 1.7579999999999998
- type: precision_at_1000
value: 0.189
- type: precision_at_20
value: 8.148
- type: precision_at_3
value: 45.263999999999996
- type: precision_at_5
value: 29.188
- type: recall_at_1
value: 41.778999999999996
- type: recall_at_10
value: 77.824
- type: recall_at_100
value: 87.90400000000001
- type: recall_at_1000
value: 94.313
- type: recall_at_20
value: 81.482
- type: recall_at_3
value: 67.896
- type: recall_at_5
value: 72.971
task:
type: Retrieval
- dataset:
config: default
name: MTEB MSMARCO (default)
revision: c5a29a104738b98a9e76336939199e264163d4a0
split: dev
type: mteb/msmarco
metrics:
- type: main_score
value: 42.120999999999995
- type: map_at_1
value: 22.736
- type: map_at_10
value: 35.092
- type: map_at_100
value: 36.230000000000004
- type: map_at_1000
value: 36.277
- type: map_at_20
value: 35.799
- type: map_at_3
value: 31.094
- type: map_at_5
value: 33.371
- type: mrr_at_1
value: 23.3810888252149
- type: mrr_at_10
value: 35.708384499931746
- type: mrr_at_100
value: 36.785903073286526
- type: mrr_at_1000
value: 36.825722574106
- type: mrr_at_20
value: 36.37977615162803
- type: mrr_at_3
value: 31.793218720152684
- type: mrr_at_5
value: 34.03677172874878
- type: nauc_map_at_1000_diff1
value: 36.139486267407925
- type: nauc_map_at_1000_max
value: 2.0214122031990587
- type: nauc_map_at_1000_std
value: -19.910962785146047
- type: nauc_map_at_100_diff1
value: 36.140766768299
- type: nauc_map_at_100_max
value: 2.020596976774594
- type: nauc_map_at_100_std
value: -19.894404573188893
- type: nauc_map_at_10_diff1
value: 36.07598204318622
- type: nauc_map_at_10_max
value: 1.8099554829591833
- type: nauc_map_at_10_std
value: -20.56954015291869
- type: nauc_map_at_1_diff1
value: 39.13795843270682
- type: nauc_map_at_1_max
value: 1.5488626830618197
- type: nauc_map_at_1_std
value: -18.834856297248827
- type: nauc_map_at_20_diff1
value: 36.08487910581468
- type: nauc_map_at_20_max
value: 1.9456332591026433
- type: nauc_map_at_20_std
value: -20.143821809894074
- type: nauc_map_at_3_diff1
value: 36.233503613892815
- type: nauc_map_at_3_max
value: 1.9619772807345706
- type: nauc_map_at_3_std
value: -20.60856230873636
- type: nauc_map_at_5_diff1
value: 35.97340075460678
- type: nauc_map_at_5_max
value: 1.8769480420927318
- type: nauc_map_at_5_std
value: -20.775503467555176
- type: nauc_mrr_at_1000_diff1
value: 36.00964169914943
- type: nauc_mrr_at_1000_max
value: 2.0640896425703765
- type: nauc_mrr_at_1000_std
value: -19.653911082078473
- type: nauc_mrr_at_100_diff1
value: 36.01143509691009
- type: nauc_mrr_at_100_max
value: 2.0670151721816086
- type: nauc_mrr_at_100_std
value: -19.63134406389329
- type: nauc_mrr_at_10_diff1
value: 35.96623584053275
- type: nauc_mrr_at_10_max
value: 1.9239076102356585
- type: nauc_mrr_at_10_std
value: -20.25471452533458
- type: nauc_mrr_at_1_diff1
value: 38.959117793598615
- type: nauc_mrr_at_1_max
value: 1.3677167801150283
- type: nauc_mrr_at_1_std
value: -18.935612195829055
- type: nauc_mrr_at_20_diff1
value: 35.94910790613082
- type: nauc_mrr_at_20_max
value: 2.0267159522212292
- type: nauc_mrr_at_20_std
value: -19.83950984848953
- type: nauc_mrr_at_3_diff1
value: 36.09825088970802
- type: nauc_mrr_at_3_max
value: 1.914144327342224
- type: nauc_mrr_at_3_std
value: -20.44286906662492
- type: nauc_mrr_at_5_diff1
value: 35.84551861223096
- type: nauc_mrr_at_5_max
value: 1.86983732222198
- type: nauc_mrr_at_5_std
value: -20.54854344258923
- type: nauc_ndcg_at_1000_diff1
value: 35.472004851727284
- type: nauc_ndcg_at_1000_max
value: 2.737377792816256
- type: nauc_ndcg_at_1000_std
value: -18.19482800058104
- type: nauc_ndcg_at_100_diff1
value: 35.46517530680521
- type: nauc_ndcg_at_100_max
value: 2.7836190775653207
- type: nauc_ndcg_at_100_std
value: -17.325512421140786
- type: nauc_ndcg_at_10_diff1
value: 35.17799161326067
- type: nauc_ndcg_at_10_max
value: 1.8621934710032726
- type: nauc_ndcg_at_10_std
value: -20.704677893177163
- type: nauc_ndcg_at_1_diff1
value: 38.90244672327909
- type: nauc_ndcg_at_1_max
value: 1.3507189870594651
- type: nauc_ndcg_at_1_std
value: -18.947293518063137
- type: nauc_ndcg_at_20_diff1
value: 35.15025708176802
- type: nauc_ndcg_at_20_max
value: 2.3625925336500946
- type: nauc_ndcg_at_20_std
value: -19.09189142369086
- type: nauc_ndcg_at_3_diff1
value: 35.41158574971149
- type: nauc_ndcg_at_3_max
value: 2.016626470801078
- type: nauc_ndcg_at_3_std
value: -21.054883496816316
- type: nauc_ndcg_at_5_diff1
value: 34.942748530346776
- type: nauc_ndcg_at_5_max
value: 1.9161622652633727
- type: nauc_ndcg_at_5_std
value: -21.276906113132515
- type: nauc_precision_at_1000_diff1
value: -1.9253565142428295
- type: nauc_precision_at_1000_max
value: 14.045425699939074
- type: nauc_precision_at_1000_std
value: 16.20785969541304
- type: nauc_precision_at_100_diff1
value: 17.346097443496493
- type: nauc_precision_at_100_max
value: 11.140141370074783
- type: nauc_precision_at_100_std
value: 14.280164624961362
- type: nauc_precision_at_10_diff1
value: 29.644284522057085
- type: nauc_precision_at_10_max
value: 1.892731779122541
- type: nauc_precision_at_10_std
value: -19.605722817038444
- type: nauc_precision_at_1_diff1
value: 38.90244672327909
- type: nauc_precision_at_1_max
value: 1.3507189870594651
- type: nauc_precision_at_1_std
value: -18.947293518063137
- type: nauc_precision_at_20_diff1
value: 26.77219179850976
- type: nauc_precision_at_20_max
value: 4.891718072273102
- type: nauc_precision_at_20_std
value: -10.612631017224652
- type: nauc_precision_at_3_diff1
value: 32.631995676805204
- type: nauc_precision_at_3_max
value: 2.130914787220238
- type: nauc_precision_at_3_std
value: -22.32778156160327
- type: nauc_precision_at_5_diff1
value: 30.93187721742059
- type: nauc_precision_at_5_max
value: 1.9956621964923527
- type: nauc_precision_at_5_std
value: -22.38641924346273
- type: nauc_recall_at_1000_diff1
value: 17.105896760286765
- type: nauc_recall_at_1000_max
value: 36.57462589387958
- type: nauc_recall_at_1000_std
value: 62.00079240400393
- type: nauc_recall_at_100_diff1
value: 31.020954943068922
- type: nauc_recall_at_100_max
value: 10.88829730164695
- type: nauc_recall_at_100_std
value: 16.616210151681816
- type: nauc_recall_at_10_diff1
value: 32.06847692573393
- type: nauc_recall_at_10_max
value: 1.7751731101901707
- type: nauc_recall_at_10_std
value: -20.93854511643462
- type: nauc_recall_at_1_diff1
value: 39.13795843270682
- type: nauc_recall_at_1_max
value: 1.5488626830618197
- type: nauc_recall_at_1_std
value: -18.834856297248827
- type: nauc_recall_at_20_diff1
value: 31.252284015750043
- type: nauc_recall_at_20_max
value: 4.142775647459482
- type: nauc_recall_at_20_std
value: -13.053410939827806
- type: nauc_recall_at_3_diff1
value: 33.081727757800536
- type: nauc_recall_at_3_max
value: 2.1476221499499597
- type: nauc_recall_at_3_std
value: -22.210127534959344
- type: nauc_recall_at_5_diff1
value: 31.79414648884249
- type: nauc_recall_at_5_max
value: 1.9631360425221316
- type: nauc_recall_at_5_std
value: -22.597007753409525
- type: ndcg_at_1
value: 23.395
- type: ndcg_at_10
value: 42.120999999999995
- type: ndcg_at_100
value: 47.625
- type: ndcg_at_1000
value: 48.769
- type: ndcg_at_20
value: 44.629000000000005
- type: ndcg_at_3
value: 34.015
- type: ndcg_at_5
value: 38.063
- type: precision_at_1
value: 23.395
- type: precision_at_10
value: 6.662
- type: precision_at_100
value: 0.943
- type: precision_at_1000
value: 0.104
- type: precision_at_20
value: 3.856
- type: precision_at_3
value: 14.427000000000001
- type: precision_at_5
value: 10.702
- type: recall_at_1
value: 22.736
- type: recall_at_10
value: 63.751999999999995
- type: recall_at_100
value: 89.265
- type: recall_at_1000
value: 97.988
- type: recall_at_20
value: 73.496
- type: recall_at_3
value: 41.791
- type: recall_at_5
value: 51.485
task:
type: Retrieval
- dataset:
config: default
name: MTEB MSMARCO (default)
revision: c5a29a104738b98a9e76336939199e264163d4a0
split: test
type: mteb/msmarco
metrics:
- type: main_score
value: 72.387
- type: map_at_1
value: 2.486
- type: map_at_10
value: 15.9
- type: map_at_100
value: 41.829
- type: map_at_1000
value: 49.701
- type: map_at_20
value: 23.980999999999998
- type: map_at_3
value: 6.704000000000001
- type: map_at_5
value: 9.913
- type: mrr_at_1
value: 95.34883720930233
- type: mrr_at_10
value: 97.09302325581395
- type: mrr_at_100
value: 97.09302325581395
- type: mrr_at_1000
value: 97.09302325581395
- type: mrr_at_20
value: 97.09302325581395
- type: mrr_at_3
value: 96.51162790697676
- type: mrr_at_5
value: 97.09302325581395
- type: nauc_map_at_1000_diff1
value: -24.043671904852435
- type: nauc_map_at_1000_max
value: 27.040649415529234
- type: nauc_map_at_1000_std
value: 44.03444305521892
- type: nauc_map_at_100_diff1
value: -15.852276195106995
- type: nauc_map_at_100_max
value: 6.322093300797476
- type: nauc_map_at_100_std
value: 17.51668982723671
- type: nauc_map_at_10_diff1
value: 7.477314872164316
- type: nauc_map_at_10_max
value: -15.799082583656809
- type: nauc_map_at_10_std
value: -21.21279591516943
- type: nauc_map_at_1_diff1
value: 23.83176861467615
- type: nauc_map_at_1_max
value: -36.61424702222069
- type: nauc_map_at_1_std
value: -37.927066893980395
- type: nauc_map_at_20_diff1
value: -2.169371886119532
- type: nauc_map_at_20_max
value: -10.0446633168474
- type: nauc_map_at_20_std
value: -11.708023341680358
- type: nauc_map_at_3_diff1
value: 17.16504844432592
- type: nauc_map_at_3_max
value: -33.146788731140774
- type: nauc_map_at_3_std
value: -36.514475090820206
- type: nauc_map_at_5_diff1
value: 19.26143280774508
- type: nauc_map_at_5_max
value: -27.74189717033227
- type: nauc_map_at_5_std
value: -33.08354526624487
- type: nauc_mrr_at_1000_diff1
value: -1.2745731776872742
- type: nauc_mrr_at_1000_max
value: 45.010228525800024
- type: nauc_mrr_at_1000_std
value: 82.11480791411624
- type: nauc_mrr_at_100_diff1
value: -1.2745731776872742
- type: nauc_mrr_at_100_max
value: 45.010228525800024
- type: nauc_mrr_at_100_std
value: 82.11480791411624
- type: nauc_mrr_at_10_diff1
value: -1.2745731776872742
- type: nauc_mrr_at_10_max
value: 45.010228525800024
- type: nauc_mrr_at_10_std
value: 82.11480791411624
- type: nauc_mrr_at_1_diff1
value: -10.5507180247667
- type: nauc_mrr_at_1_max
value: 34.505621253026
- type: nauc_mrr_at_1_std
value: 77.64350989264493
- type: nauc_mrr_at_20_diff1
value: -1.2745731776872742
- type: nauc_mrr_at_20_max
value: 45.010228525800024
- type: nauc_mrr_at_20_std
value: 82.11480791411624
- type: nauc_mrr_at_3_diff1
value: 4.909523387031578
- type: nauc_mrr_at_3_max
value: 52.01330004098173
- type: nauc_mrr_at_3_std
value: 85.09567326176314
- type: nauc_mrr_at_5_diff1
value: -1.2745731776872742
- type: nauc_mrr_at_5_max
value: 45.010228525800024
- type: nauc_mrr_at_5_std
value: 82.11480791411624
- type: nauc_ndcg_at_1000_diff1
value: -29.4861285551972
- type: nauc_ndcg_at_1000_max
value: 45.02597670219224
- type: nauc_ndcg_at_1000_std
value: 53.85407799332372
- type: nauc_ndcg_at_100_diff1
value: -24.732061539866205
- type: nauc_ndcg_at_100_max
value: 32.303183889625494
- type: nauc_ndcg_at_100_std
value: 44.659873635006456
- type: nauc_ndcg_at_10_diff1
value: -5.5662424008448905
- type: nauc_ndcg_at_10_max
value: 26.38578514340096
- type: nauc_ndcg_at_10_std
value: 24.86283081033308
- type: nauc_ndcg_at_1_diff1
value: 36.62882405696659
- type: nauc_ndcg_at_1_max
value: 28.211412260680973
- type: nauc_ndcg_at_1_std
value: 6.555794001290861
- type: nauc_ndcg_at_20_diff1
value: -18.935212746307727
- type: nauc_ndcg_at_20_max
value: 25.282877526762377
- type: nauc_ndcg_at_20_std
value: 34.501408712986326
- type: nauc_ndcg_at_3_diff1
value: 7.575900223069948
- type: nauc_ndcg_at_3_max
value: 28.619889721347835
- type: nauc_ndcg_at_3_std
value: 12.456877094803065
- type: nauc_ndcg_at_5_diff1
value: 5.943165611348846
- type: nauc_ndcg_at_5_max
value: 28.263879225638227
- type: nauc_ndcg_at_5_std
value: 13.144291627566371
- type: nauc_precision_at_1000_diff1
value: -24.648793150108457
- type: nauc_precision_at_1000_max
value: 36.80469185343303
- type: nauc_precision_at_1000_std
value: 53.550196879838985
- type: nauc_precision_at_100_diff1
value: -26.749422430833047
- type: nauc_precision_at_100_max
value: 37.35145079972374
- type: nauc_precision_at_100_std
value: 57.70428623550252
- type: nauc_precision_at_10_diff1
value: -38.005873268952115
- type: nauc_precision_at_10_max
value: 51.16779669716929
- type: nauc_precision_at_10_std
value: 65.75415144636456
- type: nauc_precision_at_1_diff1
value: -10.5507180247667
- type: nauc_precision_at_1_max
value: 34.505621253026
- type: nauc_precision_at_1_std
value: 77.64350989264493
- type: nauc_precision_at_20_diff1
value: -39.322164984171486
- type: nauc_precision_at_20_max
value: 35.76140518869625
- type: nauc_precision_at_20_std
value: 59.88144457525482
- type: nauc_precision_at_3_diff1
value: -51.50827801125146
- type: nauc_precision_at_3_max
value: 36.3553407965392
- type: nauc_precision_at_3_std
value: 43.020750110091285
- type: nauc_precision_at_5_diff1
value: -27.626241295491795
- type: nauc_precision_at_5_max
value: 47.437919946182774
- type: nauc_precision_at_5_std
value: 50.49277210796086
- type: nauc_recall_at_1000_diff1
value: -43.76386537507645
- type: nauc_recall_at_1000_max
value: 43.13591779729292
- type: nauc_recall_at_1000_std
value: 55.33804445603299
- type: nauc_recall_at_100_diff1
value: -15.909329910755282
- type: nauc_recall_at_100_max
value: 6.211365113730195
- type: nauc_recall_at_100_std
value: 11.701140658114936
- type: nauc_recall_at_10_diff1
value: 7.2700504626447335
- type: nauc_recall_at_10_max
value: -17.375284561172954
- type: nauc_recall_at_10_std
value: -23.20288495977079
- type: nauc_recall_at_1_diff1
value: 23.83176861467615
- type: nauc_recall_at_1_max
value: -36.61424702222069
- type: nauc_recall_at_1_std
value: -37.927066893980395
- type: nauc_recall_at_20_diff1
value: -3.603600554970076
- type: nauc_recall_at_20_max
value: -12.492774555512298
- type: nauc_recall_at_20_std
value: -14.555198740861197
- type: nauc_recall_at_3_diff1
value: 18.593343128896652
- type: nauc_recall_at_3_max
value: -33.422110365952626
- type: nauc_recall_at_3_std
value: -37.33498770157346
- type: nauc_recall_at_5_diff1
value: 21.12706599299733
- type: nauc_recall_at_5_max
value: -28.005654144757745
- type: nauc_recall_at_5_std
value: -34.049977102136005
- type: ndcg_at_1
value: 79.457
- type: ndcg_at_10
value: 72.387
- type: ndcg_at_100
value: 66.15100000000001
- type: ndcg_at_1000
value: 73.115
- type: ndcg_at_20
value: 69.355
- type: ndcg_at_3
value: 75.018
- type: ndcg_at_5
value: 74.29599999999999
- type: precision_at_1
value: 95.34899999999999
- type: precision_at_10
value: 80.233
- type: precision_at_100
value: 38.698
- type: precision_at_1000
value: 7.091
- type: precision_at_20
value: 70.93
- type: precision_at_3
value: 87.59700000000001
- type: precision_at_5
value: 85.58099999999999
- type: recall_at_1
value: 2.486
- type: recall_at_10
value: 17.108999999999998
- type: recall_at_100
value: 53.947
- type: recall_at_1000
value: 79.323
- type: recall_at_20
value: 26.744
- type: recall_at_3
value: 6.834999999999999
- type: recall_at_5
value: 10.280000000000001
task:
type: Retrieval
- dataset:
config: default
name: MTEB MSMARCO (default)
revision: c5a29a104738b98a9e76336939199e264163d4a0
split: train
type: mteb/msmarco
metrics:
- type: main_score
value: 39.033
- type: map_at_1
value: 20.124
- type: map_at_10
value: 32.139
- type: map_at_100
value: 33.394
- type: map_at_1000
value: 33.444
- type: map_at_20
value: 32.911
- type: map_at_3
value: 28.219
- type: map_at_5
value: 30.438
- type: mrr_at_1
value: 20.69256907895391
- type: mrr_at_10
value: 32.65586556823979
- type: mrr_at_100
value: 33.859903331429834
- type: mrr_at_1000
value: 33.90404799117744
- type: mrr_at_20
value: 33.402399052353765
- type: mrr_at_3
value: 28.80988880692012
- type: mrr_at_5
value: 30.996764352470247
- type: nauc_map_at_1000_diff1
value: 33.725693864912145
- type: nauc_map_at_1000_max
value: -0.9313190487791355
- type: nauc_map_at_1000_std
value: -19.072263227555077
- type: nauc_map_at_100_diff1
value: 33.71681815719501
- type: nauc_map_at_100_max
value: -0.9312809249177106
- type: nauc_map_at_100_std
value: -19.043814229345514
- type: nauc_map_at_10_diff1
value: 33.64027795602201
- type: nauc_map_at_10_max
value: -1.093662103807343
- type: nauc_map_at_10_std
value: -19.701316981081256
- type: nauc_map_at_1_diff1
value: 37.6475234742761
- type: nauc_map_at_1_max
value: -1.1172579533948037
- type: nauc_map_at_1_std
value: -18.42423307405879
- type: nauc_map_at_20_diff1
value: 33.67816993209406
- type: nauc_map_at_20_max
value: -1.008671531366758
- type: nauc_map_at_20_std
value: -19.30818828227061
- type: nauc_map_at_3_diff1
value: 33.948583604883346
- type: nauc_map_at_3_max
value: -1.2245037153342642
- type: nauc_map_at_3_std
value: -19.97712524394941
- type: nauc_map_at_5_diff1
value: 33.70839113129876
- type: nauc_map_at_5_max
value: -1.2040546464666357
- type: nauc_map_at_5_std
value: -20.086046366859758
- type: nauc_mrr_at_1000_diff1
value: 33.6765598428715
- type: nauc_mrr_at_1000_max
value: -0.8947712270401186
- type: nauc_mrr_at_1000_std
value: -19.054228424132294
- type: nauc_mrr_at_100_diff1
value: 33.66751649677623
- type: nauc_mrr_at_100_max
value: -0.8925074311306023
- type: nauc_mrr_at_100_std
value: -19.024986617194788
- type: nauc_mrr_at_10_diff1
value: 33.58165621319944
- type: nauc_mrr_at_10_max
value: -1.041201591295902
- type: nauc_mrr_at_10_std
value: -19.643130663802832
- type: nauc_mrr_at_1_diff1
value: 37.57249749572576
- type: nauc_mrr_at_1_max
value: -1.033405682282863
- type: nauc_mrr_at_1_std
value: -18.51556029196618
- type: nauc_mrr_at_20_diff1
value: 33.62546337983731
- type: nauc_mrr_at_20_max
value: -0.959286757562728
- type: nauc_mrr_at_20_std
value: -19.263604059725058
- type: nauc_mrr_at_3_diff1
value: 33.88354206923207
- type: nauc_mrr_at_3_max
value: -1.1588079971522036
- type: nauc_mrr_at_3_std
value: -19.969872949099017
- type: nauc_mrr_at_5_diff1
value: 33.636627542554095
- type: nauc_mrr_at_5_max
value: -1.1455854862157713
- type: nauc_mrr_at_5_std
value: -20.04209565359184
- type: nauc_ndcg_at_1000_diff1
value: 32.82308597978432
- type: nauc_ndcg_at_1000_max
value: -0.12203773198562064
- type: nauc_ndcg_at_1000_std
value: -17.229564399373693
- type: nauc_ndcg_at_100_diff1
value: 32.57166969826798
- type: nauc_ndcg_at_100_max
value: -0.01373890925713605
- type: nauc_ndcg_at_100_std
value: -16.093465438938033
- type: nauc_ndcg_at_10_diff1
value: 32.27379155337616
- type: nauc_ndcg_at_10_max
value: -0.9566233125679434
- type: nauc_ndcg_at_10_std
value: -19.561143936038498
- type: nauc_ndcg_at_1_diff1
value: 37.59387362908585
- type: nauc_ndcg_at_1_max
value: -1.0442957647404616
- type: nauc_ndcg_at_1_std
value: -18.520116150105327
- type: nauc_ndcg_at_20_diff1
value: 32.35076358048651
- type: nauc_ndcg_at_20_max
value: -0.63852068352795
- type: nauc_ndcg_at_20_std
value: -18.079761843587626
- type: nauc_ndcg_at_3_diff1
value: 32.91961454543036
- type: nauc_ndcg_at_3_max
value: -1.2446184461552787
- type: nauc_ndcg_at_3_std
value: -20.38277231661361
- type: nauc_ndcg_at_5_diff1
value: 32.477306276423406
- type: nauc_ndcg_at_5_max
value: -1.2160759325945443
- type: nauc_ndcg_at_5_std
value: -20.531368354029883
- type: nauc_precision_at_1000_diff1
value: -1.9489638449852666
- type: nauc_precision_at_1000_max
value: 12.861002515930938
- type: nauc_precision_at_1000_std
value: 13.760895468120305
- type: nauc_precision_at_100_diff1
value: 13.908852716138604
- type: nauc_precision_at_100_max
value: 9.50012447045651
- type: nauc_precision_at_100_std
value: 15.47834789121318
- type: nauc_precision_at_10_diff1
value: 26.271089856624236
- type: nauc_precision_at_10_max
value: -0.02541438089917979
- type: nauc_precision_at_10_std
value: -17.853993814393686
- type: nauc_precision_at_1_diff1
value: 37.59387362908585
- type: nauc_precision_at_1_max
value: -1.0442957647404616
- type: nauc_precision_at_1_std
value: -18.520116150105327
- type: nauc_precision_at_20_diff1
value: 23.982859950627958
- type: nauc_precision_at_20_max
value: 1.7200918681449948
- type: nauc_precision_at_20_std
value: -10.219762717296879
- type: nauc_precision_at_3_diff1
value: 29.8854159409524
- type: nauc_precision_at_3_max
value: -1.1725503295220696
- type: nauc_precision_at_3_std
value: -21.35040743421613
- type: nauc_precision_at_5_diff1
value: 28.341473659669685
- type: nauc_precision_at_5_max
value: -0.9988570074323966
- type: nauc_precision_at_5_std
value: -21.383884015668407
- type: nauc_recall_at_1000_diff1
value: 15.0972588752424
- type: nauc_recall_at_1000_max
value: 45.008064248035815
- type: nauc_recall_at_1000_std
value: 66.96866515218399
- type: nauc_recall_at_100_diff1
value: 24.231388307049734
- type: nauc_recall_at_100_max
value: 11.009794013017734
- type: nauc_recall_at_100_std
value: 21.737276638818738
- type: nauc_recall_at_10_diff1
value: 27.872938692874644
- type: nauc_recall_at_10_max
value: -0.5261627570052085
- type: nauc_recall_at_10_std
value: -18.55331296014022
- type: nauc_recall_at_1_diff1
value: 37.6475234742761
- type: nauc_recall_at_1_max
value: -1.1172579533948037
- type: nauc_recall_at_1_std
value: -18.42423307405879
- type: nauc_recall_at_20_diff1
value: 27.159324157537974
- type: nauc_recall_at_20_max
value: 1.0199280899477339
- type: nauc_recall_at_20_std
value: -11.345867427864647
- type: nauc_recall_at_3_diff1
value: 30.205972726563445
- type: nauc_recall_at_3_max
value: -1.3526411987484857
- type: nauc_recall_at_3_std
value: -21.355225675024265
- type: nauc_recall_at_5_diff1
value: 29.048273808862973
- type: nauc_recall_at_5_max
value: -1.2994416614246815
- type: nauc_recall_at_5_std
value: -21.601337732799692
- type: ndcg_at_1
value: 20.687
- type: ndcg_at_10
value: 39.033
- type: ndcg_at_100
value: 45.12
- type: ndcg_at_1000
value: 46.338
- type: ndcg_at_20
value: 41.778999999999996
- type: ndcg_at_3
value: 31.029
- type: ndcg_at_5
value: 34.989
- type: precision_at_1
value: 20.687
- type: precision_at_10
value: 6.293
- type: precision_at_100
value: 0.9329999999999999
- type: precision_at_1000
value: 0.104
- type: precision_at_20
value: 3.7150000000000003
- type: precision_at_3
value: 13.3
- type: precision_at_5
value: 9.983
- type: recall_at_1
value: 20.124
- type: recall_at_10
value: 60.425
- type: recall_at_100
value: 88.67399999999999
- type: recall_at_1000
value: 97.964
- type: recall_at_20
value: 71.101
- type: recall_at_3
value: 38.582
- type: recall_at_5
value: 48.112
task:
type: Retrieval
- dataset:
config: default
name: MTEB NFCorpus (default)
revision: ec0fa4fe99da2ff19ca1214b7966684033a58814
split: test
type: mteb/nfcorpus
metrics:
- type: main_score
value: 35.914
- type: map_at_1
value: 6.23
- type: map_at_10
value: 13.855
- type: map_at_100
value: 17.25
- type: map_at_1000
value: 18.675
- type: map_at_20
value: 15.338
- type: map_at_3
value: 10.434000000000001
- type: map_at_5
value: 11.963
- type: mrr_at_1
value: 47.05882352941176
- type: mrr_at_10
value: 55.55936409651577
- type: mrr_at_100
value: 56.10896734305185
- type: mrr_at_1000
value: 56.152100966241626
- type: mrr_at_20
value: 55.94058014063476
- type: mrr_at_3
value: 53.71517027863776
- type: mrr_at_5
value: 54.845201238390075
- type: nauc_map_at_1000_diff1
value: 22.048916658189757
- type: nauc_map_at_1000_max
value: 30.15607235167445
- type: nauc_map_at_1000_std
value: 16.742606163211338
- type: nauc_map_at_100_diff1
value: 23.377920183289515
- type: nauc_map_at_100_max
value: 28.783691100858505
- type: nauc_map_at_100_std
value: 13.017478837252122
- type: nauc_map_at_10_diff1
value: 25.927128646993104
- type: nauc_map_at_10_max
value: 21.991506013370493
- type: nauc_map_at_10_std
value: 1.2876695985618782
- type: nauc_map_at_1_diff1
value: 33.63819624372805
- type: nauc_map_at_1_max
value: 10.63972433513814
- type: nauc_map_at_1_std
value: -8.823599413443773
- type: nauc_map_at_20_diff1
value: 24.890712149522994
- type: nauc_map_at_20_max
value: 25.180597197272135
- type: nauc_map_at_20_std
value: 5.627323445078693
- type: nauc_map_at_3_diff1
value: 29.74224701878483
- type: nauc_map_at_3_max
value: 15.68065548301368
- type: nauc_map_at_3_std
value: -6.678385500066188
- type: nauc_map_at_5_diff1
value: 27.599131430348827
- type: nauc_map_at_5_max
value: 18.074085171779384
- type: nauc_map_at_5_std
value: -3.7963346347735363
- type: nauc_mrr_at_1000_diff1
value: 25.481138216138305
- type: nauc_mrr_at_1000_max
value: 49.4025864603834
- type: nauc_mrr_at_1000_std
value: 34.43075929115974
- type: nauc_mrr_at_100_diff1
value: 25.48347783962907
- type: nauc_mrr_at_100_max
value: 49.44523125924117
- type: nauc_mrr_at_100_std
value: 34.470905475648
- type: nauc_mrr_at_10_diff1
value: 25.28075660798897
- type: nauc_mrr_at_10_max
value: 49.23726620276372
- type: nauc_mrr_at_10_std
value: 34.293562054296345
- type: nauc_mrr_at_1_diff1
value: 25.954637548939214
- type: nauc_mrr_at_1_max
value: 42.65790251316441
- type: nauc_mrr_at_1_std
value: 27.317146768790057
- type: nauc_mrr_at_20_diff1
value: 25.466827368526896
- type: nauc_mrr_at_20_max
value: 49.33319883504534
- type: nauc_mrr_at_20_std
value: 34.31574919492081
- type: nauc_mrr_at_3_diff1
value: 25.120590560812783
- type: nauc_mrr_at_3_max
value: 49.094872018177085
- type: nauc_mrr_at_3_std
value: 33.02343528683664
- type: nauc_mrr_at_5_diff1
value: 25.148546642057894
- type: nauc_mrr_at_5_max
value: 48.878846118647964
- type: nauc_mrr_at_5_std
value: 33.72884725641949
- type: nauc_ndcg_at_1000_diff1
value: 19.967381926216344
- type: nauc_ndcg_at_1000_max
value: 45.87058460060273
- type: nauc_ndcg_at_1000_std
value: 36.936736559315534
- type: nauc_ndcg_at_100_diff1
value: 20.78184115615848
- type: nauc_ndcg_at_100_max
value: 41.08856508861288
- type: nauc_ndcg_at_100_std
value: 30.990523544088127
- type: nauc_ndcg_at_10_diff1
value: 17.192295354962035
- type: nauc_ndcg_at_10_max
value: 40.28247408280954
- type: nauc_ndcg_at_10_std
value: 28.64688440994157
- type: nauc_ndcg_at_1_diff1
value: 27.183052982846096
- type: nauc_ndcg_at_1_max
value: 41.536795675945534
- type: nauc_ndcg_at_1_std
value: 27.457562076625493
- type: nauc_ndcg_at_20_diff1
value: 18.421321450770613
- type: nauc_ndcg_at_20_max
value: 39.553776018660415
- type: nauc_ndcg_at_20_std
value: 28.771965761754092
- type: nauc_ndcg_at_3_diff1
value: 19.90294061664522
- type: nauc_ndcg_at_3_max
value: 43.0248266231533
- type: nauc_ndcg_at_3_std
value: 26.948716619559164
- type: nauc_ndcg_at_5_diff1
value: 17.923910079839835
- type: nauc_ndcg_at_5_max
value: 41.45509816759182
- type: nauc_ndcg_at_5_std
value: 27.989207685876043
- type: nauc_precision_at_1000_diff1
value: -15.214665617300021
- type: nauc_precision_at_1000_max
value: 13.171793905978413
- type: nauc_precision_at_1000_std
value: 38.27602755705882
- type: nauc_precision_at_100_diff1
value: -8.983955562301023
- type: nauc_precision_at_100_max
value: 26.112480717392373
- type: nauc_precision_at_100_std
value: 49.109950492594926
- type: nauc_precision_at_10_diff1
value: 1.371271849123094
- type: nauc_precision_at_10_max
value: 40.48355301137806
- type: nauc_precision_at_10_std
value: 40.068967133929654
- type: nauc_precision_at_1_diff1
value: 25.954637548939214
- type: nauc_precision_at_1_max
value: 42.65790251316441
- type: nauc_precision_at_1_std
value: 27.317146768790057
- type: nauc_precision_at_20_diff1
value: -1.6000356148225012
- type: nauc_precision_at_20_max
value: 37.15543454435496
- type: nauc_precision_at_20_std
value: 43.35178597211817
- type: nauc_precision_at_3_diff1
value: 11.833482965709617
- type: nauc_precision_at_3_max
value: 44.957544267360014
- type: nauc_precision_at_3_std
value: 30.711555798910357
- type: nauc_precision_at_5_diff1
value: 5.4677770285267595
- type: nauc_precision_at_5_max
value: 41.437825214777845
- type: nauc_precision_at_5_std
value: 34.41100928387715
- type: nauc_recall_at_1000_diff1
value: 4.212258089142459
- type: nauc_recall_at_1000_max
value: 21.08308178695022
- type: nauc_recall_at_1000_std
value: 25.337416218601504
- type: nauc_recall_at_100_diff1
value: 16.779425065215055
- type: nauc_recall_at_100_max
value: 29.033119190767277
- type: nauc_recall_at_100_std
value: 22.797391132110352
- type: nauc_recall_at_10_diff1
value: 22.316078573103635
- type: nauc_recall_at_10_max
value: 18.639375064999562
- type: nauc_recall_at_10_std
value: -1.0471307716314715
- type: nauc_recall_at_1_diff1
value: 33.63819624372805
- type: nauc_recall_at_1_max
value: 10.63972433513814
- type: nauc_recall_at_1_std
value: -8.823599413443773
- type: nauc_recall_at_20_diff1
value: 22.98174920594852
- type: nauc_recall_at_20_max
value: 22.822017926055672
- type: nauc_recall_at_20_std
value: 4.910618245400064
- type: nauc_recall_at_3_diff1
value: 27.054112895077182
- type: nauc_recall_at_3_max
value: 14.268892599954997
- type: nauc_recall_at_3_std
value: -8.290166135248048
- type: nauc_recall_at_5_diff1
value: 23.89913124441014
- type: nauc_recall_at_5_max
value: 16.035332185114015
- type: nauc_recall_at_5_std
value: -4.780578865959435
- type: ndcg_at_1
value: 45.356
- type: ndcg_at_10
value: 35.914
- type: ndcg_at_100
value: 32.531
- type: ndcg_at_1000
value: 41.146
- type: ndcg_at_20
value: 33.592
- type: ndcg_at_3
value: 41.837
- type: ndcg_at_5
value: 39.505
- type: precision_at_1
value: 47.059
- type: precision_at_10
value: 26.316
- type: precision_at_100
value: 8.207
- type: precision_at_1000
value: 2.085
- type: precision_at_20
value: 19.505
- type: precision_at_3
value: 39.422000000000004
- type: precision_at_5
value: 34.117999999999995
- type: recall_at_1
value: 6.23
- type: recall_at_10
value: 17.557000000000002
- type: recall_at_100
value: 31.631999999999998
- type: recall_at_1000
value: 63.404
- type: recall_at_20
value: 21.22
- type: recall_at_3
value: 11.546
- type: recall_at_5
value: 14.002999999999998
task:
type: Retrieval
- dataset:
config: default
name: MTEB NQ (default)
revision: b774495ed302d8c44a3a7ea25c90dbce03968f31
split: test
type: mteb/nq
metrics:
- type: main_score
value: 57.056
- type: map_at_1
value: 32.830999999999996
- type: map_at_10
value: 49.151
- type: map_at_100
value: 50.044
- type: map_at_1000
value: 50.068
- type: map_at_20
value: 49.714999999999996
- type: map_at_3
value: 44.795
- type: map_at_5
value: 47.425
- type: mrr_at_1
value: 36.848203939745076
- type: mrr_at_10
value: 51.51708694292694
- type: mrr_at_100
value: 52.164267314569265
- type: mrr_at_1000
value: 52.180196526951036
- type: mrr_at_20
value: 51.9351113026156
- type: mrr_at_3
value: 48.01081498648112
- type: mrr_at_5
value: 50.14001544998062
- type: nauc_map_at_1000_diff1
value: 37.0626379154806
- type: nauc_map_at_1000_max
value: 27.815265359052066
- type: nauc_map_at_1000_std
value: -0.0024703483347415824
- type: nauc_map_at_100_diff1
value: 37.057558292153516
- type: nauc_map_at_100_max
value: 27.8317293639744
- type: nauc_map_at_100_std
value: 0.021742851211832327
- type: nauc_map_at_10_diff1
value: 37.008318697912216
- type: nauc_map_at_10_max
value: 27.76402847756217
- type: nauc_map_at_10_std
value: -0.3496637677317498
- type: nauc_map_at_1_diff1
value: 40.29366709249466
- type: nauc_map_at_1_max
value: 22.701930006989834
- type: nauc_map_at_1_std
value: -2.5108119794353128
- type: nauc_map_at_20_diff1
value: 37.01351948636887
- type: nauc_map_at_20_max
value: 27.90193629451335
- type: nauc_map_at_20_std
value: -0.012064407820230756
- type: nauc_map_at_3_diff1
value: 37.07052207561161
- type: nauc_map_at_3_max
value: 26.870611049843884
- type: nauc_map_at_3_std
value: -1.6823761406051834
- type: nauc_map_at_5_diff1
value: 36.87828509263161
- type: nauc_map_at_5_max
value: 27.20814519969873
- type: nauc_map_at_5_std
value: -1.3031956888196534
- type: nauc_mrr_at_1000_diff1
value: 36.84284176537722
- type: nauc_mrr_at_1000_max
value: 27.621378413295055
- type: nauc_mrr_at_1000_std
value: 1.0094506914584964
- type: nauc_mrr_at_100_diff1
value: 36.8396329466549
- type: nauc_mrr_at_100_max
value: 27.635468701117134
- type: nauc_mrr_at_100_std
value: 1.0308421119518538
- type: nauc_mrr_at_10_diff1
value: 36.74896766301788
- type: nauc_mrr_at_10_max
value: 27.65282647248496
- type: nauc_mrr_at_10_std
value: 0.9154106884376038
- type: nauc_mrr_at_1_diff1
value: 40.003629960440776
- type: nauc_mrr_at_1_max
value: 23.76711836119917
- type: nauc_mrr_at_1_std
value: -1.2836461535558703
- type: nauc_mrr_at_20_diff1
value: 36.751636613584324
- type: nauc_mrr_at_20_max
value: 27.70649796383429
- type: nauc_mrr_at_20_std
value: 1.0423663510625336
- type: nauc_mrr_at_3_diff1
value: 36.65443225179595
- type: nauc_mrr_at_3_max
value: 27.175270325334207
- type: nauc_mrr_at_3_std
value: 0.0035294075560518114
- type: nauc_mrr_at_5_diff1
value: 36.57153152515841
- type: nauc_mrr_at_5_max
value: 27.336322335217396
- type: nauc_mrr_at_5_std
value: 0.2580100531459355
- type: nauc_ndcg_at_1000_diff1
value: 36.1986232613817
- type: nauc_ndcg_at_1000_max
value: 29.298820739005727
- type: nauc_ndcg_at_1000_std
value: 2.223222439674335
- type: nauc_ndcg_at_100_diff1
value: 36.17635813364367
- type: nauc_ndcg_at_100_max
value: 29.82660135655654
- type: nauc_ndcg_at_100_std
value: 3.0428734005265414
- type: nauc_ndcg_at_10_diff1
value: 35.74907741690408
- type: nauc_ndcg_at_10_max
value: 29.814329710491332
- type: nauc_ndcg_at_10_std
value: 1.9069712029711603
- type: nauc_ndcg_at_1_diff1
value: 40.003629960440776
- type: nauc_ndcg_at_1_max
value: 23.76711836119917
- type: nauc_ndcg_at_1_std
value: -1.2836461535558703
- type: nauc_ndcg_at_20_diff1
value: 35.73948116889183
- type: nauc_ndcg_at_20_max
value: 30.335483791890184
- type: nauc_ndcg_at_20_std
value: 3.0004822530123225
- type: nauc_ndcg_at_3_diff1
value: 35.88097918041399
- type: nauc_ndcg_at_3_max
value: 27.979283201946647
- type: nauc_ndcg_at_3_std
value: -0.9746291149925899
- type: nauc_ndcg_at_5_diff1
value: 35.5313972391396
- type: nauc_ndcg_at_5_max
value: 28.550369140364047
- type: nauc_ndcg_at_5_std
value: -0.3673673143765844
- type: nauc_precision_at_1000_diff1
value: -9.738411913005308
- type: nauc_precision_at_1000_max
value: 5.3313906625669105
- type: nauc_precision_at_1000_std
value: 12.682972328170672
- type: nauc_precision_at_100_diff1
value: -4.156943433455653
- type: nauc_precision_at_100_max
value: 13.657850986924522
- type: nauc_precision_at_100_std
value: 18.65622972699929
- type: nauc_precision_at_10_diff1
value: 11.224515944438068
- type: nauc_precision_at_10_max
value: 25.251830466066664
- type: nauc_precision_at_10_std
value: 12.952562722393859
- type: nauc_precision_at_1_diff1
value: 40.003629960440776
- type: nauc_precision_at_1_max
value: 23.76711836119917
- type: nauc_precision_at_1_std
value: -1.2836461535558703
- type: nauc_precision_at_20_diff1
value: 4.851855622439396
- type: nauc_precision_at_20_max
value: 23.124541998877756
- type: nauc_precision_at_20_std
value: 17.78381782746071
- type: nauc_precision_at_3_diff1
value: 24.54315060981724
- type: nauc_precision_at_3_max
value: 29.024787047578688
- type: nauc_precision_at_3_std
value: 2.9413457351587358
- type: nauc_precision_at_5_diff1
value: 18.10984788563448
- type: nauc_precision_at_5_max
value: 26.76601771821192
- type: nauc_precision_at_5_std
value: 5.542771678992256
- type: nauc_recall_at_1000_diff1
value: 8.52588694953907
- type: nauc_recall_at_1000_max
value: 73.41790358239734
- type: nauc_recall_at_1000_std
value: 69.61314473084202
- type: nauc_recall_at_100_diff1
value: 28.871721524145038
- type: nauc_recall_at_100_max
value: 59.065368924170045
- type: nauc_recall_at_100_std
value: 49.80520453249347
- type: nauc_recall_at_10_diff1
value: 29.03782265490603
- type: nauc_recall_at_10_max
value: 37.75072625287093
- type: nauc_recall_at_10_std
value: 10.62020740095944
- type: nauc_recall_at_1_diff1
value: 40.29366709249466
- type: nauc_recall_at_1_max
value: 22.701930006989834
- type: nauc_recall_at_1_std
value: -2.5108119794353128
- type: nauc_recall_at_20_diff1
value: 27.404333563376582
- type: nauc_recall_at_20_max
value: 44.726703401012806
- type: nauc_recall_at_20_std
value: 20.97357245935641
- type: nauc_recall_at_3_diff1
value: 31.63227526877648
- type: nauc_recall_at_3_max
value: 29.884975719908446
- type: nauc_recall_at_3_std
value: -0.5990776007533871
- type: nauc_recall_at_5_diff1
value: 30.05426459795973
- type: nauc_recall_at_5_max
value: 31.52192768038204
- type: nauc_recall_at_5_std
value: 0.7177246623827043
- type: ndcg_at_1
value: 36.848
- type: ndcg_at_10
value: 57.056
- type: ndcg_at_100
value: 60.792
- type: ndcg_at_1000
value: 61.329
- type: ndcg_at_20
value: 58.857000000000006
- type: ndcg_at_3
value: 49.012
- type: ndcg_at_5
value: 53.327000000000005
- type: precision_at_1
value: 36.848
- type: precision_at_10
value: 9.368
- type: precision_at_100
value: 1.1480000000000001
- type: precision_at_1000
value: 0.12
- type: precision_at_20
value: 5.122999999999999
- type: precision_at_3
value: 22.470000000000002
- type: precision_at_5
value: 15.985
- type: recall_at_1
value: 32.830999999999996
- type: recall_at_10
value: 78.616
- type: recall_at_100
value: 94.798
- type: recall_at_1000
value: 98.71799999999999
- type: recall_at_20
value: 85.236
- type: recall_at_3
value: 57.935
- type: recall_at_5
value: 67.85900000000001
task:
type: Retrieval
- dataset:
config: default
name: MTEB QuoraRetrieval (default)
revision: e4e08e0b7dbe3c8700f0daef558ff32256715259
split: dev
type: mteb/quora
metrics:
- type: main_score
value: 88.727
- type: map_at_1
value: 70.762
- type: map_at_10
value: 85.11999999999999
- type: map_at_100
value: 85.664
- type: map_at_1000
value: 85.679
- type: map_at_20
value: 85.49600000000001
- type: map_at_3
value: 82.23700000000001
- type: map_at_5
value: 84.086
- type: mrr_at_1
value: 81.44
- type: mrr_at_10
value: 87.79252380952383
- type: mrr_at_100
value: 87.8869692638498
- type: mrr_at_1000
value: 87.88761260154055
- type: mrr_at_20
value: 87.85851690174293
- type: mrr_at_3
value: 86.90000000000015
- type: mrr_at_5
value: 87.52200000000002
- type: nauc_map_at_1000_diff1
value: 75.855543672975
- type: nauc_map_at_1000_max
value: 31.902412813269077
- type: nauc_map_at_1000_std
value: -43.545213020523484
- type: nauc_map_at_100_diff1
value: 75.85787106458596
- type: nauc_map_at_100_max
value: 31.872380932295602
- type: nauc_map_at_100_std
value: -43.62670414341416
- type: nauc_map_at_10_diff1
value: 76.06994047717338
- type: nauc_map_at_10_max
value: 31.50784501542894
- type: nauc_map_at_10_std
value: -45.246286655846895
- type: nauc_map_at_1_diff1
value: 80.02300553815451
- type: nauc_map_at_1_max
value: 21.754057272930545
- type: nauc_map_at_1_std
value: -40.21688684908
- type: nauc_map_at_20_diff1
value: 75.93312323976788
- type: nauc_map_at_20_max
value: 31.750666329847178
- type: nauc_map_at_20_std
value: -44.205099175719035
- type: nauc_map_at_3_diff1
value: 76.51176248218147
- type: nauc_map_at_3_max
value: 29.57618559850948
- type: nauc_map_at_3_std
value: -46.746508593043366
- type: nauc_map_at_5_diff1
value: 76.29609472997535
- type: nauc_map_at_5_max
value: 31.148450278161345
- type: nauc_map_at_5_std
value: -46.15412773135915
- type: nauc_mrr_at_1000_diff1
value: 76.58410473550458
- type: nauc_mrr_at_1000_max
value: 34.349405634766505
- type: nauc_mrr_at_1000_std
value: -38.71411943881118
- type: nauc_mrr_at_100_diff1
value: 76.58429204389566
- type: nauc_mrr_at_100_max
value: 34.34925219271846
- type: nauc_mrr_at_100_std
value: -38.71580773780872
- type: nauc_mrr_at_10_diff1
value: 76.63814613681649
- type: nauc_mrr_at_10_max
value: 34.48850149896092
- type: nauc_mrr_at_10_std
value: -38.7768979856703
- type: nauc_mrr_at_1_diff1
value: 77.20703493053294
- type: nauc_mrr_at_1_max
value: 33.323090298596966
- type: nauc_mrr_at_1_std
value: -37.60690944443488
- type: nauc_mrr_at_20_diff1
value: 76.59666662335822
- type: nauc_mrr_at_20_max
value: 34.34258983029893
- type: nauc_mrr_at_20_std
value: -38.767765407380175
- type: nauc_mrr_at_3_diff1
value: 76.36318737637336
- type: nauc_mrr_at_3_max
value: 34.47959555949338
- type: nauc_mrr_at_3_std
value: -39.14405090748447
- type: nauc_mrr_at_5_diff1
value: 76.4642433971879
- type: nauc_mrr_at_5_max
value: 34.41499543508058
- type: nauc_mrr_at_5_std
value: -38.71845104328312
- type: nauc_ndcg_at_1000_diff1
value: 75.86336366951063
- type: nauc_ndcg_at_1000_max
value: 33.144250465211336
- type: nauc_ndcg_at_1000_std
value: -41.063832282328185
- type: nauc_ndcg_at_100_diff1
value: 75.8485208193407
- type: nauc_ndcg_at_100_max
value: 32.98328804105153
- type: nauc_ndcg_at_100_std
value: -41.52631473748263
- type: nauc_ndcg_at_10_diff1
value: 76.18012008509577
- type: nauc_ndcg_at_10_max
value: 32.974938360415535
- type: nauc_ndcg_at_10_std
value: -44.38570601633005
- type: nauc_ndcg_at_1_diff1
value: 77.28230554694761
- type: nauc_ndcg_at_1_max
value: 33.03805178290934
- type: nauc_ndcg_at_1_std
value: -37.537640383402824
- type: nauc_ndcg_at_20_diff1
value: 76.02033529740791
- type: nauc_ndcg_at_20_max
value: 32.833700044235634
- type: nauc_ndcg_at_20_std
value: -42.96157266340273
- type: nauc_ndcg_at_3_diff1
value: 75.26730760149606
- type: nauc_ndcg_at_3_max
value: 32.694778968778714
- type: nauc_ndcg_at_3_std
value: -44.5390979610003
- type: nauc_ndcg_at_5_diff1
value: 75.79190092587773
- type: nauc_ndcg_at_5_max
value: 33.06366329445576
- type: nauc_ndcg_at_5_std
value: -44.77082674014946
- type: nauc_precision_at_1000_diff1
value: -43.96311909671976
- type: nauc_precision_at_1000_max
value: -2.6621016532032535
- type: nauc_precision_at_1000_std
value: 39.67056705670567
- type: nauc_precision_at_100_diff1
value: -43.66588505776995
- type: nauc_precision_at_100_max
value: -2.6282921990659553
- type: nauc_precision_at_100_std
value: 37.054574557190094
- type: nauc_precision_at_10_diff1
value: -38.35814005471572
- type: nauc_precision_at_10_max
value: 0.7737785747028663
- type: nauc_precision_at_10_std
value: 25.250342985520945
- type: nauc_precision_at_1_diff1
value: 77.28230554694761
- type: nauc_precision_at_1_max
value: 33.03805178290934
- type: nauc_precision_at_1_std
value: -37.537640383402824
- type: nauc_precision_at_20_diff1
value: -41.30112891380059
- type: nauc_precision_at_20_max
value: -1.2060605616858489
- type: nauc_precision_at_20_std
value: 31.13148488999237
- type: nauc_precision_at_3_diff1
value: -20.140620198671257
- type: nauc_precision_at_3_max
value: 10.264712638763026
- type: nauc_precision_at_3_std
value: 7.331085213423879
- type: nauc_precision_at_5_diff1
value: -31.808640731944752
- type: nauc_precision_at_5_max
value: 5.297575496536465
- type: nauc_precision_at_5_std
value: 17.70286397606476
- type: nauc_recall_at_1000_diff1
value: 84.98581822669564
- type: nauc_recall_at_1000_max
value: 92.96636893753929
- type: nauc_recall_at_1000_std
value: 75.65227348803414
- type: nauc_recall_at_100_diff1
value: 68.36540923623107
- type: nauc_recall_at_100_max
value: 22.593370935999022
- type: nauc_recall_at_100_std
value: -70.87387778855451
- type: nauc_recall_at_10_diff1
value: 74.81283971432373
- type: nauc_recall_at_10_max
value: 32.69264273824776
- type: nauc_recall_at_10_std
value: -63.595111340014235
- type: nauc_recall_at_1_diff1
value: 80.02300553815451
- type: nauc_recall_at_1_max
value: 21.754057272930545
- type: nauc_recall_at_1_std
value: -40.21688684908
- type: nauc_recall_at_20_diff1
value: 74.80351474558462
- type: nauc_recall_at_20_max
value: 30.865356023355805
- type: nauc_recall_at_20_std
value: -64.32287367180344
- type: nauc_recall_at_3_diff1
value: 72.84287000152072
- type: nauc_recall_at_3_max
value: 28.816036268394075
- type: nauc_recall_at_3_std
value: -53.30450226597958
- type: nauc_recall_at_5_diff1
value: 71.85726172240405
- type: nauc_recall_at_5_max
value: 30.410801075288813
- type: nauc_recall_at_5_std
value: -56.351216833849406
- type: ndcg_at_1
value: 81.39999999999999
- type: ndcg_at_10
value: 88.727
- type: ndcg_at_100
value: 89.821
- type: ndcg_at_1000
value: 89.915
- type: ndcg_at_20
value: 89.34
- type: ndcg_at_3
value: 86.021
- type: ndcg_at_5
value: 87.558
- type: precision_at_1
value: 81.39999999999999
- type: precision_at_10
value: 13.41
- type: precision_at_100
value: 1.488
- type: precision_at_1000
value: 0.152
- type: precision_at_20
value: 7.061000000000001
- type: precision_at_3
value: 37.793
- type: precision_at_5
value: 24.804000000000002
- type: recall_at_1
value: 70.762
- type: recall_at_10
value: 95.846
- type: recall_at_100
value: 99.58500000000001
- type: recall_at_1000
value: 99.98700000000001
- type: recall_at_20
value: 97.773
- type: recall_at_3
value: 88.08500000000001
- type: recall_at_5
value: 92.45100000000001
task:
type: Retrieval
- dataset:
config: default
name: MTEB QuoraRetrieval (default)
revision: e4e08e0b7dbe3c8700f0daef558ff32256715259
split: test
type: mteb/quora
metrics:
- type: main_score
value: 89.003
- type: map_at_1
value: 71.419
- type: map_at_10
value: 85.382
- type: map_at_100
value: 85.976
- type: map_at_1000
value: 85.992
- type: map_at_20
value: 85.763
- type: map_at_3
value: 82.429
- type: map_at_5
value: 84.284
- type: mrr_at_1
value: 82.11
- type: mrr_at_10
value: 88.19897222222195
- type: mrr_at_100
value: 88.28645985411698
- type: mrr_at_1000
value: 88.28718663213955
- type: mrr_at_20
value: 88.26697522996409
- type: mrr_at_3
value: 87.27833333333302
- type: mrr_at_5
value: 87.91783333333296
- type: nauc_map_at_1000_diff1
value: 76.30261631592856
- type: nauc_map_at_1000_max
value: 34.980388292905786
- type: nauc_map_at_1000_std
value: -46.40601966994713
- type: nauc_map_at_100_diff1
value: 76.30695928935073
- type: nauc_map_at_100_max
value: 34.96244762174972
- type: nauc_map_at_100_std
value: -46.45454317042469
- type: nauc_map_at_10_diff1
value: 76.53649680864336
- type: nauc_map_at_10_max
value: 34.645533775546085
- type: nauc_map_at_10_std
value: -48.33008289480228
- type: nauc_map_at_1_diff1
value: 80.0109014135849
- type: nauc_map_at_1_max
value: 25.217498739752532
- type: nauc_map_at_1_std
value: -42.86995539685312
- type: nauc_map_at_20_diff1
value: 76.38636530247655
- type: nauc_map_at_20_max
value: 34.8747291704767
- type: nauc_map_at_20_std
value: -47.18148102037872
- type: nauc_map_at_3_diff1
value: 77.05849657272363
- type: nauc_map_at_3_max
value: 32.277142026099796
- type: nauc_map_at_3_std
value: -49.96987316659713
- type: nauc_map_at_5_diff1
value: 76.83629534126555
- type: nauc_map_at_5_max
value: 33.55739070811488
- type: nauc_map_at_5_std
value: -49.75050547962666
- type: nauc_mrr_at_1000_diff1
value: 77.09217511243467
- type: nauc_mrr_at_1000_max
value: 37.21107946309872
- type: nauc_mrr_at_1000_std
value: -43.530006121662296
- type: nauc_mrr_at_100_diff1
value: 77.09240459620786
- type: nauc_mrr_at_100_max
value: 37.21225118498791
- type: nauc_mrr_at_100_std
value: -43.53027090575838
- type: nauc_mrr_at_10_diff1
value: 77.06801760001858
- type: nauc_mrr_at_10_max
value: 37.286872397183544
- type: nauc_mrr_at_10_std
value: -43.617192424810156
- type: nauc_mrr_at_1_diff1
value: 78.14939860730118
- type: nauc_mrr_at_1_max
value: 36.345419894891194
- type: nauc_mrr_at_1_std
value: -41.791157198407774
- type: nauc_mrr_at_20_diff1
value: 77.0877040646715
- type: nauc_mrr_at_20_max
value: 37.22505075639358
- type: nauc_mrr_at_20_std
value: -43.56227666998121
- type: nauc_mrr_at_3_diff1
value: 76.74698348186915
- type: nauc_mrr_at_3_max
value: 37.44311099432756
- type: nauc_mrr_at_3_std
value: -43.7071965891009
- type: nauc_mrr_at_5_diff1
value: 77.0330357115108
- type: nauc_mrr_at_5_max
value: 37.52629069454836
- type: nauc_mrr_at_5_std
value: -43.73165678865622
- type: nauc_ndcg_at_1000_diff1
value: 76.13346466270058
- type: nauc_ndcg_at_1000_max
value: 36.30355111630145
- type: nauc_ndcg_at_1000_std
value: -44.567293200490255
- type: nauc_ndcg_at_100_diff1
value: 76.13003686446783
- type: nauc_ndcg_at_100_max
value: 36.24229918244899
- type: nauc_ndcg_at_100_std
value: -44.74039437592027
- type: nauc_ndcg_at_10_diff1
value: 76.15098624138862
- type: nauc_ndcg_at_10_max
value: 35.9426014751766
- type: nauc_ndcg_at_10_std
value: -47.59425076949253
- type: nauc_ndcg_at_1_diff1
value: 78.05404952179576
- type: nauc_ndcg_at_1_max
value: 36.386170501037306
- type: nauc_ndcg_at_1_std
value: -41.608020194302405
- type: nauc_ndcg_at_20_diff1
value: 76.12680119152564
- type: nauc_ndcg_at_20_max
value: 36.00419296655895
- type: nauc_ndcg_at_20_std
value: -46.42441388656108
- type: nauc_ndcg_at_3_diff1
value: 75.63820138905012
- type: nauc_ndcg_at_3_max
value: 35.289885704662126
- type: nauc_ndcg_at_3_std
value: -47.23927265939333
- type: nauc_ndcg_at_5_diff1
value: 76.12723497005791
- type: nauc_ndcg_at_5_max
value: 35.36445509855448
- type: nauc_ndcg_at_5_std
value: -48.35208930977703
- type: nauc_precision_at_1000_diff1
value: -44.30605136183297
- type: nauc_precision_at_1000_max
value: -5.750277685604905
- type: nauc_precision_at_1000_std
value: 39.95015688272993
- type: nauc_precision_at_100_diff1
value: -43.99883635441125
- type: nauc_precision_at_100_max
value: -5.404491662976336
- type: nauc_precision_at_100_std
value: 38.70820720987971
- type: nauc_precision_at_10_diff1
value: -39.281636489637265
- type: nauc_precision_at_10_max
value: -0.26078620229618693
- type: nauc_precision_at_10_std
value: 27.60706091974357
- type: nauc_precision_at_1_diff1
value: 78.05404952179576
- type: nauc_precision_at_1_max
value: 36.386170501037306
- type: nauc_precision_at_1_std
value: -41.608020194302405
- type: nauc_precision_at_20_diff1
value: -42.131775738221116
- type: nauc_precision_at_20_max
value: -2.790751064610159
- type: nauc_precision_at_20_std
value: 33.49240881529144
- type: nauc_precision_at_3_diff1
value: -19.582037618539495
- type: nauc_precision_at_3_max
value: 9.075373870080982
- type: nauc_precision_at_3_std
value: 6.435940772534901
- type: nauc_precision_at_5_diff1
value: -31.802765106740484
- type: nauc_precision_at_5_max
value: 3.543116294761505
- type: nauc_precision_at_5_std
value: 17.833033175636356
- type: nauc_recall_at_1000_diff1
value: 39.32485644932125
- type: nauc_recall_at_1000_max
value: 39.066540922772376
- type: nauc_recall_at_1000_std
value: 20.103965232560835
- type: nauc_recall_at_100_diff1
value: 70.0251051590138
- type: nauc_recall_at_100_max
value: 34.86379347929933
- type: nauc_recall_at_100_std
value: -42.106311600640375
- type: nauc_recall_at_10_diff1
value: 72.10740423001593
- type: nauc_recall_at_10_max
value: 34.886521080051295
- type: nauc_recall_at_10_std
value: -64.83306899253033
- type: nauc_recall_at_1_diff1
value: 80.0109014135849
- type: nauc_recall_at_1_max
value: 25.217498739752532
- type: nauc_recall_at_1_std
value: -42.86995539685312
- type: nauc_recall_at_20_diff1
value: 71.20093193828812
- type: nauc_recall_at_20_max
value: 33.46686672295811
- type: nauc_recall_at_20_std
value: -63.66982813610845
- type: nauc_recall_at_3_diff1
value: 72.73621188192767
- type: nauc_recall_at_3_max
value: 30.82541835421874
- type: nauc_recall_at_3_std
value: -55.217641745582725
- type: nauc_recall_at_5_diff1
value: 72.08303977300797
- type: nauc_recall_at_5_max
value: 31.70942758945847
- type: nauc_recall_at_5_std
value: -60.86547648856454
- type: ndcg_at_1
value: 82.16
- type: ndcg_at_10
value: 89.003
- type: ndcg_at_100
value: 90.109
- type: ndcg_at_1000
value: 90.205
- type: ndcg_at_20
value: 89.584
- type: ndcg_at_3
value: 86.21799999999999
- type: ndcg_at_5
value: 87.766
- type: precision_at_1
value: 82.16
- type: precision_at_10
value: 13.485
- type: precision_at_100
value: 1.529
- type: precision_at_1000
value: 0.157
- type: precision_at_20
value: 7.119000000000001
- type: precision_at_3
value: 37.657000000000004
- type: precision_at_5
value: 24.759999999999998
- type: recall_at_1
value: 71.419
- type: recall_at_10
value: 95.855
- type: recall_at_100
value: 99.57000000000001
- type: recall_at_1000
value: 99.984
- type: recall_at_20
value: 97.723
- type: recall_at_3
value: 87.914
- type: recall_at_5
value: 92.27300000000001
task:
type: Retrieval
- dataset:
config: default
name: MTEB SCIDOCS (default)
revision: f8c2fcf00f625baaa80f62ec5bd9e1fff3b8ae88
split: test
type: mteb/scidocs
metrics:
- type: main_score
value: 21.095
- type: map_at_1
value: 4.858
- type: map_at_10
value: 12.676000000000002
- type: map_at_100
value: 14.827000000000002
- type: map_at_1000
value: 15.146999999999998
- type: map_at_20
value: 13.767
- type: map_at_3
value: 8.968
- type: map_at_5
value: 10.748000000000001
- type: mrr_at_1
value: 24
- type: mrr_at_10
value: 35.44234126984123
- type: mrr_at_100
value: 36.57227499851092
- type: mrr_at_1000
value: 36.62212146550235
- type: mrr_at_20
value: 36.174427675437656
- type: mrr_at_3
value: 32.366666666666646
- type: mrr_at_5
value: 34.02166666666662
- type: nauc_map_at_1000_diff1
value: 20.4331406159799
- type: nauc_map_at_1000_max
value: 27.81489710879907
- type: nauc_map_at_1000_std
value: 20.440258897270585
- type: nauc_map_at_100_diff1
value: 20.46388418195963
- type: nauc_map_at_100_max
value: 27.75644498016236
- type: nauc_map_at_100_std
value: 20.186852999395906
- type: nauc_map_at_10_diff1
value: 20.34246509025442
- type: nauc_map_at_10_max
value: 26.09709012239771
- type: nauc_map_at_10_std
value: 16.18755590764125
- type: nauc_map_at_1_diff1
value: 30.238455629371675
- type: nauc_map_at_1_max
value: 19.771326252470793
- type: nauc_map_at_1_std
value: 9.646028512988151
- type: nauc_map_at_20_diff1
value: 20.443407591914497
- type: nauc_map_at_20_max
value: 27.219474051523424
- type: nauc_map_at_20_std
value: 18.16404214650777
- type: nauc_map_at_3_diff1
value: 23.783610410420746
- type: nauc_map_at_3_max
value: 22.806294777997753
- type: nauc_map_at_3_std
value: 10.220963066295825
- type: nauc_map_at_5_diff1
value: 22.390000403879384
- type: nauc_map_at_5_max
value: 25.124629804394328
- type: nauc_map_at_5_std
value: 12.769708421601889
- type: nauc_mrr_at_1000_diff1
value: 26.335786873134847
- type: nauc_mrr_at_1000_max
value: 23.468402127347108
- type: nauc_mrr_at_1000_std
value: 15.358807703656174
- type: nauc_mrr_at_100_diff1
value: 26.33882075438782
- type: nauc_mrr_at_100_max
value: 23.4826794899285
- type: nauc_mrr_at_100_std
value: 15.391948819124835
- type: nauc_mrr_at_10_diff1
value: 26.32246417057101
- type: nauc_mrr_at_10_max
value: 23.370043401649195
- type: nauc_mrr_at_10_std
value: 15.195884523143832
- type: nauc_mrr_at_1_diff1
value: 29.957764198418413
- type: nauc_mrr_at_1_max
value: 19.67783968368079
- type: nauc_mrr_at_1_std
value: 9.9110352264558
- type: nauc_mrr_at_20_diff1
value: 26.280188638909667
- type: nauc_mrr_at_20_max
value: 23.518821014040608
- type: nauc_mrr_at_20_std
value: 15.392630335191393
- type: nauc_mrr_at_3_diff1
value: 26.174221020389133
- type: nauc_mrr_at_3_max
value: 22.916897852904736
- type: nauc_mrr_at_3_std
value: 14.054111276765582
- type: nauc_mrr_at_5_diff1
value: 26.681081196240864
- type: nauc_mrr_at_5_max
value: 23.545290286687205
- type: nauc_mrr_at_5_std
value: 14.489255948218496
- type: nauc_ndcg_at_1000_diff1
value: 19.811292218568877
- type: nauc_ndcg_at_1000_max
value: 29.271783843120513
- type: nauc_ndcg_at_1000_std
value: 27.918160529053438
- type: nauc_ndcg_at_100_diff1
value: 20.54710323459629
- type: nauc_ndcg_at_100_max
value: 29.36192055253
- type: nauc_ndcg_at_100_std
value: 27.01510356917553
- type: nauc_ndcg_at_10_diff1
value: 20.679368997415185
- type: nauc_ndcg_at_10_max
value: 26.88373404836461
- type: nauc_ndcg_at_10_std
value: 18.962516505087503
- type: nauc_ndcg_at_1_diff1
value: 29.957764198418413
- type: nauc_ndcg_at_1_max
value: 19.67783968368079
- type: nauc_ndcg_at_1_std
value: 9.9110352264558
- type: nauc_ndcg_at_20_diff1
value: 20.770661537577425
- type: nauc_ndcg_at_20_max
value: 28.494160060607204
- type: nauc_ndcg_at_20_std
value: 21.8279540883511
- type: nauc_ndcg_at_3_diff1
value: 23.47639094473142
- type: nauc_ndcg_at_3_max
value: 23.35439798694039
- type: nauc_ndcg_at_3_std
value: 12.377439535136311
- type: nauc_ndcg_at_5_diff1
value: 22.969926936390664
- type: nauc_ndcg_at_5_max
value: 26.319631914812362
- type: nauc_ndcg_at_5_std
value: 14.911857377719349
- type: nauc_precision_at_1000_diff1
value: 3.470857299493788
- type: nauc_precision_at_1000_max
value: 23.42686850704682
- type: nauc_precision_at_1000_std
value: 40.4031184750418
- type: nauc_precision_at_100_diff1
value: 11.912902631258339
- type: nauc_precision_at_100_max
value: 27.49134680565924
- type: nauc_precision_at_100_std
value: 37.51792335831407
- type: nauc_precision_at_10_diff1
value: 14.781962138018267
- type: nauc_precision_at_10_max
value: 27.25371904985
- type: nauc_precision_at_10_std
value: 23.116512723007414
- type: nauc_precision_at_1_diff1
value: 29.957764198418413
- type: nauc_precision_at_1_max
value: 19.67783968368079
- type: nauc_precision_at_1_std
value: 9.9110352264558
- type: nauc_precision_at_20_diff1
value: 14.26689464164839
- type: nauc_precision_at_20_max
value: 28.827184932489935
- type: nauc_precision_at_20_std
value: 27.446889793638707
- type: nauc_precision_at_3_diff1
value: 20.71336447156449
- type: nauc_precision_at_3_max
value: 24.337994526159783
- type: nauc_precision_at_3_std
value: 13.397947548460653
- type: nauc_precision_at_5_diff1
value: 19.138062665558067
- type: nauc_precision_at_5_max
value: 27.822407403400668
- type: nauc_precision_at_5_std
value: 16.803790719167182
- type: nauc_recall_at_1000_diff1
value: 3.6018207583688144
- type: nauc_recall_at_1000_max
value: 23.14090214625772
- type: nauc_recall_at_1000_std
value: 42.28594835129414
- type: nauc_recall_at_100_diff1
value: 12.001413115432843
- type: nauc_recall_at_100_max
value: 27.33100640666986
- type: nauc_recall_at_100_std
value: 37.96014315095026
- type: nauc_recall_at_10_diff1
value: 14.588748658431552
- type: nauc_recall_at_10_max
value: 26.973195318900334
- type: nauc_recall_at_10_std
value: 23.07536701691167
- type: nauc_recall_at_1_diff1
value: 30.238455629371675
- type: nauc_recall_at_1_max
value: 19.771326252470793
- type: nauc_recall_at_1_std
value: 9.646028512988151
- type: nauc_recall_at_20_diff1
value: 14.196956274661462
- type: nauc_recall_at_20_max
value: 28.56336128336403
- type: nauc_recall_at_20_std
value: 27.534839405528803
- type: nauc_recall_at_3_diff1
value: 20.73309785564631
- type: nauc_recall_at_3_max
value: 24.29211194985757
- type: nauc_recall_at_3_std
value: 13.16260070090019
- type: nauc_recall_at_5_diff1
value: 19.05897924330305
- type: nauc_recall_at_5_max
value: 27.739868919446675
- type: nauc_recall_at_5_std
value: 16.676587166110888
- type: ndcg_at_1
value: 24
- type: ndcg_at_10
value: 21.095
- type: ndcg_at_100
value: 29.32
- type: ndcg_at_1000
value: 34.908
- type: ndcg_at_20
value: 24.104999999999997
- type: ndcg_at_3
value: 20.051
- type: ndcg_at_5
value: 17.424999999999997
- type: precision_at_1
value: 24
- type: precision_at_10
value: 10.99
- type: precision_at_100
value: 2.279
- type: precision_at_1000
value: 0.363
- type: precision_at_20
value: 7.234999999999999
- type: precision_at_3
value: 18.8
- type: precision_at_5
value: 15.340000000000002
- type: recall_at_1
value: 4.858
- type: recall_at_10
value: 22.252
- type: recall_at_100
value: 46.28
- type: recall_at_1000
value: 73.642
- type: recall_at_20
value: 29.304999999999996
- type: recall_at_3
value: 11.413
- type: recall_at_5
value: 15.518
task:
type: Retrieval
- dataset:
config: default
name: MTEB SciFact (default)
revision: 0228b52cf27578f30900b9e5271d331663a030d7
split: test
type: mteb/scifact
metrics:
- type: main_score
value: 72.98599999999999
- type: map_at_1
value: 59.594
- type: map_at_10
value: 68.605
- type: map_at_100
value: 69.15400000000001
- type: map_at_1000
value: 69.17699999999999
- type: map_at_20
value: 68.958
- type: map_at_3
value: 65.657
- type: map_at_5
value: 67.391
- type: mrr_at_1
value: 62.66666666666667
- type: mrr_at_10
value: 69.88624338624336
- type: mrr_at_100
value: 70.30767689431102
- type: mrr_at_1000
value: 70.32817618304256
- type: mrr_at_20
value: 70.15422258053835
- type: mrr_at_3
value: 67.66666666666666
- type: mrr_at_5
value: 68.93333333333331
- type: nauc_map_at_1000_diff1
value: 74.34793985711579
- type: nauc_map_at_1000_max
value: 55.53156114873401
- type: nauc_map_at_1000_std
value: 11.582923341718212
- type: nauc_map_at_100_diff1
value: 74.35690644257943
- type: nauc_map_at_100_max
value: 55.55273405679522
- type: nauc_map_at_100_std
value: 11.602959136140353
- type: nauc_map_at_10_diff1
value: 74.33699696988701
- type: nauc_map_at_10_max
value: 55.82035169294064
- type: nauc_map_at_10_std
value: 11.204305801905747
- type: nauc_map_at_1_diff1
value: 75.32294245224915
- type: nauc_map_at_1_max
value: 45.76150065668323
- type: nauc_map_at_1_std
value: 2.0949461653318115
- type: nauc_map_at_20_diff1
value: 74.28684168063077
- type: nauc_map_at_20_max
value: 55.704668970482786
- type: nauc_map_at_20_std
value: 11.578414086112025
- type: nauc_map_at_3_diff1
value: 74.07592950237665
- type: nauc_map_at_3_max
value: 52.00387951737786
- type: nauc_map_at_3_std
value: 7.951275505368885
- type: nauc_map_at_5_diff1
value: 74.12694586808122
- type: nauc_map_at_5_max
value: 53.89041592367371
- type: nauc_map_at_5_std
value: 9.85361506601376
- type: nauc_mrr_at_1000_diff1
value: 76.20784870583682
- type: nauc_mrr_at_1000_max
value: 57.570046380224525
- type: nauc_mrr_at_1000_std
value: 14.296417717967314
- type: nauc_mrr_at_100_diff1
value: 76.21296168893583
- type: nauc_mrr_at_100_max
value: 57.58697212436348
- type: nauc_mrr_at_100_std
value: 14.308752123915275
- type: nauc_mrr_at_10_diff1
value: 76.14459199922076
- type: nauc_mrr_at_10_max
value: 57.92786644451955
- type: nauc_mrr_at_10_std
value: 14.43313795702205
- type: nauc_mrr_at_1_diff1
value: 77.57106372375983
- type: nauc_mrr_at_1_max
value: 54.29444986607418
- type: nauc_mrr_at_1_std
value: 10.375096032059131
- type: nauc_mrr_at_20_diff1
value: 76.11275491211235
- type: nauc_mrr_at_20_max
value: 57.74339972696968
- type: nauc_mrr_at_20_std
value: 14.476904912902203
- type: nauc_mrr_at_3_diff1
value: 76.66682045455855
- type: nauc_mrr_at_3_max
value: 56.3735469447152
- type: nauc_mrr_at_3_std
value: 12.513240176317824
- type: nauc_mrr_at_5_diff1
value: 75.97086683174106
- type: nauc_mrr_at_5_max
value: 57.35499477476572
- type: nauc_mrr_at_5_std
value: 14.355734187199426
- type: nauc_ndcg_at_1000_diff1
value: 74.58448144929329
- type: nauc_ndcg_at_1000_max
value: 57.80110953681458
- type: nauc_ndcg_at_1000_std
value: 14.508773733544983
- type: nauc_ndcg_at_100_diff1
value: 74.69167071376367
- type: nauc_ndcg_at_100_max
value: 58.25671346950714
- type: nauc_ndcg_at_100_std
value: 15.047276165536436
- type: nauc_ndcg_at_10_diff1
value: 74.37843244426729
- type: nauc_ndcg_at_10_max
value: 59.80082249051946
- type: nauc_ndcg_at_10_std
value: 14.41431492985276
- type: nauc_ndcg_at_1_diff1
value: 77.57106372375983
- type: nauc_ndcg_at_1_max
value: 54.29444986607418
- type: nauc_ndcg_at_1_std
value: 10.375096032059131
- type: nauc_ndcg_at_20_diff1
value: 74.20570209855893
- type: nauc_ndcg_at_20_max
value: 59.22812358366338
- type: nauc_ndcg_at_20_std
value: 15.36619407153703
- type: nauc_ndcg_at_3_diff1
value: 74.33412987664815
- type: nauc_ndcg_at_3_max
value: 55.29579906848679
- type: nauc_ndcg_at_3_std
value: 10.47948670506532
- type: nauc_ndcg_at_5_diff1
value: 73.981771086719
- type: nauc_ndcg_at_5_max
value: 56.68614205577981
- type: nauc_ndcg_at_5_std
value: 12.629079173374707
- type: nauc_precision_at_1000_diff1
value: -18.395851293463455
- type: nauc_precision_at_1000_max
value: 28.809719521241394
- type: nauc_precision_at_1000_std
value: 51.99983961186071
- type: nauc_precision_at_100_diff1
value: -2.5512788047606887
- type: nauc_precision_at_100_max
value: 37.44561338889012
- type: nauc_precision_at_100_std
value: 51.03823752848824
- type: nauc_precision_at_10_diff1
value: 22.441073512252004
- type: nauc_precision_at_10_max
value: 59.24802281861788
- type: nauc_precision_at_10_std
value: 44.011409308958896
- type: nauc_precision_at_1_diff1
value: 77.57106372375983
- type: nauc_precision_at_1_max
value: 54.29444986607418
- type: nauc_precision_at_1_std
value: 10.375096032059131
- type: nauc_precision_at_20_diff1
value: 10.437628418310032
- type: nauc_precision_at_20_max
value: 49.2713253828718
- type: nauc_precision_at_20_std
value: 48.37583935266579
- type: nauc_precision_at_3_diff1
value: 50.803456772984426
- type: nauc_precision_at_3_max
value: 58.62730612669874
- type: nauc_precision_at_3_std
value: 26.895597721266746
- type: nauc_precision_at_5_diff1
value: 36.04230980927072
- type: nauc_precision_at_5_max
value: 56.29693445329882
- type: nauc_precision_at_5_std
value: 35.108047957498705
- type: nauc_recall_at_1000_diff1
value: 54.0149393090572
- type: nauc_recall_at_1000_max
value: 100
- type: nauc_recall_at_1000_std
value: 79.57516339869238
- type: nauc_recall_at_100_diff1
value: 73.30200897563304
- type: nauc_recall_at_100_max
value: 76.2236077226591
- type: nauc_recall_at_100_std
value: 40.6810035842295
- type: nauc_recall_at_10_diff1
value: 70.44148430346728
- type: nauc_recall_at_10_max
value: 76.05796661965898
- type: nauc_recall_at_10_std
value: 23.916714017800025
- type: nauc_recall_at_1_diff1
value: 75.32294245224915
- type: nauc_recall_at_1_max
value: 45.76150065668323
- type: nauc_recall_at_1_std
value: 2.0949461653318115
- type: nauc_recall_at_20_diff1
value: 68.40526023895922
- type: nauc_recall_at_20_max
value: 77.14770987304526
- type: nauc_recall_at_20_std
value: 32.9602859795282
- type: nauc_recall_at_3_diff1
value: 72.14683033702258
- type: nauc_recall_at_3_max
value: 54.78390680638642
- type: nauc_recall_at_3_std
value: 8.544474503532333
- type: nauc_recall_at_5_diff1
value: 70.71753125793401
- type: nauc_recall_at_5_max
value: 61.86036751930198
- type: nauc_recall_at_5_std
value: 16.623728043559797
- type: ndcg_at_1
value: 62.666999999999994
- type: ndcg_at_10
value: 72.98599999999999
- type: ndcg_at_100
value: 75.202
- type: ndcg_at_1000
value: 75.777
- type: ndcg_at_20
value: 74.126
- type: ndcg_at_3
value: 67.938
- type: ndcg_at_5
value: 70.407
- type: precision_at_1
value: 62.666999999999994
- type: precision_at_10
value: 9.6
- type: precision_at_100
value: 1.077
- type: precision_at_1000
value: 0.11199999999999999
- type: precision_at_20
value: 5.083
- type: precision_at_3
value: 26.111
- type: precision_at_5
value: 17.4
- type: recall_at_1
value: 59.594
- type: recall_at_10
value: 85.289
- type: recall_at_100
value: 94.833
- type: recall_at_1000
value: 99.333
- type: recall_at_20
value: 89.60000000000001
- type: recall_at_3
value: 71.506
- type: recall_at_5
value: 77.678
task:
type: Retrieval
- dataset:
config: default
name: MTEB SciFact (default)
revision: 0228b52cf27578f30900b9e5271d331663a030d7
split: train
type: mteb/scifact
metrics:
- type: main_score
value: 75.303
- type: map_at_1
value: 60.541999999999994
- type: map_at_10
value: 70.875
- type: map_at_100
value: 71.34899999999999
- type: map_at_1000
value: 71.363
- type: map_at_20
value: 71.173
- type: map_at_3
value: 68.219
- type: map_at_5
value: 69.76299999999999
- type: mrr_at_1
value: 63.16440049443758
- type: mrr_at_10
value: 71.95214550591557
- type: mrr_at_100
value: 72.31606780067733
- type: mrr_at_1000
value: 72.32564107428016
- type: mrr_at_20
value: 72.1512249900027
- type: mrr_at_3
value: 70.41615162752367
- type: mrr_at_5
value: 71.22579316028018
- type: nauc_map_at_1000_diff1
value: 70.88713432478681
- type: nauc_map_at_1000_max
value: 52.53874359900521
- type: nauc_map_at_1000_std
value: 16.14520822345089
- type: nauc_map_at_100_diff1
value: 70.88012809608153
- type: nauc_map_at_100_max
value: 52.54180031390171
- type: nauc_map_at_100_std
value: 16.162778040895844
- type: nauc_map_at_10_diff1
value: 70.74086998198797
- type: nauc_map_at_10_max
value: 52.42087394405289
- type: nauc_map_at_10_std
value: 15.73032520368658
- type: nauc_map_at_1_diff1
value: 74.69022426235868
- type: nauc_map_at_1_max
value: 45.482748252259306
- type: nauc_map_at_1_std
value: 9.47910187753849
- type: nauc_map_at_20_diff1
value: 70.84375569937194
- type: nauc_map_at_20_max
value: 52.65470234006966
- type: nauc_map_at_20_std
value: 16.183949074222614
- type: nauc_map_at_3_diff1
value: 70.48156645011187
- type: nauc_map_at_3_max
value: 49.10946388704575
- type: nauc_map_at_3_std
value: 13.397746184472078
- type: nauc_map_at_5_diff1
value: 70.75376736251015
- type: nauc_map_at_5_max
value: 51.05160957611554
- type: nauc_map_at_5_std
value: 14.236721249905903
- type: nauc_mrr_at_1000_diff1
value: 71.86940171966855
- type: nauc_mrr_at_1000_max
value: 54.42365652404636
- type: nauc_mrr_at_1000_std
value: 19.139697673245294
- type: nauc_mrr_at_100_diff1
value: 71.85926994801977
- type: nauc_mrr_at_100_max
value: 54.42199024809323
- type: nauc_mrr_at_100_std
value: 19.15306050030586
- type: nauc_mrr_at_10_diff1
value: 71.677284518898
- type: nauc_mrr_at_10_max
value: 54.47972900834183
- type: nauc_mrr_at_10_std
value: 19.315857249475428
- type: nauc_mrr_at_1_diff1
value: 75.9486142319394
- type: nauc_mrr_at_1_max
value: 51.90965243989473
- type: nauc_mrr_at_1_std
value: 15.395164964289846
- type: nauc_mrr_at_20_diff1
value: 71.79361460184204
- type: nauc_mrr_at_20_max
value: 54.51770420915609
- type: nauc_mrr_at_20_std
value: 19.208132629404325
- type: nauc_mrr_at_3_diff1
value: 71.52900861523246
- type: nauc_mrr_at_3_max
value: 54.89638098144882
- type: nauc_mrr_at_3_std
value: 19.734894394106117
- type: nauc_mrr_at_5_diff1
value: 71.64011255032078
- type: nauc_mrr_at_5_max
value: 54.31300515852635
- type: nauc_mrr_at_5_std
value: 19.062319641899336
- type: nauc_ndcg_at_1000_diff1
value: 70.25707604268838
- type: nauc_ndcg_at_1000_max
value: 54.24108864068546
- type: nauc_ndcg_at_1000_std
value: 19.02009727870368
- type: nauc_ndcg_at_100_diff1
value: 69.99779255019226
- type: nauc_ndcg_at_100_max
value: 54.346337390971954
- type: nauc_ndcg_at_100_std
value: 19.629886017293348
- type: nauc_ndcg_at_10_diff1
value: 69.3211450566187
- type: nauc_ndcg_at_10_max
value: 54.483831909106264
- type: nauc_ndcg_at_10_std
value: 18.753325678992873
- type: nauc_ndcg_at_1_diff1
value: 75.9486142319394
- type: nauc_ndcg_at_1_max
value: 51.90965243989473
- type: nauc_ndcg_at_1_std
value: 15.395164964289846
- type: nauc_ndcg_at_20_diff1
value: 69.71886726771191
- type: nauc_ndcg_at_20_max
value: 55.08856925504423
- type: nauc_ndcg_at_20_std
value: 19.6244990884976
- type: nauc_ndcg_at_3_diff1
value: 68.87322312255968
- type: nauc_ndcg_at_3_max
value: 51.39103027630223
- type: nauc_ndcg_at_3_std
value: 16.52935384582361
- type: nauc_ndcg_at_5_diff1
value: 69.3341220735468
- type: nauc_ndcg_at_5_max
value: 52.37066281526067
- type: nauc_ndcg_at_5_std
value: 16.404288595035418
- type: nauc_precision_at_1000_diff1
value: -27.34291495528546
- type: nauc_precision_at_1000_max
value: 21.754965941678847
- type: nauc_precision_at_1000_std
value: 39.806731788937896
- type: nauc_precision_at_100_diff1
value: -19.194290975508313
- type: nauc_precision_at_100_max
value: 27.96377640344495
- type: nauc_precision_at_100_std
value: 44.01990976097701
- type: nauc_precision_at_10_diff1
value: 5.896041466783285
- type: nauc_precision_at_10_max
value: 41.38368704222518
- type: nauc_precision_at_10_std
value: 36.763658242409406
- type: nauc_precision_at_1_diff1
value: 75.9486142319394
- type: nauc_precision_at_1_max
value: 51.90965243989473
- type: nauc_precision_at_1_std
value: 15.395164964289846
- type: nauc_precision_at_20_diff1
value: -0.8746880135384988
- type: nauc_precision_at_20_max
value: 38.55198238563861
- type: nauc_precision_at_20_std
value: 40.34293450504436
- type: nauc_precision_at_3_diff1
value: 37.74271208206134
- type: nauc_precision_at_3_max
value: 51.53293019310014
- type: nauc_precision_at_3_std
value: 31.13615617572581
- type: nauc_precision_at_5_diff1
value: 22.682077409998058
- type: nauc_precision_at_5_max
value: 45.671662469549105
- type: nauc_precision_at_5_std
value: 30.657373390819238
- type: nauc_recall_at_1000_diff1
value: 50.86308938461632
- type: nauc_recall_at_1000_max
value: 85.58163972588446
- type: nauc_recall_at_1000_std
value: 96.73157724628308
- type: nauc_recall_at_100_diff1
value: 54.20328324824615
- type: nauc_recall_at_100_max
value: 64.72579598280349
- type: nauc_recall_at_100_std
value: 61.79554052598576
- type: nauc_recall_at_10_diff1
value: 60.001499108422365
- type: nauc_recall_at_10_max
value: 59.388945417047104
- type: nauc_recall_at_10_std
value: 25.67892843978966
- type: nauc_recall_at_1_diff1
value: 74.69022426235868
- type: nauc_recall_at_1_max
value: 45.482748252259306
- type: nauc_recall_at_1_std
value: 9.47910187753849
- type: nauc_recall_at_20_diff1
value: 61.02809427841475
- type: nauc_recall_at_20_max
value: 65.73824567078161
- type: nauc_recall_at_20_std
value: 33.177127516035355
- type: nauc_recall_at_3_diff1
value: 62.59553887931729
- type: nauc_recall_at_3_max
value: 49.04871287782435
- type: nauc_recall_at_3_std
value: 16.254788998158205
- type: nauc_recall_at_5_diff1
value: 61.92441342077279
- type: nauc_recall_at_5_max
value: 50.9072289346588
- type: nauc_recall_at_5_std
value: 16.472102532060642
- type: ndcg_at_1
value: 63.164
- type: ndcg_at_10
value: 75.303
- type: ndcg_at_100
value: 77.388
- type: ndcg_at_1000
value: 77.727
- type: ndcg_at_20
value: 76.201
- type: ndcg_at_3
value: 71.044
- type: ndcg_at_5
value: 73.086
- type: precision_at_1
value: 63.164
- type: precision_at_10
value: 9.913
- type: precision_at_100
value: 1.101
- type: precision_at_1000
value: 0.11299999999999999
- type: precision_at_20
value: 5.1610000000000005
- type: precision_at_3
value: 27.894999999999996
- type: precision_at_5
value: 18.096
- type: recall_at_1
value: 60.541999999999994
- type: recall_at_10
value: 87.394
- type: recall_at_100
value: 96.88900000000001
- type: recall_at_1000
value: 99.506
- type: recall_at_20
value: 90.74600000000001
- type: recall_at_3
value: 76.393
- type: recall_at_5
value: 81.08
task:
type: Retrieval
- dataset:
config: default
name: MTEB TRECCOVID (default)
revision: bb9466bac8153a0349341eb1b22e06409e78ef4e
split: test
type: mteb/trec-covid
metrics:
- type: main_score
value: 82.838
- type: map_at_1
value: 0.251
- type: map_at_10
value: 2.165
- type: map_at_100
value: 12.354
- type: map_at_1000
value: 29.576
- type: map_at_20
value: 3.881
- type: map_at_3
value: 0.728
- type: map_at_5
value: 1.16
- type: mrr_at_1
value: 94
- type: mrr_at_10
value: 97
- type: mrr_at_100
value: 97
- type: mrr_at_1000
value: 97
- type: mrr_at_20
value: 97
- type: mrr_at_3
value: 97
- type: mrr_at_5
value: 97
- type: nauc_map_at_1000_diff1
value: -12.291556746894083
- type: nauc_map_at_1000_max
value: 58.410079466278965
- type: nauc_map_at_1000_std
value: 76.4746309144675
- type: nauc_map_at_100_diff1
value: -7.101871793520939
- type: nauc_map_at_100_max
value: 36.39771723741155
- type: nauc_map_at_100_std
value: 58.39339407401992
- type: nauc_map_at_10_diff1
value: -7.107352291658639
- type: nauc_map_at_10_max
value: -5.168031273784038
- type: nauc_map_at_10_std
value: 1.6874476582807705
- type: nauc_map_at_1_diff1
value: -6.925641373795307
- type: nauc_map_at_1_max
value: -16.441779478543896
- type: nauc_map_at_1_std
value: -10.474198968429116
- type: nauc_map_at_20_diff1
value: 1.5882506468964699
- type: nauc_map_at_20_max
value: 5.236313065500864
- type: nauc_map_at_20_std
value: 11.192711690463574
- type: nauc_map_at_3_diff1
value: -11.27725483179556
- type: nauc_map_at_3_max
value: -15.784380537162276
- type: nauc_map_at_3_std
value: -6.139446268064151
- type: nauc_map_at_5_diff1
value: -8.885561530316084
- type: nauc_map_at_5_max
value: -11.120358811665087
- type: nauc_map_at_5_std
value: -2.4135832895961995
- type: nauc_mrr_at_1000_diff1
value: -43.27731092437014
- type: nauc_mrr_at_1000_max
value: 32.25957049486496
- type: nauc_mrr_at_1000_std
value: 46.09399315281668
- type: nauc_mrr_at_100_diff1
value: -43.27731092437014
- type: nauc_mrr_at_100_max
value: 32.25957049486496
- type: nauc_mrr_at_100_std
value: 46.09399315281668
- type: nauc_mrr_at_10_diff1
value: -43.27731092437014
- type: nauc_mrr_at_10_max
value: 32.25957049486496
- type: nauc_mrr_at_10_std
value: 46.09399315281668
- type: nauc_mrr_at_1_diff1
value: -43.27731092436961
- type: nauc_mrr_at_1_max
value: 32.25957049486467
- type: nauc_mrr_at_1_std
value: 46.0939931528168
- type: nauc_mrr_at_20_diff1
value: -43.27731092437014
- type: nauc_mrr_at_20_max
value: 32.25957049486496
- type: nauc_mrr_at_20_std
value: 46.09399315281668
- type: nauc_mrr_at_3_diff1
value: -43.27731092437014
- type: nauc_mrr_at_3_max
value: 32.25957049486496
- type: nauc_mrr_at_3_std
value: 46.09399315281668
- type: nauc_mrr_at_5_diff1
value: -43.27731092437014
- type: nauc_mrr_at_5_max
value: 32.25957049486496
- type: nauc_mrr_at_5_std
value: 46.09399315281668
- type: nauc_ndcg_at_1000_diff1
value: -7.751556353880079
- type: nauc_ndcg_at_1000_max
value: 55.19026713468263
- type: nauc_ndcg_at_1000_std
value: 71.89952836445738
- type: nauc_ndcg_at_100_diff1
value: -25.04610843246315
- type: nauc_ndcg_at_100_max
value: 57.81019038299022
- type: nauc_ndcg_at_100_std
value: 76.08337029715734
- type: nauc_ndcg_at_10_diff1
value: -24.320444869696587
- type: nauc_ndcg_at_10_max
value: 45.733366593718706
- type: nauc_ndcg_at_10_std
value: 63.83197521298889
- type: nauc_ndcg_at_1_diff1
value: -2.2434179973565302
- type: nauc_ndcg_at_1_max
value: 18.89758153824172
- type: nauc_ndcg_at_1_std
value: 47.66548780052153
- type: nauc_ndcg_at_20_diff1
value: -15.739057357565608
- type: nauc_ndcg_at_20_max
value: 50.7706563011848
- type: nauc_ndcg_at_20_std
value: 63.55990312197961
- type: nauc_ndcg_at_3_diff1
value: -33.933097033234354
- type: nauc_ndcg_at_3_max
value: 26.964996703046246
- type: nauc_ndcg_at_3_std
value: 57.994876101611084
- type: nauc_ndcg_at_5_diff1
value: -22.851195043529472
- type: nauc_ndcg_at_5_max
value: 38.105128987050094
- type: nauc_ndcg_at_5_std
value: 62.943604828448585
- type: nauc_precision_at_1000_diff1
value: -16.566185636380467
- type: nauc_precision_at_1000_max
value: 54.001267515174554
- type: nauc_precision_at_1000_std
value: 48.99114275701267
- type: nauc_precision_at_100_diff1
value: -22.079909752890607
- type: nauc_precision_at_100_max
value: 60.72350574637335
- type: nauc_precision_at_100_std
value: 74.71317275812814
- type: nauc_precision_at_10_diff1
value: -24.67136917830623
- type: nauc_precision_at_10_max
value: 62.492642593544126
- type: nauc_precision_at_10_std
value: 63.282103984677974
- type: nauc_precision_at_1_diff1
value: -43.27731092436961
- type: nauc_precision_at_1_max
value: 32.25957049486467
- type: nauc_precision_at_1_std
value: 46.0939931528168
- type: nauc_precision_at_20_diff1
value: -10.837325491312193
- type: nauc_precision_at_20_max
value: 58.342424386156274
- type: nauc_precision_at_20_std
value: 62.94368290407747
- type: nauc_precision_at_3_diff1
value: -79.57580938134424
- type: nauc_precision_at_3_max
value: 48.46137407842722
- type: nauc_precision_at_3_std
value: 63.00886846885357
- type: nauc_precision_at_5_diff1
value: -29.041010931079686
- type: nauc_precision_at_5_max
value: 62.63017804997827
- type: nauc_precision_at_5_std
value: 64.82673075432224
- type: nauc_recall_at_1000_diff1
value: 0.8039533307403914
- type: nauc_recall_at_1000_max
value: 45.959450629336544
- type: nauc_recall_at_1000_std
value: 57.956398183989776
- type: nauc_recall_at_100_diff1
value: -1.2240331094549868
- type: nauc_recall_at_100_max
value: 20.244295844408505
- type: nauc_recall_at_100_std
value: 39.348498358223566
- type: nauc_recall_at_10_diff1
value: -5.246748847223987
- type: nauc_recall_at_10_max
value: -10.956091017853266
- type: nauc_recall_at_10_std
value: -5.697512598601129
- type: nauc_recall_at_1_diff1
value: -6.925641373795307
- type: nauc_recall_at_1_max
value: -16.441779478543896
- type: nauc_recall_at_1_std
value: -10.474198968429116
- type: nauc_recall_at_20_diff1
value: 3.8478123109449007
- type: nauc_recall_at_20_max
value: -4.064799516275798
- type: nauc_recall_at_20_std
value: 1.2108812991924096
- type: nauc_recall_at_3_diff1
value: -9.004282138713936
- type: nauc_recall_at_3_max
value: -18.008369184840184
- type: nauc_recall_at_3_std
value: -8.821151791530783
- type: nauc_recall_at_5_diff1
value: -4.770943590883584
- type: nauc_recall_at_5_max
value: -14.512914457838649
- type: nauc_recall_at_5_std
value: -7.116030398151972
- type: ndcg_at_1
value: 86
- type: ndcg_at_10
value: 82.838
- type: ndcg_at_100
value: 62.611000000000004
- type: ndcg_at_1000
value: 54.777
- type: ndcg_at_20
value: 79.27
- type: ndcg_at_3
value: 88.011
- type: ndcg_at_5
value: 86.52
- type: precision_at_1
value: 94
- type: precision_at_10
value: 87.2
- type: precision_at_100
value: 64.48
- type: precision_at_1000
value: 24.168
- type: precision_at_20
value: 83.7
- type: precision_at_3
value: 94
- type: precision_at_5
value: 91.60000000000001
- type: recall_at_1
value: 0.251
- type: recall_at_10
value: 2.303
- type: recall_at_100
value: 15.248000000000001
- type: recall_at_1000
value: 51.12
- type: recall_at_20
value: 4.247
- type: recall_at_3
value: 0.743
- type: recall_at_5
value: 1.201
task:
type: Retrieval
- dataset:
config: default
name: MTEB Touche2020 (default)
revision: a34f9a33db75fa0cbb21bb5cfc3dae8dc8bec93f
split: test
type: mteb/touche2020
metrics:
- type: main_score
value: 30.5
- type: map_at_1
value: 3.5700000000000003
- type: map_at_10
value: 12.913
- type: map_at_100
value: 20.259
- type: map_at_1000
value: 21.834999999999997
- type: map_at_20
value: 16.241
- type: map_at_3
value: 7.027
- type: map_at_5
value: 9.499
- type: mrr_at_1
value: 46.93877551020408
- type: mrr_at_10
value: 59.007936507936506
- type: mrr_at_100
value: 59.69658022721581
- type: mrr_at_1000
value: 59.69658022721581
- type: mrr_at_20
value: 59.455483947965135
- type: mrr_at_3
value: 55.44217687074829
- type: mrr_at_5
value: 58.29931972789115
- type: nauc_map_at_1000_diff1
value: 9.910510369504589
- type: nauc_map_at_1000_max
value: -5.800076879675597
- type: nauc_map_at_1000_std
value: 0.16322588275794053
- type: nauc_map_at_100_diff1
value: 9.656724513623926
- type: nauc_map_at_100_max
value: -6.584268120579212
- type: nauc_map_at_100_std
value: -2.6872802242536356
- type: nauc_map_at_10_diff1
value: 0.8036112204161583
- type: nauc_map_at_10_max
value: -2.160210350827288
- type: nauc_map_at_10_std
value: -20.0383888763022
- type: nauc_map_at_1_diff1
value: -14.746819188938662
- type: nauc_map_at_1_max
value: -18.36805682308219
- type: nauc_map_at_1_std
value: -22.512487408760773
- type: nauc_map_at_20_diff1
value: 8.885434478073687
- type: nauc_map_at_20_max
value: -5.604548063353215
- type: nauc_map_at_20_std
value: -15.144012685613262
- type: nauc_map_at_3_diff1
value: -7.047758972132971
- type: nauc_map_at_3_max
value: -15.561815701245369
- type: nauc_map_at_3_std
value: -30.427604119738294
- type: nauc_map_at_5_diff1
value: -1.82067867508142
- type: nauc_map_at_5_max
value: -5.262858250449523
- type: nauc_map_at_5_std
value: -24.233957421815187
- type: nauc_mrr_at_1000_diff1
value: -1.9260668149825904
- type: nauc_mrr_at_1000_max
value: -30.096543546430805
- type: nauc_mrr_at_1000_std
value: -14.970045391543408
- type: nauc_mrr_at_100_diff1
value: -1.9260668149825904
- type: nauc_mrr_at_100_max
value: -30.096543546430805
- type: nauc_mrr_at_100_std
value: -14.970045391543408
- type: nauc_mrr_at_10_diff1
value: -2.6073416358968546
- type: nauc_mrr_at_10_max
value: -30.945119000446585
- type: nauc_mrr_at_10_std
value: -15.455244376688151
- type: nauc_mrr_at_1_diff1
value: -6.281249674145899
- type: nauc_mrr_at_1_max
value: -23.95679641665857
- type: nauc_mrr_at_1_std
value: -15.909791421206295
- type: nauc_mrr_at_20_diff1
value: -1.26835503224596
- type: nauc_mrr_at_20_max
value: -30.332143078341034
- type: nauc_mrr_at_20_std
value: -14.161343406093216
- type: nauc_mrr_at_3_diff1
value: -4.088059284739747
- type: nauc_mrr_at_3_max
value: -29.245685372711044
- type: nauc_mrr_at_3_std
value: -19.21966851951373
- type: nauc_mrr_at_5_diff1
value: -1.857784149253863
- type: nauc_mrr_at_5_max
value: -29.188383926944155
- type: nauc_mrr_at_5_std
value: -14.947155704014756
- type: nauc_ndcg_at_1000_diff1
value: 10.830782596268012
- type: nauc_ndcg_at_1000_max
value: -10.76019834247598
- type: nauc_ndcg_at_1000_std
value: 22.15401405704995
- type: nauc_ndcg_at_100_diff1
value: 13.25601986425452
- type: nauc_ndcg_at_100_max
value: -15.982671587208033
- type: nauc_ndcg_at_100_std
value: 16.718111839342363
- type: nauc_ndcg_at_10_diff1
value: 5.5819315259551106
- type: nauc_ndcg_at_10_max
value: -11.952721652517152
- type: nauc_ndcg_at_10_std
value: -10.474085815762676
- type: nauc_ndcg_at_1_diff1
value: -7.87103555085007
- type: nauc_ndcg_at_1_max
value: -24.704858588989737
- type: nauc_ndcg_at_1_std
value: -14.423209358330777
- type: nauc_ndcg_at_20_diff1
value: 12.404680386239459
- type: nauc_ndcg_at_20_max
value: -14.858231638706112
- type: nauc_ndcg_at_20_std
value: -6.1945024023267985
- type: nauc_ndcg_at_3_diff1
value: 1.0264436945119229
- type: nauc_ndcg_at_3_max
value: -20.50535984245989
- type: nauc_ndcg_at_3_std
value: -21.642653573064518
- type: nauc_ndcg_at_5_diff1
value: 10.221120686687174
- type: nauc_ndcg_at_5_max
value: -14.010693047359307
- type: nauc_ndcg_at_5_std
value: -12.003565488476767
- type: nauc_precision_at_1000_diff1
value: 1.580001818770179
- type: nauc_precision_at_1000_max
value: 30.699747280072366
- type: nauc_precision_at_1000_std
value: 41.68874332471044
- type: nauc_precision_at_100_diff1
value: 22.31315643692314
- type: nauc_precision_at_100_max
value: 2.2145528979842104
- type: nauc_precision_at_100_std
value: 71.87311377290379
- type: nauc_precision_at_10_diff1
value: 15.67863381009855
- type: nauc_precision_at_10_max
value: -7.373383448098349
- type: nauc_precision_at_10_std
value: -1.5181897666741904
- type: nauc_precision_at_1_diff1
value: -6.281249674145899
- type: nauc_precision_at_1_max
value: -23.95679641665857
- type: nauc_precision_at_1_std
value: -15.909791421206295
- type: nauc_precision_at_20_diff1
value: 34.529345904394106
- type: nauc_precision_at_20_max
value: -4.595919949898392
- type: nauc_precision_at_20_std
value: 24.29710948849588
- type: nauc_precision_at_3_diff1
value: 4.667561353420005
- type: nauc_precision_at_3_max
value: -15.153394928732745
- type: nauc_precision_at_3_std
value: -26.737561411956428
- type: nauc_precision_at_5_diff1
value: 18.24121315157291
- type: nauc_precision_at_5_max
value: -5.679163199756771
- type: nauc_precision_at_5_std
value: -8.569605319267955
- type: nauc_recall_at_1000_diff1
value: 7.867956430137323
- type: nauc_recall_at_1000_max
value: 2.560546440892355
- type: nauc_recall_at_1000_std
value: 75.64459996887818
- type: nauc_recall_at_100_diff1
value: 14.288210441761784
- type: nauc_recall_at_100_max
value: -16.26081030523676
- type: nauc_recall_at_100_std
value: 34.0896349160997
- type: nauc_recall_at_10_diff1
value: 2.5271969875579505
- type: nauc_recall_at_10_max
value: -7.819191985653094
- type: nauc_recall_at_10_std
value: -16.257378614680743
- type: nauc_recall_at_1_diff1
value: -14.746819188938662
- type: nauc_recall_at_1_max
value: -18.36805682308219
- type: nauc_recall_at_1_std
value: -22.512487408760773
- type: nauc_recall_at_20_diff1
value: 16.480978109572987
- type: nauc_recall_at_20_max
value: -12.524717173853151
- type: nauc_recall_at_20_std
value: -4.285217048975553
- type: nauc_recall_at_3_diff1
value: -7.306370544521303
- type: nauc_recall_at_3_max
value: -20.06310095683196
- type: nauc_recall_at_3_std
value: -32.65155580982558
- type: nauc_recall_at_5_diff1
value: 1.2245135180902773
- type: nauc_recall_at_5_max
value: -7.751072966262725
- type: nauc_recall_at_5_std
value: -20.872612147846596
- type: ndcg_at_1
value: 43.878
- type: ndcg_at_10
value: 30.5
- type: ndcg_at_100
value: 42.821999999999996
- type: ndcg_at_1000
value: 53.6
- type: ndcg_at_20
value: 32.037
- type: ndcg_at_3
value: 36.179
- type: ndcg_at_5
value: 34.337
- type: precision_at_1
value: 46.939
- type: precision_at_10
value: 25.714
- type: precision_at_100
value: 8.469
- type: precision_at_1000
value: 1.584
- type: precision_at_20
value: 20.714
- type: precision_at_3
value: 37.415
- type: precision_at_5
value: 34.694
- type: recall_at_1
value: 3.5700000000000003
- type: recall_at_10
value: 18.536
- type: recall_at_100
value: 52.36300000000001
- type: recall_at_1000
value: 85.10600000000001
- type: recall_at_20
value: 28.322999999999997
- type: recall_at_3
value: 8.129
- type: recall_at_5
value: 12.350999999999999
task:
type: Retrieval
tags:
- mteb
license: mit
Combination of Embedding Models: Arctic S & BGE Small (en; v1.5)
Acknowledgement | Combination of Embedding Models | Usage | Citation | License
Acknowledgement
First of all, we want to acknowledge the original creators of the Snowflake/snowflake-arctic-embed-s and BAAI/bge-base-en-v1.5 models which are used to create this model. Our model is just a combination of these two models, and we have not made any changes to the original models.
Furthermore, we want to acknowledge the team of Marqo, who has worked on the idea of combining two models through concatenation in parallel to ourselves. Their initial effort allowed to re-use existing pieces of code, in particular the modeling script for bringing the combined model to HuggingFace.
Combination of Embedding Models
Overview
Embedding models have become increasingly powerful and applicable across various use cases. However, the next significant challenge lies in enhancing their efficiency in terms of resource consumption. Our goal is to experiment with combining two embedding models to achieve better performance with fewer resources.
Key Insights
- Diversity Matters: Initial findings suggest that combining models with differing characteristics can complement each other, resulting in improved outcomes. To design an effective combination, the diversity of the models—evaluated by factors like MTEB performance, architecture, and training data—is crucial.
- Combination Technique:
- We combine the embeddings of two models using the most straightforward approach: concatenation.
- Prior to concatenation, we normalize the embeddings to ensure they are on the same scale. This step is vital for achieving coherent and meaningful results.
Implementation
We combined the following models:
Model Details
- Output Embedding Dimensions: 768 (384 + 384)
- Total Parameters: 66M (33M + 33M)
Future Directions
While the results are promising, we acknowledge the complexity of model combinations and the importance of focusing on more than leaderboard rankings. The simplicity of concatenating embeddings yielding tangible gains emphasizes the potential for further exploration in this area.
We look forward to conducting additional experiments and engaging in discussions to deepen our understanding of effective model combinations.
Usage
Please refer to our first concatenated model, Arctic M (v1.5) & BGE small, for more details on how to use this model.
Citation
@misc{https://doi.org/10.48550/arxiv.2407.08275,
doi = {10.48550/ARXIV.2407.08275},
url = {https://arxiv.org/abs/2407.08275},
author = {Caspari, Laura and Dastidar, Kanishka Ghosh and Zerhoudi, Saber and Mitrovic, Jelena and Granitzer, Michael},
title = {Beyond Benchmarks: Evaluating Embedding Model Similarity for Retrieval Augmented Generation Systems},
year = {2024},
copyright = {Creative Commons Attribution 4.0 International}
}
License
Notice that Arctic S is licensed under Apache-2.0 and BGE Small (en; v1.5) is licensed under MIT license. Please refer to the licenses of the original models for more details.