Add Data use & integrity rule: prohibit overlapping external data (HeshamHaroon/Arabic_Function_Calling); verification of top submissions
Browse files- index.html +16 -0
index.html
CHANGED
|
@@ -1548,6 +1548,22 @@ print(ds[<span class="s">"train"</span>][<span class="s">0</span>])</div>
|
|
| 1548 |
<div class="domain-tag"><div class="d-name">🔧 Utilities</div><div class="d-tools">translate_text · calculate_end_of_service</div></div>
|
| 1549 |
</div>
|
| 1550 |
<p class="section-sub" style="margin-top:24px;">The full tool registry has 27 tools: 20 are actively called in gold labels, and 7 appear only as distractors in candidate sets: <code>calculate_inheritance</code>, <code>get_directions</code>, <code>get_current_time</code>, <code>get_time</code>, <code>get_population</code>, <code>get_birthday</code>, and <code>currency_conversion</code>.</p>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1551 |
</div>
|
| 1552 |
</section>
|
| 1553 |
|
|
|
|
| 1548 |
<div class="domain-tag"><div class="d-name">🔧 Utilities</div><div class="d-tools">translate_text · calculate_end_of_service</div></div>
|
| 1549 |
</div>
|
| 1550 |
<p class="section-sub" style="margin-top:24px;">The full tool registry has 27 tools: 20 are actively called in gold labels, and 7 appear only as distractors in candidate sets: <code>calculate_inheritance</code>, <code>get_directions</code>, <code>get_current_time</code>, <code>get_time</code>, <code>get_population</code>, <code>get_birthday</code>, and <code>currency_conversion</code>.</p>
|
| 1551 |
+
|
| 1552 |
+
<div style="margin-top:48px; border:1px solid var(--red); border-radius:14px; padding:26px 30px; background:rgba(255,129,129,0.06);">
|
| 1553 |
+
<h3 style="font-size:20px; font-weight:700; margin:0 0 10px 0;">🔒 Data use & integrity (important)</h3>
|
| 1554 |
+
<p style="color:var(--muted); font-size:15px; margin:0 0 12px 0;">
|
| 1555 |
+
The test phase is <strong>blind</strong>. To keep the competition fair for everyone, the following rules apply to all teams and all tracks:
|
| 1556 |
+
</p>
|
| 1557 |
+
<ul style="color:var(--muted); font-size:15px; line-height:1.7; margin:0 0 12px 20px;">
|
| 1558 |
+
<li>You <strong>may</strong> use general-purpose pretrained models and resources that do <strong>not</strong> contain the test items and their gold labels.</li>
|
| 1559 |
+
<li>You <strong>may not</strong> train on, query, or otherwise use any function-calling dataset that overlaps with the task data — in particular <code>HeshamHaroon/Arabic_Function_Calling</code> and any dataset derived from the same source — because these contain test items together with their labels.</li>
|
| 1560 |
+
<li>Looking up or reconstructing the gold answers for test items from any external source is <strong>prohibited</strong>.</li>
|
| 1561 |
+
<li>Systems should be trained on the provided <code>train</code> and <code>dev</code> splits; any additional resources must be disclosed in your system-description paper.</li>
|
| 1562 |
+
</ul>
|
| 1563 |
+
<p style="color:var(--muted); font-size:15px; margin:0;">
|
| 1564 |
+
Top-ranked submissions will be <strong>verified</strong> by reproducing predictions from the submitted model and code, and we run integrity checks against label leakage. Violations invalidate a submission and may be reported to the ArabicNLP 2026 Shared Task Chairs.
|
| 1565 |
+
</p>
|
| 1566 |
+
</div>
|
| 1567 |
</div>
|
| 1568 |
</section>
|
| 1569 |
|