Migrate model card from transformers-repo
Browse filesRead announcement at https://discuss.huggingface.co/t/announcement-all-model-cards-will-be-migrated-to-hf-co-model-repos/2755
Original file history: https://github.com/huggingface/transformers/commits/master/model_cards/elgeish/cs224n-squad2.0-roberta-base/README.md
README.md
ADDED
|
@@ -0,0 +1,86 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
## CS224n SQuAD2.0 Project Dataset
|
| 2 |
+
The goal of this model is to save CS224n students GPU time when establishing
|
| 3 |
+
baselines to beat for the [Default Final Project](http://web.stanford.edu/class/cs224n/project/default-final-project-handout.pdf).
|
| 4 |
+
The training set used to fine-tune this model is the same as
|
| 5 |
+
the [official one](https://rajpurkar.github.io/SQuAD-explorer/); however,
|
| 6 |
+
evaluation and model selection were performed using roughly half of the official
|
| 7 |
+
dev set, 6078 examples, picked at random. The data files can be found at
|
| 8 |
+
<https://github.com/elgeish/squad/tree/master/data> — this is the Winter 2020
|
| 9 |
+
version. Given that the official SQuAD2.0 dev set contains the project's test
|
| 10 |
+
set, students must make sure not to use the official SQuAD2.0 dev set in any way
|
| 11 |
+
— including the use of models fine-tuned on the official SQuAD2.0, since they
|
| 12 |
+
used the official SQuAD2.0 dev set for model selection.
|
| 13 |
+
|
| 14 |
+
## Results
|
| 15 |
+
```json
|
| 16 |
+
{
|
| 17 |
+
"exact": 75.32082922013821,
|
| 18 |
+
"f1": 78.66699523704254,
|
| 19 |
+
"total": 6078,
|
| 20 |
+
"HasAns_exact": 74.84536082474227,
|
| 21 |
+
"HasAns_f1": 81.83436324767868,
|
| 22 |
+
"HasAns_total": 2910,
|
| 23 |
+
"NoAns_exact": 75.75757575757575,
|
| 24 |
+
"NoAns_f1": 75.75757575757575,
|
| 25 |
+
"NoAns_total": 3168,
|
| 26 |
+
"best_exact": 75.32082922013821,
|
| 27 |
+
"best_exact_thresh": 0.0,
|
| 28 |
+
"best_f1": 78.66699523704266,
|
| 29 |
+
"best_f1_thresh": 0.0
|
| 30 |
+
}
|
| 31 |
+
```
|
| 32 |
+
|
| 33 |
+
## Notable Arguments
|
| 34 |
+
```json
|
| 35 |
+
{
|
| 36 |
+
"do_lower_case": true,
|
| 37 |
+
"doc_stride": 128,
|
| 38 |
+
"fp16": false,
|
| 39 |
+
"fp16_opt_level": "O1",
|
| 40 |
+
"gradient_accumulation_steps": 24,
|
| 41 |
+
"learning_rate": 3e-05,
|
| 42 |
+
"max_answer_length": 30,
|
| 43 |
+
"max_grad_norm": 1,
|
| 44 |
+
"max_query_length": 64,
|
| 45 |
+
"max_seq_length": 384,
|
| 46 |
+
"model_name_or_path": "roberta-base",
|
| 47 |
+
"model_type": "roberta",
|
| 48 |
+
"num_train_epochs": 4,
|
| 49 |
+
"per_gpu_train_batch_size": 16,
|
| 50 |
+
"save_steps": 5000,
|
| 51 |
+
"seed": 42,
|
| 52 |
+
"train_batch_size": 16,
|
| 53 |
+
"version_2_with_negative": true,
|
| 54 |
+
"warmup_steps": 0,
|
| 55 |
+
"weight_decay": 0
|
| 56 |
+
}
|
| 57 |
+
```
|
| 58 |
+
|
| 59 |
+
## Environment Setup
|
| 60 |
+
```json
|
| 61 |
+
{
|
| 62 |
+
"transformers": "2.5.1",
|
| 63 |
+
"pytorch": "1.4.0=py3.6_cuda10.1.243_cudnn7.6.3_0",
|
| 64 |
+
"python": "3.6.5=hc3d631a_2",
|
| 65 |
+
"os": "Linux 4.15.0-1060-aws #62-Ubuntu SMP Tue Feb 11 21:23:22 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux",
|
| 66 |
+
"gpu": "Tesla V100-SXM2-16GB"
|
| 67 |
+
}
|
| 68 |
+
```
|
| 69 |
+
|
| 70 |
+
## How to Cite
|
| 71 |
+
```BibTeX
|
| 72 |
+
@misc{elgeish2020gestalt,
|
| 73 |
+
title={Gestalt: a Stacking Ensemble for SQuAD2.0},
|
| 74 |
+
author={Mohamed El-Geish},
|
| 75 |
+
journal={arXiv e-prints},
|
| 76 |
+
archivePrefix={arXiv},
|
| 77 |
+
eprint={2004.07067},
|
| 78 |
+
year={2020},
|
| 79 |
+
}
|
| 80 |
+
```
|
| 81 |
+
|
| 82 |
+
## Related Models
|
| 83 |
+
* [elgeish/cs224n-squad2.0-albert-base-v2](https://huggingface.co/elgeish/cs224n-squad2.0-albert-base-v2)
|
| 84 |
+
* [elgeish/cs224n-squad2.0-albert-large-v2](https://huggingface.co/elgeish/cs224n-squad2.0-albert-large-v2)
|
| 85 |
+
* [elgeish/cs224n-squad2.0-albert-xxlarge-v1](https://huggingface.co/elgeish/cs224n-squad2.0-albert-xxlarge-v1)
|
| 86 |
+
* [elgeish/cs224n-squad2.0-distilbert-base-uncased](https://huggingface.co/elgeish/cs224n-squad2.0-distilbert-base-uncased)
|