Commit ·
54aad73
1
Parent(s): 3b2aa31
Update README.md
Browse files
README.md
CHANGED
|
@@ -24,7 +24,7 @@ The details are available at [Github:FS-ABSA](https://github.com/nustm/fs-absa)
|
|
| 24 |
|
| 25 |
To bridge the domain gap between general pre-training and the task of interest in a specific domain (i.e., `laptop` in this repo), we conducted *domain-adaptive pre-training*,
|
| 26 |
i.e., continuing pre-training the language model (i.e., T5) on the unlabeled corpus of the domain of interest (i.e., `laptop`) with the *text-infilling objective*
|
| 27 |
-
(corruption rate of 15% and average span length of 1). We collect relevant 100k unlabeled reviews from Amazon Electronics for the laptop domain
|
| 28 |
For pre-training, we employ the [Adafactor](https://arxiv.org/abs/1804.04235) optimizer with a batch size of 84 and a constant learning rate of 1e-4.
|
| 29 |
|
| 30 |
Our model can be seen as an enhanced T5 model in the laptop domain, which can be used for various NLP tasks related to the laptop domain,
|
|
|
|
| 24 |
|
| 25 |
To bridge the domain gap between general pre-training and the task of interest in a specific domain (i.e., `laptop` in this repo), we conducted *domain-adaptive pre-training*,
|
| 26 |
i.e., continuing pre-training the language model (i.e., T5) on the unlabeled corpus of the domain of interest (i.e., `laptop`) with the *text-infilling objective*
|
| 27 |
+
(corruption rate of 15% and average span length of 1). We collect relevant 100k unlabeled reviews from Amazon Electronics for the laptop domain.
|
| 28 |
For pre-training, we employ the [Adafactor](https://arxiv.org/abs/1804.04235) optimizer with a batch size of 84 and a constant learning rate of 1e-4.
|
| 29 |
|
| 30 |
Our model can be seen as an enhanced T5 model in the laptop domain, which can be used for various NLP tasks related to the laptop domain,
|