baltop/mistral-instruct-finetune

README.md

@@ -16,7 +16,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [mistralai/Mistral-7B-Instruct-v0.2](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2) on an unknown dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.0305
+- Loss: 0.0307
 
 ## Model description
 
@@ -48,18 +48,18 @@ The following hyperparameters were used during training:
 
 | Training Loss | Epoch | Step | Validation Loss |
 |:-------------:|:-----:|:----:|:---------------:|
-| 1.1899        | 0.17  | 25   | 0.3116          |
-| 0.1795        | 0.33  | 50   | 0.1088          |
-| 0.0819        | 0.5   | 75   | 0.0425          |
-| 0.0453        | 0.67  | 100  | 0.0419          |
-| 0.0534        | 0.83  | 125  | 0.0382          |
-| 0.0338        | 1.0   | 150  | 0.0315          |
-| 0.0358        | 1.17  | 175  | 0.0345          |
-| 0.0336        | 1.33  | 200  | 0.0334          |
-| 0.0401        | 1.5   | 225  | 0.0322          |
-| 0.0326        | 1.67  | 250  | 0.0308          |
-| 0.0396        | 1.83  | 275  | 0.0309          |
-| 0.0307        | 2.0   | 300  | 0.0305          |
+| 1.1806        | 0.17  | 25   | 0.3104          |
+| 0.1789        | 0.33  | 50   | 0.1074          |
+| 0.0665        | 0.5   | 75   | 0.0420          |
+| 0.0444        | 0.67  | 100  | 0.0414          |
+| 0.05          | 0.83  | 125  | 0.0351          |
+| 0.0322        | 1.0   | 150  | 0.0311          |
+| 0.0361        | 1.17  | 175  | 0.0343          |
+| 0.0338        | 1.33  | 200  | 0.0319          |
+| 0.039         | 1.5   | 225  | 0.0322          |
+| 0.0324        | 1.67  | 250  | 0.0304          |
+| 0.0392        | 1.83  | 275  | 0.0309          |
+| 0.0305        | 2.0   | 300  | 0.0307          |
 
 
 ### Framework versions

adapter_config.json

@@ -19,13 +19,13 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
+    "v_proj",
+    "down_proj",
     "k_proj",
     "up_proj",
-    "down_proj",
     "q_proj",
     "lm_head",
     "gate_proj",
-    "v_proj",
     "o_proj"
   ],
   "task_type": "CAUSAL_LM",

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:8818e6ea4830b84464c905ac6f94cfe80a63f77e51c2638d2bd0bcefa4dc7252
+oid sha256:6fc6264c501532bf987ee0a4a8b3abe92f6cd9f3401bd7a759de754e4ec0e795
 size 864513616

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3590f49eb926244738eba44f5a2a5b93f3455bb0c8fe39c90b692c6cf1ea1736
+oid sha256:e21b1c43d06392b0c9b0f537763c90b02516bd21d3eed626cd2a59d8380ef01a
 size 4664