End of training

README.md

@@ -1,69 +1,50 @@
 ---
 library_name: transformers
-license: cc-by-4.0
+license: mit
 base_model: roberta-base
-metrics:
-- accuracy
 tags:
 - generated_from_trainer
-- text-classification
-- classification
-- nlp
-- vulnerability
+metrics:
+- accuracy
 model-index:
 - name: vulnerability-severity-classification-roberta-base
   results: []
-datasets:
-- CIRCL/vulnerability-scores
 ---
 
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
 
-# VLAI: A RoBERTa-Based Model for Automated Vulnerability Severity Classification
-
-# Severity classification
-
-This model is a fine-tuned version of [roberta-base](https://huggingface.co/roberta-base) on the dataset [CIRCL/vulnerability-scores](https://huggingface.co/datasets/CIRCL/vulnerability-scores).
-
-The model was presented in the paper [VLAI: A RoBERTa-Based Model for Automated Vulnerability Severity Classification](https://huggingface.co/papers/2507.03607) [[arXiv](https://arxiv.org/abs/2507.03607)].
-
-**Abstract:** VLAI is a transformer-based model that predicts software vulnerability severity levels directly from text descriptions. Built on RoBERTa, VLAI is fine-tuned on over 600,000 real-world vulnerabilities and achieves over 82% accuracy in predicting severity categories, enabling faster and more consistent triage ahead of manual CVSS scoring. The model and dataset are open-source and integrated into the Vulnerability-Lookup service.
-
-You can read [this page](https://www.vulnerability-lookup.org/user-manual/ai/) for more information.
+# vulnerability-severity-classification-roberta-base
 
+This model is a fine-tuned version of [roberta-base](https://huggingface.co/roberta-base) on an unknown dataset.
+It achieves the following results on the evaluation set:
+- Loss: 2.0348
+- Accuracy: 0.8126
+- F1 Macro: 0.7423
+- Low Precision: 0.6555
+- Low Recall: 0.4770
+- Low F1: 0.5522
+- Medium Precision: 0.8428
+- Medium Recall: 0.8545
+- Medium F1: 0.8486
+- High Precision: 0.8081
+- High Recall: 0.8194
+- High F1: 0.8137
+- Critical Precision: 0.7545
+- Critical Recall: 0.7552
+- Critical F1: 0.7548
 
 ## Model description
 
-It is a classification model and is aimed to assist in classifying vulnerabilities by severity based on their descriptions.
-
-## How to get started with the model
-
-```python
-from transformers import AutoModelForSequenceClassification, AutoTokenizer
-import torch
+More information needed
 
-labels = ["low", "medium", "high", "critical"]
+## Intended uses & limitations
 
-model_name = "CIRCL/vulnerability-severity-classification-roberta-base"
-tokenizer = AutoTokenizer.from_pretrained(model_name)
-model = AutoModelForSequenceClassification.from_pretrained(model_name)
-model.eval()
+More information needed
 
-print("Model revision:", model.config._commit_hash)
+## Training and evaluation data
 
-test_description = "SAP NetWeaver Visual Composer Metadata Uploader is not protected with a proper authorization, allowing unauthenticated agent to upload potentially malicious executable binaries \
-that could severely harm the host system. This could significantly affect the confidentiality, integrity, and availability of the targeted system."
-inputs = tokenizer(test_description, return_tensors="pt", truncation=True, padding=True)
-
-# Run inference
-with torch.no_grad():
-    outputs = model(**inputs)
-    predictions = torch.nn.functional.softmax(outputs.logits, dim=-1)
-
-# Print results
-print("Predictions:", predictions)
-predicted_class = torch.argmax(predictions, dim=-1).item()
-print("Predicted severity:", labels[predicted_class])
-```
+More information needed
 
 ## Training procedure
 
@@ -78,37 +59,20 @@ The following hyperparameters were used during training:
 - lr_scheduler_type: linear
 - num_epochs: 5
 
-It achieves the following results on the evaluation set:
-- Loss: 2.0472
-- Accuracy: 0.8159
-- F1 Macro: 0.7492
-- Low Precision: 0.6724
-- Low Recall: 0.5028
-- Low F1: 0.5753
-- Medium Precision: 0.8380
-- Medium Recall: 0.8688
-- Medium F1: 0.8531
-- High Precision: 0.8177
-- High Recall: 0.8111
-- High F1: 0.8144
-- Critical Precision: 0.7620
-- Critical Recall: 0.7459
-- Critical F1: 0.7538
-
 ### Training results
 
 | Training Loss | Epoch | Step  | Validation Loss | Accuracy | F1 Macro | Low Precision | Low Recall | Low F1 | Medium Precision | Medium Recall | Medium F1 | High Precision | High Recall | High F1 | Critical Precision | Critical Recall | Critical F1 |
 |:-------------:|:-----:|:-----:|:---------------:|:--------:|:--------:|:-------------:|:----------:|:------:|:----------------:|:-------------:|:---------:|:--------------:|:-----------:|:-------:|:------------------:|:---------------:|:-----------:|
-| 2.5002        | 1.0   | 18542 | 2.5700          | 0.7344   | 0.6361   | 0.6175        | 0.3019     | 0.4055 | 0.7650           | 0.8301        | 0.7962    | 0.7315         | 0.7099      | 0.7206  | 0.6356             | 0.6096          | 0.6223      |
-| 2.2754        | 2.0   | 37084 | 2.3349          | 0.7628   | 0.6641   | 0.6756        | 0.3151     | 0.4297 | 0.7848           | 0.8521        | 0.8171    | 0.7549         | 0.7564      | 0.7556  | 0.7050             | 0.6100          | 0.6540      |
-| 2.0351        | 3.0   | 55626 | 2.1389          | 0.7886   | 0.7097   | 0.6592        | 0.4275     | 0.5186 | 0.8057           | 0.8674        | 0.8354    | 0.7884         | 0.7760      | 0.7821  | 0.7461             | 0.6641          | 0.7027      |
-| 1.5727        | 4.0   | 74168 | 2.0768          | 0.8055   | 0.7373   | 0.6553        | 0.4882     | 0.5596 | 0.8383           | 0.8535        | 0.8458    | 0.8089         | 0.7971      | 0.8030  | 0.7167             | 0.7671          | 0.7410      |
-| 1.4295        | 5.0   | 92710 | 2.0472          | 0.8159   | 0.7492   | 0.6724        | 0.5028     | 0.5753 | 0.8380           | 0.8688        | 0.8531    | 0.8177         | 0.8111      | 0.8144  | 0.7620             | 0.7459          | 0.7538      |
+| 2.5193        | 1.0   | 18744 | 2.6088          | 0.7325   | 0.6353   | 0.5418        | 0.3410     | 0.4186 | 0.7928           | 0.7822        | 0.7874    | 0.6917         | 0.7826      | 0.7343  | 0.6912             | 0.5313          | 0.6008      |
+| 2.6298        | 2.0   | 37488 | 2.3400          | 0.7594   | 0.6742   | 0.6229        | 0.3490     | 0.4474 | 0.8133           | 0.8029        | 0.8081    | 0.7517         | 0.7594      | 0.7555  | 0.6361             | 0.7436          | 0.6856      |
+| 1.6998        | 3.0   | 56232 | 2.2209          | 0.7833   | 0.7045   | 0.5831        | 0.4344     | 0.4979 | 0.8010           | 0.8566        | 0.8279    | 0.7940         | 0.7648      | 0.7791  | 0.7316             | 0.6952          | 0.7129      |
+| 1.7861        | 4.0   | 74976 | 2.0459          | 0.8048   | 0.7275   | 0.6711        | 0.4268     | 0.5217 | 0.8382           | 0.8505        | 0.8443    | 0.7942         | 0.8163      | 0.8051  | 0.7438             | 0.7342          | 0.7389      |
+| 1.4938        | 5.0   | 93720 | 2.0348          | 0.8126   | 0.7423   | 0.6555        | 0.4770     | 0.5522 | 0.8428           | 0.8545        | 0.8486    | 0.8081         | 0.8194      | 0.8137  | 0.7545             | 0.7552          | 0.7548      |
 
 
 ### Framework versions
 
-- Transformers 5.16.1
+- Transformers 5.17.0
 - Pytorch 2.14.0+cu130
 - Datasets 4.8.5
-- Tokenizers 0.23.1
+- Tokenizers 0.23.2

config.json

@@ -34,7 +34,7 @@
   "pad_token_id": 1,
   "problem_type": "single_label_classification",
   "tie_word_embeddings": true,
-  "transformers_version": "5.16.1",
+  "transformers_version": "5.17.0",
   "type_vocab_size": 1,
   "use_cache": true,
   "vocab_size": 50265

emissions.csv

@@ -1,2 +1,2 @@
 timestamp,project_name,run_id,experiment_id,duration,emissions,emissions_rate,cpu_power,gpu_power,ram_power,cpu_energy,gpu_energy,ram_energy,energy_consumed,water_consumed,country_name,country_iso_code,region,cloud_provider,cloud_region,os,python_version,codecarbon_version,cpu_count,cpu_model,gpu_count,gpu_model,longitude,latitude,ram_total_size,tracking_mode,cpu_utilization_percent,gpu_utilization_percent,ram_utilization_percent,ram_used_gb,on_cloud,pue,wue
-2026-09-03T11:11:27,VulnTrain,d718941b-bdf5-4cee-ac36-9c6ccece885e,5b0fa12a-3dd7-45bb-9766-cc326314d9f1,16507.520108614997,0.5036411007268355,3.0509797802032888e-05,70.00021457465269,904.5119904215752,70.0,0.32093893680077556,4.143460128932118,0.32019856806849734,4.78459763380139,0.0,Luxembourg,LUX,luxembourg,,,Linux-6.8.0-138-generic-x86_64-with-glibc2.39,3.12.3,3.3.0,224,Intel(R) Xeon(R) Platinum 8480+,4,4 x NVIDIA L40S,6.1327,49.6098,2015.3353996276855,machine,0.9777136258660508,77.09917041448888,1.0799987844900938,21.39229411367108,N,1.0,0.0
+2026-09-10T16:45:23,VulnTrain,e2a30913-f7d2-4cf6-9a0d-547d4b34bb9f,5b0fa12a-3dd7-45bb-9766-cc326314d9f1,16676.899105929,0.5094324424343436,3.0547192208725984e-05,70.00021580851913,906.4022709612171,70.0,0.32423673863741953,4.1919406713276315,0.32343805258440234,4.839615462549457,0.0,Luxembourg,LUX,luxembourg,,,Linux-6.8.0-139-generic-x86_64-with-glibc2.39,3.12.3,3.3.1,224,Intel(R) Xeon(R) Platinum 8480+,4,4 x NVIDIA L40S,6.1327,49.6098,2015.3354377746582,machine,1.0158316754084522,76.79354313619099,1.0693374329294028,21.204635001467906,N,1.0,0.0

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:bfdef81f818bda4c1d0ce9f6bc23b00ade4137afbf1947580f41f41f49a31f54
+oid sha256:ffc2260dd1e49c9dcb1a6e7ab4c4f7e67ff602ac8742a5e94c10d6b4489bbdfc
 size 498618976