Model Card for Model raicrits/OpenLLama13b_Loquace_ITA

An open-source LLaMa language model of 13b parameters fine-tuned to follow instructions in italian.

Model Description

This model is an open-source LLM of 13b parameters based on OpenLLaMA, an open-source replica of Meta AI's LLaMA. The model was fine-tuned in order to follow instructions, as proposed in Alpaca, but using LoRA technique and a bigger dataset of instruction/answers in italian, cosimoiaia/Loquace-102k.

This repository contains the model merged with the LoRA adapters obtained in the fine-tuning procedure.

• Developed by: Stefano Scotta (stefano.scotta@rai.it)
• Model type: LLM fine-tuned to follow instructions
• Language(s) (NLP): Italian
• License: Other
• Finetuned from model: openlm-research/open_llama_13b

Uses

The model can be used as is to respond to simple instructions in Italian or can be further fine-tuned to perform specific tasks.

Bias, Risks, and Limitations

As any other LLM it is possible that the model generates content which does not correspond to the reality as well as wrong, biased, offensive and inappropriate answers.

How to Get Started with the Model

Prompt template:

"Di seguito è riportata un'istruzione che descrive un compito, abbinata a un input che fornisce un ulteriore contesto. Scrivete una risposta che completi in modo appropriato la richiesta.

### Istruzione:
{instruction}

### Input:
{input}

### Risposta:"

Usage: Use the code below to get started with the model.

import os
import torch
import sys
from transformers import LlamaTokenizer, LlamaForCausalLM

if torch.cuda.is_available():
   device = "cuda"
else:
   device = "cpu"

def generate_prompt(instruction, input=None):
   if input:
       return f"""Di seguito è riportata un'istruzione che descrive un compito, abbinata a un input che fornisce un ulteriore contesto. Scrivete una risposta che completi in modo appropriato la richiesta.

### Istruzione:
{instruction}

### Input:
{input}

### Risposta:"""
   else:
       return f"""Di seguito è riportata un'istruzione che descrive un compito. Scrivete una risposta che completi in modo appropriato la richiesta..

### Istruzione:
{instruction}

### Risposta:"""

model_name = "raicrits/OpenLLama13b_Loquace_ITA"

model = LlamaForCausalLM.from_pretrained(
   model_name,
   torch_dtype=torch.float16,
   device_map="auto"
)

tokenizer = LlamaTokenizer.from_pretrained(model_name)

instruction = "qual'è la relazione tra i seguenti oggetti"
input = "sedia, tavolo, divano"
prompt = generate_prompt("instruction", input)
inputs = tokenizer(prompt, return_tensors="pt")
input_ids = inputs["input_ids"].to(device)

generation_output = model.generate(
   input_ids=input_ids,
   max_new_tokens=256,
)


output = tokenizer.decode(generation_output[0])
output = output.split("### Risposta:")[1].strip().replace("</s>","")
print(output)

"Sedia, tavolo e divano sono tutti oggetti che possono essere utilizzati per creare un'atmosfera rilassante in una stanza."

Training Details

Training Data

The model was fine-tinuned on cosimoiaia/Loquace-102k, a dataset of 102k question/answer pairs in italian.

Training Procedure

The fine-tuning procedure was done using LoRA approach following closely what done for fine-tuning models like Alpaca-LoRA.

Training Hyperparameters

Training setting:

• train epochs=3,

• learning_rate=3e-4,

• optimizer="adamw_hf"

• mixed precision training: float16

LoRA configuration:

• r= 8

• lora_alpha=16

• target_modules=["q_proj","v_proj"]

• lora_dropout=0.05

• bias="none"

• task_type=TaskType.CAUSAL_LM

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

• Hardware Type: 1 NVIDIA A100/40Gb
• Hours used: 68
• Cloud Provider: Private Infrastructure
• Carbon Emitted: 7.34 kg eq. CO2

Model Card Authors

Stefano Scotta (stefano.scotta@rai.it)

Model Card Contact

stefano.scotta@rai.it