Image-Text-to-Text
GGUF
Norwegian
Norwegian Bokmål
Norwegian Nynorsk
conversational
instruct
borealis
gemma3
quantized
norwegian
norwegian-bokmal
norwegian-nynorsk
full-release
🇪🇺 Region: EU
Instructions to use NbAiLab/borealis-27b-gguf with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use NbAiLab/borealis-27b-gguf with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf NbAiLab/borealis-27b-gguf:Q4_K_M # Run inference directly in the terminal: llama cli -hf NbAiLab/borealis-27b-gguf:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf NbAiLab/borealis-27b-gguf:Q4_K_M # Run inference directly in the terminal: llama cli -hf NbAiLab/borealis-27b-gguf:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf NbAiLab/borealis-27b-gguf:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf NbAiLab/borealis-27b-gguf:Q4_K_M
Use Docker
docker model run hf.co/NbAiLab/borealis-27b-gguf:Q4_K_M
- Llama new
- LM Studio
- Jan
- vLLM
How to use NbAiLab/borealis-27b-gguf with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "NbAiLab/borealis-27b-gguf" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "NbAiLab/borealis-27b-gguf", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/NbAiLab/borealis-27b-gguf:Q4_K_M
- Ollama
How to use NbAiLab/borealis-27b-gguf with Ollama:
ollama run hf.co/NbAiLab/borealis-27b-gguf:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use NbAiLab/borealis-27b-gguf with Docker Model Runner:
docker model run hf.co/NbAiLab/borealis-27b-gguf:Q4_K_M
- Lemonade
How to use NbAiLab/borealis-27b-gguf with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull NbAiLab/borealis-27b-gguf:Q4_K_M
Run and chat with the model
lemonade run user.borealis-27b-gguf-Q4_K_M
List all available models
lemonade list
- Atomic Chat
File size: 11,198 Bytes
5f48712 fcc768d 5f48712 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 | ---
license: other
datasets:
- NbAiLab/aurora-sft
language:
- 'no'
- nb
- nn
base_model:
- NbAiLab/borealis-27b
pipeline_tag: image-text-to-text
library_name: gguf
tags:
- conversational
- instruct
- borealis
- gemma3
- gguf
- quantized
- norwegian
- norwegian-bokmal
- norwegian-nynorsk
- full-release
---

# Borealis 27B GGUF
## Model Summary
**NbAiLab/borealis-27b-gguf** is a GGUF release of **27B-parameter** instruction-tuned **full release** model in the Borealis family from the National Library of Norway (Nasjonalbiblioteket, NB).
This is the first Borealis release to incorporate data made available under the agreement between rights-holder organizations in Norway and the Norwegian government. To date, only a limited subset of the material has been used, specifically to teach the model how to generate news article titles and ingress texts. Models with the suffix `-open`, do not include any material from the agreement.
All Borealis released models are based on the **Gemma 3** family. This GGUF repository is based on the corresponding [NbAiLab/borealis-27b](https://huggingface.co/NbAiLab/borealis-27b) safetensors release.
### Sizes
| Size | Full release | Open release |
|---:|---|---|
| 27B | [NbAiLab/borealis-27b](https://huggingface.co/NbAiLab/borealis-27b) | [NbAiLab/borealis-open-27b](https://huggingface.co/NbAiLab/borealis-open-27b) |
| 12B | [NbAiLab/borealis-12b](https://huggingface.co/NbAiLab/borealis-12b) | [NbAiLab/borealis-open-12b](https://huggingface.co/NbAiLab/borealis-open-12b) |
| 4B | [NbAiLab/borealis-4b](https://huggingface.co/NbAiLab/borealis-4b) | [NbAiLab/borealis-open-4b](https://huggingface.co/NbAiLab/borealis-open-4b) |
| 1B | [NbAiLab/borealis-1b](https://huggingface.co/NbAiLab/borealis-1b) | [NbAiLab/borealis-open-1b](https://huggingface.co/NbAiLab/borealis-open-1b) |
| 270M | [NbAiLab/borealis-270m](https://huggingface.co/NbAiLab/borealis-270m) | [NbAiLab/borealis-open-270m](https://huggingface.co/NbAiLab/borealis-open-270m) |
## Training Data
Supervised fine-tuning (SFT) uses instruction data prepared by the National Library of Norway for Norwegian-centric assistant behavior, writing, summarization, question answering, and related tasks.
The SFT dataset for this model is [NbAiLab/aurora-sft](https://huggingface.co/datasets/NbAiLab/aurora-sft). [NbAiLab/aurora-sft-open](https://huggingface.co/datasets/NbAiLab/aurora-sft-open) is the open version of the SFT dataset. The only difference between [NbAiLab/aurora-sft-open](https://huggingface.co/datasets/NbAiLab/aurora-sft-open) and [NbAiLab/aurora-sft](https://huggingface.co/datasets/NbAiLab/aurora-sft) is the addition of 10k tasks derived from copyright-protected newspapers material.
The Norwegian government has entered into an agreement, through the National Library of Norway, to access copyrighted press material via Kopinor on behalf of the Norwegian Media Businesses' Association (MBL), enabling the lawful training, development, maintenance, and public release of Norwegian language models. The agreement covers copyright-protected press publications published in Norway up to one year from the date of publication of the model, effectively creating a rolling cutoff date rather than a fixed historical cutoff. For this release, the cutoff date is January 1, 2025.
## Evaluation
<figure>
<img src="./borealis_evals_202605.png" alt="Borealis evaluation results">
<figcaption>Borealis evaluation results on selected tasks (best score among {0-5}-shot).</figcaption>
</figure>
We evaluate Borealis with NorEval, MMLU-English, and nb-gpt-bench, our own evaluation suite, which will be published and described in an upcoming paper. Despite using only around 10k newspaper-derived tasks from the abovementioned agreement, we observe a slight performance increase in some key metrics. We hope to further increase the difference by incorporating proper pre-training on the newspaper material.
## Safety and Alignment
The Borealis family of models are aligned for safety using prompt baking and weighted merging of SFT and aligned models. The goal of this process is to balance model quality, usefulness, and safer behavior.
As with all generative models, outputs can still be incorrect, biased, harmful, or inappropriate. Do not use the model for safety-critical or high-stakes applications without additional evaluation and safeguards.
### Prompt Baking
To align the Borealis models, we employ *prompt baking*, a procedure that distills the behavior induced by a system prompt directly into the model weights using [`bakery`](https://github.com/marksverdhei/bakery). Specifically, we train a LoRA adapter to minimize the KL-divergence between two model distributions: Borealis conditioned on the system prompt, and the same base model augmented with the LoRA adapter but evaluated without the system prompt in context. This objective encourages the adapter to reproduce the behavioral effects of the prompt without requiring the prompt to be present at inference time.
To reduce degradation on downstream tasks and preserve general model utility, we merge the resulting prompt adapter into the base model using a scaling factor of `0.25`, which we found to provide the best empirical trade-off.
## Intended Use
- Norwegian-centric assistant-style tasks, including drafting, summarization, Q&A, and light reasoning (this is not a reasoning model).
- Assessment and improvement of Norwegian writing style and quality.
- Evaluation of behavior and language coverage for Norwegian, Bokmål, and Nynorsk.
## Usage
This repository contains GGUF files for `NbAiLab/borealis-27b`. For Transformers or
vLLM serving from safetensors, use the main model repository: `NbAiLab/borealis-27b`.
### llama.cpp
Run a GGUF directly from the Hub:
```bash
llama-server -hf NbAiLab/borealis-27b-gguf --port 8080
```
Or download a GGUF file and run it locally:
```bash
llama-cli -m borealis-27b-Q4_K_M.gguf \
-p "Skriv et kort sammendrag av hva Nasjonalbiblioteket gjør." \
-n 256
```
For image-text use with llama.cpp, use the matching projector file from this
repository, for example `mmproj_borealis-27b-BF16.gguf`, together with a model GGUF.
### Ollama
Run the GGUF repository from Hugging Face:
```bash
ollama run hf.co/NbAiLab/borealis-27b-gguf
```
For a local GGUF file, create a minimal `Modelfile`:
```text
FROM ./borealis-27b-Q4_K_M.gguf
```
Then create and run the local Ollama model:
```bash
ollama create borealis-27b -f Modelfile
ollama run borealis-27b "Skriv tre korte punkter om norsk språkteknologi."
```
## Limitations
- The model may hallucinate or produce incorrect information.
- Safety alignment reduces but does not eliminate harmful or inappropriate outputs.
- Performance outside Norwegian and English use cases has not been fully characterized.
## EU AI Act
The model is a fine-tune of Gemma 3. Using Gemma 3 27B as a conservative upper-bound reference, the original Gemma 3 27B training compute is estimated at approximately 2.1-2.3 x 10^24 FLOPs, based on the disclosed 14T training-token budget and the 27B parameter scale. The fine-tuning run used approximately 3.4 x 10^20 FLOPs, or about 0.015% of the estimated original training compute. This is substantially below the European Commission's indicative one-third threshold for treating a downstream modification as a significant modification that would make the modifier the provider of the modified General Purpose AI (GPAI) model.
On that basis, the fine-tuning activity is preliminarily assessed as not constituting a substantial modification for the purpose of becoming the provider of a new modified GPAI model under the compute-based criterion. However, the resulting model remains derived from a generative general-purpose AI model and may still be subject to downstream AI-system obligations under the EU AI Act.
For additional model-level documentation, see the [Model Documentation Form](./Model_Documentation_Form.pdf).
## License
The license of this model is an adaptation of the Apache 2.0 license with additional use-based restrictions. In particular, users of the model are required to refrain from intentionally using the model to recreate data the model has been trained on. The license also requires users not to use the model or its output to provide end-user services whose primary purpose is to give access to licensed press publications in the training data.
For more information, see the [LICENSE](https://huggingface.co/NbAiLab/borealis-27b-gguf/blob/main/LICENSE) and the [License FAQ](./LICENSE_FAQ.pdf).
## Authenticity
This model release is signed by the National Library of Norway. The signed manifest in `signing/SHA256SUMS` covers the model-runtime artifacts, including model weights, configuration, tokenizer files, and chat template.
To verify model authenticity and file integrity after downloading the repository, run:
```bash
bash signing/verify.sh
```
For more verification instructions, see [ai.nb.no/verify](https://ai.nb.no/verify).
## Weights
This repository contains the Transformers (safetensors) release of **NbAiLab/borealis-27b-gguf**.
Companion formats:
- GGUF: [NbAiLab/borealis-27b-gguf](https://huggingface.co/NbAiLab/borealis-27b-gguf)
- MLX: [NbAiLab/borealis-27b-mlx](https://huggingface.co/NbAiLab/borealis-27b-mlx)
- MLX 8-bit: [NbAiLab/borealis-27b-mlx-8bits](https://huggingface.co/NbAiLab/borealis-27b-mlx-8bits)
## Citation and Contributors
The Borealis family of models is a joint effort of multiple teams at the National Library of Norway. Led by Javier de la Rosa ([@versae](https://huggingface.co/versae)), key contributors include (in alphabetical order) Rolv-Arild Braaten, Magnus Breder Birkenes, Lucas Charpentier, Pawel Cyrta, Tita Enstad, Markus Sverdvik Heiervang, Arne Martinus Lindstad, Marthe Løken Midtgaard, Marie Roald, Marie Røsok, Thea Tollersrud, and Angelina Zanardi. Olaus Ingskog Bergstrøm contributed with legal advice. And Yngvil Beyer, Svein Arne Brygfjeld, and Wilfred Østgulen helped with strategic oversight.
A tecnical report will be released soon.
## Acknowledgements
Thanks to the Gemma team at Google for releasing Gemma 3, and to everyone contributing to the Norwegian language technology ecosystem.
## Disclaimer
The models published in this repository are intended for a generalist purpose and are available to third parties. These models may have bias and/or any other undesirable distortions. When third parties, deploy or provide systems and/or services to other parties using any of these models (or using systems based on these models) or become users of the models, they should note that it is their responsibility to mitigate the risks arising from their use and, in any event, to comply with applicable regulations, including regulations regarding the use of artificial intelligence. In no event shall the owner of the models (The National Library of Norway) be liable for any results arising from the use made by third parties of these models.
## Contact
For feedback, technical concerns, or collaboration inquiries, please contact <a rel="noopener nofollow" href="mailto:ailab@nb.no">ailab@nb.no</a>.
|