---
license: apache-2.0
---

## Quantization Details

- **Quantization Method:** AWQ
- **Bits:** 4
- **Group Size:** 128
- **Quantization Tool:** [ms-swift](https://github.com/modelscope/ms-swift/blob/main/examples/export/quantize/mllm/awq.sh)

## Memory Usage

| **Type** | **GUI-Owl-7B** | **GUI-Owl-7B-AWQ-4bit** |
|:---------------:|:----------------:|:----------------:|
| **Memory Size** | 16G GB |  6.5 GB | 

## Inference

Run with:

```bash
vllm serve JessieWei/GUI-Owl-7B-AWQ-4bit
```

Detailed parameter tuning refer to [GUI-Owl-7B](https://huggingface.co/mPLUG/GUI-Owl-7B).