Instructions to use airesearch/WangchanLion7B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use airesearch/WangchanLion7B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="airesearch/WangchanLion7B", trust_remote_code=True)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("airesearch/WangchanLion7B", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("airesearch/WangchanLion7B", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use airesearch/WangchanLion7B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "airesearch/WangchanLion7B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "airesearch/WangchanLion7B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/airesearch/WangchanLion7B

SGLang

How to use airesearch/WangchanLion7B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "airesearch/WangchanLion7B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "airesearch/WangchanLion7B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "airesearch/WangchanLion7B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "airesearch/WangchanLion7B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use airesearch/WangchanLion7B with Docker Model Runner:
```
docker model run hf.co/airesearch/WangchanLion7B
```

SR commited on Feb 27, 2024

Commit

f0a4dc0

verified ·

1 Parent(s): 729a5f4

Update README.md

Browse files

Files changed (1) hide show

README.md +11 -11

README.md CHANGED Viewed

@@ -92,19 +92,19 @@ Finetuning datasets are sourced from [LAION OIG chip2 and infill_dbpedia (Apache
 # Evaluation
 We performed human and machine evaluations on XQuAD zero-shot and one-shot settings:
 ## XQuAD
-|      Model     | Exact Match (Zero-shot) | F1 (Zero-shot) | Exact Match (One-shot) | F1 (One-shot) |
-|:--------------:|:-----------------------:|:--------------:|:----------------------:|:-------------:|
-| openthaigpt7B  |         18.57           |     28.40      |         30.42          |    39.76      |
-| SeaLLM7B       |           6.30          |    21.67       |      34.45         |  47.80    |
-| Typhoon-7b     |         -               |     34.46      |            -           |    **54.03**      |
-| WangchanLion7B |        **37.56**        |   **49.84**    |         **39.24**          |    51.07      |
 ## iAPP Wiki QA
-|      Model     | Exact Match (Zero-shot) | F1 (Zero-shot) | Exact Match (One-shot) | F1 (One-shot) |
-|:--------------:|:-----------------------:|:--------------:|:----------------------:|:-------------:|
-| openthaigpt7B  |         22.06         |     40.07    |         31.39        |    47.98    |
-| SeaLLM7B       |          8.25         |     34.40    |         40.05        |    58.27    |
-| WangchanLion7B |       **55.48**       |   **67.93**  |       **56.43**      |  **68.85**  |
 # What WangchanLion offers:
 - Transparent pretrained model: The development of SEA-LION is community-driven, with different ASEAN collaborators contributing pretraining datasets. The SEA-LION developers ensure that all datasets are safe and can be utilized without commercial restrictions. This transparency extends to the provision of pretraining code, ensuring anyone can replicate SEA-LION using the provided datasets.

 # Evaluation
 We performed human and machine evaluations on XQuAD zero-shot and one-shot settings:
 ## XQuAD
+|      Model     | F1 (Zero-shot) | F1 (One-shot) |
+|:--------------:|:--------------:|:-------------:|
+| openthaigpt7B  |     27.3487      |    34.3104      |
+| SeaLLM7B V2      |    16.1104       |  25.7399    |
+| Typhoon-7b     |     34.46      |    **54.03**      |
+| WangchanLion7B |   **45.8763**    |    49.9145      |
 ## iAPP Wiki QA
+|      Model     | F1 (Zero-shot) |  F1 (One-shot) |
+|:--------------:|:--------------:|:-------------:|
+| openthaigpt7B  |     40.0614    |    46.6883    |
+| SeaLLM7B V2      |     23.6425    |    28.9934    |
+| WangchanLion7B |   **58.9051**  |  **62.9776**  |
 # What WangchanLion offers:
 - Transparent pretrained model: The development of SEA-LION is community-driven, with different ASEAN collaborators contributing pretraining datasets. The SEA-LION developers ensure that all datasets are safe and can be utilized without commercial restrictions. This transparency extends to the provision of pretraining code, ensuring anyone can replicate SEA-LION using the provided datasets.