Instructions to use Ellbendls/Qwen-2.5-3b-Quran with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Ellbendls/Qwen-2.5-3b-Quran with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Ellbendls/Qwen-2.5-3b-Quran")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Ellbendls/Qwen-2.5-3b-Quran")
model = AutoModelForCausalLM.from_pretrained("Ellbendls/Qwen-2.5-3b-Quran")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use Ellbendls/Qwen-2.5-3b-Quran with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Ellbendls/Qwen-2.5-3b-Quran"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Ellbendls/Qwen-2.5-3b-Quran",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Ellbendls/Qwen-2.5-3b-Quran

SGLang

How to use Ellbendls/Qwen-2.5-3b-Quran with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Ellbendls/Qwen-2.5-3b-Quran" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Ellbendls/Qwen-2.5-3b-Quran",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Ellbendls/Qwen-2.5-3b-Quran" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Ellbendls/Qwen-2.5-3b-Quran",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Ellbendls/Qwen-2.5-3b-Quran with Docker Model Runner:
```
docker model run hf.co/Ellbendls/Qwen-2.5-3b-Quran
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Model Card for Fine-Tuned Qwen2.5-3B-Instruct

This is a fine-tuned version of the Qwen2.5-3B-Instruct model. The fine-tuning process utilized the Quran Indonesia Tafseer Translation dataset, which provides translations and tafsir in Bahasa Indonesia for the Quran.

Model Details

Model Description

Base Model: Qwen2.5-3B-Instruct
Fine-Tuned By: Ellbendl Satria
Dataset: emhaihsan/quran-indonesia-tafseer-translation
Language: Bahasa Indonesia
License: MIT

This model is designed for NLP tasks involving Quranic text in Bahasa Indonesia, including understanding translations and tafsir.

Uses

Direct Use

This model can be used for applications requiring the understanding, summarization, or retrieval of Quranic translations and tafsir in Bahasa Indonesia.

Downstream Use

It is suitable for fine-tuning on tasks such as:

Quranic text summarization
Question answering systems related to Islamic knowledge
Educational tools for learning Quranic content in Indonesian

Biases

The model inherits any biases present in the dataset, which is specific to Islamic translations and tafsir in Bahasa Indonesia.

Recommendations

Users should ensure that applications using this model respect cultural and religious sensitivities.
Results should be verified by domain experts for critical applications.

How to Get Started with the Model

from transformers import AutoModelForCausalLM, AutoTokenizer

# Load the tokenizer and model
tokenizer = AutoTokenizer.from_pretrained("Ellbendls/Qwen-2.5-3b-Quran")
model = AutoModelForCausalLM.from_pretrained("Ellbendls/Qwen-2.5-3b-Quran")

# Move the model to GPU
model.to("cuda")

# Define the input message
messages = [
    {
        "role": "user", 
        "content": "Tafsirkan ayat ini اِهْدِنَا الصِّرَاطَ الْمُسْتَقِيْمَۙ"
    }
]

# Generate the prompt using the tokenizer
prompt = tokenizer.apply_chat_template(messages, tokenize=False, 
                                       add_generation_prompt=True)

# Tokenize the prompt and move inputs to GPU
inputs = tokenizer(prompt, return_tensors='pt', padding=True, 
                   truncation=True).to("cuda")

# Generate the output using the model
outputs = model.generate(**inputs, max_length=150, 
                         num_return_sequences=1)

# Decode the output
text = tokenizer.decode(outputs[0], skip_special_tokens=True)

# Print the result
print(text.split("assistant")[1])

Downloads last month: 3

Safetensors

Model size

3B params

Tensor type

F16

Model tree for Ellbendls/Qwen-2.5-3b-Quran

Base model

Qwen/Qwen2.5-3B

Finetuned

Qwen/Qwen2.5-3B-Instruct

Finetuned

(1278)

this model

Quantizations

1 model

Ellbendls
/

Qwen-2.5-3b-Quran