Instructions to use abhi9ab/DeepSeek-R1-Distill-Llama-8B-finance-v1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use abhi9ab/DeepSeek-R1-Distill-Llama-8B-finance-v1 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="abhi9ab/DeepSeek-R1-Distill-Llama-8B-finance-v1")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("abhi9ab/DeepSeek-R1-Distill-Llama-8B-finance-v1")
model = AutoModelForCausalLM.from_pretrained("abhi9ab/DeepSeek-R1-Distill-Llama-8B-finance-v1")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use abhi9ab/DeepSeek-R1-Distill-Llama-8B-finance-v1 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "abhi9ab/DeepSeek-R1-Distill-Llama-8B-finance-v1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "abhi9ab/DeepSeek-R1-Distill-Llama-8B-finance-v1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/abhi9ab/DeepSeek-R1-Distill-Llama-8B-finance-v1

SGLang

How to use abhi9ab/DeepSeek-R1-Distill-Llama-8B-finance-v1 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "abhi9ab/DeepSeek-R1-Distill-Llama-8B-finance-v1" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "abhi9ab/DeepSeek-R1-Distill-Llama-8B-finance-v1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "abhi9ab/DeepSeek-R1-Distill-Llama-8B-finance-v1" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "abhi9ab/DeepSeek-R1-Distill-Llama-8B-finance-v1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Unsloth Studio new

How to use abhi9ab/DeepSeek-R1-Distill-Llama-8B-finance-v1 with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for abhi9ab/DeepSeek-R1-Distill-Llama-8B-finance-v1 to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for abhi9ab/DeepSeek-R1-Distill-Llama-8B-finance-v1 to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for abhi9ab/DeepSeek-R1-Distill-Llama-8B-finance-v1 to start chatting

Load model with FastModel

pip install unsloth
from unsloth import FastModel
model, tokenizer = FastModel.from_pretrained(
    model_name="abhi9ab/DeepSeek-R1-Distill-Llama-8B-finance-v1",
    max_seq_length=2048,
)

Docker Model Runner
How to use abhi9ab/DeepSeek-R1-Distill-Llama-8B-finance-v1 with Docker Model Runner:
```
docker model run hf.co/abhi9ab/DeepSeek-R1-Distill-Llama-8B-finance-v1
```

Uploaded model

Developed by: abhi9ab
License: apache-2.0
Finetuned from model : unsloth/DeepSeek-R1-Distill-Llama-8B

This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.

Model Card

The goal of this model is to enhance the base model's performance on financial tasks by fine-tuning it on a specialized financial dataset. Using LoRA, this model has been optimized for low-rank adaptation, allowing efficient fine-tuning with fewer resources.

Model Details

Base Model: unsloth/DeepSeek-R1-Distill-Llama-8B
Model Type: Language Model (Distilled)
Fine-Tuning Technique: LoRA (Low-Rank Adaptation)
Fine-Tuned Model: DeepSeek-R1-Distill-Llama-8B-finance-v1
Dataset: Josephgflowers/Finance-Instruct-500k (reduced to 5k JSONL entries)
Platform: Free-tier Kaggle Notebook
Library: Hugging Face Transformers, Unsloth and Pytorch

This model is a fine-tuned version of the unsloth/DeepSeek-R1-Distill-Llama-8B, utilizing LoRA for efficient parameter adaptation. It has been specifically tuned on a reduced version (5k) of the Josephgflowers/Finance-Instruct-500k dataset to enhance performance in finance-related tasks.

Intended Use

The model is intended for tasks related to financial question answering, generation, and instructions that require domain-specific knowledge in finance. It can also be used in other natural language understanding and generation tasks that benefit from fine-tuning on a finance-specific dataset.

Dataset

The model was fine-tuned on a subset of the Finance-Instruct-500k dataset from Hugging Face, specifically reduced to 5,000 JSONL entries for the fine-tuning process. This dataset contains financial questions and answers, providing a rich set of examples for training the model.

Training Data

Dataset Name: Josephgflowers/Finance-Instruct-500k
Data Size: 5k samples (subset from original dataset)
Domain: Finance
Task: Instruction-based fine-tuning for financial information retrieval and generation.

Notes

This fine-tuning was performed on the free-tier of Kaggle Notebook, so training time and available resources are limited.
Ensure that your runtime in Colab/Kaggle is set to a GPU environment to speed up the training process.
The reduced 5k dataset is a smaller sample for experimentation. You can scale this up depending on your needs and available resources.

Performance

The model performs well in financial instruction tasks, delivering accurate responses based on the reduced dataset. Performance can be further evaluated through specific finance-related benchmarks.

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("abhi9ab/DeepSeek-R1-Distill-Llama-8B-finance-v1")
model = AutoModelForCausalLM.from_pretrained("abhi9ab/DeepSeek-R1-Distill-Llama-8B-finance-v1")

inputs = tokenizer("Example finance-related query", return_tensors="pt")
outputs = model.generate(inputs['input_ids'])

Acknowledgement

Josephgflowers for the dataset.
Hugging Face Transformers library for model implementation and Unsloth for LoRA-based fine-tuning.

Downloads last month: 9

Model tree for abhi9ab/DeepSeek-R1-Distill-Llama-8B-finance-v1

Base model

deepseek-ai/DeepSeek-R1-Distill-Llama-8B

Finetuned

unsloth/DeepSeek-R1-Distill-Llama-8B