Instructions to use unsloth/DeepSeek-V4-Pro with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use unsloth/DeepSeek-V4-Pro with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="unsloth/DeepSeek-V4-Pro")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("unsloth/DeepSeek-V4-Pro") model = AutoModelForCausalLM.from_pretrained("unsloth/DeepSeek-V4-Pro", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use unsloth/DeepSeek-V4-Pro with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "unsloth/DeepSeek-V4-Pro" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "unsloth/DeepSeek-V4-Pro", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/unsloth/DeepSeek-V4-Pro
- SGLang
How to use unsloth/DeepSeek-V4-Pro with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "unsloth/DeepSeek-V4-Pro" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "unsloth/DeepSeek-V4-Pro", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "unsloth/DeepSeek-V4-Pro" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "unsloth/DeepSeek-V4-Pro", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use unsloth/DeepSeek-V4-Pro with Docker Model Runner:
docker model run hf.co/unsloth/DeepSeek-V4-Pro
Mirror worker 6
Browse files- model-00006-of-00064.safetensors +3 -0
- model-00014-of-00064.safetensors +3 -0
- model-00022-of-00064.safetensors +3 -0
- model-00030-of-00064.safetensors +3 -0
- model-00038-of-00064.safetensors +3 -0
- model-00046-of-00064.safetensors +3 -0
- model-00054-of-00064.safetensors +3 -0
- model-00062-of-00064.safetensors +3 -0
model-00006-of-00064.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:cf7e1aaa849312f1d9f31ddb8309b48483a14bd594310db99c548085c6b6d78a
|
| 3 |
+
size 13903802792
|
model-00014-of-00064.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:1830a51d66eaf06b22e1c58329d93b59c7281340bf3d8f851fa9e78b35edab28
|
| 3 |
+
size 13903805136
|
model-00022-of-00064.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:8c0b74723a79f8cc6dea610709ce55336a4f1ba3ba8bf6739869e8764e769bc2
|
| 3 |
+
size 13903805136
|
model-00030-of-00064.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ae1b865dd129d58bb741763eb9714ada226d978161f76beee08740bea0c7afb6
|
| 3 |
+
size 13903805136
|
model-00038-of-00064.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6d88fe9ea60374a3ba960c38e27cc2361a0db3a54148a77bf59067077d5808b6
|
| 3 |
+
size 13903805136
|
model-00046-of-00064.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:4a385c12ab746b513a5590707d5919e713ef9dc7a8bd51d26828fa6578f7c268
|
| 3 |
+
size 13903805136
|
model-00054-of-00064.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a230b4c58df07517a18ed495a86755861ed20eb57b2580a63f1339b3608a0309
|
| 3 |
+
size 13903805136
|
model-00062-of-00064.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:36a8b9ab83f3bc084d4fa0fbbe56418b94b35721aa7743ddcc7c09973dfd0004
|
| 3 |
+
size 13903805136
|