Instructions to use zhengchenphd/ICE-GRT with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use zhengchenphd/ICE-GRT with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="zhengchenphd/ICE-GRT")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("zhengchenphd/ICE-GRT") model = AutoModelForCausalLM.from_pretrained("zhengchenphd/ICE-GRT", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use zhengchenphd/ICE-GRT with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "zhengchenphd/ICE-GRT" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "zhengchenphd/ICE-GRT", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/zhengchenphd/ICE-GRT
- SGLang
How to use zhengchenphd/ICE-GRT with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "zhengchenphd/ICE-GRT" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "zhengchenphd/ICE-GRT", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "zhengchenphd/ICE-GRT" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "zhengchenphd/ICE-GRT", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use zhengchenphd/ICE-GRT with Docker Model Runner:
docker model run hf.co/zhengchenphd/ICE-GRT
Commit History
update readme 97666a0
Chen Zheng commited on
update readme 9509278
Chen Zheng commited on
update readme 4783bb2
Chen Zheng commited on
update readme e8b9c07
Chen Zheng commited on
update readme a6b787e
Chen Zheng commited on
update readme f917f60
Chen Zheng commited on
readme 9e40036
Chen Zheng commited on
update pytorch_model.bin 0fc9b92
Chen Zheng commited on
update config and tokenizer c5b20a0
Chen Zheng commited on
update readme 0025a8a
Chen Zheng commited on
old ckpt 863e0c0
Chen Zheng commited on
update pytorch_model.bin 3a32f07
Chen Zheng commited on
upload model card 1e89e95
Chen Zheng commited on
update readme 86b9ff4
Chen Zheng commited on
write readme 2ecd103
Chen Zheng commited on