Instructions to use IndexTeam/Index-1.9B-Chat with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use IndexTeam/Index-1.9B-Chat with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="IndexTeam/Index-1.9B-Chat", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("IndexTeam/Index-1.9B-Chat", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use IndexTeam/Index-1.9B-Chat with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "IndexTeam/Index-1.9B-Chat"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "IndexTeam/Index-1.9B-Chat",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/IndexTeam/Index-1.9B-Chat

SGLang

How to use IndexTeam/Index-1.9B-Chat with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "IndexTeam/Index-1.9B-Chat" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "IndexTeam/Index-1.9B-Chat",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "IndexTeam/Index-1.9B-Chat" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "IndexTeam/Index-1.9B-Chat",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use IndexTeam/Index-1.9B-Chat with Docker Model Runner:
```
docker model run hf.co/IndexTeam/Index-1.9B-Chat
```

Index-1.9B-Chat / README_zh.md

AsirAsir

Upload 2 files

3b64139 verified about 2 years ago

preview code

Raw

History Blame Contribute Delete

2.68 kB

metadata

license: other
license_name: license
license_link: LICENSE

Index-1.9B-Chat

模型介绍

我们很高兴首次发布Index系列模型中的轻量版本：Index-1.9B系列本次开源的Index-1.9B 系列包含以下模型：

Index-1.9B base : 基座模型，具有 19亿非词嵌入参数量，在2.8T 中英文为主的语料上预训练，多个评测基准上与同级别模型比处于领先。
Index-1.9B pure : 基座模型的对照组，与base具有相同的参数和训练策略，不同之处在于我们严格过滤了该版本语料中所有指令相关的数据，以此来验证指令对benchmark的影响。
Index-1.9B chat(本仓库模型) : 基于index-1.9B base通过SFT和DPO对齐后的对话模型，我们发现由于预训练中引入了较多定向清洗的对话类语料，聊天的趣味性明显更强。
Index-1.9B character : 在SFT和DPO的基础上引入了RAG来实现fewshots角色扮演定制。

已适配llamacpp和Ollama，详见Index-1.9B-Chat-GGUF

更多细节详见我们的GitHub和Index-1.9B技术报告

Transformers 加载方式

可通过以下代码加载 Index-1.9B-Chat 模型来进行对话：

import argparse
from transformers import AutoTokenizer, pipeline

# 注意！目录不能含有"."，可以替换成"_"
parser = argparse.ArgumentParser()
parser.add_argument('--model_path', default="IndexTeam/Index-1.9B-Chat", type=str, help="")
parser.add_argument('--device', default="cpu", type=str, help="") # also could be "cuda" or "mps" for Apple silicon
args = parser.parse_args()

tokenizer = AutoTokenizer.from_pretrained(args.model_path, trust_remote_code=True)
generator = pipeline("text-generation",
                    model=args.model_path,
                    tokenizer=tokenizer, trust_remote_code=True, 
                    device=args.device)


system_message = "你是由哔哩哔哩自主研发的大语言模型，名为“Index”。你能够根据用户传入的信息，帮助用户完成指定的任务，并生成恰当的、符合要求的回复。"
query = "续写 天不生我金坷垃"
model_input = []
model_input.append({"role": "system", "content": system_message})
model_input.append({"role": "user", "content": query})

model_output = generator(model_input, max_new_tokens=300, top_k=5, top_p=0.8, temperature=0.3, repetition_penalty=1.1, do_sample=True)

print('User:', query)
print('Model:', model_output)