Instructions to use Ddavidich/LOMONOSOV-ZENIT-27B-1M-INDEV with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Ddavidich/LOMONOSOV-ZENIT-27B-1M-INDEV with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Ddavidich/LOMONOSOV-ZENIT-27B-1M-INDEV", trust_remote_code=True)
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForImageTextToText

processor = AutoProcessor.from_pretrained("Ddavidich/LOMONOSOV-ZENIT-27B-1M-INDEV", trust_remote_code=True)
model = AutoModelForImageTextToText.from_pretrained("Ddavidich/LOMONOSOV-ZENIT-27B-1M-INDEV", trust_remote_code=True, device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Ddavidich/LOMONOSOV-ZENIT-27B-1M-INDEV with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Ddavidich/LOMONOSOV-ZENIT-27B-1M-INDEV"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Ddavidich/LOMONOSOV-ZENIT-27B-1M-INDEV",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Ddavidich/LOMONOSOV-ZENIT-27B-1M-INDEV

SGLang

How to use Ddavidich/LOMONOSOV-ZENIT-27B-1M-INDEV with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Ddavidich/LOMONOSOV-ZENIT-27B-1M-INDEV" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Ddavidich/LOMONOSOV-ZENIT-27B-1M-INDEV",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Ddavidich/LOMONOSOV-ZENIT-27B-1M-INDEV" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Ddavidich/LOMONOSOV-ZENIT-27B-1M-INDEV",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Ddavidich/LOMONOSOV-ZENIT-27B-1M-INDEV with Docker Model Runner:
```
docker model run hf.co/Ddavidich/LOMONOSOV-ZENIT-27B-1M-INDEV
```

LOMONOSOV-ZENIT-27B-1M-INDEV / receipts /kv_precision /fp8.json

Ddavidich

Ship the receipts the card cites, and fix the 4090 citation

0d63b24 verified 2 months ago

Raw History Blame Contribute Delete

4.47 kB

	{
	"schema": "lomonosov_zenit_retention_probe_v1",
	"label": "kv_fp8",
	"model": "Ddavidich/LOMONOSOV-ZENIT-27B-1M-INDEV",
	"corpus_sha256": "66a5b0f2eed04fce33318cd4f295c4ea7bc25f3116b00798284c20576d61bb6e",
	"corpus_books": 3,
	"prompt_tokens": 250334,
	"needles_scored": 14,
	"correct": 11,
	"accuracy": 0.7857142857142857,
	"kv_cache_dtype": "fp8",
	"native_rope_requested": true,
	"rope_effective": {
	"top": {
	"rope_type": "default",
	"factor": null,
	"original_max_position_embeddings": null,
	"partial_rotary_factor": 0.25,
	"attention_factor": null
	},
	"text_config": {
	"rope_type": "default",
	"factor": null,
	"original_max_position_embeddings": null,
	"partial_rotary_factor": 0.25,
	"attention_factor": null
	}
	},
	"position_plan": {
	"ZENIT_NOOSPHERE_MODE": "raw"
	},
	"load_seconds": 60.18490409798687,
	"first_question_seconds": 141.53339269699063,
	"later_question_seconds_mean": 2.0742292880761224,
	"results": [
	{
	"index": 1,
	"depth_fraction": 0.06666666666666667,
	"token_position": 16666,
	"expected": "BQ-13682",
	"answer_text": "BQ-13682",
	"correct": true,
	"seconds": 141.53339269699063,
	"was_first_question": true
	},
	{
	"index": 2,
	"depth_fraction": 0.13333333333333333,
	"token_position": 33357,
	"expected": "HB-24619",
	"answer_text": "<think>\n\n</think>\n\nLOMONOSOV-ZENITLOMONOSOV-ZENITLOMONOSOV-ZEN",
	"correct": false,
	"seconds": 4.365870871988591,
	"was_first_question": false
	},
	{
	"index": 3,
	"depth_fraction": 0.2,
	"token_position": 50047,
	"expected": "RJ-46395",
	"answer_text": "RJ-46395",
	"correct": true,
	"seconds": 1.438419154001167,
	"was_first_question": false
	},
	{
	"index": 4,
	"depth_fraction": 0.26666666666666666,
	"token_position": 66736,
	"expected": "ZK-83394",
	"answer_text": "ZK-83394",
	"correct": true,
	"seconds": 1.5427661190042272,
	"was_first_question": false
	},
	{
	"index": 5,
	"depth_fraction": 0.3333333333333333,
	"token_position": 83428,
	"expected": "TR-15255",
	"answer_text": "<think>\n\n</think>\n\nTR-15255",
	"correct": true,
	"seconds": 1.7670672749809455,
	"was_first_question": false
	},
	{
	"index": 6,
	"depth_fraction": 0.4,
	"token_position": 100120,
	"expected": "MK-13191",
	"answer_text": "<think>\n\n</think>\n\nMK-13191",
	"correct": true,
	"seconds": 1.871128962986404,
	"was_first_question": false
	},
	{
	"index": 7,
	"depth_fraction": 0.4666666666666667,
	"token_position": 116810,
	"expected": "PW-31226",
	"answer_text": "<think>\n\n</think>\n\nPW-31226",
	"correct": true,
	"seconds": 2.0053434440051205,
	"was_first_question": false
	},
	{
	"index": 8,
	"depth_fraction": 0.5333333333333333,
	"token_position": 133500,
	"expected": "CY-84017",
	"answer_text": "<think>\n\n</think>\n\nNT-82593",
	"correct": false,
	"seconds": 1.9487302799825557,
	"was_first_question": false
	},
	{
	"index": 9,
	"depth_fraction": 0.6,
	"token_position": 150191,
	"expected": "GD-41058",
	"answer_text": "<think>\n\n</think>\n\nNT-82593",
	"correct": false,
	"seconds": 1.8654474189970642,
	"was_first_question": false
	},
	{
	"index": 10,
	"depth_fraction": 0.6666666666666666,
	"token_position": 166882,
	"expected": "QX-74073",
	"answer_text": "<think>\n\n</think>\n\nQX-74073",
	"correct": true,
	"seconds": 2.2831903980113566,
	"was_first_question": false
	},
	{
	"index": 11,
	"depth_fraction": 0.7333333333333333,
	"token_position": 183573,
	"expected": "NT-82593",
	"answer_text": "<think>\n\n</think>\n\nNT-82593",
	"correct": true,
	"seconds": 2.178749257000163,
	"was_first_question": false
	},
	{
	"index": 12,
	"depth_fraction": 0.8,
	"token_position": 200264,
	"expected": "VM-48579",
	"answer_text": "VM-48579",
	"correct": true,
	"seconds": 1.781214969989378,
	"was_first_question": false
	},
	{
	"index": 13,
	"depth_fraction": 0.8666666666666667,
	"token_position": 216954,
	"expected": "LF-80058",
	"answer_text": "<think>\n\n</think>\n\nLF-80058",
	"correct": true,
	"seconds": 2.106787228025496,
	"was_first_question": false
	},
	{
	"index": 14,
	"depth_fraction": 0.9333333333333333,
	"token_position": 233644,
	"expected": "XS-27918",
	"answer_text": "<think>\n\n</think>\n\nXS-27918",
	"correct": true,
	"seconds": 1.81026536601712,
	"was_first_question": false
	}
	]
	}