Instructions to use LatitudeGames/Wayfarer-12B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use LatitudeGames/Wayfarer-12B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="LatitudeGames/Wayfarer-12B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("LatitudeGames/Wayfarer-12B")
model = AutoModelForCausalLM.from_pretrained("LatitudeGames/Wayfarer-12B", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use LatitudeGames/Wayfarer-12B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "LatitudeGames/Wayfarer-12B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LatitudeGames/Wayfarer-12B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/LatitudeGames/Wayfarer-12B

SGLang

How to use LatitudeGames/Wayfarer-12B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "LatitudeGames/Wayfarer-12B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LatitudeGames/Wayfarer-12B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "LatitudeGames/Wayfarer-12B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LatitudeGames/Wayfarer-12B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use LatitudeGames/Wayfarer-12B with Docker Model Runner:
```
docker model run hf.co/LatitudeGames/Wayfarer-12B
```

Sample of training data format?

by jaypickle - opened Feb 21, 2025

Discussion

jaypickle

Feb 21, 2025

Hi, first of all thank you so much for this model! The first time it killed me I was overjoyed.

I understand why you wouldn't want to release the full data set, but could we get a few samples? I just want to be sure I'm formatting my prompts in an optimal way for this model.

Thank you!

Gryphe

Latitude org Feb 21, 2025

•

edited Feb 21, 2025

I can share the system prompt that was used for the text adventure data, sure!


Describe scenes, characters, and events with rich sensory detail and specificity. Use diverse speech patterns and vocabulary fitting each character and setting. Create quirky, subversive characters with unusual traits. Develop intriguing plot hooks and campaign stories within the given world lore.

Balance combat, exploration, social interaction, and intrigue. Make combat brutal and consequential. Don't describe thoughts, emotions, or decisions directly - reveal character through action and dialogue. Evaluate attempted actions (preceded by >) for realistic outcomes.

Write concisely with everyday words, avoiding similes and metaphors. Vary sentence structure to control pacing. Always write out speech in full, using quotation marks. Create ongoing adventures with new plot points as needed, focusing on the immediate moment without summarizing or repeating player actions.

Limit responses to 200 tokens. Never write endings or generate > or #, and do not create lists nor end your responses with a question.

Show realistic consequences for dangerous actions. Describe injuries and trauma appropriately for the genre. If the player character dies with no chance of survival, describe their death in detail and end your response with GAME OVER.

[Scenario details go here]

This is followed by a user message with their character's details, followed by an assistant message with the opening narrative ("You are xxx"). And then it's back-and-forth from there!

jaypickle

Feb 21, 2025

Thank you!

Gryphe

Latitude org Feb 21, 2025

No problem - Leaving this thread open since it's useful info.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment