Instructions to use OpenYourMind/Minimax-M3-abliterated-clean with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use OpenYourMind/Minimax-M3-abliterated-clean with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="OpenYourMind/Minimax-M3-abliterated-clean", trust_remote_code=True)
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("OpenYourMind/Minimax-M3-abliterated-clean", trust_remote_code=True)
model = AutoModelForMultimodalLM.from_pretrained("OpenYourMind/Minimax-M3-abliterated-clean", trust_remote_code=True, device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use OpenYourMind/Minimax-M3-abliterated-clean with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "OpenYourMind/Minimax-M3-abliterated-clean"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpenYourMind/Minimax-M3-abliterated-clean",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/OpenYourMind/Minimax-M3-abliterated-clean

SGLang

How to use OpenYourMind/Minimax-M3-abliterated-clean with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "OpenYourMind/Minimax-M3-abliterated-clean" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpenYourMind/Minimax-M3-abliterated-clean",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "OpenYourMind/Minimax-M3-abliterated-clean" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpenYourMind/Minimax-M3-abliterated-clean",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use OpenYourMind/Minimax-M3-abliterated-clean with Docker Model Runner:
```
docker model run hf.co/OpenYourMind/Minimax-M3-abliterated-clean
```

Please, make GGUF Q2 version

by denisvasi - opened Jun 17

Discussion

denisvasi

Jun 17

I have only 4 A100 40Gb. Now i use unsloth version

OpenYourMind

Owner Jun 17

Thanks for the interest, currently i am not making any quantizations of models as there usualy is someone doing them. Usualy a GGUF Q2 version should be doable on CPU / RAM yourself easily.

SicariusSicariiStuff

Jun 18

https://huggingface.co/SicariusSicariiStuff/Minimax-M3-abliterated_GGUF

Notice: this is an experimental version, haven't tested it, please let me know if there are any issues
Q3 is up, soon Q2

OpenYourMind

Owner Jun 18

Thanks man, much appreciate you taking the time to do GGUF's

denisvasi

Jun 18

@SicariusSicariiStuff Can you provide smaller model like IQ2_M, because Q2_K too big for my server (160GB). It will be great

morikomorizz

Jun 19

•

edited Jun 19

@OpenYourMind If you don't mind, could you share the total refusal rate and KL divergence of the model? Additionally, may I have your permission to create a GGUF variant?

OpenYourMind

Owner Jun 19

•

edited Jun 19

@morikomorizz
well, i am not too much of a fan of Refusal rate reporting (as everyone else does) just because the refusal rate differs as everyone uses different dattasets and they just mean "nothing". We use our Severe Harmfull prompt set for refusal detection and a clean ablation is always 0/1300 after ablation with 0 garble on 500 non harmfull counter tested.

my ablation technique i am not checking for KL divergence (i am not using a framework like heretic abliterix obliterus or any of those). If you want to measure around that feel free to run / compare for KL youself. You can do as many GGUF as you like ... its open weights and i dont care much. if u add a credit line somewhere would be nice .... maybe someone would like to buy me a coffe then ;)

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment