Granite 4.0 MLX-LM 4-bit
Collection
Granite 4.0 Language Models for macOS (MLX-LM) - 4-bit Quantization • 16 items • Updated
How to use ethicalabs/granite-4.0-h-micro-base-MLX with MLX:
# Make sure mlx-lm is installed
# pip install --upgrade mlx-lm
# if on a CUDA device, also pip install mlx[cuda]
# Generate text with mlx-lm
from mlx_lm import load, generate
model, tokenizer = load("ethicalabs/granite-4.0-h-micro-base-MLX")
prompt = "Once upon a time in"
text = generate(model, tokenizer, prompt=prompt, verbose=True)How to use ethicalabs/granite-4.0-h-micro-base-MLX with MLX LM:
# Install MLX LM uv tool install mlx-lm # Generate some text mlx_lm.generate --model "ethicalabs/granite-4.0-h-micro-base-MLX" --prompt "Once upon a time"
This model ethicalabs/granite-4.0-h-micro-base-MLX was converted to MLX format from ibm-granite/granite-4.0-h-micro-base using mlx-lm version 0.28.2.
4-bit
Base model
ibm-granite/granite-4.0-h-micro-base