Skip to content

mlx-lm

Outlines provides an integration with mlx-lm, allowing models to be run quickly on Apple Silicon via the mlx library.

Installation

In addition to outlines, you must install mlx-lm and mlx libraries. You must use a device which supports Metal.

Using models.mlxlm

from outlines import models

model = models.mlxlm("mlx-community/Meta-Llama-3-8B-Instruct-8bit")

With the loaded model, you can generate text or perform structured generation, e.g.

from outlines import models, generate

model = models.mlxlm("mlx-community/Meta-Llama-3-8B-Instruct-8bit")

phone_number_pattern = "\\+?[1-9][0-9]{7,14}"
generator = generate.regex(model, phone_number_pattern)

model_output = generator("What's Jennys Number?\n")
print(model_output)
# '8675309'