If you have looked into running AI locally on a Mac, you have almost certainly run into the letters MLX, usually next to a model name on a download page or in an app's backend settings. So what is MLX? In plain terms, MLX is Apple's open-source array and machine-learning framework, built specifically for Apple Silicon (the M-series chips in modern Macs, and increasingly the chips in iPhone and iPad). It is the toolkit Apple's own researchers use to run models efficiently on Apple hardware, and it has quietly become one of the main ways people run local large language models on a Mac. This guide explains what MLX actually is, the two ideas that make it fast, how it compares to PyTorch and to the GGUF and llama.cpp stack, and, just as importantly, what MLX is not.
Want the short version? Jump to the comparison table. If you would rather skip frameworks entirely and just chat privately on your phone, PocketLLM runs the model fully on-device with no account and zero telemetry: join the launch list. Our roundup of the best local AI apps for Mac and our guide to LM Studio both cover apps that can run MLX models.
MLX is a framework, not a file format and not an app. It is Apple's NumPy-like array library with machine learning built in, designed to exploit Apple Silicon's unified memory so the CPU and GPU share one pool of data with no copying, and it uses lazy evaluation to optimize work before it runs. On a Mac it powers local LLMs through tools like mlx-lm, and it now reaches iPhone and iPad through MLX Swift. Its closest counterpart for running local models is the llama.cpp plus GGUF stack.
PocketLLM is launching soon. Private, on-device AI, starting on iPhone and iPad with more platforms planned. No account, no tracking, no cloud. Join the launch list and be first in.
Join the launch listWhat is MLX, exactly?
MLX is an array framework: at its core it gives you multi-dimensional arrays (tensors) and a large set of math operations on them, much like NumPy or the tensor part of PyTorch. On top of that it adds the machinery for machine learning: automatic differentiation, neural-network building blocks, and optimizers. What sets it apart is that it was designed from scratch for Apple Silicon rather than adapted from an NVIDIA-first world. Apple released it as open source, with both a Python API and a Swift API (MLX Swift). When someone calls a model "an MLX model," they mean its weights have been converted into the layout MLX expects, usually from PyTorch or safetensors, and often quantized to 4-bit.
Unified memory: MLX's core advantage
On a typical PC with a discrete graphics card, the CPU has its own system RAM and the GPU has its own separate VRAM, so to run a model on the GPU you copy data into VRAM and are capped by how much the card has. Apple Silicon works differently: it has unified memory, a single physical pool of RAM shared by the CPU and the GPU. MLX was built around this. Its arrays live in that shared memory, so an operation can run on the CPU or the GPU without moving the data first. For large language models this matters twice: there is no copy overhead between processor and accelerator, and how much model you can load is bounded by total system RAM, not a smaller VRAM budget. A 32 GB Mac can hold a model that would not fit on a mainstream 8 GB graphics card.
Lazy evaluation and why it helps
The second idea is lazy evaluation. In an eager framework, each line of math runs the instant you write it. MLX instead records what you asked for and does not compute until you force it, by reading a result or calling an explicit evaluate step. Building the plan first lets MLX optimize the whole chain: fusing steps, skipping work whose result is never used, and scheduling across the CPU and GPU. It also offers composable transformations, so you can take gradients, vectorize over a batch, or compile a function into a faster graph. You do not need to understand any of this to run a model, but it is a big part of why MLX is fast for the size of its codebase.
How MLX powers local LLMs (on Mac, and now iPhone and iPad)
On a Mac, most people meet MLX through a companion library called mlx-lm, which wraps the framework with helpers to load a model and generate text or run a small local server. A community on Hugging Face publishes converted, quantized versions of popular open models in the MLX layout, so you can download one and run it in a couple of commands. Several desktop apps also expose an MLX backend as an alternative to loading a GGUF file. On iPhone and iPad, MLX Swift lets developers run smaller models fully on-device. The realistic ceiling on a phone is a small model, roughly 1B to 3B at 4-bit, because everything must fit in the memory the OS allows an app.
MLX vs PyTorch
PyTorch is the dominant deep-learning framework in the wider world. It runs across NVIDIA GPUs, CPUs, and Apple's own Metal backend, and almost every research paper and pretrained model ships in it first. MLX deliberately does less: it targets Apple Silicon only, is lazy by default, and is native to unified memory. The upside of that focus is a smaller codebase and, on a Mac, often faster or simpler local inference with a familiar NumPy and PyTorch-like API. PyTorch's upside is portability: to train across a rack of NVIDIA GPUs or deploy on many platforms, it is the safe choice. In practice the two coexist, because most MLX language models started life as PyTorch weights.
MLX vs GGUF and llama.cpp
This is where category confusion creeps in, so it is worth being precise. GGUF is a file format. llama.cpp is the runtime that reads a GGUF file and runs the model. MLX is a framework with its own model layout. So "MLX vs GGUF" is not a like-for-like matchup; the honest framing is the MLX stack versus the llama.cpp plus GGUF stack, two ways to run the same quantized models on a Mac. The GGUF and llama.cpp side wins on portability (Windows, Linux, CPUs, and phones), on the number of ready-to-run models, and on being the format most local-AI apps import. The MLX side wins on being native to Apple Silicon and tightly integrated with Python and Swift. For the deeper head-to-head, see our GGUF vs MLX comparison and our primer on what GGUF models are.
The comparison table
| Aspect | MLX | llama.cpp + GGUF | PyTorch | PocketLLM |
|---|---|---|---|---|
| What it is | ML framework | Runtime + file format | ML framework | Consumer app |
| Runs on | Apple Silicon (Mac, iPhone, iPad) | Mac, Windows, Linux, phones | NVIDIA, CPU, Apple Metal | iPhone, iPad, Mac (more planned) |
| Model format | MLX (converted) | GGUF | PyTorch / safetensors | GGUF (via llama.cpp) |
| Who it is for | Developers on Apple hardware | Developers and app builders | ML engineers everywhere | Everyday users |
| Setup effort | Install and some code | App or command line | Code | Download, no config |
Example models people run this way
Whether you use MLX or the GGUF and llama.cpp stack, the models and sizing rules are the same. The headline is the 4-bit file size, but file size is not RAM need: add headroom for the KV cache (which grows with conversation length), the runtime, and the OS, and on iPhone the OS caps app memory. Treat these as approximate.
| Model | Approx params | ~4-bit size | Comfortable RAM | Source |
|---|---|---|---|---|
| SmolLM2 | 135M–1.7B | very small | ~1–2 GB | Model card |
| Llama 3.2 3B | ~3B | ~2 GB | ~4 GB or more | Model card |
| Phi-3.5 Mini | ~3.8B | ~2.2 GB | ~4 GB or more | Estimate |
| Qwen 2.5 7B | ~7B | ~4.5 GB | ~8 GB | Estimate |
Community benchmarks put a 3B model at roughly 30 or more tokens per second on an M2-class Mac, responsive enough for chat, though the figure depends on the model, quantization, and hardware. The comfortable phone-class range is 1B to 3B; a 7B is happier on a Mac with 16 GB or more. At the far end, OpenAI's open-weight gpt-oss models (about 20B needing roughly 16 GB of memory, about 120B needing roughly 80 GB) are workstation class, not something a phone runs.
What MLX is NOT
Because the name shows up in so many places, MLX gets miscast. To keep it straight:
- Not a file format. GGUF and safetensors are formats. MLX is the framework; the format confusion is the single most common mistake.
- Not an app you download to chat. MLX has no chat window. It is the plumbing that apps and scripts build on.
- Not a model. MLX does not "know" anything. It runs models that you load into it.
- Not the same as Apple Intelligence. That and the on-device Foundation Models are shipped product features; MLX is the open research framework developers use directly.
Where PocketLLM fits
MLX and the GGUF plus llama.cpp stack both answer the same question: how do I run a capable model on my own hardware so my data never leaves it? PocketLLM answers it for people who do not want to touch a framework. Under the hood it uses llama.cpp with GGUF models, alongside a CoreML path, rather than MLX, but the outcome is the design that matters: the model runs on your iPhone, iPad, or Mac, there is no account, and we collect zero telemetry on your prompts or responses. You never convert weights or pick a quantization; the app handles that for you.
Frequently asked questions
What is MLX in simple terms?
MLX is Apple's open-source array and machine-learning framework, built specifically for Apple Silicon (the M-series chips in Macs and the chips in recent iPhones and iPads). Think of it as a NumPy-like library with machine learning built in, tuned so the CPU and GPU share one pool of memory with no copying. It is what many people use to run local LLMs on a Mac, and it now reaches iPhone and iPad through MLX Swift. It is a framework, not a file format and not a chat app.
Is MLX a file format like GGUF?
No, and this is the most common mix-up. GGUF is a model file format that llama.cpp-style runtimes load. MLX is a framework, a set of libraries you write or run code against, and it has its own model layout that is usually converted from PyTorch or safetensors weights. So the fair comparison is not MLX versus GGUF but the MLX stack versus the llama.cpp plus GGUF stack: two ways to run the same quantized models locally on a Mac.
Can MLX run LLMs on iPhone and iPad?
Yes, within limits. MLX has a Swift interface (MLX Swift) that lets developers run models on iOS and iPadOS, not just macOS. The practical limit is memory: a phone comfortably runs small models in roughly the 1B to 3B range at 4-bit, because the file plus the KV cache, runtime, and operating system must all fit in the RAM the OS lets an app use. Workstation-scale models do not run on a phone regardless of the framework.
MLX vs llama.cpp: which is faster on a Mac?
It depends on the model, the quantization, and the version of each project, so treat any single benchmark with caution. Both are well optimized for Apple Silicon and run popular models at usable speeds; community benchmarks show MLX ahead on some models and llama.cpp ahead on others. llama.cpp with GGUF wins on portability and the number of ready-to-run models and apps. MLX wins on tight Python and Swift integration on Apple hardware. For most users, quantization choice matters more.
Does PocketLLM use MLX?
PocketLLM's on-device models run through llama.cpp using the GGUF format, alongside a CoreML path, rather than through MLX. The goal is the same as any MLX setup: run the model fully on your device so your conversations never leave it. The difference is that PocketLLM handles the format and memory sizing for you, so you never convert weights or pick a quantization by hand. It is pre-launch, with an email waitlist and no App Store link yet.