How fast is on-device AI on an iPhone? The honest answer is that it depends, and this page is the hub that explains what it depends on. iPhone LLM tokens per second is shaped by five things: the model size, the quantization, the chip and its Neural Engine, how long your context is, and whether the phone has started to thermally throttle. Understand those five levers and you can predict, within a reasonable range, how fast any small model will feel on your phone. For a single-model deep dive with device-by-device numbers, see our Llama 3.2 iPhone benchmarks. For picking a model, our best GGUF models for iPhone and best on-device LLM apps guides go deeper.
Want the cheat sheet? Jump to the summary table. Want a private model tuned for exactly this? PocketLLM runs GGUF models fully on-device with zero telemetry on your conversations. Join the launch list.
On a recent iPhone, a 1B model at Q4 typically runs in the tens of tokens per second (roughly 30 to 55 tok/s in community benchmarks), a 3B model lands lower (roughly 15 to 25 tok/s), and a 7B model is memory-constrained and slower still. Numbers vary by chip, quantization, context length, and thermal state, so treat any single figure as a range, not a guarantee. Every range on this page is labeled with its source, and none were measured by us.
PocketLLM is launching soon. Private, on-device AI, starting on iPhone and iPad with more platforms planned. No account, no tracking, no cloud. Join the launch list and be first in.
Join the launch listWhat "tokens per second" actually measures
A token is a chunk of text, roughly three quarters of a word, so a 100-token reply is about 75 words. Tokens per second (tok/s) is the rate at which the model generates those chunks after it starts replying. It is the decode speed, the number people usually mean when they call a model fast or slow.
Two speeds matter. Time to first token is the gap between pressing send and seeing the first character, dominated by the prefill step where the model reads your prompt. Throughput is the tok/s that follows. A model can have modest throughput and still feel snappy if it starts quickly. For a rule of thumb, average silent reading is around 5 to 8 words per second, roughly 7 to 11 tok/s, so anything sustaining 15 tok/s or more reads like a fast, competent typist.
What drives iPhone LLM tokens per second
Five levers explain almost all of the variation, and together they let you estimate a model's speed before you download it.
- Model size (parameters). Bigger models do more math per token, so a 3B is slower than a 1B on the same phone, and a 7B slower again. This is the single biggest lever. Our small language models guide covers why the phone-comfortable range is 1B to 3B.
- Quantization. Storing each weight in fewer bits (about 4 bits at Q4 instead of 16) shrinks the file and, more importantly, cuts the memory bandwidth needed per token. Bandwidth is usually the bottleneck on a phone, so a Q4 model is both smaller and faster than full precision. Our quantization explainer walks through the levels.
- Chip and Neural Engine. A newer Apple silicon generation has more memory bandwidth and faster GPU and Neural Engine cores, so the same model runs faster on a newer iPhone. PocketLLM's GGUF models run through a llama.cpp-style runtime with Metal acceleration, and a CoreML path can also use the Neural Engine. One clarification people get wrong: GGUF is a model file format used by llama.cpp-style runtimes, while MLX is Apple's array and machine-learning framework for Apple Silicon with its own model format. They are different categories, so the fair comparison is GGUF plus llama.cpp versus MLX. Our GGUF explainer has the details.
- Context length. The longer the conversation, the more tokens the model attends to on every step, and the larger the key-value cache grows. Both add cost, so tok/s drifts down as a chat gets long, and the extra memory matters on a phone.
- Thermal state. Sustained generation heats the chip, and iOS reduces clock speed to protect the device, which lowers tok/s. Short chat turns rarely trigger this, but many minutes of continuous generation can.
Estimated tokens per second by model class
The table below gives rough ranges by model class at Q4 on a recent iPhone. These are estimates and community benchmark figures, not our own measurements, and they are wide on purpose.
| Model class (Q4) | Approx file size | RAM you want free | Typical iPhone throughput | Source |
|---|---|---|---|---|
| 1B | ~0.6–0.8 GB | ~1.5–2 GB | ~30–55 tok/s | Community benchmark |
| 3B | ~2 GB | ~3–4 GB | ~15–25 tok/s | Community benchmark |
| 7B | ~4–4.5 GB | ~6–8 GB | ~4–12 tok/s | Estimate |
A critical caveat: file size is not RAM need. Weights are only part of the footprint: the runtime, the OS, and the key-value cache (which grows with context length) all need headroom, and iOS caps how much memory an app may use, so a 2 GB model wants meaningfully more than 2 GB of RAM free. Our guide on how much RAM a local LLM needs works through the math: roughly 0.6 GB of weights per 1B parameters at Q4, plus headroom.
Note the ceiling. A 7B model is roughly the top of what a phone can hold, and it is tight. Much larger open models, such as OpenAI's gpt-oss releases at 20B and 120B (which need roughly 16 GB and 80 GB of memory), are desktop and workstation class and do not run on a phone at all.
How we will measure
We have not run our own iPhone benchmark suite yet, because PocketLLM is pre-launch. When we do, we will publish the full method so the numbers are reproducible. Here is what we will record for every result:
- Device: exact iPhone model and RAM (e.g. iPhone 16 Pro, 8 GB).
- OS: the iOS version.
- Build: PocketLLM version and build number.
- Backend: llama.cpp GGUF runtime with Metal, or CoreML.
- Model id: exact repository and file name, not just a family name.
- Quantization: precise quant, e.g. Q4_K_M.
- Context: context length, e.g. 4096 tokens.
- Run count: runs per figure, so no single lucky pass.
- Statistic: median (resists outliers), prefill and decode reported separately.
- Thermal state: reported state (nominal, fair, serious), plus battery level and charge state.
- Date: the test date.
Until then, we point to community benchmarks and model-card figures and label them as such. The post closest to first-party numbers is our Llama 3.2 iPhone benchmarks; older, model-specific posts point up to this hub for the general framework.
Summary: the levers at a glance
Here is the whole picture in one table, including how PocketLLM approaches each lever.
| Lever | Effect on speed | PocketLLM approach |
|---|---|---|
| Model size | Bigger is slower | Ships phone-class 1B–3B GGUF by default |
| Quantization | Fewer bits is faster and smaller | Q4 GGUF quants tuned for iPhone |
| Chip / Neural Engine | Newer silicon is faster | CoreML plus llama.cpp with Metal acceleration |
| Context length | Longer context is slower, more RAM | Sensible defaults, on-device KV cache |
| Thermal state | Heat throttles sustained speed | Short chat turns stay cool; no background grind |
| Where it runs | Cloud adds a network round trip | Fully on-device, no account, no network hop |
How to get more tokens per second on your iPhone
If a model feels slow, you have real levers to pull before blaming the phone:
- Drop a size class. On a 6 GB iPhone, a 1B model usually feels better than forcing a 3B, with less memory pressure and throttling. On 8 GB, 3B is the sweet spot.
- Use a Q4 quant. The standard on-device trade-off: faster and smaller with a minor quality cost that rarely shows in everyday chat.
- Keep chats focused. Long conversations grow the context and KV cache, slowing every new token. Start a fresh chat for a new topic.
- Let a hot phone cool. After a long session, throughput recovers once the chip returns to a nominal thermal state.
- Match the model to the task. Rewriting, summarizing, and casual chat run great on small models; the hardest multi-step reasoning is where a bigger cloud model pulls ahead. See our comparison of on-device versus cloud AI.
Frequently asked questions
What is a good iPhone LLM tokens per second?
For chat, anything faster than your reading speed feels good, and most people read around 5 to 8 words per second, which is roughly 7 to 11 tokens per second. So a model that sustains 15 tokens per second or more will feel like a fast, competent typist. In community benchmarks, a 1B model at Q4 on a recent iPhone often runs in the 30 to 55 tokens per second range, a 3B model in the 15 to 25 range, and a 7B model lower and memory-constrained. Treat these as ranges, not guarantees, because iPhone LLM tokens per second depends on the chip, the quantization, the context length, and whether the phone has warmed up.
How many tokens per second do I need for a chat to feel fast?
Faster than you can read, which is the key threshold. Average silent reading is roughly 5 to 8 words per second, and one token is about three quarters of a word, so 10 tokens per second is already at the edge of comfortable and 15 or more feels quick. Time to first token matters just as much as raw throughput: under about 400 milliseconds from pressing send to the first character feels instant. A model can generate at a modest tokens per second and still feel responsive if it starts quickly.
Does quantization make an iPhone LLM faster?
Yes, in two ways. Quantization shrinks each weight from 16 bits to, commonly, about 4 bits, which cuts the model file size and the amount of memory bandwidth needed per token, and memory bandwidth is usually the bottleneck on a phone. So a Q4 model is both smaller and faster than the same model at full precision. The trade-off is a small quality loss, which for 4-bit quants is usually minor for everyday chat.
Why does my on-device model slow down during a long session?
Two reasons, usually. First, thermal throttling: sustained generation heats the chip, and iOS reduces clock speed to protect the device, which lowers tokens per second. Short chat turns rarely trigger this, but many minutes of continuous generation can. Second, context length: as a conversation grows, the model has more tokens to attend to on every step, and the key-value cache grows, so each new token costs a little more compute and memory. Starting a fresh chat or letting the phone cool restores full speed.
Can an iPhone run a 7B model at usable speed?
On a recent iPhone with 8 GB of RAM, a 7B model at Q4 (roughly 4 to 4.5 GB on disk) can load, but it is tight on memory and noticeably slower than a 3B, often in the single digits to low tens of tokens per second in estimates. For most people a 3B model is the sweet spot on a phone, with 7B reserved for iPad or Mac. Much larger open models, such as the 20B and 120B gpt-oss releases, are desktop and workstation class (needing roughly 16 GB and 80 GB of memory) and do not run on a phone at all.