"It's only a 2 GB model, and my phone has 8 GB of RAM, so I'm fine" is the reasoning that trips up almost everyone who tries local AI for the first time. The download size tells you how big the weights are, not how much memory the model actually needs while it runs. This guide answers the real question, how much RAM for a local LLM, with a sizing table you can use instead of guesswork. If the format side is new to you, our GGUF explainer and quantization guide cover why a Q4 file is so much smaller than the original model.
Want the numbers now? Jump to the sizing table. Want a phone that picks a model sized to its own memory automatically? PocketLLM does that on-device with zero telemetry — join the launch list.
A rough rule is about 0.6 GB of weights per 1 billion parameters at Q4, so a 3B model is around 2 GB. But file size is not RAM need: add the KV cache (which grows with context length), the runtime's compute buffers, and the operating system on top, and phones also cap how much memory one app may use. In practice, budget the weight size plus 50 to 100 percent headroom. That keeps 1B to 3B models comfortable on a modern phone, 7B on a 16 GB laptop, and the largest open models on a desktop.
PocketLLM is launching soon. Private, on-device AI, starting on iPhone and iPad with more platforms planned. No account, no tracking, no cloud. Join the launch list and be first in.
Join the launch listThe quick rule: about 0.6 GB per 1B parameters
At Q4 quantization — the default most on-device apps use — a model needs roughly 0.6 GB of memory for its weights per 1 billion parameters. That gives you a fast mental model: a 1B model is around half a gigabyte, a 3B is about 2 GB, and a 7B is roughly 4.5 GB. It is an estimate, not a guarantee, because exact quant recipes and architectures vary, but it is close enough to plan around. The catch is that the weights are only the floor. What you actually have to fit in RAM is always larger, and the rest of this guide is about that gap.
Why file size is not the same as RAM need
The single most useful thing to internalize about local models is that the number on the download button is a lower bound. Four separate costs stack on top of it.
Weights are just the floor
The GGUF file on disk contains the quantized weights and nothing else. Loading it into memory reproduces that size, so a 2 GB file starts by occupying about 2 GB. Everything below is added on top of that baseline, not folded into it.
The KV cache grows with context length
As the model generates, it keeps a KV cache — the attention keys and values for every token in the conversation. This cache scales with context length and model size. In a short chat it is small, but as the conversation or document gets longer it grows, and at long context it can approach the size of the weights themselves. A model that fits in a quick exchange can run out of memory in a 20,000-token session. Treat the KV cache as a variable cost you have to leave room for, not a fixed one you can ignore.
Runtime, compute buffers, and the OS
The inference runtime (llama.cpp on PocketLLM's on-device GGUF path) allocates compute and scratch buffers while running. On top of that, the operating system, background apps, and the UI are already using memory before your model ever loads. "Free RAM" is never the whole number printed on the spec sheet, which is why targeting 100 percent of nominal memory is a recipe for stutter.
iOS caps how much memory an app can use
On iPhone and iPad there is an extra ceiling: the system limits how much memory any single app may allocate, and it will terminate an app that exceeds it. So even if a device physically has 8 GB, an app cannot use all of it for a model. This is why on-device apps stay conservative, and why a "2 GB model" wants meaningfully more than 2 GB of free headroom to run reliably on a phone. For the models that fit this envelope well, see our best GGUF models for iPhone.
How much RAM for a local LLM, tier by tier
Putting the pieces together, here is what each class of device can comfortably handle. These are ranges, not promises. Apple does not always publish exact per-model RAM, so the phone figures are approximate and teardown-sourced.
| Device tier | Typical RAM | Comfortable model (Q4) | Source |
|---|---|---|---|
| Recent iPhone / iPad (base) | ~6–8 GB | 1B–3B | Estimate |
| Recent iPhone / iPad Pro | ~8 GB | up to ~3B; small 7B is tight | Estimate |
| 8 GB Mac or laptop | 8 GB | 3B comfortably; 7B tight | Estimate |
| 16 GB Mac or laptop | 16 GB | 7B comfortably | Estimate |
| 32 GB+ desktop / workstation | 32–128 GB | large models and gpt-oss-20b | Estimate |
If you are shopping for hardware to run bigger models, our best Mac for local LLMs guide breaks down where the memory tiers land, and running a local LLM on iPad covers the tablet side.
The model-size sizing table
This is the table to bookmark. It maps common model sizes to the free RAM you should aim for, which device tier they suit, and whether they fit PocketLLM's on-device, phone-first envelope. The Source column marks whether a figure comes from a published model card or is our own estimate.
| Model (example) | Approx Q4 weights | Free RAM to aim for | Best device tier | PocketLLM fit | Source |
|---|---|---|---|---|---|
| Tiny (SmolLM2 135M–1.7B) | ~0.1–1 GB | ~1–2 GB | any recent phone | Yes | Model card |
| Small (Llama 3.2 3B) | ~2 GB | ~3–4 GB | phone / any laptop | Yes | Model card |
| Phi-3.5 Mini (~3.8B) | ~2.2 GB | ~4–5 GB | high-RAM phone / laptop | Larger devices | Model card |
| Mid (Qwen 2.5 7B) | ~4.5 GB | ~8 GB | 16 GB laptop / desktop | Desktop-class | Model card |
| gpt-oss-20b | open-weight | ~16 GB | desktop / workstation | No | Model card |
| gpt-oss-120b | open-weight | ~80 GB | workstation / server | No | Model card |
Two honest caveats. First, prefer a bigger model at a lower quant over a smaller model at a higher quant when memory is tight — the extra parameters usually help more than the extra precision, as our quantization breakdown shows. Second, always leave headroom: a model that exactly fills your free RAM will stutter or get killed the moment context grows.
What about the big open models?
OpenAI released open-weight models as gpt-oss-20b and gpt-oss-120b. They are genuinely runnable locally, but not on a phone: gpt-oss-20b needs on the order of 16 GB of memory and gpt-oss-120b around 80 GB, which is desktop and workstation territory. Do not let a headline about "open models on your device" convince you a 120B model will fit in a pocket. On-device phone AI lives in the 1B to 3B range; the giant models are for machines with a lot of RAM. If you want to browse what actually fits well, our best local LLM models roundup ranks them by size and speed.
How to actually pick a model for your RAM
Work backwards from memory, not from the model you wish you could run. Start with your device's real free RAM, subtract a buffer for the OS and other apps, then choose the largest model whose weights plus KV-cache headroom fit inside what is left. On a phone that almost always lands you on a 1B to 3B model at Q4; on a 16 GB laptop it opens up 7B; on a 32 GB or larger desktop you can reach the heavier open-weight models. The memory budget is the constraint, and the model follows from it.
How PocketLLM handles this for you
You should not have to do this arithmetic by hand every time. PocketLLM is built to detect your device's memory and offer a model and quant sized to fit, then run inference fully on-device through its CoreML and llama.cpp backends, with no account and zero telemetry on your conversations. Instead of downloading a 4 GB file and discovering it will not load, you get a model that already fits your phone's real, headroom-adjusted budget.
Want the right-sized model chosen for you and run privately on iPhone? PocketLLM handles the memory math on-device with zero telemetry. Join the launch list.
Frequently asked questions
How much RAM do I need for a local LLM?
There is no single number, but a useful starting rule is that a model needs roughly 0.6 GB of weights per 1 billion parameters at Q4 quantization, plus meaningful headroom on top. So a 3B model has about 2 GB of weights but wants closer to 3 to 4 GB of free RAM once you add the KV cache, the runtime, and the operating system's own needs. When people ask how much RAM for a local LLM, the honest answer is: take the weight size, then add 50 to 100 percent for headroom, and make sure the total fits with room to spare. On a phone you also have to respect the memory cap iOS places on any single app.
Does a model's file size equal the RAM it needs?
No, and this is the most common mistake. The file on disk holds only the weights. When the model runs it also allocates a KV cache that grows with your context length, plus compute buffers for the runtime and the memory the operating system already uses. A 2 GB model file can therefore need noticeably more than 2 GB of RAM in practice, especially at long context. Always budget for the weights plus headroom, not just the download size.
Can I run a local LLM on a phone with 6 GB or 8 GB of RAM?
Yes. Recent iPhones and iPads sit in roughly the 6 to 8 GB range (Apple does not always publish exact per-model figures, so treat these as approximate), which is comfortable for models in the 1B to 3B range at Q4. A 3B model with around 2 GB of weights leaves enough room for the KV cache and the system on an 8 GB device. Larger 7B models are usually better suited to 16 GB laptops and desktops than to phones.
How much does the KV cache add to RAM as context grows?
The KV cache stores the attention keys and values for every token in the conversation, so it grows with context length and with the size of the model. At a few thousand tokens on a small model it is modest, often a few hundred megabytes, but at long context it can climb toward the size of the weights themselves. That is why a model that fits comfortably in a short chat can run out of memory in a very long one, and why headroom matters more than matching the file size exactly.
How much RAM does a 7B model or gpt-oss need?
A 7B model at Q4 has roughly 4.5 GB of weights and wants around 8 GB of free RAM, which points to a 16 GB laptop or desktop rather than a phone. The larger open-weight models are heavier still: OpenAI's gpt-oss-20b needs on the order of 16 GB of memory and gpt-oss-120b around 80 GB, putting them firmly in desktop and workstation territory. None of these are phone-friendly, so on a phone you stay in the 1B to 3B range.