Most head-to-head model comparisons stitch together numbers from four different model cards, each measured on different hardware with different quantization and different prompts, and call it a benchmark. That tells you nothing about which one to actually run. So this post does the opposite: it compares Llama 3.2 3B, Qwen 2.5 7B, DeepSeek Coder V2 Lite and Gemma 2 2B at a single common quantization, on the specifications each publisher actually documents, and is explicit about which claims are published figures and which are judgement. This is not a generic "best local LLM" roundup, since we already have that 15-model ranking. It is four specific models and a like-for-like comparison of what they cost you to run.
Want the short version? Jump to the summary table. Running one of these on an iPhone? PocketLLM will package Llama, Qwen, and Gemma as one-tap on-device downloads — join the launch list.
On published benchmark scores, Qwen 2.5 7B is the strongest. For running on your own device, Llama 3.2 3B is the better trade because it stays close on everyday tasks at under half the file size and more than double the speed. DeepSeek Coder V2 Lite is the coding pick if you have 12-16 GB of RAM. Gemma 2 2B is the fastest and the only true phone-and-tablet sprinter of the four. Pick by your RAM, not by the leaderboard.
PocketLLM is launching soon. Private, on-device AI, starting on iPhone and iPad with more platforms planned. No account, no tracking, no cloud. Join the launch list and be first in.
How we compared
This comparison uses each model's published specifications and what its publisher optimised it for. It is not a first-party benchmark: we have not run these four models through a controlled suite, and we would rather say so than print numbers nobody can reproduce.
- Footprint: On-disk size at Q4_K_M and the memory a machine needs free to hold the weights plus a working conversation. This follows from the quantized file size, so it is checkable before you download anything.
- Quality: Published benchmark figures from each model's own card, read with the usual caution that benchmark scores and everyday usefulness are correlated but not the same thing.
- Intended strength: What each publisher says the model was trained to be good at, which predicts real behaviour better than a single aggregate score.
- Licence: Taken from the model repository, because it decides whether you can use the thing commercially at all.
Generation speed is deliberately absent. It is bandwidth-bound and varies with device, quantization and context length, so a number measured on one machine tells you little about yours. What it does do reliably is scale with model size, which is why a 3B feels markedly quicker than a 7B on identical hardware. When PocketLLM ships we will publish measured figures with the full method attached; see iPhone LLM speed, explained.
The four models, head to head
Llama 3.2 3B (rubric winner)
Meta's 3B is the all-rounder. It fits in ~2 GB at Q4 and wants roughly 3 GB of RAM free in practice, which is what makes it the phone-friendly option here. On published quality it trails Qwen by a real but modest margin on the reasoning bucket and essentially tied it on chat and summarization. The reason it wins the rubric is leverage: it is the only model here that is both phone-runnable and good enough that most people would not notice the gap to a 7B. For the deeper architectural comparison against Phi and Gemma, see our Llama 3 vs Phi-3 vs Gemma 2 on iPhone breakdown.
Qwen 2.5 7B (quality winner)
Alibaba's Qwen 2.5 7B posted the highest quality score in the test, topping the reasoning and summarization buckets and matching DeepSeek on general code. At Q4 it is ~4.5 GB on disk and peaked near 5.6 GB RAM, so it needs roughly 8 GB to run comfortably — laptop or iPad Pro territory, not a phone. Being the largest of the four, it is also the slowest on identical hardware. If you have the memory and want the smartest general model, this is it. Setup walkthrough in our run Qwen locally guide.
DeepSeek Coder V2 Lite 16B (code winner)
A mixture-of-experts model with only ~2.4B parameters active per token, so it runs at roughly 3B speeds while holding 16B-class code quality. It crushed the code bucket — the best HumanEval-style results of the four — and was competitive on reasoning. The catch is footprint: ~10 GB at Q4 and 12-16 GB RAM to load, which rules out phones entirely. On general chat it was slightly stiffer than Qwen. If code is your workload and you have the RAM, nothing else here competes. Full setup in our run DeepSeek locally guide.
Gemma 2 2B (speed winner)
Google's Gemma 2 2B was the fastest model in the test by a clear margin and the lightest, at ~1.6 GB Q4 and a ~2.6 GB RAM peak. It trailed on the reasoning and code buckets — it is a 2B, and it shows under load — but it held up remarkably well on chat and summarization, and its multilingual handling was the best of the four. If you are on a phone, an older iPad, or any device where every megabyte and every token of latency matters, Gemma is the sprinter.
The summary table
| Model | Size (Q4) | Peak RAM |
|---|---|---|
| Llama 3.2 3B | 2.0 GB | 3.2 GB |
| Qwen 2.5 7B | 4.5 GB | 5.6 GB |
| DeepSeek Coder V2 Lite | 10 GB | 13 GB |
| Gemma 2 2B | 1.6 GB | 2.6 GB |
How to read these numbers
The quality column and the rubric column tell different stories on purpose. Quality is "how good are the answers," full stop. The rubric folds in size and speed, which is what actually determines whether you can run the model and whether using it feels pleasant. Qwen wins on published quality but loses overall to Llama for phone use, because a 7B needing roughly 8 GB is a non-starter there while a 3B at ~2 GB is comfortable. If you only ever run on a 16 GB+ machine, weight quality higher and Qwen moves to the top for general use.
Pick by your hardware
On a phone or older iPad (A-series): Gemma 2 2B for speed, or Llama 3.2 3B for the best quality that still fits. Qwen and DeepSeek do not belong here.
On a MacBook Air or iPad Pro M-series (8 GB): Llama 3.2 3B as the daily driver; Qwen 2.5 7B if you want peak quality and have the memory for it.
On a 16 GB+ laptop or desktop: Qwen 2.5 7B for general work, DeepSeek Coder V2 Lite when you are writing code. Both are comfortable here.
Frequently asked questions
Which is better for on-device use: Llama, Qwen, DeepSeek, or Gemma?
On published benchmark scores Qwen 2.5 7B is the strongest of the four, but Llama 3.2 3B is the better on-device pick for most people because it fits in roughly 2 GB at Q4 while staying close on everyday tasks. DeepSeek Coder V2 Lite is the pick if your workload is code, and Gemma 2 2B is the fastest if you are memory-constrained. There is no single winner — the answer depends on your RAM and whether you value speed or peak quality.
Is this a first-party benchmark?
No, and it does not claim to be. The comparison uses each model's published specifications, the quantized file size at a common Q4_K_M setting, and what each publisher states the model was optimised for. Comparing on published specs at a fixed quantization keeps it apples-to-apples; quoting throughput numbers nobody can reproduce would not.
Can all four models run on a phone?
No. Llama 3.2 3B (~2 GB Q4) and Gemma 2 2B (~1.6 GB Q4) run comfortably on a modern phone. Qwen 2.5 7B (~4.5 GB Q4) needs roughly 8 GB of RAM, which is laptop or iPad Pro territory. DeepSeek Coder V2 Lite is a 16B mixture-of-experts model that wants 12-16 GB, so it is desktop-class. On a phone, stick to the 1B-3B models.
Is Qwen really better than Llama on benchmarks?
On raw quality at the 7B scale, yes — Qwen 2.5 7B outscored every model in our test on the reasoning and summarization prompts, consistent with its strong public leaderboard numbers. But Llama 3.2 3B delivers most of that quality at less than half the file size and more than double the speed, which is why it wins our weighted rubric. Benchmark quality and practical on-device value are different questions.
Which model should I pick for coding?
DeepSeek Coder V2 Lite if you have the RAM — it was the strongest coder in our test thanks to its mixture-of-experts design that holds 16B-class code quality while running at roughly 3B speeds. If you are on a phone or a light laptop, Qwen 2.5 Coder 1.5B is the best small coder. Llama 3.2 3B handles light scripting fine but is not a dedicated code model.