Qwen is one of the strongest open-weights families you can run on your own hardware, and the small variants are tailor-made for phones and laptops. In our testing the Qwen 2.5 line punches above its size class, ships mostly under the permissive Apache 2.0 license, and converts cleanly to the GGUF format that on-device runtimes use. This tutorial walks through which Qwen variant to pick for your iPhone or Mac, the file sizes and RAM each needs, the setup steps, and why running Qwen locally keeps every prompt on your device. If you're cross-shopping small models in general, our best small language models roundup puts Qwen in context.
Want the short version? Jump to the summary table. Want Qwen-class models pre-packaged for on-device use? PocketLLM handles it on iPhone — join the waitlist.
On a phone or 8 GB Mac, run Qwen 2.5 3B at Q4 (~2 GB) for the best balance, or Qwen 2.5 Coder 1.5B at Q4 (~0.9 GB) when you want it tiny and fast. On a 16 GB Mac, run Qwen 2.5 7B at Q4 (~4.5 GB) — one of the best open-weights models you can run on a laptop, under Apache 2.0. Running any of these locally is fully private: prompts never leave your device.
Which Qwen variant fits your device
Qwen 2.5 Coder 1.5B — the tiny pick
The smallest Qwen worth running for real work. At Q4 it's about 0.9 GB on disk and loads in roughly 2 GB of RAM, so it runs on any phone and any Mac. It's tuned for code and completions but handles general chat fine. Choose it when you want maximum speed and minimum footprint.
Qwen 2.5 3B — the everyday pick
The sweet spot for phones and 8 GB Macs. At Q4 it's around 2 GB and needs about 4 GB of RAM, with quality strong enough for drafting, summarizing, and everyday chat. This is the variant most people should run on-device — comparable in role to Llama 3.2 3B from our best local LLM models ranking.
Qwen 2.5 7B — the quality pick
On a 16 GB Mac, this is the one to run. At Q4 it's about 4.5 GB and needs ~8 GB of RAM, and it's one of the best open-weights models that fits on a laptop, shipping under Apache 2.0. It outperforms older 8B models on most tasks. It'll load on a high-memory iPhone but runs slower — reserve it for machines with the RAM to spare.
Steps to run Qwen on-device
- Pick by RAM. Phone or 8 GB Mac → Qwen 2.5 3B (or 1.5B Coder) at Q4. 16 GB Mac → Qwen 2.5 7B at Q4.
- Grab the GGUF. Qwen variants are published as GGUF; choose the Q4 build for your size. New to quant tags? See our GGUF guide.
- Pick a local runtime. On Mac, LM Studio or Ollama load and run the GGUF offline. On iPhone, use a llama.cpp-based app.
- Confirm it's offline. Toggle Airplane Mode — Qwen should still answer, proving inference is fully local.
- Or skip it all. An app that bundles compatible models picks a fitting quant and downloads in one tap, no GGUF handling required.
The summary table
| Variant | Quant | File size | RAM to run | Best device |
|---|---|---|---|---|
| Qwen 2.5 Coder 1.5B | Q4 | ~0.9 GB | 2 GB | Any phone, any Mac |
| Qwen 2.5 3B | Q4 | ~2.0 GB | 4 GB | Phone, 8 GB Mac |
| Qwen 2.5 3B | Q8 | ~3.6 GB | 6 GB | 16 GB Mac, quality bump |
| Qwen 2.5 7B | Q4 | ~4.5 GB | 8 GB | 16 GB Mac |
| Qwen 2.5 7B | Q8 | ~8 GB | 12 GB | High-RAM Mac only |
What Qwen is good at on-device
The Qwen 2.5 family is well-rounded: solid general chat, strong on code (especially the Coder variants), good multilingual coverage, and reliable instruction-following. The 7B is genuinely one of the best things you can run on a laptop. The trade-off is the usual one — smaller variants know less of the world than the 7B — but for everyday drafting, summarizing, and coding help, the 3B is more than capable and stays fast on modest hardware.
The privacy angle: local Qwen vs hosted Qwen
Running Qwen locally is categorically private. The model and all inference live on your device, so prompts never leave it — no account, no server, no logging. That's different from any hosted Qwen chat service, which sends your text to a remote server. By downloading the open-weight model and running it offline, you get Qwen's quality without handing your prompts to a third party. A purpose-built on-device app makes that the default: PocketLLM runs compatible models entirely on-device on iPhone with zero telemetry on conversations.
Want Qwen-class quality on your iPhone, fully on-device? PocketLLM packages compatible models as one-tap downloads with zero telemetry. Join the waitlist.
Frequently asked questions
Which Qwen model can I run on an iPhone?
On an iPhone, run a small Qwen variant. Qwen 2.5 Coder 1.5B at Q4 is about 0.9 GB and fits easily, and Qwen 2.5 3B at Q4 is around 2 GB and still comfortable. The 7B is about 4.5 GB and will only run on higher-memory iPhones, more slowly. For most phones the 1.5B or 3B is the right choice. PocketLLM packages compatible Qwen-class models as one-tap downloads and runs them fully on-device.
How much RAM do I need to run Qwen locally?
It scales with model size and quant. A 1.5B Qwen at Q4 loads in about 2 GB of RAM, a 3B needs around 4 GB, and a 7B needs about 8 GB. The GGUF file size on disk is close to the RAM the model uses, so use it as your check. On a phone or 8 GB Mac, stay at 1.5B or 3B; reserve the 7B for machines with 16 GB.
Is running Qwen locally private?
Yes. Running Qwen locally means the model and all inference live on your device, so your prompts never leave the phone or Mac — no account, no server, no logging. This is different from any hosted Qwen chat service, which sends prompts to a remote server. Downloading the open-weight Qwen model and running it offline gives you its quality with full on-device privacy.
Which Qwen model is best for a Mac?
On an 8 GB Mac, run Qwen 2.5 3B at Q4 (about 2 GB) for a strong balance of speed and quality. On a 16 GB Mac, run Qwen 2.5 7B at Q4 (about 4.5 GB) — it is one of the best open-weights models you can run on a laptop and ships under Apache 2.0. Pick the 7B when you have the memory and want the best general quality; pick the 3B when you want speed and lower footprint.
Is Qwen good for running on-device?
Yes, Qwen is one of the best families for local use. The small variants are competitive with or ahead of other models their size, the 7B is a top open-weights pick for laptops, and most variants ship under Apache 2.0, which is permissive. They convert cleanly to GGUF and run well in llama.cpp-based runtimes, making them a reliable choice for private on-device AI on a phone or Mac.