How to Run Gemma Locally on iPhone & Mac (2026)

Gemma is Google's family of open-weight models, and its smaller members are a genuinely good fit for phones and laptops. Because the weights are openly published, you can download a compact Gemma, grab it in the GGUF format that on-device runtimes read, and run it entirely on your own hardware. This tutorial walks through how to run Gemma locally on an iPhone or Mac: which small variant to pick, roughly how large each one is, how much memory it wants, and the steps to get a GGUF quant running offline. If you are cross-shopping small models in general, our best local LLM models ranking puts Gemma next to the alternatives, and if the GGUF and quant jargon is new, start with our guide to GGUF models. Prefer a different family? We have companion guides to running Qwen locally and running DeepSeek locally too.

Want the short version? Jump to the summary table. Want Gemma-class models pre-packaged so you skip the GGUF handling entirely? PocketLLM is built to run compatible models on-device on iPhone. Coming soon, join the launch list.

Which Gemma variant fits your device

Google has shipped several Gemma generations (the original Gemma, Gemma 2, and a more recent Gemma 3 line), and the exact sizes on offer shift between releases. Treat the specific variant names below as examples and confirm the current lineup and license terms on the official model card before downloading. The rule that matters for on-device use is simple: match the parameter count to your memory.

A small Gemma (around 1B to 2B): the phone pick

The comfortable phone-class range is roughly 1B to 3B parameters. A recent 1B Gemma at Q4 is under about 1 GB on disk and loads on almost any iPhone, while a 2B such as Gemma 2 2B at Q4 sits around 1.5 GB and gives you a bit more quality. Both are fine for chat, drafting, and summarizing. Pick the 1B when you want maximum speed and the smallest footprint, the 2B when you have the memory to spare.

A 4B-class Gemma: the everyday pick

A roughly 4B Gemma (such as a Gemma 3 4B build) at Q4 is around 2.5 GB and is a strong everyday choice for a high-memory iPhone or any Mac. It is noticeably more capable than the 1B to 2B tier while still staying light enough to run responsively on modest hardware. This is the size most people should target on a laptop that does not have a lot of spare RAM.

A 9B or larger Gemma: the quality pick

On a 16 GB Mac, a mid-size Gemma like a 9B at Q4 (around 5 to 6 GB) is the best-quality option that still fits comfortably. Larger variants (12B and up, and the big 27B-class model) are desktop and workstation territory: a 27B at Q4 alone is on the order of 16 GB of weights before headroom, so it is not phone-friendly and wants a high-RAM machine. Match the variant to your memory rather than reaching for the biggest one.

How to run Gemma locally

1. Pick by RAM. Phone or 8 GB Mac to a 1B to 4B Gemma at Q4. 16 GB Mac to a 9B at Q4.
2. Get the GGUF. Download the open-weight Gemma in the GGUF format and choose the Q4 build for your size. New to quant tags? See our GGUF guide and the deeper quantization explainer.
3. Pick a local runtime. On Mac, LM Studio or Ollama load and run the GGUF offline; on Apple Silicon you can also use an MLX-based path (MLX is Apple's ML framework, a different category from the GGUF file format). On iPhone, use a llama.cpp-based app.
4. Confirm it is offline. Toggle Airplane Mode. Gemma should still answer, which proves inference is fully local.
5. Or skip the manual steps. An app that bundles compatible models picks a fitting quant and downloads it in one tap, no GGUF wrangling required.

The summary table

Variant (example)	Quant	Approx file size	RAM to run	Best device	Source
Gemma ~1B	Q4	~0.7-1 GB	~1.5 GB	Any phone, any Mac	Estimate
Gemma 2 2B	Q4	~1.5-1.7 GB	~3 GB	Phone, 8 GB Mac	Estimate
Gemma ~4B	Q4	~2.5 GB	~4-5 GB	High-mem phone, any Mac	Estimate
Gemma ~9B	Q4	~5-6 GB	~8 GB	16 GB Mac	Estimate
Gemma ~27B	Q4	~16 GB	~20+ GB	High-RAM desktop only	Estimate

Every size above is an estimate derived from a rough rule of thumb (roughly 0.6 GB of weights per 1B parameters at Q4), not a measured figure, so confirm the real numbers on the model card. And remember that these file sizes are not the same as the memory the model needs to run. The weights are only part of it: the KV cache grows with your context length, and the runtime and the operating system each take their share, so plan for meaningfully more free RAM than the file size suggests. On iPhone there is an extra constraint, because iOS limits how much memory a single app may use.

Local Gemma vs hosted Gemma vs PocketLLM

The same Gemma weights can run in very different places, and where inference happens is what decides your privacy. Here is how a hosted Gemma service, a self-managed local Gemma, and PocketLLM compare.

Aspect	Hosted Gemma (cloud)	Local Gemma (GGUF + llama.cpp)	PocketLLM
Where inference runs	Remote server	Your device	Your device
Account required	Usually yes	No	No
Prompts leave device	Yes	No	No
Works offline	No	Yes	Yes
Setup effort	Low (sign in)	Moderate (pick GGUF + runtime)	One-tap (planned)
Telemetry on chats	Per provider policy	None (local)	Zero

What Gemma is good at on-device

The small Gemma variants are well-rounded: capable general chat, decent instruction-following, and reasonable multilingual coverage for their size. Some larger Gemma 3 variants also add image understanding, though the tiny text-only builds are what you will run on a phone. The trade-off is the familiar one: a 1B to 4B model simply knows less than a frontier cloud model, so for the hardest reasoning you will still reach for something bigger. For everyday drafting, rewriting, and quick questions, though, an on-device Gemma is fast and more than good enough. For a head-to-head against other phone-class models, see our Llama vs Phi vs Gemma on iPhone comparison and the broader small language models roundup.

The privacy angle: local Gemma vs hosted Gemma

Running Gemma locally is private by construction. The weights and all inference live on your device, so your prompts never leave it: no account, no server, no logging. That is categorically different from a hosted Gemma chat service, which sends your text to a remote server governed by that provider's data policy. By downloading the open-weight model and running it offline, you keep Gemma's quality without handing your prompts to a third party. A purpose-built on-device app makes that the default: PocketLLM is designed to run compatible models entirely on-device on iPhone, with no account and zero telemetry on your conversations.

Want Gemma-class quality on your iPhone, fully on-device? PocketLLM is designed to package compatible models as one-tap downloads with zero telemetry. Coming soon, join the launch list.

Frequently asked questions

Which Gemma model can I run on an iPhone?

On an iPhone, stick to a small Gemma variant. A roughly 1B Gemma (such as a recent Gemma 3 1B build) at Q4 is under about 1 GB on disk and fits almost any phone, and a 2B Gemma like Gemma 2 2B at Q4 lands around 1.5 GB and stays comfortable. The comfortable phone-class range is roughly 1B to 3B parameters. Larger 9B and up variants are laptop and desktop class, not phone-friendly. Remember that the file size is not the full memory need: the runtime, the OS, and the growing KV cache all want headroom on top, and iOS caps how much memory an app can use. Verify the exact variant lineup on the model card, since Google's Gemma family updates over time.

How do I run Gemma locally on a Mac?

To run Gemma locally on a Mac, download an open-weight Gemma model in the GGUF format (choose a Q4 build to start) and load it in a llama.cpp-based runtime such as LM Studio or Ollama. That path runs entirely offline. Apple Silicon Macs can alternatively use an MLX-based path, since MLX is Apple's on-device ML framework with its own model format, while GGUF is the file format that llama.cpp-style runtimes read. Both keep inference on your machine. To confirm it is local, turn on Airplane Mode and check that Gemma still answers.

How much RAM do I need to run Gemma locally?

It scales with the variant and quantization. As a rough rule at Q4, weights take roughly 0.6 GB per 1B parameters, so a 2B Gemma is around 1.2 to 1.7 GB of weights and a 9B is around 5 to 6 GB. You then need meaningful headroom on top for the KV cache (which grows with context length), the runtime, and the OS, so a model whose file is about 2 GB wants noticeably more than 2 GB of free RAM. On a phone or an 8 GB Mac, stay in the 1B to 4B range; reserve 9B and larger for machines with 16 GB or more.

Is running Gemma locally private?

Yes. When you run Gemma locally, the model weights and all inference sit on your own device, so your prompts never leave the phone or Mac: no account, no server round-trip, no logging. That is different from any hosted Gemma chat service, which sends your text to a remote server governed by that provider's policy. Downloading the open-weight model and running it offline gives you Gemma's quality with on-device privacy by construction.

Is Gemma good for on-device use?

Gemma is a solid choice for local use. It is Google's open-weight family, the small variants are designed to be efficient, and the models are published in or convert cleanly to GGUF for llama.cpp-based runtimes, which is what most on-device apps use. The usual trade-off applies: a 1B to 4B on-device Gemma knows less and reasons less deeply than a large cloud model, but for everyday chat, drafting, and summarizing on a phone or laptop it is genuinely useful and fully private.