中文
AI Tools / llm-chat

Gemma

Reviewed by AIX Cove ·
Gemma
llm-chat

Gemma at a glance

Gemma is Google DeepMind's family of open-weight models. It is the download-and-run counterpart to the hosted Gemini apps and API. You get the weights, and you decide where they run, on a phone, a laptop, a server, or a cloud VM. The family has been through several generations, and the current one is Gemma 4, released in April 2026, with the 12B unified model following in June.

I checked the official Gemma site, the Hugging Face model cards, and the terms page before writing this. The practical question for most readers is which size fits their hardware and whether their use case is allowed under the license. Both have real answers, and they change between generations, so the current docs are the source of truth.

The Gemma 4 lineup

Gemma 4 ships in five sizes, spanning two architectures. E2B and E4B are compact models built for maximum compute and memory efficiency, aimed at phones and IoT devices. The 12B, 26B A4B, and 31B are the reasoning-focused tier for laptops and servers. The 26B A4B is a Mixture-of-Experts model, which means only a fraction of its parameters activate per token, so it runs in far less memory than a dense 26B would need.

All five are multimodal. They take text and image input, and the E2B, E4B, and 12B additionally handle video and audio natively. Context windows run to 128K tokens on the small models and 256K on the medium ones. Multilingual support covers more than 140 languages. The models have configurable thinking modes, so you can trade latency against reasoning depth, and native function calling for agent workflows. The 12B is described as encoder-free, meaning it processes audio and vision without separate encoder modules, which keeps the deployment footprint smaller.

Beyond the core sizes, Google publishes specialist variants. DiffusionGemma is a text diffusion model aimed at generation speed. T5Gemma and T5Gemma 2 are encoder-decoder models for deep contextual understanding. MedGemma targets medical text and image comprehension. EmbeddingGemma, FunctionGemma, VaultGemma, TranslateGemma, and ShieldGemma 2 cover embeddings, edge function calling, differentially private LLMs, translation, and safety classification respectively.

How people actually run it

Gemma runs through most mainstream tooling. Ollama and LM Studio cover the quick local path, Gemma.cpp and Google AI Edge handle on-device deployment, and Hugging Face Transformers, Keras, and JAX cover training and fine-tuning. The Hugging Face downloads are gated, so you agree to Google's license terms before accessing the weights.

The community reception has been strongly oriented toward local deployment. The Gemma 4 release thread on Hacker News drew over 1,800 points, and follow-ups showed people running the 26B model in 2 GB of RAM on M-series Macs, running Gemma 4 on iPhones, and even getting 26B to 5 tokens per second on a 13-year-old Xeon with no GPU. Those are the enthusiast edge cases, not typical results, but they explain what the small sizes are for.

Typical workflow

A common starting point is Ollama. Install it, pull a Gemma model by name, and chat with it from the terminal. On a recent laptop, the 4B or 12B size is the sane default. For a phone or embedded target, the E2B or E4B models are the ones to evaluate. For a server with a serious GPU, the 31B or 26B A4B becomes viable.

When a plain chat model is not enough, the next step is fine-tuning. The official guides cover QLoRA tuning with Hugging Face Transformers, tuning with the Gemma library, and full fine-tuning. A small tuned model usually beats prompt-engineering a large base model for a narrow internal task, and it costs less to serve.

Where it fits, and where it does not

Gemma fits teams that want model weights under their own control, that need offline or on-device inference, or that want to fine-tune on proprietary data without sending it to a hosted API. It also fits researchers comparing compact models and builders who want Google's model lineage without being tied to Gemini's hosted product.

It is the wrong choice if you want a managed assistant with no infrastructure work. For that, Gemini is the hosted path. Gemma also is not a drop-in guarantee of quality at every size, the small models trade capability for efficiency, and any self-hosted deployment still carries hardware, evaluation, safety, and maintenance costs that the download page does not show.

Practical tips

  • Pick the size by hardware, not by benchmark. A 31B model on a machine it does not fit will be slower than a 12B that fits comfortably. Check memory requirements against your actual device before downloading.
  • Use the quantized and QAT variants for tight memory. Google publishes quantization guides and Gemma 4 QAT models for mobile and laptop efficiency. A quantized model is usually the difference between fitting and not fitting on a consumer machine.
  • Match context length to the job. The small models cap at 128K and the medium ones at 256K. Long-document work should use the medium tier, while a 128K window covers most chat and RAG workloads.
  • Toggle thinking modes deliberately. Configurable thinking trades latency for reasoning depth. Turn it on for hard reasoning and analysis, off for high-volume generation where speed matters more.
  • Test function calling before building an agent on it. Native function calling exists across the family, but tool-calling reliability varies by size and quantization. Run your actual tool schema through the model before committing.
  • Check the license for your use case. Gemma 4 weights are distributed under the Gemma 4 license, and earlier generations use the Gemma Terms of Use. Distribution and hosted-service obligations differ between them, so read the current terms rather than assuming Apache-style freedom.

Limits and risks

  • Open weight is not open everything. Gemma is distributed under Google's own terms, not a standard permissive license, and the terms have changed between generations. Review them before commercial distribution or hosted offerings.
  • Hardware planning is real work. The largest models need serious GPUs or heavy quantization. The enthusiast posts about running 26B in 2 GB of RAM are engineering feats, not typical setups.
  • Small models have small ceilings. The E2B and E4B sizes are efficiency-first. They are impressive for their footprint, but they are not substitutes for the large models on complex reasoning.
  • Quality claims move fast. Model generations and benchmark stories change every few months. Decisions should be based on the current model card and your own evaluation set, not on launch-day reporting.
  • Not a hosted service. There is no official managed Gemma API in the same sense as Gemini. Running it well, serving it, and keeping it patched is your responsibility.

Related tools on AIXCove

  • Gemini if you want the hosted Google assistant and API instead of self-hosting weights
  • Qwen for another open-weight family with a wide size range and strong coding focus
  • AnythingLLM if you want a ready-made private document chat workspace around local models
  • Browse all AI chat tools, or read our Open WebUI and Ollama guide for a local chat setup

Sources and further reading

Sources: official docs & pricing pages, hands-on testing where noted, and community feedback. Prices verified August 2026 and may change.