Gemma 3: Google's 'DeepSeek' Announced! 27B Model Runs on a Single GPU, Outperforming Full-Spec DeepSeek V3/o3-mini

Key Points

• Google has announced its latest AI model, 'Gemma 3,' offering a highly efficient model capable of running on a single GPU.

• The 27B model demonstrates performance exceeding DeepSeek V3, o3-mini, and Llama-405B, earning high praise second only to R1.

• It is extremely efficient, requiring up to 32 times less computational resources compared to other models of similar performance.

• It has entered the top 10 on the ChatBot Arena, exhibiting top-tier performance even as an open-source model.

• Supports a 128K context window and offers multilingual and multimodal capabilities (35+ languages, image, text, and video analysis).

• Available on platforms such as Hugging Face, Ollama, and Kaggle.

• NVIDIA contributed to the optimization, ensuring high performance across a wide range of GPUs.

• ShieldGemma 2 was also announced, featuring image safety check capabilities.

• Optimized for Google's proprietary TPUs and AMD GPUs, making it easy for developers to adjust and customize.

• Celebrating its first anniversary, the Gemma series has surpassed 100 million downloads, with over 60,000 variants created.

Main Text

The strongest model capable of running on a single GPU has arrived! Google's latest model, Gemma 3, is characterized by its open-source nature and high efficiency. The 27B model easily outperformed the 671B full-spec DeepSeek V3 model, the o3-mini model, and the Llama-405B model, showing performance second only to R1.

In terms of required computational resources, other models would need at least 10 times the resources to achieve similar performance, and in some cases, up to 32 times as much.

In the ChatBot Arena large model competition, Gemma 3 ranked in the top 10, achieving SOTA (State-of-the-art) among models with a similar parameter count, and boasting performance second only to R1 among open-source models.

Net users have expressed surprise one after another, with many noting they were particularly impressed by the fact that it runs on a single GPU and features a 128K context window.

However, while some feel a sense of déjà vu regarding this development, there are also voices questioning why there is no comparison with QwQ 32B.

Currently, Gemma 3 can be experienced directly in Google AI Studio and is also available for download on platforms such as Hugging Face, Ollama, and Kaggle.

The most powerful model that can run on a single GPU

Gemma 3 comes in four versions: 27B is the largest, with 1B, 4B, and 12B versions also available. Some are pre-trained models, while others are fine-tuned to follow general instructions.

Looking at the deployment, the main features are as follows:

Runs on a single GPU/TPU: When running on a single GPU/TPU, Gemma 3 demonstrated performance exceeding the Llama-405B, DeepSeek-V3, and o3-mini models in initial human preference evaluations on the LMArena leaderboard.

It would be a waste not to utilize such efficient model performance for deployment on edge devices. Google has also prepared dedicated tutorials and case studies, providing helpful, step-by-step guidance.

As an example, here is how a quantized version of the 1-billion parameter model runs on a smartphone. (Please refer to the original article for the video.)

The performance on a Samsung Galaxy S24 Ultra is as follows:

We also introduce an example of creating social media content locally on a PC web browser. (Please refer to the original article for the video.)

The performance on a MacBook Pro 2023 (Apple M3 Pro chip) is as follows:

Multimodal and multilingual: Capable of analyzing modalities such as images, text, and short videos.

Regarding language support, it supports over 35 languages out of the box, and also supports pre-trained models for over 140 languages.

Supports diverse tools and flexible development: Supports platforms such as Hugging Face Transformers, Ollama, JAX, Keras, PyTorch, Google AI Edge, UnSloth, vLLM, and Gemma.cpp.

128k token context window.

Furthermore, it supports function calling and structured output, assisting in task automation and agent construction. The official team has also introduced quantized versions.

Google also announced ShieldGemma 2: A 4-billion parameter image safety checker built on Gemma 3. Developers can customize ShieldGemma 2 to meet their own security needs.

Technical details

There is also a 25-page technical report released alongside the model, which contains many details.

How is it trained?

This time, the pre-training and post-training processes of Gemma were optimized by combining techniques such as distillation, reinforcement learning, and model integration.

Gemma 3 uses the same tokenizer as Gemma 2 and supports over 140 languages. Furthermore, using the JAX framework, the 1 billion parameter model was trained on 2 trillion tokens, the 4 billion parameter model on 4 trillion tokens, the 12 billion parameter model on 12 trillion tokens, and the 27 billion parameter model on 14 trillion tokens on Google TPUs.

In post-training, Gemma 3 utilized the following four elements:

• Distilling Gemma 3's pre-trained checkpoints from larger instruction models.

• RLHF (Reinforcement Learning from Human Feedback): Aligning model predictions with human preferences.

• RLMF (Reinforcement Learning from Machine Feedback): Enhancing mathematical reasoning capabilities.

• RLEF (Reinforcement Learning from Execution Feedback): Improving coding capabilities.

This approach improved performance in mathematics, coding, and instruction following, with the LMArena score reaching 1338 points.

Comparison with previous versions:

The final page of the original text lists the key authors, with a total of 16 co-first authors.

OMT: NVIDIA directly involved in optimization

The technical report also noted the following points of interest:

First, NVIDIA was directly involved in optimizing the Gemma 3 models, ensuring they achieve peak performance on all GPUs, from the Jetson Nano to the latest Blackwell chips.

Additionally, Gemma 3 is listed in the NVIDIA API catalog, allowing for rapid prototyping simply by calling the API.

Gemma 3 is also optimized for Google's proprietary TPUs and is integrated with AMD GPUs via the open-source ROCm™ stack. CPU solutions are also available.

Gemma 3 comes with an improved code library containing recipes for efficient fine-tuning and inference. Developers can customize and adjust the model according to their specific needs.

With other models thriving, the Gemma series has reached its first anniversary, surpassing 100 million downloads, and over 60,000 variants have been created by the community.

Currently, competition in the open-source model field is intensifying more and more.

Finally, for those interested, please click the links below directly.

Reference links: [1]https://blog.google/technology/developers/gemma-3/ [2]https://x.com/sundarpichai/status/1899779090472644881?s=46 [3]https://developers.googleblog.com/en/introducing-gemma3/

Original Article