On Google Cloud TPU v6e accelerators, Gemma 3 27B generation plateaued beyond 64 concurrent users; Gemma 3 12B scaled further.

Google benchmarked Gemma 3 12B and Gemma 3 27B on a Google Kubernetes Engine Autopilot cluster connected to a single-host Tensor Processing Unit v6e node pool. The TPU configuration used a 2-by-2 chip topology, with vLLM from the vllm-project/tpu-inference stack serving the models.

For decode-heavy text generation, Gemma 3 27B reached a scaling limit after 64 concurrent users. At 128 users, its normalized throughput plateaued at 4.12 times the baseline, while Gemma 3 12B reached an 8.19-times normalized throughput multiplier.

For prefill-heavy classification workloads, the two models produced more similar scaling. At 128 concurrent users, peak normalized throughput was approximately 6.0 to 6.4 times baseline for both models, without saturating the TPUs in that test.

The tests covered 16, 32, 64 and 128 concurrent users. Google reported that saturation could appear as latency spikes and silent request drops, and recommended using end-to-end latency rather than standard CPU or memory metrics for scaling decisions; these recommendations apply to the described benchmark configuration.