Japanese telecom operator KDDI reduced its Buffmee AI app’s response latency by 38% and improved time-to-first-token by nearly 18%.

Google Cloud’s customer account reports that KDDI reduced total application response latency by 38% and improved time-to-first-token, or TTFT, by nearly 18% for Buffmee. These are separate application-performance metrics, and the report says the work brought the service to KDDI’s target response performance.

Buffmee is a consumer retrieval-augmented generation, or RAG, application built around the concept of “AI that helps you grow.” It grounds responses in more than 100 sources, including books, magazines, and web media, and supports information search, summarization, personalized learning, and exploration of hobbies.

The development team faced latency problems while grounding a large collection of proprietary content, including books and magazines. It used the Gemini Enterprise Agent Platform Evaluation Service, an LLM-as-a-Judge approach—using a large language model to evaluate outputs—and the Rule of Hundreds framework to replace labor-intensive manual testing with automated evaluation.

KDDI constructed hundreds of automated evaluation tests and a benchmark dataset to assess answer reliability across use cases, reporting a 25% improvement in groundedness scores. Those results come from Google Cloud’s customer case study; the supplied excerpt does not give absolute latency values, the groundedness scoring scale, or evidence that errors or hallucinations were eliminated.