Researchers introduced BOOST, an inference runtime providing concurrent, proportional access to host memory and GPU high-bandwidth memory (HBM).

Researchers at Georgia Tech, NVIDIA Research, and Stanford University published a technical paper titled “BOOST: Concurrent Access to Host Memory and HBM to Accelerate LLM Inference.” Semiconductor Engineering reports that BOOST is a runtime system targeting large language model (LLM) inference.

The supplied abstract describes concurrent and proportional access to host memory and GPU high-bandwidth memory (HBM), seeking to extract their combined bandwidth. The contribution concerns access to two memory tiers, rather than the introduction of a new memory chip or additional GPU hardware.

The abstract calls BOOST the first runtime to provide this capability; that priority claim belongs to the paper’s authors. Although the article headline reports improved inference throughput, the supplied excerpt does not give speedup figures, model sizes, GPU configurations, host-memory bandwidth, or benchmark conditions.