NVIDIA Dynamo’s Shadow Engine Recovery restores large-language-model inference capacity within seconds following an engine-process failure.

NVIDIA introduced Shadow Engine Recovery for NVIDIA Dynamo, its software framework for large-language-model inference serving. The feature is designed to restore serving capacity within seconds after an inference-engine process fails.

The conventional recovery path uses a cold restart. NVIDIA noted that this can require model weights to be loaded from storage into high-bandwidth memory and kernels to be compiled before the engine resumes operation.

Shadow Engine Recovery addresses the recovery of an engine process rather than a failed GPU, server rack, data center or power system. The supplied source does not claim that it repairs failed hardware or restores an entire facility.

No universal recovery-time benchmark, model size, cluster configuration or hardware platform was included in the supplied description. The stated timing is therefore NVIDIA’s general characterization of recovery occurring “within seconds.”