0
siddhantkhare.com · 1d · discuss
An LLM serving simulator drained all four GPU replicas at its first autoscaler tick, before any request arrived. 0 of 200 requests completed. The repro, the root cause, the fix, and how real autoscalers set a floor.