LLM inference begins with prefill, which processes the prompt and builds a key and value cache. Decode then generates new tokens using that cached history. Grouped query attention, used in models such as Llama 3.1 and Qwen3, reduces the cache by letting several query heads share a KV head. Reading that cache is still a recurring cost at every attention layer and generation step. The Colfax post on S/P ping pong describes a scheduling improvement for FA4 decode on Blackwell. In the original schedule, QKT for the next KV block waits for softmax on the current block, despite having no...