shelf.cafe
top new
submit Sign in

Your LLM's time to first token might be measuring your HTTP client

0

Siddhant Khare · 1mo · discuss

An 8 KB HTTP read hid visible output for 185 ms in a controlled test. The client boundary changed the result.

Sign in
shelf.cafe - just a place to share interesting blog posts and articles.
about

We can't find the internet

Attempting to reconnect

Something went wrong!

Attempting to reconnect