We studied time to first token (TTFT) and how it scales with increasing context length for GPT and Claude models. We found a significant difference, with GPT showing a noticeable quadratic component, while Claude models remain closer to linear.
Private to this visit. Text and context are processed by AI.