Why branching saves tokens

The token-savings figure on the Tree-insights panel is worth understanding, because the intuition most people start with — “tokens get spent every turn anyway, this just trims a little” — has the mechanism backwards.

The cost of a turn is not flat

A language model is stateless. It doesn’t remember your last message — on every turn you resend the entire conversation so far as its input. So the total input across a whole chat is the sum of every growing prefix, which grows quadratically with length. The last turn of an 86-message thread, on its own, resends the 85 messages before it. Every turn pays, again, for the entire past.

That’s why a long linear chat is expensive out of proportion to its content — and why the saving is not “a little”.

Branching cuts it two compounding ways

1. Each branch is shorter. A quadratic cost on 14 messages is far below the same cost on 86. A turn deep in a branch resends only that branch’s own messages plus a short frozen summary of its parent — not the whole tree.

2. Siblings never contaminate each other. This is the bigger effect. In a linear chat, a tangent you explored around message 30 and abandoned becomes a permanent tax: turns 31 through 86 each resend those dead messages, forever. In a tree, that tangent lives in its own branch and never enters any other branch’s context.

The meter is honest

The comparison charges the branching overhead against Cymose — every turn in a branch is counted as also carrying its frozen parent summary. The number is the real token count of the tree structure versus the same content in one flat thread, not a figure tilted in the product’s favour.

Why it matters — and cost is the smaller half

Money. Input tokens are billed. Halving the context roughly halves the input cost of every turn.

Quality — the larger reason. Every model has a finite context window, and well before the hard limit its answers degrade as the window fills with irrelevant history. A discussion about your database schema shouldn’t be in context when you later ask about CSS. In a linear chat it is; in a branch it isn’t. The model sees only what’s relevant — cheaper and better answers.