LangDAG Lab

An LLM conversation is a tree, not a list. Three captured ones: a model asking for a tool, one answer continued two ways, and a routing policy whose preferred deployment stops answering halfway through.

The finding. A chat transcript is usually drawn as a line, and the moment you want to try a second answer to the same question that shape stops working. Storing the conversation as a tree instead makes branching free — both lines keep the whole history above the point they left from — and it makes two other things visible that a flat list hides: a tool call is a node with a reply node beneath it, and the deployment that served each answer is a property of the node rather than of the conversation.

Nothing on this page touches a chain.

Captured trees, not a model answering

No model is called when you open this page. No key is read, nothing is fetched and nothing is generated. Every answer below was scripted by hand in the capture harness — you can read all three conversations in one file of its source — and recorded by driving the real library against mock providers.

What makes the trees worth showing is that they are not drawings of a data structure. The harness runs langdag itself, with an in-memory store and no database file, and commits whatever it produces. The node identifiers, the sequence numbers, the parent links, the stop reasons and the token counters below are the library's own.

The harness is pigfox/langdag-lab, the fixtures live at commit 209710f, and they were captured against langdag v0.10.0.

That commit is checkable rather than merely cited. The harness has a verify step that re-runs every capture and compares it byte for byte with what is committed, which is what turns a JSON file in a repository from something anyone can edit into a claim anyone can test.

What was normalized, and what was not

The library stamps a fresh identifier on every node, another on every output group, a timestamp, and a measured latency. None of those describe the conversation and all of them change between two runs of the same code, so the harness projects them away: identifiers become ordinals in walk order, and timestamps and latencies are zeroed.

Nothing else was touched. The text, the sequence numbers, the parent links, the stop reasons, the token counts and the model resolution are exactly as the library wrote them. The risk in a normalization like that is doing too much rather than too little — one that flattened enough would make any two conversations compare equal while still reporting success — so the harness tests every rule in both directions: a re-run's different identifiers must not read as a difference, and a moved node, a renamed deployment or an edited word must.

Tool round trip

The model asks for a tool, the caller answers, the model uses the answer. langdag stores the request as content blocks and indexes the tool id; running the tool is the caller's job, which is why this is four nodes and the second user turn holds a tool result rather than typed text.

Nodes: 4
Turns deep: 3
Served by: anthropic
Tokens in / out: 910 / 91

n0 user

What is the current UTC offset for Tokyo?
n1 assistant anthropic
tool call — lookup_timezone toolu_tokyo_1
```
{
  "city": "Tokyo"
}
```
claude-haiku-4-5 tool_use 412 in / 57 out
n2 user
tool result — answers toolu_tokyo_1
```
Asia/Tokyo, UTC+09:00, no daylight saving
```
n3 assistant anthropic

Tokyo is on Asia/Tokyo, which is UTC+09:00 all year — Japan does not observe daylight saving, so the offset never shifts.

claude-haiku-4-5 end_turn 498 in / 34 out

Two continuations of one node

One assistant answer, continued twice with the same follow-up and answered differently each time. Both continuations keep the whole path above them, so the two lines of reasoning share a history and diverge only below the node they left from.

Nodes: 6
Turns deep: 3
Served by: anthropic
Tokens in / out: 398 / 132

n0 user

I need to rate limit an HTTP API. What shape should it be?
n1 assistant anthropic 2 continuations

Two shapes fit: a token bucket, which allows a burst up to its capacity and refills at a steady rate, or a leaky bucket, which drains at a fixed rate and shapes the burst away entirely.

claude-haiku-4-5 end_turn 96 in / 41 out
n2 user

Go with the token bucket. What should I watch out for?
n3 assistant anthropic

Token bucket. Size the capacity to the burst you are willing to absorb and the refill rate to the sustained limit; a client that has been quiet can spend its savings immediately, which is usually what an API caller expects.

claude-haiku-4-5 end_turn 151 in / 48 out
n4 user

Go with the token bucket. What should I watch out for?
n5 assistant anthropic

Token bucket, but keep one bucket per client rather than one for the process, and evict an idle bucket on a timer. A single shared bucket makes one noisy caller everyone else's rate limit.

claude-haiku-4-5 end_turn 151 in / 43 out

Weighted policy, preferred route declines

A policy that sends most traffic to one deployment and keeps a second in reserve. The preferred route answers the first turn, then declines the second, and the standby answers it. Each answer carries the name of the deployment that served it; the declined attempt is in the attempt log rather than the tree, because a call that returned nothing produces no node.

Nodes: 4
Turns deep: 3
Served by: anthropic, backup
Tokens in / out: 232 / 69

The declared policy

Weights are a share of traffic, not an order of preference — the router draws against them. The capture fixes the draw so the recorded conversation is reproducible; the harness proves the weighting is real separately, by driving every value across the range and checking the split.

anthropic 80%
backup 20%

3 attempts, 1 declined

These are not nodes, and that is the point. The library stores what a provider returned, and a refused call returns nothing — so a declined attempt produces no node and cannot appear in the tree. A page showing only the tree could tell you which deployment answered and would have no way to show that another one was asked first.

anthropic served
anthropic declined 529 overloaded: upstream declined the request
backup served

n0 user

Where should I start optimizing a service that is slow under load?
n1 assistant anthropic

Start with the read path. A cache in front of the slowest query buys the most headroom for the least risk, because a stale read is recoverable and a stale write is not.

claude-haiku-4-5 end_turn 88 in / 36 out
n2 user

And the write path?
n3 assistant backup

For the write path, batch on the client and make the handler idempotent before you touch the database. Batching without idempotency turns one retry into a duplicate row.

claude-haiku-4-5 end_turn 144 in / 33 out

Branch one of them live

Everything above is captured. This one thing is not. Pick a node from any of the three trees, write a follow-up, and it goes to claude-haiku-4-5-20251001 as a single call. What comes back is appended below the node you chose, tagged as live, and it is gone when you reload — nothing you write is stored and no captured tree is changed by it.

To be exact about what is and is not langdag here: the three trees were built by the library in the capture harness, and this one call is a direct request to the Anthropic Messages API from this server. The answer is put into a node of langdag's own shape so it sits in the tree correctly, but no library built it and nothing persisted it.

3 branches per visitor per day, answers capped at 400 tokens, and a ceiling on what the whole site spends in a day. When either is reached the page says so and everything above keeps working, because none of it needs a model.

Reading the trees

Every node above is in the document when the page loads, indented by how far below the root it sits. The script draws the connecting lines and adds a control that steps through a tree one turn at a time; without it you get the same nodes in the same order, which is why the capture can be the only source for this page.

Across the three captures there are 14 nodes in 3 trees.