Module 7 of 11 · Expert · 12 min read

Graphs: Graph of Thoughts and agent workflow graphs

Branch, merge and loop with shared state, from Graph of Thoughts to stateful agent graphs you can build and debug.

Key takeaways

1Graph of Thoughts (Besta et al., 2023) lets thoughts be combined and refined, not just branched.
2Workflow graphs model a system as nodes, edges (including conditional ones) and shared state.
3Parallel branches need an explicit merge step; loops need explicit exits.
4Keep state small and typed, log every node, and checkpoint so runs can resume.

01Why graphs

Trees branch but never recombine. Real work often does: three researchers investigate different questions and their notes are merged; two drafts are combined into one that keeps the best of each; a reviewer sends a draft back for another pass. Graph of Thoughts (Besta et al., 2023) models reasoning as an arbitrary graph, adding operations such as aggregating several thoughts into one and refining a thought in a loop. The paper reported better quality at lower cost than Tree of Thoughts on tasks such as sorting, where splitting, solving and merging fits the problem.

02Graph of Thoughts operations

Divide-and-conquer tasks are the natural fit: split into parts, solve each, merge the partial results.

Generate: create one or more new thoughts from an existing one (branching).
Aggregate: merge several thoughts into a single, better one, the step trees lack.
Refine: improve a thought by looping on it.
Score and keep: evaluate thoughts and keep the best for further work.

Example

Merge prompt: Here are three summaries of the same 60-page report, each covering a different third: <summary_a>, <summary_b>, <summary_c>. Combine them into one summary that keeps every distinct key finding, removes duplicates, resolves contradictions by checking the quotes provided, and flags any disagreement the quotes can't settle.

03Agent workflow graphs

The same idea underpins how many production agent systems are built. Frameworks such as LangGraph, and many visual workflow tools, describe an application as a graph: nodes are steps (an LLM call, a tool, code or a person), edges say what runs next, conditional edges route on results, and cycles create loops. A shared, typed state object passes through the graph and each node updates it. Checkpointing saves that state so runs can pause for human input and resume.

State: a small schema, such as {question, plan, notes[], draft, critique, iteration}.
Nodes: plan, research, merge, draft, critique, revise, human_review, finish.
Edges: plan → research (fanned out in parallel) → merge → draft → critique.
Conditional edge: critique → finish if passed; → revise if failed and iteration < 3; → human_review otherwise.
Interrupt: pause before publishing for human approval.

04Worked example: a research report graph

Each node gets a small, focused prompt that reads from and writes to state.

Example

plan: Given state.question, write 3 independent research questions that together answer it. Output JSON: {"questions": [...]}. research (once per question, in parallel): Answer state.questions[i] using search. Return notes with source URLs and quotes. merge: Combine all notes into one deduplicated list grouped by theme; mark conflicts. draft: Write the report from the merged notes only; cite sources. critique: Check the draft against the rubric; return {"passed": true or false, "issues": [...]}. router: passed → finish; not passed and iteration < 3 → revise; otherwise → human_review.

05Design rules for graphs

A graph you can't trace is a graph you can't debug. Observability is part of the design, not an afterthought.

Give each node one job and a clear input and output contract.
Keep state lean: store references and summaries, not every intermediate token.
Make merges explicit: decide how conflicts between branches are resolved.
Bound every cycle with an iteration counter in state.
Log each node's input, output and timing so you can trace failures.
Put human-approval interrupts before external or irreversible actions.

06Pitfalls

Start with the smallest graph that works, measure, and add nodes only where tests show a gap.

Over-engineering: a twelve-node graph for a task one prompt could do.
State bloat: passing everything everywhere until context windows overflow.
Ambiguous ownership: two nodes that both 'fix' the same field and undo each other.
Silent merges: a merge step that quietly drops a branch's findings. Ask it to report what it discarded.