A branched study

Some questions are answered by experiment rather than by reading. This example is a real recorded session in which the CLIO Science agent, asked a plain question, ran a small study: it proposed competing answers, tested them side by side and reported which ones the evidence can and cannot separate. You can open the whole session below and follow every claim to the games behind it.

The question

The session started from this request, with no tools, steps or hypotheses named:

What is the best approach to solving Wordle? Use the word list from the wordle
tools, not the official one. Play every game yourself instead of writing a solver.

The agent used a Wordle plugin that serves games from its own synthetic word list and checks each guess. The plugin’s answers stay on the server, so the agent could learn about the list only by playing.

How the study ran

1. The agent wrote the question in the Science log and proposed three competing strategies as hypotheses: an opener chosen for this list’s common letters, the same opener followed by a fixed second word, and an English-style opener. Each hypothesis states the result that would show it wrong.
2. It started a workflow with one branch per hypothesis, all on the same twenty seeds, and a merge step after them.
3. Each branch handed its twenty games to four helper agents, five games each, so twelve players worked at once and every guess was still chosen by a model.
4. The merge compared the strategies seed by seed in its own analysis script and wrote the conclusion, with two recorded conflicts about what the sample cannot show.

Graph: the whole study, from the question down to the conclusion. Each experiment's observations are drawn as one group. Click to enlarge.

What it found

All three strategies solved all sixty games, in 3 to 5 guesses. Their means were 3.75 guesses for the English-style opener and 3.65 for each of the other two. The paired comparisons over the shared seeds all had intervals that include zero, so the study concluded that twenty seeds cannot separate the strategies. It estimated that about 150 paired seeds would be needed to detect a difference of a quarter of a guess. A reference solver run separately on the same seeds averaged 3.65, in line with the study.

Views

A study’s views are dashboards its agent builds and attaches to the study. They read the study’s measurements and records live, so they fill in as branches log games. Nothing about them is fixed by the plugin: in a later run of the same request the agent built a seed-by-seed comparison of its three strategies over the study’s measurements, looked at its own chart and revised it. Asked for a way to look at the games themselves, it added a second view with a seed picker that replays one game’s boards. The Views mode shows one view at a time, switched by title. The session you can open below ran before views existed, so the captures here come from that later run.

A view the agent designed: guesses per game for each strategy on the same ten seeds, read live from the study's measurements. Click to enlarge.

The second view, added when asked for a look at the games: pick a seed to replay its boards. The agent's own chart has flaws it did not catch: the heading reads undefined, and the three strategies' boards are drawn on top of each other. Click to enlarge.

Check a claim

Every claim in the conclusion cites the entries it rests on. Each game is an observation that points to the recorded calls that played it, so you can follow a number from the conclusion to the evidence, to the observation, and to the guesses in the transcript.

Select a group or an entry to read its records: here one experiment's games, each with its measurements, the evidence that cites it and its sources. Click to enlarge.

Open the session

Open the Wordle studyThe recorded session: the conversation, the workflow and the Science log with every branch. 48 MB.

The page is an interactive archive of the actual session. Opening it does not run the agent or contact any service. The session ran with tool approvals turned off (Bypass checks), so the agent did not stop to ask before each game.

Model choices vary from run to run. In other runs of the same request the agent played every game itself or enlarged the sample after a first merge, so treat this session as one example of how a study can unfold, not as a fixed recipe.