A study tells you where you stand. Monitoring tells you whether what you did worked. Same frozen bank, same models, comparable numbers over time.
One instrument, run again
The study ends with a frozen prompt bank. Monitoring re-runs that same bank: same questions, same models, same codebook, read the same way. That is what makes two runs comparable. Without the frozen bank, a second measurement is a new anecdote, not a trend.
Between the two runs sits the work.
- Study, ending in a frozen bank
- The work on the sources
- Monitoring re-runs the same bank
The order is study, then the work on the sources the report points at, then monitoring: the re-run measures the effect of that work. A second measurement with nothing in between still compares — it just has nothing to attribute the movement to but time.
Recommended
Invented sampleSame frozen bank, same models. The difference does not prove causation.
44 of 396 responses in the first run, 75 of 396 in the second, after the work. The same 396 responses stand behind both numbers, so the difference is readable. Change the bank and the two numbers stop meaning anything together.
What breaks comparability:
Editing questions after the baseline
New questions start their own baseline, with their own first run. They never join the existing line.
Changing models or repetitions mid-trend
The instrument changed, so the numbers changed for reasons that are not yours.
Splicing banks into a trend
Never. We document the bank so any later edit is deliberate, not accidental.
Cadence, agreed per client
No fixed package. The cadence depends on how fast your category moves and when your decisions happen: budget cycles, campaign windows, retailer negotiations.
Two rhythms combine:
Periodic runs
- With the frozen bank, to keep the line moving between decisions.
Deeper re-runs
- At longer intervals, more models or more repetitions, for the numbers that support bigger decisions.
Alerts run between and across runs:
Drop
Your visibility drops
Against the previous comparable run.
Pattern
A specific critique repeats across assistants
One negative answer is noise; the same critique in several answers is a pattern with a source attached.
Rival set
A competitor enters or leaves the answer set
For questions that used to be stable.
Every alert names the question, the model and the responses behind it. Alerts are measurement with a citation, not a newsfeed.
What monitoring does not tell you
Kept visible on purpose:
It does not prove causation
A model update or new third-party content can move the same number. The comparison tells you what changed. It does not prove you caused it.
If nothing moved, the report says nothing moved
No chart gets stretched to show progress.
Third parties still write most of what gets cited
Part of the movement, up and down, sits outside anyone's control.
What the trend panels show
One panel per outcome rate (Mentioned, Recommended, Named as the best, With a purchase path). In each panel:
Points
Two points, one per run, connected by a straight line
Labels "run 1" and "run 2" with the run date.
Denominator
The rate above each point, the denominator printed under each point
"n = 396 responses". Denominators are part of the visual, not footnotes.
Delta
The delta labeled between the two points
In percentage points, with a plus or minus sign. Zero movement is drawn as zero.
Scale
A fixed scale per panel
So panel heights are not stretched to exaggerate change.
Caption
A fixed caption under every panel
"Same frozen bank, same models. The difference does not prove causation."
The panel above this section is that specification, drawn.
The honest annotation is the caption. It ships with every panel, including the ones where nothing moved.
Start with the baseline, or continue it
No baseline yet
- The study comes first: monitoring without a frozen bank has nothing to compare.
You already have one
- Ours or from another provider, monitoring starts with a conversation about the decision the numbers need to inform: is what we are doing working, and how do we know?