A written hypothesis
Record what you are testing, which variable you are changing, and which you are holding constant — before you generate.
Loading...
Loading...
Documentation
Most generative work is already an experiment — the findings just get thrown away. Mark any collection as an experiment and Numonic keeps the hypothesis, the conditions, and the verdicts attached to the assets themselves.
When you create a collection, switch the type toggle from Standard to Experiment. Everything a normal collection does still works.
Each asset you add can carry a condition, variant label, seed, prompt, model, and free-form run parameters.
Rate runs on dimensions you define, flag common defects, and judge seed-matched pairs head to head.
An experiment is a collection extension, not a separate area of the product. Experiment collections still nest, still publish, still export to PDF, and still do everything Collections does. The experiment layer only adds structure on top.
Record what you are testing, which variable you are changing, and which you are holding constant — before you generate.
There is no fixed rubric. Define the dimensions that matter for this experiment and score every run against them.
Numonic suggests pairs that share a seed, so you compare the variable you changed rather than generation noise.
Conditions, seeds, models, parameters, and outcome tags are queryable from the main search bar alongside every other asset.
Every experiment collection carries a context record. All fields are optional — fill in what is true and leave the rest blank. It shows in the app as a banner at the top of the collection, and is readable over the API.
Designed but not yet generating. Nothing registered yet.
Actively generating and registering runs.
Finished, with a conclusion you can point at.
Stopped without a conclusion. Kept on purpose — a dead end you can prove you already walked.
Numonic supports two kinds of judgement, because they answer different questions. Scoring asks “how good is this run?”; comparison asks “which of these two is better?” — a question people answer far more consistently.
There is no built-in rubric. Type the dimensions this experiment cares about — anatomy, prompt adherence, lighting, brand fit — and score each run against them. Dimension names are normalised to lowercase with underscores so they stay comparable across runs.
Alongside scores, four common defect flags ship by default — blurry, artifacts, off_prompt, and clipped — so recurring failure modes become countable rather than remembered.
The comparison builder groups your runs by seed and proposes every pair that shares one. Comparing two runs from the same seed isolates the variable you changed instead of the randomness you did not.
Scores are never edited in place. A new evaluation supersedes an older one and both stay in the history, so you can see how your judgement of a run changed over the life of the experiment.
Run metadata is part of the main search grammar, not a separate filter panel. These fields combine with each other and with every other search term.
condition:cond2bRuns belonging to one condition of the experiment.
seed:101Every run generated from a given seed, across conditions.
run_model:flux-devRuns produced by a specific model, independent of the asset.
experiment:lighting-sweepScope a search to one experiment.
run_param:sampler:dpmpp_2mQuery any recorded run parameter by key and value, without a schema change.
outcome:hands_brokenRuns carrying a given outcome tag.
Combine them freely: condition:cond2b seed:101 outcome:text_literal returns one condition’s runs from a single seed that carried a given outcome tag. The full grammar is on the Search Syntax page.
The AI Librarian can read an experiment’s runs and record its own observations against them. Machine-assisted judgements are stored with an evaluator type of vlm_assisted, kept distinct from your own human evaluations rather than blended into them.
This matters when you later ask what the experiment concluded: you can always tell which verdicts a person actually made.