How we test AI tools
Directories summarize. Rating sites crowdsource. AI blogs paraphrase. Our bench feeds a tool one published brief and publishes the real result — and until a tool has been through it, its page says so. Here’s exactly how, and why you can trust it.
Six steps, every tool we run
None of the 8 live tools has first-party output published yet. Every tool page carries a RESEARCH PROFILE banner: verified pricing, policy and feature research with dated primary sources, and provisional scores. We publish this number rather than let “tested” stand in for it.
Four axes, not one star
Borrowed from the best of the rating sites — the real buying signal lives in the sub-scores, especially value.
A measured bench composite is the weighted sum above: Value 35% + Quality 30% + Ease 20% + Speed 15%, identical for every tool in a category. Scores labeled “provisional” on a page are editor estimates from pricing, policy and feature research — they are not produced by this formula, and each one is replaced by the weighted composite when its category’s test round publishes.
Our exact test inputs, published in full
No review site shows you what it actually fed the tools — which makes “we tested it” unfalsifiable. Ours is published in full, so every bench result is reproducible: paste the same brief into the same tool and check it. The brief embeds deliberate traps that separate good tools from great ones.
A finance-explainer script rendered 9:16 with default settings plus one round of obvious fixes. It exposes caption accuracy, scene-matching judgment (does the b-roll match the sentence?), avatar lip-sync on numerals, and render-queue honesty.
Scene 1 (hook, 0–8s): "If you got a 10% raise tomorrow, you'd be broke again by March. Here's why."
Scene 2 (concept, 8–25s): "It's called lifestyle creep. Every time your income rises, your spending quietly rises to meet it — the streaming upgrade, the nicer gym, the third delivery app. None of it feels like a decision."
Scene 3 (mechanism, 25–42s): "The fix isn't a budget. It's automation: the day a raise lands, route half of the increase to an account you never see. You can't spend what never arrives."
Scene 4 (proof, 42–52s): "People who automate savings save roughly three times more than people who rely on willpower — same income, same city, same rent."
Scene 5 (CTA, 52–60s): "Follow for one money mechanism a day — no hustle, no hype."
- Percentage + month name + invented compound ("lifestyle creep") for caption accuracy
- Abstract concepts (automation, willpower) — stock-footage engines must find non-literal b-roll
- Tight per-scene timing — tests whether the tool respects pacing or pads scenes
- A statistics sentence — tests on-screen text/number rendering