A/B test ideas with hypotheses, metrics and priorities
Testing button colors one after another wastes months. Good tests start from a real problem on the page. Your todo.is agent reviews what you want to improve, looks at your data if you share it, and writes a ranked list of test ideas, each with a clear hypothesis and how to measure it.
The prompt
- Give me A/B test ideas for [WHAT TO TEST]. The goal is to improve [GOAL METRIC]. About [TRAFFIC] visitors or recipients see it each month. Review it as a first-time visitor and use this data if attached: [DATA]. Write 12 test ideas. For each give: the problem you noticed, a hypothesis ("If we…, then…, because…"), version A and version B in detail (including the exact new copy), the main metric and a guardrail metric, and an ICE score (impact, confidence, ease, 1–10 each). Tell me which tests my traffic can realistically detect and roughly how long each would need to run. Rank the list and write a one-page test plan for the top 3. Save it as an Excel file and a short Word summary.
What to change
- [WHAT TO TEST]: A page URL, an email, a checkout or an app screen, e.g. "our pricing page at plotwise.io/pricing".
- [GOAL METRIC]: e.g. "trial sign-ups from the pricing page" or "email click rate".
- [TRAFFIC]: e.g. "8,000 visitors a month". It decides what you can test.
- [DATA]: Optional: attach analytics exports, heatmap screenshots or survey answers, or write "none".
Example result
- A/B test ideas · plotwise.io/pricing (example)
- Goal: trial sign-ups · Traffic: about 8,000 visitors a month
- Top 3 (full plans in the Word file)
- 1. Show the annual price as a monthly amount · ICE 8 / 7 / 9
- • Problem: the annual plan shows "$468/year", which looks expensive next to "$49/month"
- • Hypothesis: if we show "$39/month, billed yearly", more visitors will choose annual, because the saving is visible
- • A: $468/year · B: $39/month billed yearly (save 20%)
- • Main metric: trial starts · Guardrail: refund requests
- 2. Answer the top objection beside the button · ICE 7 / 7 / 10
- • Problem: survey answers say people fear being charged after the trial
- • A: "Start free trial" · B: "Start free trial" + "No card needed. Cancel in one click."
- • Main metric: trial starts
- 3. Recommend one plan · ICE 7 / 6 / 8
- • Problem: three plans look the same; heatmap shows visitors scrolling up and down between them
- • B: highlight "Team" as "Most popular for teams of 3–10"
- Other ideas (ranked)
- • Move the customer logos above the plans
- • Replace the feature table with "Who each plan is for"
- • Add a short FAQ under the plans
- • Swap the stock photo for a product screenshot
- What your traffic can detect
- With about 8,000 visitors a month, test big, visible changes one at a time. Small tweaks like button color would need far more traffic to show a reliable result. Plan for each test to run at least two full weeks, ideally until it reaches the sample size your testing tool calculates.
How to do it with todo.is
- Copy the prompt and add the page or email, your goal and your monthly traffic.
- Paste it into todo.is or send it to your agent on Telegram. Attach analytics or survey data if you have it.
- Your agent reviews the page, writes hypotheses with exact copy for each variant and scores them.
- Start with the top test in your testing tool, then ask your agent to analyze the results when it ends.
Tips for a better result
- Write the hypothesis before you design the variant. If you cannot say why it should win, skip it.
- Low traffic? Test bold changes like a new headline or offer, not small details.
- Decide the run time and main metric before starting, and do not stop the test the day it looks good.
- Send your agent the finished test results as a CSV and ask it to check whether the difference is meaningful.
A/B test ideas: FAQ
- What should I A/B test first? Start where many people drop off and the change is easy: headline, offer wording, the call to action and the form. Test one main change at a time.
- How long should an A/B test run? Until it reaches the sample size your tool calculates, and at least one or two full weeks to cover weekday and weekend behavior.
- Can I A/B test with low traffic? Yes, but only big changes will show a clear result. With very low traffic, user feedback and expert reviews can be more useful than tests.
- Does todo.is run the A/B test? No. It plans the tests, writes the variant copy and analyzes results you export. You run the test in your website or email tool.
JavaScript is required to use the todo.is app.