Product experiment design

Research

Use when the user names an A/B test, an experiment, a growth bet, or a before-and-after read: the plan with one primary metric, guardrails, a computed sample size, precommitted interpretation bands, then the scale-extend-or-kill readout.

by AutoGPTNewUpdated 12d ago

Instructions

Instructions your experts follow when using this skill.

Use this when the user names an A/B test, an experiment, a growth bet, or a before-and-after read. The output is a plan whose verdict is decided before the data arrives, and then the readout against it.

What you need first

The belief under test, the decision it informs, and the traffic or users available. No traffic numbers means you size with bands and name the minimum sample before launch. No belief stated means you name the bottleneck belief first and get a yes on it.

Design it

Name the one bottleneck belief the test exists to move. A test that tries to prove three things proves none.
List two to four candidate tests and compare them on information gain, cost, time, and reversibility. Pick one and say in a line why it won.
Write the design: control and variant; ONE primary success metric; guardrail metrics that catch harm elsewhere; the minimum sample computed from the baseline plus the minimum detectable effect at 95% significance and 80% power; a duration of at least one to two full weeks; pre-declared segments; the business-meaningful threshold stated beside the p-value; and the flag configuration. Guardrails are not optional.
Precommit the interpretation bands with observable numbers: the upgrade band where you scale, the ambiguous band where you extend or rethink, and the downgrade band where you kill. No vague words like "several" or "good".
State plainly what the test cannot establish, then launch only on a yes.
Read it out: the band it landed in, the guardrail check, the sample-ratio mismatch check, and the call — scale, extend, or kill.

Output

The dated experiment plan and, after the run, the dated readout with the call and the follow-up offered on a yes.

When the traffic is not there

Say the sample will not reach the minimum and propose the alternative — a longer run, a bigger effect to look for, a painted-door test, or a qualitative read instead. Never move the bands after seeing the data, and never call a test before one to two full weeks and the precomputed sample size.

Triggers8

Phrases that help your experts find this skill.

design an A/B testexperiment planminimum detectable effecthow long should the test runscale or killexperiment readoutguardrail metricsprecommit the bands

More skills

Research

Product marketing context

by AutoGPTFrom coreyhaines31/marketingskills/skills/product-marketing

When the user wants to create or update their product marketing context document. Also use when the user mentions 'product context,' 'marketing context,' 'set up context,' 'positioning,' 'who is my target audience,' 'describe my product,' 'ICP,' 'ideal customer profile,' or wants to avoid repeating foundational information across marketing tasks. Use this at the start of any new project before using other marketing skills — it creates `.agents/product-marketing.md` that all other skills reference for product, audience, and positioning context.

1 installedView

Research

Competitor profiling

by AutoGPTFrom coreyhaines31/marketingskills/skills/competitor-profiling

When the user wants to research, profile, or analyze competitors from their URLs. Also use when the user mentions 'competitor profile,' 'competitor research,' 'competitor analysis,' 'profile this competitor,' 'analyze competitor,' 'competitive intelligence,' 'competitor deep dive,' 'who are my competitors,' 'competitor landscape,' 'competitor dossier,' 'competitive audit,' or 'research these competitors.' Input is a list of competitor URLs. Output is structured competitor profile markdown files. For creating comparison/alternative pages from profiles, see competitors. For sales-specific battle cards, see sales-enablement.

1 installedView

Research

Customer insight research

by AutoGPTFrom coreyhaines31/marketingskills/skills/customer-research

When the user wants to conduct, analyze, or synthesize customer research. Use when the user mentions "customer research," "ICP research," "talk to customers," "analyze transcripts," "customer interviews," "survey analysis," "support ticket analysis," "voice of customer," "VOC," "build personas," "customer personas," "jobs to be done," "JTBD," "what do customers say," "what are customers struggling with," "Reddit mining," "G2 reviews," "review mining," "digital watering holes," "community research," "forum research," "competitor reviews," "customer sentiment," "PMF survey," "product/market fit survey," "customer interview questions," "interview outreach," "Sales Safari," or "find out why customers churn/convert/buy." Use for analyzing existing research assets, mining online sources, AND running primary research (interviews and surveys). For writing copy informed by research, see copywriting. For acting on research to improve pages, see cro.

1 installedView