What this template is for
Experiments fail two ways: bad design (under-powered, no clear metric, no decision criteria) and bad ops (launched without review, results never written up, ship decision never recorded). This template forces the design inputs to be explicit before launch and forces the decision to be recorded after.
It is built for growth, product, and experimentation teams running a continuous test cadence.
When to use it
Use this template for any controlled experiment - A/B, multivariate, holdout - that affects the production user experience. Skip it for internal-only tests, ad creative tests, or marketing-only splits which belong in the marketing-campaign-brief template.
How to set it up in Jira
Create an experimentation project. Add the custom fields above to the Create Issue screen and use Custom Fields to mark Hypothesis, Primary Metric, Sample Size, and Approver as required-on-create so reviewers cannot approve incomplete plans.
Sub-task breakdown
- Review hypothesis and sample size is the design gate.
- Build variants and instrument the metric is the engineering build.
- Review-board approval is the explicit go-ahead.
- Launch and monitor guardrails is the live experiment.
- Analyse results and write the readout is the analysis.
- Decide ship / kill / iterate is the explicit decision that turns the experiment into action.
Use STM Issue Templates to auto-attach these sub-tasks on every new experiment so no step is skipped - see /stm-overview/.
Fields to add to your Jira create screen
These are the fields a project admin should make sure exist on the Create Issue screen for this issue type (Project settings → Screens). Without these on the screen, reporters can't provide the information triage needs - and STM can't reference them either.
| Field | Example value | Required |
|---|---|---|
Summary | A/B test - new onboarding checklist vs control | Yes |
Hypothesis (custom) | If we show a 3-step checklist on first login, then activation rate will increase by 5%. | Yes |
Primary Metric (custom) | 7-day activation rate | Yes |
Secondary Metrics (custom) | D30 retention, feature engagement | No |
Variant Description (custom) | Variant B: 3-step checklist on first login | No |
Audience / Targeting (custom) | New signups, US/EU, web only | No |
Sample Size Required (custom) | 12,000 per variant | Yes |
Expected Duration (custom) | 14 days | No |
Minimum Detectable Effect (custom) | 5% relative | No |
Approver (custom) | Head of Growth | No |
Component/s | experimentation | No |
Labels | experiment, onboarding | No |
Attachments | Mockups, statistical plan, dashboard link | No |
Note on custom fields. STM currently supports up to 5 custom fields per template. You can add as many custom fields as you like to your Jira Create Issue screen - the 5-field limit only applies if you want STM to set or update those custom fields itself.
Sub-tasks STM creates automatically
Build an STM sub-task template containing the items below, then wire it to an On Create Issue Executor scoped to this issue type. Whenever a new issue of this type is created in the project, STM creates the full sub-task set in one step - with assignee, due date, and components inherited from the parent unless you override them.
- Review hypothesis and sample-size calculation
- Build variants and instrument the metric
- Get experimentation review-board approval
- Launch the experiment and monitor guardrails
- Analyse results and write the readout
- Decide ship / kill / iterate and document the decision
Common questions
What is a Jira A/B test plan template?
It is a Jira task that captures the design, approval, and analysis of a single experiment. It records hypothesis, primary and secondary metrics, audience, sample size, expected duration, and minimum detectable effect, plus sub-tasks for design review, instrumentation, approval, launch, analysis, and ship/kill decision. The point is to prevent unreviewed experiments and to make analysis traceable.
How do you document an A/B test in Jira?
Create one Jira issue per experiment using this template before any code is written. Populate the hypothesis, primary metric, and sample size first - those are the inputs the review board needs. Use sub-tasks to walk the experiment through design, build, approval, launch, analysis, and decision. Link the issue to any feature-request or product-launch Epic it informs.
Should A/B tests be tracked in Jira or in an experimentation platform?
The experimentation platform (LaunchDarkly, Optimizely, Statsig) runs the experiment. Jira captures the workflow around it - hypothesis approval, analysis sign-off, ship decision. Keeping the workflow in Jira gives you cycle-time and throughput metrics across the whole experimentation program, which platforms alone do not.
What custom fields are required for an A/B test ticket?
Hypothesis, primary metric, sample size required, and approver are non-negotiable. Secondary metrics, audience targeting, expected duration, and minimum detectable effect should be on every ticket but can be filled during design review. Configure these via Custom Fields with required-on-screen rules so reviewers cannot approve incomplete plans.
Automate the sub-tasks with STM
STM Issue Templates saves the sub-task list above as a reusable template and creates them on every new issue of this type - via an Executor on issue creation, on status transition, or triggered manually from the issue's "Create bulk sub-tasks" menu. STM does not change the parent issue's create screen (that's a Jira project-settings job) but it removes the manual work of creating the sub-tasks every time.
Try STM on the Atlassian Marketplace ↗ See how STM templates are built →