An Amazon product-image A/B test should answer one commercial question, not provide a prettier excuse for changing the whole listing. Use one eligible ASIN, one approved control, one source-faithful treatment, one declared image role, Amazon's randomized comparison, and a decision rule written before the first result arrives.
That makes this a different job from recovering a suppressed Amazon main image. Recovery restores factual eligibility. Experimentation begins only after the listing, selected image, offer, and inventory are stable enough to support a trustworthy comparison.
Evidence boundary: the planter is a fictional controlled product. The cover places one extracted product source into two newly generated empty bays. The comparison board and both square candidates reuse the same source pixels at measured frame widths of 71.75% and 84.75%. They were not uploaded to Amazon, did not run in Manage Your Experiments, and do not establish product accuracy, policy approval, statistical significance, conversion lift, or profit.
Confirm that Amazon can run the test
Amazon's current Manage Your Experiments page says access requires a Professional selling account and the Brand Representative role for a brand enrolled in Amazon Brand Registry. A product is eligible when it belongs to the enrolled brand and has received enough recent traffic to support a valid result. Version A is the currently published content; Version B is the new content. Check Amazon's current eligibility and setup guidance.
That produces a hard preflight:
| Gate | Evidence required | If it fails |
|---|---|---|
| Account | Professional selling account | Do not build a Seller Central experiment |
| Brand role | Brand Representative for the enrolled brand | Resolve brand access; do not duplicate the ASIN |
| ASIN | Exact marketplace, child ASIN, sold variant, and current image contribution | Stop if the wrong product or variation owns the media |
| Eligibility | ASIN appears for the selected experiment type | Record “ineligible”; do not fake a before/after comparison |
| Stability | Stable price, offer, inventory, traffic plan, and live control | Delay or explicitly model the confound |
| Product authority | Approved source, checksum, sold configuration, and current live image | Do not generate a treatment |
| Measurement ability | Amazon result plus contribution and customer-harm guardrails | Do not auto-publish a winner |
Low traffic is not a reason to compare last month with this month. That sequential shortcut mixes the image with seasonality, ads, organic rank, price, reviews, inventory, and competitor movement. If the ASIN is ineligible, keep the control, improve the evidence package, and prioritize a higher-traffic SKU or another channel where assignment and exposure can be measured correctly.
Choose the image role from the bottleneck
Amazon now publicly describes testing product images and has announced image-gallery experiments for supporting images. It has also announced simultaneous experiments across multiple content types. Review the supporting-image announcement and the simultaneous-experiments announcement.
More available test surfaces do not mean “change everything.” Select the role that matches the decision:
| Observed bottleneck | Candidate experiment | Primary interpretation |
|---|---|---|
| Search impressions but weak product-page entry | One approved main-image treatment | Does clearer identification change the listing outcome? |
| Product-page traffic but recurring scale doubt | One supporting image that shows verified scale evidence | Does answering the scale question improve qualified orders? |
| Confusion about included items | One supporting image with the exact sold configuration | Does product clarity improve outcomes without more returns? |
| Strong traffic but weak A+ engagement | Two approved source-preserving A+ versions | Which module hypothesis performs better? |
| Several coordinated changes form one strategy | A predeclared simultaneous package of image, title, or other fields | Does the package win? It does not isolate each component |
Use a simultaneous experiment only when the business decision concerns the combined package. If title, image, bullets, and A+ all change, a winning result supports that bundle—not a claim that the image alone caused the difference.
Current seller discussion supports the operational pain, not a universal benchmark. Merchants ask whether image testing produces meaningful gains, which image role to prioritize, and how much traffic or time is enough. Replies disagree and some appear synthetic, so they should shape the questions rather than supply expected lift. Read the current seller discussion.
Pre-register one visual hypothesis
A useful hypothesis names the audience problem, treatment, primary metric, guardrails, and decision:
For eligible visitors to child ASIN
EXAMPLE-NTH-PLN-001, increasing the same approved product's width within the white main-image frame from 71.75% to 84.75% will improve units sold per unique visitor because the item will be easier to identify at search and detail-page thumbnail sizes, without increasing wrong-item contacts, product-truth defects, or matured return rate.
That is not a claim that tighter framing wins. It is a falsifiable contract. Amazon's current role and category rules still control whether either candidate is allowed, and Manage Your Experiments may decide two versions are too similar to produce a useful result.
The full-resolution control candidate and treatment candidate make the review reproducible. They are evidence artifacts, not recommended Amazon uploads.
Download the Amazon image-experiment ledger. It keeps the ASIN, versioned assets, hypothesis, immutable fields, metric, guardrails, dates, result, and signed decision in one row per experiment stage.
Keep product truth outside the experiment
Customer response cannot authorize a false product. Before Version B reaches Seller Central, compare it with the approved source at full resolution:
- same child ASIN, sold variant, quantity, included items, and package;
- same geometry, openings, seams, fasteners, materials, finish, and color;
- same printed text, marks, dimensions, and visible construction;
- no invented benefit, use, scale cue, accessory, certification, or result;
- current marketplace, category, and image-role rules passed;
- previous published asset, checksum, and rollback owner retained.
For a main image, the treatment may need an exact source-preserving crop, frame, background, or approved alternate photograph. For a supporting image, AI can help create a bounded environment candidate, but the product source and factual evidence remain authoritative. The Amazon AI image-rules guide covers disclosure and ordinary image requirements; the source-preserving A+ workflow covers deterministic modules after the image itself is approved.
Reject a treatment when the model changes the product even if the change looks more premium. The experiment's job is to compare valid merchandising hypotheses, not to discover whether customers notice a fabricated feature.
Let Amazon randomize and finish the comparison
Amazon says customers are randomly split between the versions and results update during the run. The current default starts after validation, runs to significance, and can auto-publish a significant winner. Amazon says a significance-based result can sometimes arrive in about four weeks; if a seller chooses a fixed duration, it recommends eight to ten weeks. Review Amazon's current duration and result guidance.
Use those mechanics instead of building a second traffic router. Before scheduling:
- upload the reviewed Version B under a versioned filename;
- record the exact ASIN, marketplace, experiment type, slot, control, and treatment;
- write the hypothesis, primary metric, guardrails, and minimum commercial effect worth acting on;
- choose “to significance” or a fixed duration for a documented reason;
- disable auto-publish when product, contribution, return, or support guardrails require human review;
- freeze unrelated listing changes where practical and log unavoidable interventions;
- wait for completion unless a predeclared harm or validity condition requires rollback.
Do not call a winner from the first favorable weekly update. Do not extend a losing test until it wins. Do not silently swap Version B during the run. A changed candidate is a new experiment.
Decide from Amazon's result and merchant economics
Amazon's public tool page lists units sold per unique visitor, conversion, units sold, sales, sample size, and projected annual sales impact among the result fields. Use Amazon's completed randomized result for the comparison, but do not confuse sales with contribution or a projection with cash already earned.
A transparent planning translation is:
future eligible unique visitors × (units per unique visitor B − units per unique visitor A) × contribution per unit
Run that as a scenario with observed Amazon rates and a merchant-controlled contribution figure. It is not an estimate of historical causal profit, and it becomes invalid when price, fees, discounts, return costs, traffic quality, or stock conditions change.
| Completed result | Decision |
|---|---|
| B is better and product, contribution, and trust guards pass | Publish B, retain A and the result, then observe matured returns |
| A is better | Keep A; record the rejected hypothesis instead of deleting the evidence |
| Inconclusive | Keep A; improve the contrast or choose a higher-traffic decision |
| B lifts conversion but contribution deteriorates | Do not call it a business winner |
| B wins before returns or support outcomes mature | Record the Amazon result; defer the durable merchant decision |
| Simultaneous package wins | Publish or retest the package; do not assign the effect to one component |
| Product-truth or policy defect appears | Stop, restore the known-good asset, and treat the issue as an incident |
Returns and support contacts can arrive after Amazon declares an experiment winner. Keep two dates: the platform result date and the merchant decision date after the relevant guardrails mature. Report the probability and sample Amazon supplies, the raw commercial delta, and the limits. Do not invent a universal confidence threshold outside the live tool.
The operating rule
For one Amazon image experiment:
- confirm account, role, ASIN, traffic, and experiment-type eligibility;
- diagnose the image role and buyer problem before producing Version B;
- preserve the exact product source and declare one visual hypothesis;
- review Version B for product truth, current policy, and rollback readiness;
- pre-register the metric, commercial threshold, guardrails, and stopping rule;
- let Amazon randomize eligible customers and complete the declared comparison;
- interpret a simultaneous test as a package when multiple attributes move;
- translate the result into contribution carefully and wait for delayed trust outcomes;
- sign keep, publish, iterate, or rollback while retaining both assets and the result.
For a merchant-owned storefront where Amazon's assignment and result are unavailable, use the separate Shopify product-image A/B testing workflow. For the broader sequence from catalog truth through marketplace content, ads, lifecycle, and measurement, use the AI ecommerce workflow map.
