A/B testing is something our brands really love as a service - GTI has me do testing against product images all the time. I would like to bring in sessions to the mix. One thing to note about GTI - this works best in their wholesale markets - as we cannot capture clicks and ATC - is this something we might be able to do in the future? Here is the report we use: https://janetechnologies.thoughtspot.cloud/?utm_source=welcome-email&utm_medium=email/&orgId=508446133#/insights/pinboard/61a18591-3644-4688-9bb4-5ae712754f74 Table we use is called: Product Performance Problem The brand performance reports I use to evaluate brand-partner A/B-style tests (photo updates, content changes, etc.) only contain absolute event counts — Product Detail Page Views, Add to Carts, Checkouts, Units Sold. Without denominators, I can't tell whether a lift came from the change we made or from confounding factors. Brand partners have started asking sophisticated questions I can't currently answer, which weakens the story we can tell them about Jane's value. Example report attached (DankVapesTest.xlsx): Store, Brand, Category, Subcategory, Product, Views, ATC, Checkouts, Units Sold, weight breakdowns, Date. All useful, but missing the two columns I need most. What I'm asking for Add two things to the existing brand performance export, at the same Store + Brand + Product + Date grain that's already there: Sessions — unique visit counts so I can compute conversion rates (Views / Sessions, ATC / Sessions, Checkouts / Sessions) instead of just absolute counts. If true sessions aren't available, a reasonable proxy works (unique visitors, page loads, anything denominator-shaped). Inventory signal — ideally an "in-stock" flag per product-day (was this SKU available on this date, yes/no), plus a count of in-stock SKUs per brand-day. This lets me normalize for inventory changes between test periods. Why both, not just one (but will take sessions if that is all I can get) Sessions tells me whether traffic changed between my A and B periods. Inventory tells me whether the brand's shelf footprint changed. Both can move a brand's numbers independently of any actual performance change, and either one being missing means I can't isolate what my photo update (or whatever I'm testing) actually did. Concrete example: a brand could 2x their checkouts in Period 2 purely by launching two new SKUs or by a competitor going out of stock — nothing to do with what I tested. Today I have no way to detect or correct for that. Impact * Every brand-partner A/B test I run today (and the report I send the partner) ships with a measurement gap. Partners are starting to notice. * This is the single biggest analytical limitation I have. Most other improvements (statistical significance, control brands, segmentation) can be built on top of sessions + inventory — they can't be built without them. * Limited to non-headless / non-Roots stores is fine — for headless markets we've already accepted that Checkouts is the only reliable signal. This is about getting better data where the data exists. Acceptance criteria * Sessions column appears in the brand performance export at the same grain as existing columns * Inventory signal (in-stock flag + SKU count per brand-day) available in the same export, OR available as a separate file I can join on by Store + Product + Date * Documentation on how sessions are defined (cookie? device ID? timeout window?) * Documentation on inventory source and refresh cadence Out of scope (for this ticket) * Real concurrent A/B testing infrastructure — that's a separate, larger conversation with product * Statistical significance calculations — I can layer those on once the data exists * Headless / Roots stores — known limitation, not asking to solve here Attached: DankVapesTest.xlsx — example of current report format, showing what columns exist today and where the new ones would slot in.
