General rule of thumb: Don't work with people who want your results but not your process.

running a test with 7 alts and all 7 are positive is a great problem to have, but it can still be a problem...

Tactics and Strategy get confused for each other. Heat Maps, Surveys, User Feedback are TACTICS. Usually poor ones. Strategy is putting a plan together that gets results. It may or may not use those tactics but ultimately it's about discovery and exploitation of information.

"I woke with this conclusion from Skin in the Game: 1) Things work thanks to a handful of people; the rest operates by situational imitation, narrow mimicry, and semi-conscious role-playing." - Nassim Nicholas Taleb

Something New: Bandit Based Yield Theory testingdiscipline.com/2023/01/08/som…

Something New: Simpson’s Paradox testingdiscipline.com/2023/01/16/som…

Something New: Rant – Product Management Best Practices testingdiscipline.com/2023/01/16/som…

Something New: Segmentation and Dunning-Kruger testingdiscipline.com/2023/01/23/som…

Something New: Rant – Gaussian Distribution testingdiscipline.com/2023/01/30/som…

Something New: Forer Effect and You testingdiscipline.com/2023/02/06/som…

So GA4 is kind of terrible, isn't it?

Something New: Beat the Prisoner’s Dilemma testingdiscipline.com/2023/02/13/som…

Study analytics not to strengthen your argument but instead to find the truth. Too many marketers will try to use data to demonstrate how smart they are instead of realizing that the smartest marketers use data to identify their blindspots.

Something New: SEO and AB Testing testingdiscipline.com/2023/02/20/som…

Something New: Rant – Experts Suck testingdiscipline.com/2023/02/27/som…

It's hard to predict the magnitude of success of any given campaign, blog post, or experiment. The question is: how can you lower the "cost" of each action or mistake in order to maximize the chances of big wins?

Something New: Overcoming Organizational Inertia testingdiscipline.com/2023/03/20/som…

John James Ah awesome, thanks! Reminds me of how similar things have played out in A/B testing creative online — marketers & designers generally can’t pick winners. Folks like Andrew Anderson wrote cxl.com/blog/the-disci… in response, but too hard to internalize so everyone just ignored! 😓

This.