Absolutely. If you're the business owner, selling fewer widgets is Very Bad!
But in my post, I specifically called out a line in OP's article that I disagreed with: (paraphrasing) "Your startup deserves the same rigor as medical testing."
To clarify -- and support your point --, we're shipping software, not irreversible medical procedures. If you get it wrong, you sell fewer widgets /temporarily/ and you revert back to a known better solution. With medicine, there aren't necessarily take-backsies -- but there absolutely are in software. Reverting deploys is something all of us do quite regularly!
Is it A/B testing? Maybe, maybe not. I'm not a data scientist. But I think saying that your startup deserves the same rigor as a medical test is misleading at best and harmful at worst.
I just think companies should be more okay with educated risks, rather than waiting days, weeks, months for statistical significance on a feature that has little chance of actually having a negative impact. As you said elsewhere in the thread, for startups, stasis is death.
(BTW, I've read a lot of your other comments in the thread. I think we're pretty well aligned!)
But in my post, I specifically called out a line in OP's article that I disagreed with: (paraphrasing) "Your startup deserves the same rigor as medical testing."
To clarify -- and support your point --, we're shipping software, not irreversible medical procedures. If you get it wrong, you sell fewer widgets /temporarily/ and you revert back to a known better solution. With medicine, there aren't necessarily take-backsies -- but there absolutely are in software. Reverting deploys is something all of us do quite regularly!
Is it A/B testing? Maybe, maybe not. I'm not a data scientist. But I think saying that your startup deserves the same rigor as a medical test is misleading at best and harmful at worst.
I just think companies should be more okay with educated risks, rather than waiting days, weeks, months for statistical significance on a feature that has little chance of actually having a negative impact. As you said elsewhere in the thread, for startups, stasis is death.
(BTW, I've read a lot of your other comments in the thread. I think we're pretty well aligned!)