Ask HN: Do you run A/B Tests?

I was introduced to ridiculously simplified A/B testing while working at BigTech. I wanted to understand the landscape of running A/B tests outside of BigTech. As an example, I recently worked with a Fintech startup that was not running true A/Bs via randomised user buckets, but they did have a setup to test out a feature with a particular cohort before a larger rollout.

So, do you do A/B testing or feature flagging in any way? If yes, can you also mention

* Daily Active Users / Monthly Active users? Trying to understand at what “scale” does A/B testing make sense for organisations

* What’s the infrastructure and plumbing look like? Inhouse setup (add a flag to a user profile or if conditions in code?) or do you pay for a SaaS that helps?

* What’s the overhead like? Do you feel it’s worth it?

If you are not using A/B testing, why not? Is it the initial investment or the followup maintenance that is the headache?