Sign up FAST! Login

Andreessen and Mixpanel call for end to “Bullshit Metrics”...

Andreessen and Mixpanel call for end to Bullshit Metrics


Stashed in: Startups, Traction, Marc Andreessen, @lizgannes, Mixpanel, @a16z, Pants on fire!

To save this post, select a stash from drop-down menu or type in a new one:

One person's bullshit is another person's fertilizer.

Liz Gannes writes:

Mixpanel and its investor Andreessen Horowitz (a16z) are trying to persuade the tech world to be more honest with itself by reporting numbers that are far more informative: engagement and retention.

Yes, those happen to be the very things Mixpanel measures, but as I’ve written before, they make a lot of sense.

It’s not just start-ups that try to push big numbers over reality; last year Google famously obfuscated Google+ user counts as it was getting its social efforts off the ground.

The problem is, once one company reports a bullshit metric — like how many registered users it has — its competitors don’t want to appear smaller by reporting something real — like how many monthly active users it has. So the bullshit metric shaming needs to be a broader movement for it to work.

(To which I say, where are the bumper stickers?)

“We can do better as an industry. We should do better because collectively we’re not benefiting — we’re all just fooling each other,” Mixpanel founder Suhail Doshi wrote in a blog post he’s publishing today.

Two thoughts.

1. It's self-serving of Mixpanel to be pushing this agenda. I'm already hurting from Mixpanel's lack of introductory pricing, and all this agenda does is make me look bad for not wanting to pay Mixpanel's prices.

2. OF COURSE everyone wants engagement and retention. Stop making people feel bad for having something in their product that works! Engagement and retention are things that can be continually improved. Investors should treat them as such, and not require that a startup do EVERYTHING at once. Monthly actives, Engagement, Retention: Pick ONE to improve at any given time or your scientific method gets screwed up.

Case point: As a startup they want me to pay $150/month?! Ouch.

And it only gets more expensive from there...

Mix panel pricing

Jay Jamison seconds that emotion:

To me, the first thing I want to see is evidence of what I think of as early product market fit. Rather than showing me installs, show me that any small number of users that truly, with concrete evidence, really want your product. This is actually more difficult to do than you might think. It requires that the startup can show engagement and retention over a period of time. And it requires that the startup is thinking about what how it measures user engagement in a manner that’s specific to that specific company.

This perspective seems aligned with Doshi’s feedback. One thing that I liked a great deal in his post was his recommendation that startups focus on One Key Metric (OKM). The idea is that tracking 1 actionable metric that “they can literally bet their business on.” Companies picking OKM have to deeply understand their business and what is driving growth and success in order to do so.

Read more about One Key Metric here.

The only challenge with One Key Metric is that a business has to pick ONE particular activity to optimize on at a time: Attract New Users, Convert New Users to Engaged Active Users, Retain Active Users, Monetize Active Users.

No one metric can satisfy all of them. One Key Metric changes as what you optimize changes.

At PandaWhale we have three types of users: Writers, Readers who "Hit It and Quit It", and Readers who "Stay and Play".

With any given change to our software, we're optimizing for one particular activity with one of these groups.

It's a balancing act to say the least.

You May Also Like: