Being a self-proclaimed community data nerd sometimes brings some funny interactions. A direct, a colleague, a peer from elsewhere excitedly shows you a dashboard with an averaged out metric and says “So, what do you think? How does this look?”
Me: “What exactly am I looking at and what are you actually averaging?”
I’m such a kill joy.
Averages are the strangest of metrics in my opinion because they’re often quite opinionated. And opinions are decisions.
Someone somewhere chose which population to blend together before the math ever happened - and that choice right there is doing a ton more work than the resulting number ever will.
To really illustrate how ridiculous just saying an average is:
Over 99% of all humans have more than the average number of human legs (2).
The average temperature in the United States in March is 41.5F.
Imagine just leaving those there. No explanation. No deeper context.
I mean - sure, yeah both averages are very real and very accurate. But the numbers are actually kind of useless. Do we design the majority of pants and bicycles for less than 2 legs? Do we need a heavy jacket for Phoenix, Arizona?
With what I’ve given you above, it’s a big ol’ shrug.
And needless to say, communities are no different. Blending Alaska and Arizona’s late winter weather is the same as doing an average across your entire community. The averaged number tells you the temperature of a place or time that doesn’t exist.
Take the metric “average posts per member.” Sounds like a health metric. But it’s actually two very different populations all mashed into one. It’s new members still finding their footing - but also the small maintenance crew who’ve been here since the beginning and post constantly because the place would go quiet without them. But average those together gets a number that doesn’t describe either group. And maybe it looks fine. But in reality, it’s describing absolutely nobody.
Same with “average time to resolution” on a support community. Blend the veteran members who know exactly where to look for answers with the brand new members who don’t even know what to search for yet. Get a resolution time that flatters the community’s onboarding while it’s actually terrible or makes the community’s speed look like absolute garbage while it’s actually fine - all of course depending on which cohort happens to be larger that month. You can’t tell which. That’s the whole problem. The average doesn’t know which cohort it’s hiding.
So what do you actually do differently?
First, before you average anything, name the darn unit. Not the metric… the population underneath it. If you can’t say out loud what group of members this number is describing, you’re not ready to average it yet. You’re just doing fancy math and hoping it counts as insight.
Second, segment before you aggregate - not after! Cohort by tenure at minimum: week one, month one, month six, year one plus. A single “engagement score” split into those four buckets will tell you more in five minutes than the blended number told you in a year. Why? Now you’re looking at four small, coherent temperatures instead of one big meaningless one.
Third, treat any metric someone hands you pre-averaged with some serious suspicion. Ask what’s inside it. If the answer is “everybody,” ask again.
Of course, none of this is really about statistics. It’s all about admitting that “the community” isn’t one big population. It’s cohorts, arriving at different times, learning at different speeds, needing different things from the same room. Averaging across all of that and you won’t find the center of anything. You’ve just found the spot exactly between two places, neither of which anyone actually lives.
Oh and that long awaited explanation: Some humans have 1 or no legs, which therefore lowers the average below 2, despite the majority of humans having all their allocated legs.