As the social media analytics field is pretty knew, it’s really important to understand how the metrics are computed.
Let’s take for example, one of the main metrics in measuring one’s page performance : the “Engagement Rate”.
What is supposed to be measured with the Engagement Rate ? “The Engagement Rate measures how well your Fans interact with your content”. That’s a common definition.
What’s that supposed to mean? If you have a 0.02% Engagement Rate, it means 0.02% of your fans have engaged with your content, isn’t it?
Only the formula commonly used does not measure that. Not at all.
Another commonly used formula is this one :
Don’t you think, there’s something missing?
What about the interactions when your fans play your videos, look at your photos or click on your links? Aren’t these actions a BIG part of your engagement? For some Brands or Pages (fashion, beauty, e-commerce or even cars manufacturers for example), we may even argue that these interactions are close to be the most valuable ones.
But in the commonly used formulas, these interactions are nowhere to be found! For some Pages, it may represent close to 80% or more of the total interactions, and they’re not included in the calculation! The Engagement Rate is supposed to measure how well your fans respond to your content, but a big part of their response is not measured.
What kind of insights do you get with that? No wonder the average engagement rate is below 0.2%.
Brands need to focus on the number of people reached, not the number of fans!
Facebook’s great promise is in viral marketing (touching friends of fans), not in the number of fans you have. Especially when you know that close to 84% of your Facebook Fans don’t ever see your content…
Let’s compare two Pages.
Which one is the most successful on Facebook ?
The second one for sure : close to two more times fans, a bigger engagement rate.
Well. We’re not so sure about this. Let’s insert another metric.
If you’re measuring your engagement rate based on the number of fans, you may be hiding the hard truth : what counts is not the number of fans, but the number of users reached (who cares if they’re fans or friends of fans anyway?).
A visibly smaller page with less overall interactions but with more shares and/or with fans with a wider friends network may well outperform another page with way more fans or more interactions, when it comes to number of users reached.
So, to help you measure how well your content engage your readers, these formulas include all your fans, even if more than 80% of them simply can’t engage with your content (if they don’t see it how could they like it, comment it or share it ?) and, icing on the cake, it’s not including the users who are not fans but are able to interact with your content because they see it in their newsfeed?
It’s a bit like comparing apples and pears. The formulas make a ratio out of interactions (done by fans and non-fans alike) with the total number of fans only. This creates a bias towards a bigger engagement rate. To be accurate, the formula would need to exclude all likes, comments and shares from non-fans.
Want to trick your Engagement Rate further? Follow this advice : post more. It’s quite simple and it works.
1. You’ll reach more fans, it means that more of them will have the opportunity to interact with your content, thus increasing your interactions count
2. Every user reached (fans or not) will have more opportunities to like, comment or share your content (a post can only be liked or shared once!). This will increase your interactions count as well.
These formulas clearly favor Pages with high publishing activities, even if, on average, their posts perform less than posts on other Pages with lower publishing activity levels.
We want to know the proportion of our fans that interact with our content, but we’re not measuring unique users, we’re measuring the number of interactions. One single fan may be responsible for 28 likes, 13 comments and 6 shares. In the formulas above, this will be considered as 47 “users”…
These are inappropriate formulas to get unbiased insights on whether your community on Facebook enjoys your content or not.
What remains is a ridiculously low Engagement Rate that makes your boss wonder if Facebook works at all.
The formulas may still be useful, nonetheless, for benchmarking purpose for example, even if it they are biased. But, in our opinion,they should be called Activation Rate, or something similar, not Engagement Rate. They certainly do not help you evaluate if your community enjoys your content.
Based on our previous comments, we’ve devised the following formula :
And if you’re doing an Engagement Rate benchmark between several Pages, we advise you to use the following formula :
The formulas present the following benefits which makes it a lot more accurate :
- It includes all type of interactions : photo views, video plays, link clicks, quotes… Not only likes, comments and shares.
- It compares fans and non-fans engagement (Users engaged may or may be not fans) with total number of fans and non-fans reached
- Only people who had the opportunity to see your content (thus the opportunity to interact with it) are included
- The impact of publishing rate is isolated (for the second one)
On top of that, this is probably the way Facebook itself would measure Engagement. Have you seen how “Virality rate” is measured by Facebook ? It’s the number of people creating stories/Number of People reached. Not the number of shares/number of fans.
And at the end, it really answers the question “Does my content engage my community or not ?”
Using the first formula you may expect Engagement Rates of 30,40, 50% or more.
Now the boss is happy.
EDIT : Added the fifth problem on the 4th of July
EDIT : Added the Engagement Rate formula for single Pages, thanks to our readers comments, on the 24th of September
EDIT : Added the Engagement Rate formula based on PTAT, on 11th of October
49 Comments
Mat Morrison
Some excellent work here; many thanks. I agree entirely with your overall thesis, and love your final metric (I assume that you’re using the “page_engaged_users” data — but have a few (tiny) points of information:
# Problem One
First of all, we need to be very careful distinguishing between “exposure” and “view” (particularly since impressions on the News Ticker aren’t separated out from impressions in the News Feed.) This is just a language issue, but it can be important.
Secondly, we’re talking about SOCIAL media. If we’re doing our job right, we’re looking at those public actions that a viewer takes that can be (through organic or paid channels) shared with their friends.
Most of the time I don’t give a hoot whether someone has engaged with my content; I care about the way their endorsement has been (or can be) shared with their social graph. This isn’t about fans; it’s about reaching through the fans to their wider social graph. Otherwise, it’s just inefficient CRM!
# Problem Two
Again – agree. But we should be careful waving Facebook’s 16% number around. It’s an old number for one thing (a whole year old!). And the variance is huge; I’ve seen reach as low as 5%, and as high as 25%. Thanks to Facebook’s editorial algorithms, there seems to be a nice – fairly straight line – correlation between {likes + comments + shares} and exposure. (Which brings me to another point: the reason that the {likes + comments + shares} engagement model works is because that’s what Facebook seems to use to power its algorithm.)
More importantly, the “16%” generally a daily reach figure. Reach over a week or a month is always much higher than daily reach. Once again – this doesn’t affect your calculations, and I sound like I’m picking nits. I do think it’s important, however, that people like you who clearly understand Insights data communicate this kind of story carefully!
These two problems aren’t really problems. They represent some slight quibbles about language and a possible shift in focus. I think that this article is fabulous stuff; and I’ll be happy to share it with everyone I know.
03 Jul 2012 02:07 pm (@Twitter)
Steve
Mat, thank you so much for your participation on this blog.
First problem : can’t agree more
Seconde problem : you’re absolutely right. Brands may reach a lot more fans than the 16% average. That’s the reason why Wisemetrics is useful!
For what I know (but, hey, I didn’t produce the Edgerank), Facebook do take into account clicks on photos/videos and links. Even if the weight is lighter than on shares/comments/likes indeed.
Thanks a lot for your support
Steve
PS : yes, this is the “page_engaged_users” metric
03 Jul 2012 03:07 pm (@@Wisemetrics)
Wise Metrics Proposes New Formula For Measuring Facebook Engagement – AllFacebook
[...] Wise Metrics pointed out four flaws with the current method of calculating Facebook engagement, as well as providing a formula of its own, in a post on its blog. [...]
Guille
Perfect article!
Only one doubt… should’t both parts of the formula be divided by the number of posts… not only the number of engaged users but also the total number of reached users / number of posts that you are measuring.
I think.
04 Jul 2012 04:07 pm (@Twitter)
Steve
Thanks!
You might well be right... Let me think a bit about it...
04 Jul 2012 04:07 pm
Jeff Werkheiser
I like your thinking on this. I completely agree the denominator should be users reached, not total fans. This way true engagement is only measured against people that actually came in some kind of contact with your content. Whether or not the 16% is still completely accurate, it’s close. I typically see around that number on standard posts, but up to 25% on more viral posts. So, I know that at the very least around 75% of our fans are not seeing our content on a post.
While your new way of measuring engagement might very well be correct, I don’t see Facebook using ‘users reached’ as a denominator anytime soon for engagement. The higher the percentage of engagement looks to the brand, the less likely brands would be to purchase the (relatively) new promoted posts to bump up their reach. Something Facebook wouldn’t look too fondly on I’m sure. Or the shareholders.
I also recently noticed that the new engagement detail listed on each individual post now just says the number of people that saw your post. They already did away with the percentage of fans that saw it?
Great blog with very interesting insight. Thanks!
04 Jul 2012 06:07 pm (@@J_Werky)
Steve
Hello Jeff, nice talking to you on that subject!
Facebook is already using “Users reached” for posts for example. And for the whole page, the number is available through the API.
Have a nice day!
04 Jul 2012 06:07 pm (@wisemetrics)
Wise Metrics Proposes New Formula For Measuring Facebook Engagement
[...] Wise Metrics forked out 4 flaws with a stream process of calculating Facebook engagement, as good as providing a regulation of a own, in a post on a blog. [...]
Johanne
The images do not display for me…
05 Jul 2012 08:07 pm (@Twitter)
Steve
I'm sorry Johanne. Don't know what's happening to you ? Do you use a firewall/proxy that might prevent pictures from showing up ?
09 Jul 2012 11:07 am
Facebook’s engagement rate metric – Does it need to be re-evaluated? | IndiaSocial – social media open
[...] metrics, a provider of Facebook analytics for brands and agencies, suggests that this formula doesn’t present a clear enough picture and gives an alternate formula to calculate engagement. Here are the reasons [...]
Two New Metrics For Assessing User Engagement on Social Media Found
[...] Wisemetrics point out, this formula is flawed in that it finds how engaged a page’s ‘fans’ [...]
Edward j Bass
I’ve tried this formula out and what concerns me is that posts supported by ads tend to skew the results here somewhat.
For instance a post also used as a Facebook ad would generate a much higher reach but not always a proportionate increase in engagement.
Be interested to hear your thoughts on this…
16 Jul 2012 11:07 pm (@@audiosilver)
Steve
You're absolutely right Edward. To limit the impact of paid campaigns, your reached users count should only include users reached via Organic and viral.
Paid campaigns may still have an impact on viral reach though.
17 Jul 2012 11:07 am
Current Facebook Engagement Rate Calculation Needs Revision
[...] above formula has been popular for a while but has some hidden problems that was recently listed by WiseMetrics. WiseMetrics, a company that is also in the business of measuring the ROI for Facebook brands has [...]
O Facebook errou ao criar sua Taxa de Engajamento « Head of Digital Strategy
[...] chegarão à 30,40, 50% ou mais. ** Essa análise foi originalmente publicada no blog da Wise e traduzida e adaptada para realidade Brasil. Share [...]
Madhulika
Assume I’m doing an analysis of my brand page, how do I know if my average engagement rate is good or bad? How do I get to know of an industry benchmark?
25 Jul 2012 07:07 pm (@@CMadhulika)
Steve
Good point. Until there are some benchmark available, you can use some rule of thumb.
50% engagement rate sounds a good performance, as one out of two users who've seen your posts considered them interesting enough to act on them.
Compare this ratio to email open rates for example, or display banners engagement rate.
25 Jul 2012 07:07 pm
More on metrics..by Prasant Naidu | @stuartgh (Plectic Ltd)
[...] above formula has been popular for a while but has some hidden problems that was recently listed by WiseMetrics. WiseMetrics, a company that is also in the business of measuring the ROI for Facebook brands has [...]
Stuart G Hall
Hi
Did I understand your formula correctly as the following, expressed using FB’s own terms, below?
‘Talking about this’/ number of your page posts/ ‘Reach’ x 100
29 Jul 2012 03:07 pm (@@stuartgh)
Stuart G Hall
Thanks Steve, I see the FB data in Insights in Page level data under 'Daily/Weekly/28 days Page engaged users' and for Reach under the same time ranges as Totals, which combines Organic and Paid reach.
07 Aug 2012 11:08 am (@@stuartgh)
Steve
Nope. We're using Facebook's own terms. The data may not be available directly on Insights, so you need to export Insights' data into Excel.
"People Talking about this" and "engaged users" are totally different metrics and we don't advise using PTAT in any engagement rate calculation as it contains flaws the same way the "Likes+comments+shares" has.
31 Jul 2012 11:07 am
Stas
Hi!
My name is Stas and I am community manager at russian social media monitoring service Yousan.ru.
You’ve done really great job in this post and we want ask your permision to translate it into russian and add some relevant to our market examples.
Certainly we will link to you as authors of the original one.
Thanks!
04 Aug 2012 01:08 pm (@https://twitter.com/youscan)
Steve
Hi Stas,
please go ahead.
Thanks for asking us and for the link to our article.
Have a nice day,
Steve
04 Aug 2012 10:08 pm
Measuring the Value of Social Media | Useful Social Media Blog
[...] current measurement of Facebook engagement for instance has been called into question. Wisemetrics have analysed the current Facebook metric and found it to be lacking in some areas, proposing that [...]
^_^
Tell me please, do I understand correctly: ‘users engaged’ we divide by the amount of posts, and then the result we divide by ‘users reached’ and then we multiply the result with 100?
I get in results – 0,50%. that’s weird
(961 / 23 / 8249 *100 = 0,50%)
is it wrong, isn’t it?
but if I divide the ‘users engaged’ by ‘the users reached’ and then multiply with 100 I get the: 11,64%
that’s sounds ok
what do you think about it?
14 Aug 2012 11:08 am (@Twitter)
Steve
Hi,
it seems correct to me.
I suppose this is due to the pretty high number of posts compared to the number of users engaged.
The second formula you use (users engaged/users reached) is totally acceptable though.
It doesn't take into account the number of posts, but it is a really minor flaw, especially if you're not using this metric as a comparison between different pages.
23 Aug 2012 10:08 am
О Facebook метрике «talking about this» | Блог компании Netpeak – раскрутка сайтов
[...] из Wisemetrics предлагают следующую формулу для подсчета engagement [...]
» Social Media Engagement Metrics { Vervely }
[...] calculations give too much weight on existing fans or likes. One of the beauties of social media is its ability to go past one’s existing networks. To [...]
Alejandro
I like the formula’s concept, but at the same time I’m quite confused. Which value do you use from the exported Facebook Data? daily? weekly? 28 days? do you use one specific day or do you use an average?
Thanks in advance!
25 Sep 2012 12:09 am (@@rancilyo)
Steve
Hello Alejandro,
You can use the formula for any date range, it would work, but we recommend that you use it on a longer period of time, like 28 days, because most users engage from time to time and not every week (unfortunately!).
The idea behind this metric is also to measure progress, so you may want to use the weekly value if you want to measure progress from one week to another.
25 Sep 2012 10:09 pm
Omkar Mishra
Something i have written on Similar lines to calculate Engagement Rate for Facebook:
http://csharpdflat.in/calculating-engagement-rate-on-facebook/
Let me know if it helps you guys
25 Sep 2012 11:09 pm (@csharpdflat)
MARINA
Hi Steve, Please We do not find the “engagenet user”? How we can get it? Where does it?
Kindest Regards
Marina
27 Sep 2012 11:09 pm (@@marinapulido)
Steve
Hello Marina, you can find "Engaged users" in the excel extract of your Insights, or through 3d party solutions using Facebook's API like ours.
27 Sep 2012 11:09 pm
Marwann
Engaged Users / Reached Users is more or less how you calculate Virality Rate (Post People Talking about this / Post Reach). A little confusing…
01 Oct 2012 12:10 pm (@marwannas)
Steve
People Talking About This and Engaged Users are really different metrics.
"People Talking About This" only includes people creating stories. Thus people looking at your pictures, playing your videos or tagging your posts, for exemple, are not included. But they are engaged nevertheless.
01 Oct 2012 12:10 pm
Liensdumatin 10/02/2012 « Ray's liens du matin
[...] Why The Current Facebook Engagement Rate Calculation Is Inaccurate | Wisemetrics – The News Fe… [...]
Mary
Question…
What is my edge rank perecentage wise if I have 135 likes and 42 fans talking about this ?
01 Nov 2012 09:11 pm (@Twitter)
Egle
Hey, thank you for clearing things out
However, I’m still missing something here. My numbers are following: 28 days engaged people – 36654, 28 days total reached – 655963, so if I put in in my formula
36654/655963*100=559%
Is this % number realistic?
If I include number of posts, then ROI is 186%, still very big….
21 Nov 2012 10:11 am (@Twitter)
Egle
OK, I guess I shouldn’t multiply by 100 while doing it on excel
, then I got 5,6%. Now seems ok 
21 Nov 2012 10:11 am (@Twitter)
Steve
Looks ok indeed!
26 Nov 2012 11:11 am (@@wisemetrics)
Donna
Hi Steve, i agree with you way of more accurately measuring engagement… however does the benchmark of good engagement rate then change? A 1-2% engagement rate when looking at the “Talking about this” / # of fans is considered good. In otherwords, if using your metric – how can I determine if I have a decent engagement rate vs. industry averages?
10 Jan 2013 04:01 pm (@Twitter)
Steve
Hi Donna,
There is no perfect solution right now.
A first step is to look at the raw number. A 40% engagement rate means 40% of those who have seen your content have clicked on it. Seems like a pretty good number isn't it ?
The last option is too look only at what we call "public interactions" (likes, comments, shares). They will give you an imperfect picture as you've seen before, but the biases affect all the Pages in more or less the same manner. Just keep in mind this is a raw estimate and should not be considered as an Engagement Rate, more a "Benchmark rate".
10 Jan 2013 04:01 pm
karen
Thanks for putting a better solution to the doubts I have and unclear understanding of the FB calculation. I am working on preparing a KPI for a client now. Do you have any suggestions on how I should peg a KPI?
Assuming xx is the current metric from your formulas above taken from FB insights (of which we are an admin), how much do you think I should increase by the end of 12 months?
21 Feb 2013 05:02 am (@Twitter)
Steve
Hi Karen,
sorry for this very late answer.
Unfortunately there is no easy answer to your question. I would suggest you have a look at the trend you've been experiencing the last year as well as the last few months in order to extrapolate.
Best,
Stephane
22 Mar 2013 10:03 am
Facebook in Links (Februar 2013) |
[...] Why The Current Facebook Engagement Rate Calculation Is Inaccurate [...]
Engagement significa compromiso
[...] [Wisemetrics] [...]
Pourquoi le taux d'engagement est un mauvais indicateur de benchmark | Emarketinglicious
[...] Il y a de cela quelques mois, j’avais déjà évoqué les nombreux biais qui rendait le calcul du taux d’engagement imprécis pour ne pas d…. [...]
Pourquoi le taux d’engagement est un mauvais indicateur de benchmark | The social guideline
[...] Il y a de cela quelques mois, j’avais déjà évoqué les nombreux biais qui rendaient le calcul du taux d’engagement impr&eacu…. [...]
Leave a Comment