Are they good or bad? Vanity metric or valuable metric?
“Why does Seth Godin not have comments? Seth is awesome, I want to be awesome…Do I need to get rid of my comment section as well?
And what about Copyblogger? And Michael Hyatt?
Dang. This blog thing is hard!”
^^^ Recent conversation I had with myself.
That conversation with myself led me to crunch some numbers.
Just last week we wrapped up the launch of our core product, 10ksubs.
And as the launch wound down, I started taking notes in order to write a detailed recap post like we did for the Spring 2015 launch. We got so much positive feedback on that article that we’ve decided to do a full recap of every major launch we do. That way we can share all the stuff that worked and didn’t work and lessons we learned so you can benefit from them as well. These recap posts also have a nice side benefit of forcing us to fully document what we do so we can replicate it with the next launch.
As I was taking notes and outlining the launch recap article, I ran across this tweet.
Earlier this week, I announced that I removed the comments section from my blog. Here’s why: http://t.co/zyY80gjLDj
— Michael Hyatt (@MichaelHyatt) January 15, 2015
How did I run across a 11-month-old tweet? I have no idea. #twitter
But that triggered a thought…
How much is a comment really worth?
Are people who comment on blog posts more or less likely to buy our products?
If they are more likely to buy, by how much?
Every article I read on the topic cited more internal-process-type reasons why the blog owners killed the comments than hard, analytical, reader-focused-type of reasons.
Internal-Process Reason: “We get lots of spam. That’s annoying.” (Meh, sounds lame.)
Reader-Based Reason: “Our readers genuinely hate them.” (Ok, sounds legit.)
Had anyone run the numbers to see if comments actually HELP their readers? I spent the next 5 hours Googling around trying to answer that question. And I got nothing.
Had no one run a legitimate split test to figure this out?
So I tweeted the smartest conversion rate optimization person I know, Devesh, to ask him if it was possible to do it.
Here is what I sent him…
Long story short…
It was possible! (woot)
So a few Skype calls with Jeff (Videofruit technical wizard) and 8-ish hours of slaving over Excel spreadsheets and pivot tables later, we had the answer to our question.
When we did the math, we found that someone who commented on at least one blog post in the 60 days leading up to the 10ksubs launch was 5x more likely to buy than someone who didn’t.
Here is how we did the math…
(Note: If your eyes glaze over when someone pulls out a calculator, feel free to skip this part.)
First, we added up the total number of people who bought the course: 387
Then, we made a list of all the people who commented on blog posts in the 60 days prior to the launch: 759
Then, we cross-referenced the buyers list and the commenters list to see how many of the commentators bought: 51
Then, we checked to see how many people went through the actual launch sequence: 26,640
Then we did some division:
51 out of the 759 commentators bought. That means 6.71% of all commenters turned into customers.
Then we removed all of those folks from the data set (the 51 commenting customers and 701 non-buying commenters) to figure out the buying % across the entire list. A comparison of these two numbers would then show us if commenting folks buy at a higher or lower rate.
Here are the general list numbers:
362 out of the 26,460 non-commenters on the launch list bought the course. That means 1.36% of people who went through the launch sequence but didn’t comment bought.
If your head hurts, that’s ok. Mine does too.
Here is the short version:
6.71% of the people who commented bought the course.
1.36% of the people who didn’t comment bought.
That means commenters are 5x more likely to buy than regular subscribers.
If you extrapolate further, things get even crazier.
The average value of someone who is on the VF email list and actively comments is: $97.43
The value of someone who is on the VF email list and doesn’t actively comment is: $19.83
What does all this mean?
Honestly, I don’t know. My head still hurts from doing the math.
I do know one thing: These numbers are extremely compelling.
After seeing these numbers, we ran a few more calculations on other engagement points: live workshops, selling webinars, installing our app, etc.
The goal was to see if these other engagement points increased the chances of people buying like they did with comments on blog posts. And of course, the answer was yes. For each of the engagement points, conversion percentages went up drastically.
Moral of the story: TBD
For the next 10-ish days I’m working on writing the Fall 10ksubs launch recap post. It’s going to be large and I think it’ll be the most helpful post ever written on this blog.
Keep your eyes out for it in a few days.
One more thing: What do you want to know about the launch? What’s one thing you’re curious about (yes, we’ll share revenue), something you want to learn how to do or a number you’d like to see broken down?
Share it in the comment section below and we’ll do our best to answer them all in the article.
Footer Note #1: This wasn’t a true split test and the numbers are not statistically relevant. But they are extremely compelling. I’d like to run a true A/B test on this in the Spring 2016 launch.
Footer Note #2: I absolutely love both Michael Hyatt and the crew at Copyblogger. They are good people. Follow them.