Exploring the differences in social media measurement tools

For the last couple of years, I have used a combination of three different social media measurement services to help me gather information about Bemidji State’s presences on Twitter and Facebook — I started by using CrowdBooster, added Sprout Social a bit later, and also spend some time with Twitter’s own analytics and Facebook Insights.

I always ended up incredibly frustrated by the process, because while each service provided some small interesting element that the other didn’t, the basic data was never consistent across platforms.

Let’s take a look at Twitter, and I’ll show you what I mean. Let’s look back to the week of April 26-May 2, 2015 — Finals Week for BSU, and the week leading up to Commencement on May 8, which meant it was a heavy week for us. So let’s look at what each of these three measurement tools shows for our activity during that week.


You can click to enlarge and see everything, but to hit the high points Crowdbooster claims that from April 26-May 2, we tweeted 127 times with 75 favorites, 68 retweets, and 4 replies.

Sprout Social:

This is from Sprout Social’s brand-new Twitter reports, which they launched on July 30. Again, click to enlarge; it shows that we tweeted 82 times, and elsewhere in the report that we had 123 favorites, 70 retweets and 10 replies.

Twitter Analytics:

I exported Twitter’s data as a .csv and put it into Google Sheets for this report. Twitter’s data shows 82 tweets, 113 favorites, 69 retweets and 7 replies.

It doesn’t take much data analysis to see that these numbers are all over the board. Sprout Social and Twitter agree that we sent 82 tweets; Crowdbooster’s number of 127 isn’t even in the same neighborhood. Favorites are different across all three reports, as are retweets and replies, but the Crowdbooster numbers are in some cases lower in spite of its claim that we sent 45 more tweets — it reports 4 replies, for instance, while Sprout reports 10 and Twitter reports 7.

For a long time I have just written this off as “this is why you cannot take social media metrics at all seriously.” It never occurred to me until recently that at least Crowdbooster and Sprout were likely measuring entirely different things — in ways that on the surface are essentially invisible. It’s a puzzle to be solved, and while Google searches have turned up feature comparisons of Sprout vs Crowdbooster, I haven’t found anything that dives into the differences in what the tools report.

So if this is a puzzle, let’s try to solve it.

I have created three Google Sheets for this examination containing exports of the data each of the three services generated for my April 26-May 2 tweets: one for Crowdbooster, one for Sprout Social and the previously-mentioned one for Twitter Analytics. These will be our starting point.

The first discrepancy I want to explore is the difference in the number of tweets being reported by Crowdbooster, 127, as compared to Sprout and Twitter, which both agreed was 82. The first step there is to make sure Sprout and Twitter really are in agreement about what we sent, and the numbers are not just coincidentally the same. To do this, I simply reviewed both lists of tweets side by side in two Sheets windows. They did, in fact, completely agree; every tweet in the Twitter record was also listed in Sprout, and vice versa. So, to explore why Crowdbooster is reporting so many tweets, I did the same thing and compared that list side-by-side with Twitter’s.

Those lists are more difficult to compare. Twitter’s list starts with the most-recent tweet at the top, then descends to the oldest tweet in the time period you’re measuring. Crowdbooster starts with the oldest tweets at the top. Immediately, a discrepancy – Crowdbooster did not include this tweet in its list which was on both the Twitter and Sprout Social lists. But, that tweet was auto-posted by Sprout Social at 8 p.m. on April 25 — the day before we said we wanted to begin our measurement. It shouldn’t have been on any of the lists, but both Twitter and Sprout had it.

Five tweets into the list, it’s clear why Crowdbooster is reporting a heavier tweet load – it’s including retweets. I can see both sides of the argument on whether retweets count as “our” content – on one hand, it’s not original content posted by us, but content that originated elsewhere. On the other, we’ve made a concious decision to put those tweets into our timeline and share them with our followers. This is a decision that is actively made for every retweet, and retweets are something that we discuss in our overall talks about how we’re using Twitter. By giving someone a retweet, we’re bringing them into our timeline and making them part of our content stream. I believe they should count as “our” tweet for the purposes of counting our monthly activity.

But, Twitter doesn’t; my next step was to count the retweets in Crowdbooster’s list and see if there were 45. If there are, the entire discrepancy between tweet activity can be attributed to our RTs — which would be fantastic, because segmenting out that information could be incredibly useful.

However, only 40 of our tweets started with an RT during that week. Five are still missing. And, I discovered that Crowdbooster is including tweets we deleted (like this one, which had a typo (you’ll have to take my word for that since the tweet no longer exists) and was immediately replaced with this one) in its total. I counted five deleted tweets that were being included in the Crowdbooster report – that means the entirety of the discrepancy between Crowdbooster, Sprout and Twitter has been identified.

Twitter and Sprout Social are including only original tweets that are currently active and sourced from your account in its list of your activity for the month. Crowdbooster is including all of those, plus any retweets and it’s including deleted tweets.

This discovery actually leads to a significantly more-important question – if we deleted a tweet in April, why is Crowdbooster still able to find that tweet and include it in a report that I ran on June 30? Twitter’s own support documents say:

  • When you delete a Tweet, it is removed from your account, the timeline of any accounts you follow, and also Twitter search results.

Obviously, these tweets are not actually deleted in the sense that anyone thinks “deleted” means. Twitter is preventing a deleted tweet from being displayed on your timeline, but it obviously still exists on Twitter’s servers if Crowdbooster can parse and find it. That’s a problem.

I also discovered that Twitter and Sprout don’t necessarily report times of tweets accurately – for example, this tweet was posted at 10 p.m. on April 30. Crowdbooster reports it as having been posted on April 30. Twitter reported it in its analytics as being posted at 3 a.m. on May 1. That’s +5 hours, which indicates a time zone might be set incorrectly somewhere; however, all are well – Twitter is correctly set to GMT-5, as is Crowdbooster, and Sprout Social is correctly set to UTC-6. In some cursory explorations I haven’t yet been able to find out why this is. It bears further exploration.

However, none of the mystery time zone issues or philosophies behind whether RTs should count as “our” content are as important in my mind as discovering the fact that deleted tweets are, in fact, not deleted.

MnSCU social media update for August 2015

Here’s the Storify recap of my Twitter stream from Monday, Aug. 3, with my monthly (well, mostly monthly) update on Twitter, Facebook and Instagram follower totals for the seven state universities in the Minnesota State Colleges & Universities system.

MnSCU Change Leadership workshop, Day 1

Today I wrapped up Day 1 at the Change Leadership workshop to support the Minnesota State Colleges and Universities system office’s Charting the Future strategic planning inititaive. The plan has been in development for several years, but hit a rough patch last year when faculty unions across the state revolted against it, leading to a series of no-confidence votes for the system’s chancellor. This Star-Tribune editorial from April covers the system’s work to restart the initiative after efforts were made to bring the faculty unions back into the project.

I haven’t been involved in Charting the Future up until this point, and part of the reason I wanted to explore this now was to get a different view of how the system would be moving forward with this initiative after what felt like a lost year in 2014.

Today’s first day of the workshop focused on organizational psychology and change management; there was a lot of incredibly interesting information, some things I want to research when I get home, and a book written by one of the presenters that I bought for later reading.

Plus, there was a taco bar for lunch — and nothing is bad when you have a taco bar.

Looking forward to Day 2 tomorrow.

What I’m Playing: Mass Effect

Somehow, BioWare’s Mass Effect series is one that slipped past me during my gaming time on Playstation 3; it seems as good a time as any to backtrack and fill in that row on my Gamer Merit Badge sash (which really should be a real thing). Sony’s PSN Flash Sale brought me Mass Effect for $4.50; I found a used copy of Mass Effect 2 at GameStop for $4, and I’ve borrowed Mass Effect 3 from a friend here in town. I’m tackling them in order, starting yesterday.

So far, I’m not far enough into Mass Effect to have a good feeling about it either way — I have just reached the point where Shepherd has interacted with the Prothean beacon and has awoken back on his ship after being rendered unconcious. It’s a lot of dialogue options and infinite frustration with how difficult it is to use the game’s early sniper rifles.

I’ll report back on how things are progressing as I make my way through these games.

Blogo “Blogger Edition”

Blogo released a big update last week; let’s try out their new embed option for Instagram.

(hey that’s neat)


I know Storify works…

#BartlettMetrics for March, 2015

 I’ve not been great about backtracking to ab.com and posting an update about my monthly #BartlettMetrics reports on social media follower updates for MnSCU’s state colleges and universities. For this month, I thought I’d try using Storify to save an archive of the tweets I posted about the update.

Destiny is not its concept art

Back in January, Erik Kain wrote a piece on Destiny at Forbes called “Why Destiny was the best and worst game of 2014.” I just read it today, though, and spent some time discussing it afterward with my go-to teammate for my adventures in this game.

In brief, Kain says Destiny is the best game of the year because:

The gameplay is terrific, and the progression system, however bizarre and confusing it may be, works to keep you playing and grinding away to your heart’s content.

However repetitive the levels can get, there’s no denying that scrapping your way through a challenging mission with a pair of Guardians at your side, and then maybe getting some sweet loot at the end, can be a lot of fun. All the little pieces here, taken individually, are fantastic.

This sounds like a game you’d want to play. And, he’s not wrong — the shooting mechanics in this game are tight, the controls are slick and easy to manage, the game is visually beautiful, and the music is excellent. Existing in the game’s environment as a player is incredibly fun.

However, on the flip side of that “game of the year” coin, and using a series of concept-art illustrations as supporting material, it’s the worst game of the year because “…the game Bungie advertised early on was far more imaginative and inspiring than the final product.” 

That’s obviously a debatable, subjective point. Much of the concept art is indeed significantly different from what can be seen in the version of Destiny that we now get to play. It does not stand to reason, however, that different automatically means better.

This piece in particular was pointed out in the Forbes story:

About it, Kain writes:

Perhaps the most frustrating of all, a giant frog with goblins on its back. And a Guardian wielding a broadsword. Why are these things not in the game?

Well, because it’s a giant frog, and fighting a giant frog in Destiny as the game exists right now would be incredibly stupid. And, given how the universe has been constructed and the direction Bungie seems to be taking the game, fighting a giant frog in Destiny at any point in the game’s future would still be incredibly stupid. Nothing in that illustration remotely resembles anything that’s in the game. And while there are things in the sky on Venus that look like dragons, that doesn’t mean an honest-to-goodness giant frog belongs anywhere in this game as it exists today

This is the folly of getting lost in a sea of concept art that is floating around on the Internet completely void of context. Hundreds of concepts are developed for games like this; some of them make it through to the final game, and a significantly larger percentage get pushed to the side. Some likely get pushed to the side because the scope and scale of the game changes, and reality simply demands that not every awesome thing can go into a game. Some are likely pushed aside because they’re terrible. But even the ones that are amazing may not necessarily work with the current direction of the game and still find themselves on the sidelines. Maybe the concept is so good that it’s revisited in the future as part of an expansion, or as part of the already-in-development Destiny 2. Or maybe Bungie takes that idea and eventually builds another game out of it entirely. The ultimate lesson to be learned in this sea of beautiful concept art is that we’ll probably never know which category any particular piece falls into unless something happens to look familiar in a future expansion.

As an aside, this is why Dark Horse’s The Star Wars project — an eight-issue mini-series based on George Lucas’ first draft of Star Wars with visuals based on early Ralph McQuarrie concept art — has always bothered me. The source material for this series all should’ve been left in the scrap heap of history — interesting relics of what later became Star Wars, but not anything that should’ve been the basis for a full-blown adaptation. There’s a reason Lucas moved on from those concepts — some of them were really bad. This is particularly true of the story elements — a lot of what was in the main plot for The Star Wars involving trade disputes and political gamesmanship actually became most of the main plot for Star Wars Episode I: The Phantom Menace. And, as you recall, that movie was terrible

Fans’ current obsession with Destiny concept art feels like a similar exercise in futility — they’re latching on to things that may look cool in a vacuum but lacking in any sense of how those things may or may not be remotely connected to the current state of the game and its universe, or to the plans that guide how that universe will grow and change in the future. But it helps fuel the conspiracy theories about how the game was gutted during the final year of its development, and so these images are added to the crackling bonfire of reasons Destiny is terrible.

As for me, I’ve been playing since the day the game launched and my friend can attest to the fact that I still giggle like an absolute idiot every time I stick a fusion grenade to a Wizard’s head and blow it up. This game is incredibly fun, and it’s a level of fun that some gaps in a story can’t diminish. I still enjoy looking at this art and appreciating it for the creativity and vision; but I refuse to make the false connection between these images and some imagined deficiency in the game.

Saving old buildings

There’s an old building in downtown Bemidji that used to house the Masonic lodge that has been vacant for a long time. It was announced earlier this week that Bemidji’s Watermark Art Center has received a grant that will allow it to buy — and then demolish — the building. You can read about that here at the Bemidji Pioneer.

The immediate reaction is totally predictable — and it falls along the lines of “why put in the effort to save buildings like this when you can just spend three months replacing it with something made of styrofoam and spackle?”

I just wish we could maybe drill into Americans’ heads that if you want to save buildings like this, efforts have to be made to take care of them all along. Too often you have these remarkable old places that are just left derelict for decades, and then saving them becomes this amazingly expensive proposition. But had they only found some way to take care of them all along… It reminds me of that fabulous old hospital building in downtown Ely  That place should be amazing, and it’s just sitting there empty and rotting away. But you watch, when the time comes that a decision is made to tear it down, people will come out of the woodwork trying to save that building. But right now nobody seems to give a crap about it.

People like to point to Europe and all of the hundreds-of-years-old buildings they have over there as an indication that we’re doing it wrong. But what they totally gloss over is the fact that those centuries-old palaces and castles over there have had *armies* of caretakers for each of those centuries. They stand because people have made great and enduring efforts to make sure they stand, not because they’ve just magically willed it to be so. THAT is what we haven’t gotten right — it’s not that we lack the desire to preserve history, but not enough people have the will to do the work along the way to make sure it happens. They just want to swing in at the end of a building’s life, waive a wand and save it, regardless of its condition or ability to be saved.

Previewing The Birthday Massacre’s “Superstition”


A couple of days ago, a five-minute video featuring 30-second samples from each of the 10 tracks on the upcoming The Birthday Massacre album Superstition, which releases on Nov. 11 (pre-order from iTunes), was uploaded to YouTube. You can check out the preview here:


Here’s what I took from the preview of each of the album’s 10 tracks, to the extent that you can take anything out of some random 30-second excerpt from a song:

01. Divide (0:00)
This song reminds me quite a bit of Down, the second track on TBM’s last album that’s probably my favorite thing they’ve ever done (it’s not an exaggeration to suggest I’ve probably listened to Down over a thousand times). The sound is different – Down is more hard-driving – but the lyrics seem similar, both thematically and structurally. I’d like to hear more. At the tail end of this snippet there’s a lyric “…rising up away from the Earth” that takes me right back to some ‘80s synth pop one-hit wonder that as I write this is driving me insane through my inability to properly recall it. To this mystery song I say: I will find you.

02. Diaries (0:30)
The lyrics that are in this snippet are a little weak: “Close your eyes | You’re dreaming, so am I. You know I can’t stay here and I won’t pretend | All that begins never comes to an end.” I won’t pass too much judgment on this until I hear the whole thing, but at first blush it feels like this could be a “skip track” candidate like One Promise turned out to be on their last album. 

03. Superstition (1:00)
When I first heard this I swore TBM was covering Girls Just Wanna Have Fun. Seriously, listen to this and replace the lyrics with “I come home in the morning light | My mother yells when you gonna live your life right…” It’s almost perfect. I want to hear the rest of this song, but I’m going to be entirely unable to shake the vision of ‘80s Cyndi Lauper when I listen to it. It’s definitely distracting, but there’s no way to know yet if that’s going to be good or bad.

04. Destroyer (1:30)
Destroyer seems to be the obligatory “monster voice” track that TBM has been busting out sparingly on its recent albums; the sound apes the monster-voice parts of Lover’s End from the band’s second album, Violet. It’ll probably be cool, but it feels like the kind of sound you could’ve called for this album the minute it was announced as being a thing. That’s not a slam; this type of thing is one of the band’s signature sounds, and they always make it work. I’m looking forward to this track.

05. Surrender (2:00)
This is one of my favorite of the 30-second previews; somehow it reminds me of Always  which ranks highly on the list of my favorite TBM tracks that I’ve never actually compiled. 

06. Oceania (2:30)
This feels like one of those songs that I’m not going to love, but it’s not going to be a track-skipper either. It sounds like a lot of the other things they’ve done, but from this little sample anyway there isn’t a hook that made me want to stop and listen to it. In fact, although it’s the sixth track in the album it’s the 10th out of 10 snippets I’ve written about for this particular post; I just kept glossing over it while thinking about the other samples. I’m quite interested to know how I’ll feel about this song after listening to the entire thing in the context of the complete album a few times over. Because I’ve just written five times more about this than I did about the immediately preceeding track, which I told you I liked quite a bit.

07. Rain (3:00)
This is going to be the first single from Superstition, and the band has already teased that a video is coming. They also released an instrumental-only track of Rain as one of the promos for its PledgeMusic campaign that’s supporting the album. You can listen to that here:
…and there was another teaser from the campaign that featured some of the behind-the-scenes work on the vocal mixing for this track, which you can also find on YouTube, here:

…so hearing a bit of all of that coming together in the final track is pretty cool. This should be a good song.

08. Beyond (3:30)
I can envision playing this song a ton, and I will say right now that I’m probably going to like singing along to this song in my car. I’m not really sure what else to say about it beyond that.

09. The Other Side (4:00)
This is an interesting track to me in that it seems to have the melodic-to-more hard-driving transition that would have made me guess this was going to be Superstition’s monster-voice track, only without evidence of the monster voice. After listening to it a few times, it sort of reminds me of their Red Stars track from Walking with Strangers — which is good because I think Red Stars is one of the band’s best songs.

10. Trinity (4:30)
This is an instrumental-only snippet, the only one of the 10 previews in this sampler to not include vocals. Awesomely, this sounds like it’s been ripped straight from the Blade Runner soundtrack (listen to the end title theme for Blade Runner) and seems to perfectly encapsulate the heavy ‘80s influences that run through most of what The Birthday Massacre has done with their music (and which you can hear repeatedly in nearly every track of this album — I’ve only mentioned a few of the ways this pulled me back to that era as I listened to this sampler. Seriously, it’s in just about every track). I’m very intrigued by this and will be curious to hear the entire track — I’m definitely wondering if there are lyrics or if this will end up being a purely instrumental piece (I could Google this and get an answer, I’m sure, but I choose to not find this out until the album drops on Nov. 11).

Overall this feels like it could be a strong album; as a followup to Hide and Seek from 2012 it’s got its work cut out for it, to be sure, but other than Diaries there doesn’t seem like there’s a particularly weak song in the bunch. I’m very much looking forward to Superstition’s Nov. 11 release date, and I fully expect to listen to that on a constant rotation — and not much else — between then and Dec. 13, when I’m going to have my first chance to see TBM in concert in Minneapolis.