Report Finds X May Be Inflating Its Ad Performance Results (original) (raw)

There’s an anomaly in X’s web traffic data that’s been hurting my head in recent months.

As per data from SEMRush, which measures referral traffic from Google, X saw a very sudden, and massive increase in web referrals in April last year.

X traffic jump

As you can see in this chart, organic traffic to "www.twitter.com" went from around a billion per day on average up to April 5th,2023, to more than 2 billion on April 7th. And since then it’s increased even further, and is now up to an average of around 3 billion per day.

Which is a huge increase, and there’s nothing that I can find to suggest why Twitter, or X, would have suddenly seen such a big jump in web referrals at this time (Note: I asked SEMRush about this, and they said that a change in its reporting methodology correlates with this spike, though it still seems like a big jump).

Maybe, then, I figured, it could be due to X limiting access to its API (which went into effect at the end of March), which could have then prompted a big uptick in researchers accessing the platform in different ways. Maybe it’s related to the rate limits that X implemented to stop AI companies from scraping its platform, though those only came into effect in July.

Worth noting, too, that "www.x.com" has also seen a similar traffic spike very recently, going from organic traffic of 390k on February 14th, 2024, to 1.35m just a day later.

X traffic jump

The Super Bowl was on the 12th, so that’s not it, and it’s also a day after Valentine’s Day. And the change is huge here, a 3.4x multiple.

So why is this happening? What’s changing within X’s system, or Google’s, to drive such big shifts in these numbers?

Maybe, in the case of x.com at least, the spike is a result of the company's ongoing work to switch over to the new domain?

It's also worth noting that other reporting providers have not recorded the same increases, and in fact, show that X traffic, on both web and mobile, is in decline, year-over-year.

As per this chart from Similarweb:

Similarweb X.com traffic

None of these systems has 100% data access, but both generally provide solid indicators of trends and general performance. And as noted, SEMRush has explained that they did start tracking "SERP Features" in April 2023, which could explain some of the volatility in its figures.

But it just seems not quite right, which has made it hard to get a real read on X's true performance.

Then today, I saw this report from Mashable, which, quoting data from ad measurement provider CHEQ, found that of all of the referral traffic driven via X ads during the weekend of the Super Bowl, almost 76% of it came from sources that CHEQ identified as likely fake.

As explained by Mashable:

“CHEQ monitors bots and fake users across the internet in order to minimize online ad fraud for its clients. The company accomplishes this by tracking how visitors from different sources, such as X, interact with a client's page after they click one of their links. The company can also tell when a bot is passing itself off as a real user, such as when a fraudulent user is faking what type of operating system they are using to view a website.”

Now, while there is definitive method here, the findings are also anecdotal, as CHEQ only has access to this type of insight for its own client list. But for comparison, CHEQ also reported that the exact same measurement report for last year’s Super Bowl found that only 2.8% of the traffic from X was likely fake, out of 159,000 visits.

For clarity, this year, CHEQ’s analysis looked at 144,000 visits to its clients' sites that came from X during Super Bowl weekend,

So could X be inflating its numbers with fakes and spam sources, in order to boost its own reach figures, as a means to potentially lure more ad spend?

That still seems unlikely, given the business risk of doing so. But then again, X owner Elon Musk did make fakes on the platform a key focus, as part of his efforts to get out of buying it, which was ultimately rejected by the courts.

Back in July 2022, as Musk sought to renege on his $44 billion offer for the app, Musk argued that the platform wasn’t actually worth that price due to the high amount of bot profiles, which Twitter had continually included in its active user figures.

Twitter had long held that the number of fake profiles in the app didn’t exceed 5% of its total mDAU count, based on its own sampling. But Musk claimed that it was actually much, much more, with his own team’s analysis finding that around 33% of Twitter’s active profiles were likely fake. Musk eventually settled on it being a more modest 20%, while noting that it was likely much higher.

20% fake/spam accounts, while 4 times what Twitter claims, could be *much* higher.

My offer was based on Twitter’s SEC filings being accurate.

Yesterday, Twitter’s CEO publicly refused to show proof of <5%.

This deal cannot move forward until he does.

— Elon Musk (@elonmusk) May 17, 2022

So Musk himself claimed that at least 20% of X’s active usage was via bots and spam. Yet since taking over the platform, Musk has made no mention of this, instead quoting “record high” usage figures, while also claiming to have defeated bots in the app.

And seemingly, both of these stats can’t be true.

For example, if bots accounted for 20% of Twitter’s mDAU count at the time of Musk's purchase, as Elon claimed, that would mean that X would have had to eradicate around 50.6 million bot accounts to clean it up and start afresh, without bots included in its active user figures. X’s user count has since climbed from 238 million daily actives, to more than 250 million today, a count that Musk initially reported in November 2022, just a month after he took over at the app. Which would mean that, accounting for bot removals, X would have had to have added around 62 million more users in that short period after Musk took over.

The platform has never added more than 30 million more actives over a 12-month period, so either X saw astronomical growth as a result of Elon taking over, or Elon’s bot estimates were wrong, and were likely juiced as part of a bid to wriggle out of the X deal.

Or X still has a heap of bots, around 20% of its user base, and Musk and Co. have simply opted to keep counting them in its usage stats, just as Twitter management allegedly (according to Musk) did before him.

But then again, X does also have ad verification partners who are able to provide third-party analysis on this front, and would likely be able to confirm whether X ads are being seen by real people or not. And if X is actually bulking up its numbers with bot traffic, those verification partners would detect this.

Right?

Well, it depends on what, specifically, each of its verification partners is measuring.

So, for the most part, X’s ad verification partners are focused on ad placement, not on audience verification, and they only do this for commissioning partners, not all X traffic. So it is possible that they wouldn’t be able to provide definitive data to dispel this concern either.

Does that mean that X is juicing its numbers, in an effort to make its performance seem better than it is?

No, it doesn’t. Again, all of these data points could have alternative explanations (again, SEMRush has provided more insight on this), and because CHEQ only has access to a selection of X traffic data, it’s not clear if this is happening across the board.

The real insight will come from your own ad performance data, and what you’re seeing. If you’re noting that your X insights include a lot of clicks and engagement, but you’re not seeing the same reflected in your own analytics and performance data, then it could be that something’s not right, and that X ads could be gathering bot traffic.

Essentially, as with all things digital marketing, individual results will vary. If you’re still seeing strong ad performance on X, it’s probably worth sticking with it.

But it could be worth keeping an eye on your X performance metrics moving forward, in case of anomalies.