Bot traffic is a plague.
Sometimes is astonishing to think about think about the impact it can have on the online ad industry.
Now, a new video created by Forensiq shows just how quickly just a single infected computer can add to the tally of worthless clicks.
Remember, this is just one computer over a 24-hour period.
Read more about this video from Business Insider and the Wall Street Journal.
Some are already hailing private marketplaces as the “future of programmatic,” while others are lamenting their growing pains. There are “big changes” coming in the world of ad fraud. This viewability stuff needs to get sorted out, because there are some killer metrics waiting.
Here are a few highlights from some of the industry news we came across this week.
Reading The Information's piece on Facebook's reported re-introduction of the Atlas ad-serving technology , I wondered – Does the market really need six or more full stack adtech solutions? Google is the undisputed leader in the field – it's spent…
Major talking points this week involved the continued uncertainty around the transition to a viewability standard, the absolute certainty around the growth in spending expected across the programmatic landscape, and what publishers are going to have to be ready for to be successful going forward.
Posted by Cyrus-Shepard
The folks at Groupon surprised us earlier this summer when they reported the results of an experiment that showed that up to 60% of direct traffic is organic.
In order to accomplish this, Groupon de-indexed their site, effectively removing themselves from Google search results. That's crazy talk!
Of course, we knew we had to try this ourselves.
We rolled up our sleeves and chose to de-index Followerwonk, both for its consistent Google traffic and its good analytics setup—that way we could properly measure everything. We were also confident we could quickly bring the site back into Google's results, which minimized the business risks.
(We discussed de-indexing our main site moz.com, but... no soup for you!)
We wanted to measure and test several things:
Here's what happened.
The fastest, simplest, and most direct method to completely remove an entire site from Google search results is by using the URL removal tool.
CAUTION: Removing any URLs from a search index is potentially very dangerous, and should be taken very seriously. Do not try this at home; you will not pass go, and will not collect $200!
After submitting the request, Followerwonk URLs started disappearing from Google search results in 2-3 hours.
The information needs to propagate across different data centers across the globe, so the effect can be delayed in areas. In fact, for the entire duration of the test, organic Google traffic continued to trickle in and never dropped to zero.
In the Groupon experiment, they found that when they lost organic traffic, they actually lost a bunch of direct traffic as well. The Groupon conclusion was that a large amount of their direct traffic was actually organic—up to 60% on "long URLs".
At first glance, the overall amount of direct traffic to Followerwonk didn't change significantly, even when organic traffic dropped.
In fact, we could find no discrepancy in direct traffic outside the expected range.
I ran this by our contacts at Groupon, who said this wasn't totally unexpected. You see, in their experiment they saw the biggest drop in direct traffic on long URLs, defined as a URL that is at least as long enough to be in a subfolder, like http://followerwonk.com/bio/?q=content+marketer.
For Followerwonk, the vast majority of traffic goes to the homepage and a handful of other URLs. This means we didn't have a statistically significant sample size of long URLs to judge the effect. For the long URLs we were able to measure, the results were nebulous.
Conclusion: While we can't confirm the Groupon results with our outcome, we can't discount them either.
It's quite likely that a portion of your organic traffic is attributed as direct. This is because of different browsers, operating systems and user privacy settings can potentially block referral information from reaching your website.
After waiting 2 hours, we deleted the request. Within a few hours all traffic returned to normal. Whew!
If the time period is short enough, and you used the URL removal tool, apparently not.
In the case of Followerwonk, Google removed over 300,000 URLs from its search results, and made them all reappear in mere hours. This suggests that the domain wasn't completely removed from Google's index, but only "masked" from appearing for a short period of time.
In both the Groupon and Followerwonk experiments, the sites were only de-indexed for a short period of time, and bounced back quickly.
We wanted to find out what would happen if you de-indexed a site for a longer period, like two and a half days?
I couldn't convince the team to remove any of our sites from Google search results for a few days, so I choose a smaller personal site that I often subject to merciless SEO experiments.
In this case, I de-indexed the site and didn't remove the request until three days later. Even with this longer period, all URLs returned within just a few hours of cancelling the URL removal request.
In the chart below, we revoked the URL removal request on Friday the 25th. The next two days were Saturday and Sunday, both lower traffic days.
Likely, the URLs were still in Google's index, so we didn't have to wait for them to be recrawled.
Here's another shot of organic traffic before and after the second experiment.
For longer removal periods, a few weeks for example, I speculate Google might drop these semi-permanently from the index and re-inclusion would comprise a much longer time period.
Moz community member Adina Toma wrote an excellent YouMoz post on the re-inclusion process using the same technique, with some excellent tips for other, more extreme situations.
Big thanks to Peter Bray for volunteering Followerwonk for testing. You are a brave man!
Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!
Get Social