More Light, Less Heat on Click Fraud at SES

In past meetings of the Search Engine Strategies conferences, the session on click fraud has often resembled a steel-cage grudge match, with representatives from click-auditing firms and SEM agencies holding up examples of the major search engines’ high-handed indifference to their marketer customers’ concerns, and mouthpieces for the engines asserting that they were doing all they could to detect and root out bogus clicks—without offering any substantiating evidence.

By comparison, the session on “Auditing Paid Listings & Click Fraud Issues” at SES New York earlier this month was more like Wimbledon than the World Wrestling Federation.

Part of that difference may have been due to a change in the roster. Rather than pitting a tag team of click auditors against the major engines, the dais this time had only one, Tom Cuthbert, president and CEO of Click Forensics. He was joined by Shuman Ghosemajumder, Google’s business product manager for trust and safety and something of a “Mr. Invalid Clicks” at these events, and by Reggie Davis, making his first appearance at a search gathering since being named Yahoo! Search Marketing’s vice president of marketplace quality. Also participating in the discussion was John Marshall, CEO of Click Tracks, a Web analytics firm.

The change of structure seemed to indicate a desire to get away from the harsh accusations of past years and get down to talking in greater detail about where click fraud is being successfully detected and where it’s still an active concern. That more level tone suits the current agendas of both Google and Yahoo!, who in the weeks leading up to the conference were more forthcoming than they’ve ever been about the scope of the problem on their search networks.

As a result, Ghosemajumder was able to assume that his audience had already absorbed the information Google had released on the levels of detection it deploys to spot invalid clicks and what those filters find: namely, that less than 10% of all the clicks on its network are flagged as invalid by automated detection and not billed to advertisers at all, with a further 0.02% detected after billing and credited to marketers’ accounts.

Google deploys a three-stage system for detecting fraud, using real-time automated filters to pick up suspect click patterns on its search network and offline traffic analysis to flag problems on its AdSense network of partner Web sites. If those proactive levels don’t pick up all the suspect clicks, then Google deploys reactive live investigation, usually in response to an advertiser report.

As Ghosemajumder explained it, Google has a strong incentive to spot as many invalid clicks as it can during the first proactive stage of this detection process. Catching and filtering fraud in real time isn’t possible in most other industries subject to fraud; credit card companies can detect fraud after the fact, but that still carries some cost, in terms of deactivating cards, issuing new ones and handling refunds to consumers and chargebacks to merchants.

As a result, he said, Google purposely sets its click detection levels high to make sure invalid clicks don’t go through and get charged to advertisers’ accounts—even if that means the company fails to bill advertisers for some perfectly fine clicks. “We cast our net wide enough to have a high confidence of catching the actual malicious clicks,” Ghosemajumder said. “The clicks themselves aren’t blocked. If a legitimate click gets detected as invalid, that click can still wind up converting for the advertiser.”

The process of detecting invalid clicks differs from spam e-mail detection in one fundamental way, Ghosemajumder told the SES audience. In spam filtering, false positives are an unacceptable result, and sending a legitimate e-mail to a spam folder is as wrong as letting actual spam get through.

But Google would rather err on the side of caution and judge some valid clicks as bogus, even though the company loses revenue by not charging for these false positives. “That’s okay from a competitive standpoint,” Ghosemajumder said. “We ultimately compete on [return on investment]. We compete on the basis of results that we deliver for advertisers. So if we’re giving some legitimate clicks away for free, we’re simply providing an enhanced ROI for advertisers.

“We want to see more over-reporting [of click fraud] rather than under-reporting.”

Ghosemajumder said that while publishers taking part in Google’s AdSense network would also take a revenue hit from these false positives, that risk is “part of the bargain” of being in the network. “We’re going to look after our advertisers first and foremost because they’re the ones who fund the network,” he said.

That argument seems designed to rebut past criticisms that the big search engines lack a financial incentive to attack click fraud because they stand to benefit from the revenue it brings in.

Of course, saying that over-rigorous fraud detection makes good business sense doesn’t make it so. But as further proof of Google’s good intentions, Ghosemajumder pointed to a number of measures the company has begun evangelizing about that should help reduce the instances of real or perceived fraud. These include the use of auto-tagging on AdWords ads, giving each click a unique ID; that way, advertisers will be able to distinguish an actual click from a simple page reload, something they can’t do using only their Web logs.

Last year, Google also introduced reporting on both the number and percentage of invalid clicks in advertisers’ campaigns, Ghosemajumder pointed out. The company plans to launch enhanced click reporting this quarter and will roll out an online resource center on invalid clicks. Google will also host an Invalid Clicks Advertiser Forum on May 2 to discuss best prevention practices and get feedback from 200 large search marketers.

Yahoo! vice president Reggie Davis conceded that his company had not done a good job in the past of communicating its anti-click fraud initiatives to advertisers—partly because of pending litigation, which he helped direct in his earlier job as the company’s associate general counsel.

“There’s a significant disconnect between what we do inside the company to manage click fraud and traffic quality and what’s know external to the company,” he told that audience. “It’s important for us to step forward and say, ‘We can do a better job.’ We’ve learned from our challenges around litigation and it’s clear that we need to have better disclosure, better communication.”

Yahoo!’s aim, Davis said, is to shake up the existing process of filtering clicks on the front end and investigating and issuing refunds on the back end. “We want to offer greater instrumentation, greater functionality and greater interactivity with our advertisers so we can work together to shape the quality of our traffic,” he told the audience.