Microsoft and Ask Jeeves have thrown paid inclusion links out of their search engines in recent moves that could bring new pressure on Yahoo to reconsider its fee-based indexing policies.
Microsoft on Thursday said its redesigned MSN Search site would no longer display links obtained through paid inclusion, a controversial arrangement in which Web publishers pay to have their sites indexed and frequently refreshed.
Microsoft said it made the change to clarify the difference between its commercial and noncommercial results. MSN, which relies on Yahoo to power its search, now filters paid inclusion links out of Yahoo’s results.
Yahoo is now the only major search engine championing paid inclusion.
“We remain committed to our content-acquisition programs, and we will continue to work with content providers to evolve and improve those programs,” Yahoo spokeswoman Stephanie Ichinose said.
Microsoft’s decision came after Ask Jeeves shut down one of its paid inclusion programs earlier this year and announced in late June that it would start winding down the other this month. Google has never offered paid inclusion, maintaining that its search technology can index the Web without outside help, and that such a program would undermine confidence in its search results.
But Google was long the exception. In the aftermath of the dot-com burnout, cash-hungry search engines turned to paid inclusion as a new source of revenue.
Unlike paid-placement programs, paid inclusion does not guarantee a top ranking in search results, only that a site will appear in a search engine’s index; after that, the normal ranking algorithms are supposed to apply.
But critics said the practice is of questionable value and creates at least the suspicion of impropriety. The FTC issued a 2002 advisory to search companies, recommending that they do a better job disclosing their commercial relationships, including their paid-inclusion programs.
The turning point for the industry may have come in March, when Yahoo rolled out its Site Match paid-inclusion program through its subsidiary, Overture. Many marketers loudly protested the terms of the new agreement, which charges not only for quality review and indexing but for each click-through.
Danny Sullivan, editor of SearchEngineWatch.com, said the controversy over Site Match likely contributed to Ask Jeeves’ and MSN’s decisions to retreat from paid inclusion.
“If Yahoo had rolled out paid inclusion and no one had batted an eye, it’s possible it would still be operating on Ask Jeeves and Microsoft,” he said. But in Sullivan’s view, paid inclusion has always presented a confusing proposition. “You tell users you have the most comprehensive and freshest search engine out there, and you tell advertisers they should do paid inclusion because you might not do all their pages or you might not get to their pages as often as they might like. How do you reconcile those two things?” he said.
Jim Lanzone, senior vice president of search at Ask Jeeves, agreed. “We think that it would be wrong to draw an artificial distinction between commercial and noncommercial pages, when in order to build the best search engine, we have to have those pages anyway,” he said.
Ask Jeeves ended its XML-based, pay-per-click Index Connect program in March, citing technical conflicts with its main search algorithm. Last week, the company announced that it is winding down its flat-rate Site Submit program as well.
In defense of fee-based indexing, Yahoo’s Ichinose said that even the most advanced search engines can benefit from supplementary data gleaned from paid inclusion.
“Crawlers have some inherent challenges with updating data on a timely basis,” she said. “A direct relationship allows us to ensure that the information we have is fresh and relevant.”
She said Yahoo saw no evidence of a consumer rebellion against paid inclusion. “We haven’t heard from consumers that paid inclusion is something that they are upset about or complaining about. It’s really been much more in the media space where the concern has arisen.”
There are signs that Microsoft privately agrees. Its recent moves represent only small steps away from paid inclusion and from its partnership with Yahoo. Besides getting search results from Yahoo, MSN Search continues to display text advertising from the Yahoo-owned Overture network. And MSN product manager Justin Osmer carefully left open the possibility that Microsoft might reverse its course on paid inclusion.
“Although we are discontinuing the program at this time, we will continue to evaluate options both independently and in conjunction with our partners,” he said.