Google gets caffeine injection as search speeds up

[Al-khiyal]

Google gets caffeine injection as search speeds up

August 11, 2009 -- Google has unveiled a new version of its search engine which it says will be faster and more accurate than ever before. The upgrade, which insiders have dubbed "caffeine", was announced on Monday after the company opened up access to web developers. It is intended to replace the technology giant's main search engine after tests have been completed.

Although little about the surface appearance of the new version has changed, engineers promised that radical changes behind the scenes would vast improvements for ordinary users. "For the past several months a large team of Googlers has been working on a secret project: a next-generation architecture for Google's web search," said the company in a statement on one of its blogs. "It's the first step in a process that will let us push the envelope on size, indexing speed, accuracy, comprehensiveness and other dimensions."

The company claims that significant changes to the way the system works will improve the experience for users – although it will also send shockwaves through the community of marketers who try and optimise their results to appear higher up in Google's index. The move comes despite Google's extensive lead in the search engine market – a domination which has provided it with billions in profits. Despite that lead, a series of threats have emerged recently that have seen the company redouble its efforts. Chief among them is Microsoft's continuing effort to break into the top tier of the search business, which has so far included rebranding its search engine as Bing.com and scoring a deal to take control of Yahoo's search business. Other possible contenders for the future of the business have emerged, too, including a "knowledge engine" called Wolfram Alpha, designed by British-born physicist Stephen Wolfram; and the emergence of so-called "real-time search" through websites like Facebook and Twitter.

Caffeine allows Google to index the web at a higher pace - gathering more information and doing it faster - but the company's search quality specialist, Matt Cutts, rejected claims that it was developed in response to the actions of rivals. "I love competition in search and want lots of it, but this change has been in the works for months," he wrote on his blog. I think the best way for Google to do well in search is to continue what we've done for the last decade or so: focus relentlessly on pushing our search quality forward."

Whether the upgrade will have a significant impact on Google's business has yet to emerge, but Martin McNulty, director of search marketing specialist Trafficbroker, said that it could give it a significant boost. "Google's Caffeine is undoubtedly faster, almost twice as fast at times. It's like a Google Gti," he said. "The launch of Bing has been a good thing, although unfortunately more for Google than Bing if the level of innovation continues at this frantic pace. Google is still very much in the driving seat and is still setting the pace."

[Al-khiyal]

Search for answers to Google's power leaves UK internet firm baffled

August 17, 2009 -- A British husband and wife team have been waging a three-year battle to get their price comparison website recognised by Google in a saga that sheds new light on the power of the world's largest search engine Foundem.co.uk directs shoppers to online deals for goods such as TVs or flights, but has struggled since one day it suddenly disappeared from Google search results for these categories. There is no evidence that Google is in any way being dishonest or unfair in the way that it ranks such websites, but Foundem's fight to discover what happened has highlighted the ever-growing influence of its mysterious search algorithms.

Many consumers believe Google's search engine works on a formula that was created by founders Sergey Brin and Larry Page and that was that: they set it running and the rest is history. In fact, as those in the internet industry know, Google carries out regular "tweaks" of its algorithm. About 450 a year in fact. When they are made, the sheer scale of Google – it has an estimated 90% market share in Britain – means these can have huge and often unintended consequences. Another example occurred about the time Foundem saw its traffic from the site drop off, when Google stopped returning results for Amazon.com for more than a day. The internet retailer is a household name, so not appearing in Google's results for a time is not the end of the world. However, for a start-up company, getting into Google's results is a question of survival.

"A typical website in the UK receives around two in every five visits from search engines and obviously, the vast majority of those come via Google," according to Robin Goad, research director at Hitwise UK. "For a site that is selling things like Moneysupermarket or Foundem then it is up to 60% to 70% of their traffic that comes from Google. It is definitely true that if you fall foul of the rules ... it can have a massive impact on your business." Exactly what those rules are is far from clear because Google does not want to give too much information away in order to avoid web developers "gaming" the system and promoting sites that are not, in fact, relevant to users. That would lead users to try other search engines and affect its traffic. Trying to find out why exactly your site has dropped off is difficult. You can fill out an online reconsideration request, but its wording makes it quite clear that Google assumes you have done something untoward and tried to game its system.

Foundem's Shivaun Raff explains: "We sent reconsideration requests then started sending emails to as many people as we could find, expecting each time that this was just a failure of process, expecting that once our case was in front of people with the power to do something it would get fixed." But Google insists its search rankings are only ever driven by a desire to make results useful. "We can't comment on individual cases," said a spokesman. "But our systems are designed to produce the most relevant and useful results for the people who use Google search. Where sites are adding little value or original content, they are likely to fall in our ranking. Surveys of our users show that what they most dislike when they search is to receive multiple results from sites showing the same or very similar content."

Foundem has been able to make revenues for the operation – which consists of just the two of them and a computer programmer friend – by offering their technology to other web publishers. Foundem, for instance, currently runs the shopping comparison service for the website of Bauer Media's Motorcycle News. To become more well known and achieve a higher position on Google, as more people link to you, of course, takes advertising, but again the might of Google dominates the online advertising market in the UK, which is why the changes that happened in the summer of 2006 to Foundem's scores within Google AdWords platform were so worrying for the company. AdWords allows web users to buy keywords in an online auction environment. When the keyword is used in a search by a Google user, that auction decides which advert appears either alongside or above Google's "natural" search results. It is, however, not quite as simple as that. A web owner could, in fact, outbid everyone else for a keyword and still not appear on Google if it does not have a high enough "quality score" (see below).

What suddenly happened to Foundem a few weeks after it disappeared from Google's search results was its quality scores on Adwords dropped dramatically. "Overnight all our quality scores went from excellent or very good to poor and very poor, and our cost per click went from 5p to £5," according to Shivaun. Google's search team had alerted the AdWords team to the fact that Foundem was a poor quality website. As AdWords is the part of Google that has to deal with paying customers, its processes for dealing with potential problems is far more straightforward than in search. Foundem contacted the AdWords team and was eventually able to persuade them that its website was relevant to Google's users. It took a year and then suddenly its quality scores went back up.

"Google uses algorithmic approaches to evaluate the site quality of AdWords landing pages," explained a Google spokesman. "No algorithm is perfect, so we have systems in place for the review of penalties based on human judgment and user feedback when appropriate. All of our algorithms and decisions are focused on improving user experience." However, it is odd that the fact that the Google AdWords team thinks a site is relevant to Google users has no bearing on whether its search business does. The connection only works the other way: if the search team does not think a website is relevant to users, they alert AdWords. Google maintains that this one-way street is critical: "One of the core principles underpinning Google's web search has always been that natural search rankings [are] made independently of whether a website advertises with us. Both natural search and AdWords make automatic quality assessments to help the ranking process, but each system looks primarily at different signals, which we publish in our guidelines."

Exactly how AdWords operates is to be tested in the U.S. courts by a company called TradeComet.com, owner of search engine SourceTool.com, which has accused Google of "engaging in predatory conduct to block search traffic by imposing massive, unjustified price increases" through the system. That case, and now the three-year saga of Foundem are likely to raise concerns about the power of Google, something already worrying politicians in Britain. "There is a real risk that Google, entirely unintentionally, could limit innovation simply because of its dominance," according to Peter Luff, the Conservative chairman of the Business and Enterprise Committee. "I don't ... believe that the organisation ever seeks to behave in anything but the most socially responsible way – but monopolies will always act in their own best interests, and those interests may not coincide with those of the rest of us. Google is just too dominant for any of us to feel entirely comfortable."

Algorithm and blues: How to win with Google

Google's search algorithm relies on more than 200 individual signals to decide what sorts of site are relevant to its web searchers. These start with PageRank, the breakthrough bearing the surname of Google's co-founder Larry Page that measures a website's relevance by the number of other sites linked to it, and extend to measures of the unique content on the site itself and whether the text on the page is replicated – either on other parts of the site or elsewhere on the web – and even whether it is spelt correctly. One problem faced by Foundem is that as a price comparison service its raison d'etre is to pull in information from elsewhere on the web so a lot of the text – such as product descriptions – will be replicated. Google says its users do not want to be presented with a list of options on the site they visit, while the Foundem pair point out that is, in essence, all that Google itself does. Meanwhile, Foundem results are appearing relatively highly on Yahoo and Bing – Microsoft's search engine . Relevance is also crucial for Google's AdWords platform. It assigns a quality score for each bid for a keyword – the word used in a user's search query that will trigger an advert. A high-quality score means a lower cost per keyword. That, plus a high click-through rate, will get the advert placed higher on the Google page than other adverts even though those rival advertisers may have spent more on their keywords. Working out what works with Adwords is very complex and like Google's search system has spawned an industry of people who claim to know exactly how to place advertisers high on Google's results.

[Al-khiyal]

Google introduces new search options

October 2, 2009 -- Google has introduced new options to search that are intended to make life easier for users. The new options - nine in total - are hidden in the left sidebar and allow users to filter results by the past hour or a specific date range; they can be specified to look only within blogs or for reviews; and enable the user increase or decrease the number of shopping sites they get in the results. Up till now the search results could only be reduced to news, images, videos or certain languages. Product manager Nundu Janakiram and software engineer Patrick Riley announced yesterday in a post on the Googleblog that the changes would be rolled out over the day. They can be found by searching with Google and clicking "Show options" in the blue bar under the logo.

The option of showing fewer shopping sites is an important new feature as searching for products has become nearly impossible on Google, since the results are full of search engine optimised shopping sites. It has also made it possible to rediscover pages you've visited before by clicking "Visited pages," or filter out the websites you've seen by clicking "Not yet visited." And it presents a better integration of specific views of results like the "Timeline" or the "Wonder wheel" rehash the data further to bring the historic or contextual information to the front if wanted.

On Monday, Google announced that it was adding its "Hot Trends" feature to its main page search, allowing users to see how popular their search is at any given hour. Now, when you search Google and your query matches one of the top 100 fastest-rising search terms, it shows you a graph at the bottom of page, with more information – like how popular the query is, how fast it's rising over time, and other useful data. Google's new search is another setback for Microsoft's search engine Bing, which still delivers better image results, but doesn't allow you to refine your search results in the same interface. Bing recently faced its first monthly decline as its share fell to 3.25%. Google globally has a quasi-monopoly with a market "share" of 90.54%.