Single Google Query uses 1000 Machines in 0.2 seconds

Interesting article, I know how Google processes a search query, but they keep the index in memory too, hmm, nowadays I find that all my new posts are updated in a couple of hours, things have advanced a lot in the over the decade.

Google is growing so rapidly, don’t know how many machines they will use for a search hit 4 -5 years down the line. These numbers may create problem for Google because Environmentalist are looking for these subjects which are related with a bigger company like Google. The CO2 emission thing was also not good for them.

Anyway, thanks for sharing these numbers, now we know what is happening behind the scene.

I don’t think you’re reaching the right conclusion. Marissa was not saying that 1000 machines are dedicated for 0.2 seconds to a single query. It’s, “Over the course of 0.2 seconds, 1000 machines are involved in the processing of each query.” But the amount of time spent on each machine is probably much less. See this Reddit comment: link

Honestly, it is a comment on your disregard for truth (or your credulity, not sure which one is worse) that you could publish this article without realizing that it’s absolutely impossible for a company of Google’s size to spend 200 machine seconds on each query. It would require millions of dedicated machines at a cost of billions of dollars. It might not even be possible if web search was the only thing Google did with all their machines, and it completely outside the realm of possibility when you consider that not all their machines are dedicated to search.

The figure for energy per query isn’t necessarily wrong just because everything is held in memory. DRAM doesn’t use much energy.

This article will definitely cast a grey cloud over Google’s eco-friendly policies.Hope they have an explaination for the use of such huge machines for the queries………I wonder how much Yahoo & MSN use????

So the index is stored in 1000 machines. Do you honestly think your search query will touch each of those 1000 machines? You search for “ponies” and your search goes to the appropriate servers for ponies, and other horse-like creatures (incidentally, the same servers for unicorns too)

Also, funny thing about these computer things, they can do more than one thing at a time! *gasp*

Use scroogle instead. Google keeps logs of your search terms.

Remember, RAM density is still increasing exponentially. Ten years down the road, we could have had a breakthrough that allows Google to keep the entire index in the RAM of each machine.

It is possible that their per-transaction power usage has actually gone down as a result. What is really important to know is what the ceiling is on the number of transactions in a given bucket of time an be, and what that is relative to the previous number.

I’m guessing they’re talking about 1000 servers with at least 256GB of ram each, and running a distributed hash table that is stored in ram. This effectively gives them (256 TERABYTES – redundant copy) of ultra high speed key/value lookups. This is nothing new; many companies with a substantial online presence use this to scale.

In 2005, Google entered into partnerships with other companies and government agencies to improve production and services. Google announced a partnership with NASA Ames Research Center to build up 1,000,000 square feet (93,000 m2) of offices and work on research projects involving large-scale data management, nanotechnology, distributed computing, and the entrepreneurial space industry. Google also entered into a partnership with Sun Microsystems in October to help share and distribute each other’s technologies. The company entered into a partnership with AOL of Time Warner, to enhance each other’s video search services.

Google is growing so rapidly, don’t know how many machines they will use for a search hit 4 -5 years down the line. These numbers may create problem for Google because Environmentalist are looking for these subjects which are related with a bigger company like Google. The CO2 emission thing was also not good for them.

Anyway, thanks for sharing these numbers, now we know what is happening behind the scene.

The author, Amit Agarwal, states that, “This revelation may be a bit embarrassing for Google, which has defended its ecological record in the past, claiming that a single Google query takes just 0.0003KWh of energy…”

On what information does the author assume that Google was not truthful about their previous statement? Retrieving data from memory is a *much* less expensive operation from both a computation and power standpoint. Those same 1000 servers may have been taking longer, using more power per query, to handle the same number of users under the old architecture. You also don’t know if they are spinning down their drives now (which would use a lot less energy) since everything is loaded in RAM, and only spinning up to persist data.

The author makes a statement based on no critical thinking with no evidence to back it up.

Sorry but please check your math.

1000 machines are involved in responding a query, but those machines are involved in answering millions of queries per day, so energy consumption per query is very very low.

To Author:
1000 machines to perform a query in .2 seconds off of an index stored in RAM vs. one machine taking minutes to do the query from a disk based index.

Google shouldn’t be embarrassed. You should be for writing a tech article on a massive data center and making a comment about it’s power usage without understanding what economy of scale means.

The amount of energy consumed and the eco impact of the query spanning 1000 machines in 2/10ths of a second is practically nonexistent.

wow a bunch of tree huggers are threatening to shut down google? heh what the hell

Who gives a shit about the energy consumption? It’s miniscule, and besides, can you do any better?

Ryan,

ms is milliseconds not machine seconds. 200 ms for a query.

Embarrassed? This was a good article until you added that bit of sensationalist bullshit at the end.

I can’t believe the horrible quality of interpretation the author of this article did. If each query used .2 seconds of 1000 CPUs, Google would be in bankrupt. First, a query takes only some microseconds for processing (in the case of Google’s DB). Even in a standard server with an average 2gb of database a query shouldn’t take more than half a second unless it’s very complex and it’s not correctly indexed. Why would the biggest expert of databases (well, competes with Oracle) waste 1000 machines for .2 seconds only for a single query?

Shame on you for publishing something that’s obviously flawed on your side.

How much energy is required to get in your car and go down to the local library? Or to print, publish, distribute and sell those reference books we used to have? Google is *saving* the world a great deal of energy.

Pffft. That’s nothing. I used to be a homeless rodeo clown but now I am a world class magician !

This article is needlessly sensationalist. You give no proof that Google’s querys are actually inefficient.

F-

This article is utter rubbish. The “conclusions” it draws are not borne out by the information given in the article.

First of all, 1000 energy efficient machines using RAM CAN use less electricity than 12 energy inefficient machines using hard disk drives.. those platters don’t spin by themselves you know, and the motors use HUGE amounts of energy.

Secondly you suggest that the former also somehow means that Google’s datacentre is NOT the most energy efficient… of course without ANY EVIDENCE AT ALL. Even if it took the entire power output of a small country to power Google’s datacentre, as long as they use LESS for MORE they can still be the most efficient.

The severe lack of logic demonstrated in this article exposes this as nothing more than lame Google bashing.

By the way, anyone who believes that ANY search engine doesn’t log search terms etc… well you are an idiot pal, because I guarantee you EVERYONE DOES – even if they are willing to lie to your face about it.

One query >>> 1000 servers >>> 0.0003KWh of energy unbeleiveable

Joe Jance makes a good point. The article author’s comment claiming that these numbers are embarrassing don’t seem very credible.

For example, there is no reason at all to assume that the 1000 servers fully occupy their CPU for the whole 0.2s for this one query. In fact to assume that would be somewhat ridiculous, since I assume one computer needs to collate and return the results, and that also happens inside the 0.2s.

“This revelation may be a bit embarrassing for Google”

I was also chime in that this statement shows a total lack of understanding of massively scalable architectures.

A single query may in fact be split amongst 1000+ machines. However that query is using a miniscule percentage of each of those machines total cpu power which is simultaneously handling hundreds or thousands of similar requests. In addition, though your query may touch 1,000 machines, those represent a small fraction of the total number of google machines handling Google’s business. The total number of machines google has in its hosted data centers is estimated to be in the hundreds of thousands.

Your attempt to couch this somehow as an embarrassment related to how ‘eco-friendly’ or not they are really is an embarrassment for a ‘tech blog’ which should know better.

If you wanted to learn more you could have just Google’d for it:

link

link

And finally, if you are so eco-friendly… How many watts were wasted on publishing this link-bait article of yours? Maybe you’ll publish that next time… Or turn off your computer instead and save some power, and our time.

I was going to add this blog to my Google reader, then I read this nonsensical statement:

“a single search query on Google can consume the processing power of 1000 machines”.

Goodbye.

One need not worry about Environmentalists ‘going after’ Google because, they hold all the right cards. Their papers are definitely in order and their social ideology is in line with Party doctrine.

Kinda sounds like Chinese political alignment doesn’t it? I guess it’s no coincidence regarding Google’s position towards freedom of expression in China as well?

Oh well, no problem. Kilowatts/hr. are of no consequence when you’re building dossiers on your visitors, so that when the time is opportune, they can be turned over to the Party leadership to see who the dissenters are.

We would want everyone to be on the same page, wouldn’t we? Just like they are in China.

This sounds amazing. As i keep hitting through google all day, i trouble oen thousand servers lying in some corner of the world. and their servers breath more co2 than my car!

I think that the search engine must must be retrieving first few pages of search results on query execution. After all, the total number of search results that Google displays is only an approximation, so its not like Google has gone through all the search results.

I was going to add this blog to my Google reader, then I read this nonsensical statement:

“a single search query on Google can consume the processing power of 1000 machines”.

Goodbye.

While I don’t think one query uses much power, I bet the entire google operation consumes a huge amount.

Well I was pretty surprised with the question of Eco Friendly Search engines. Well we flocks only have an idea that the Industries, Mills, chemical factories, verticals etc pollute the atmospheres. Now it seems like websites are also indirectly polluting the atmosphere……

This sounds amazing. As i keep hitting through google all day, i trouble oen thousand servers lying in some corner of the world. and their servers breath more co2 than my car!!

Interesting that only “NotMyName” spotted the real error: by using Google, how much energy do we actually save? I.e. has the energy consumption of the average person gone down since Google? I bet it has. To find information we now use fewer books, make fewer phone calls, drive around less. Investigating the trade-off should have been the objective.