Comments on: How Google processes 20 petabytes of data each day http://ebiquity.umbc.edu/blogger/2008/01/09/how-google-processes-20-petabytes-of-data-each-day/ EBB is the ebiquity research group\\\'s blog at the University of Maryland, Baltimore County (UMBC). We focus on technologies that facilitate the design, implementation and control of distributed, intelligent information systems -- mobile and pervasive computing, ad hoc networking, multiagent systems, knowledge representation and reasoning, and the semantic web. As the tides of technology ebb and flow, we hope the good ideas wash up on our beach and the bad ones drift back out to sea. Sat, 05 Jul 2008 03:52:50 +0000 http://wordpress.org/?v=2.5.1 By: omar fidel http://ebiquity.umbc.edu/blogger/2008/01/09/how-google-processes-20-petabytes-of-data-each-day/#comment-18127 omar fidel Sun, 02 Mar 2008 21:55:52 +0000 http://ebiquity.umbc.edu/blogger/2008/01/09/how-google-processes-20-petabytes-of-data-each-day/#comment-18127 the wasted electricity to run these amusements must cost millions, when will people come to terms with efficiency. If the internet becomes the new TV, will it be as mindless, how can it not. Consumption of mass media includes infotainment, music and visual popular culture. Use sparingly, consume less and do more. Afterall everything is an abstraction from something, find the source. the wasted electricity to run these amusements must cost millions, when will people come to terms with efficiency. If the internet becomes the new TV, will it be as mindless, how can it not. Consumption of mass media includes infotainment, music and visual popular culture. Use sparingly, consume less and do more. Afterall everything is an abstraction from something, find the source.

]]> By: Nick B http://ebiquity.umbc.edu/blogger/2008/01/09/how-google-processes-20-petabytes-of-data-each-day/#comment-17824 Nick B Mon, 18 Feb 2008 23:52:22 +0000 http://ebiquity.umbc.edu/blogger/2008/01/09/how-google-processes-20-petabytes-of-data-each-day/#comment-17824 @Simon: A petabyte can be described as 10^15 bytes or 2^50 bytes, depending on your standard... so obviously 10*2^51 = 20 * 2^50 = 20 petabytes in that particular notation. Some light reading on the topic: http://en.wikipedia.org/wiki/Binary_prefix http://en.wikipedia.org/wiki/Petabyte And this is how I finally realised why my new 4GB iPod only holds 3.73GB according to Windows... Apple describes it in base 10 (so 4 * 10^9 bytes) and Windows describes it in base 2 (about 3.73 * 2^30). I feel cheated :( @Simon: A petabyte can be described as 10^15 bytes or 2^50 bytes, depending on your standard… so obviously 10*2^51 = 20 * 2^50 = 20 petabytes in that particular notation.

Some light reading on the topic:
http://en.wikipedia.org/wiki/Binary_prefix
http://en.wikipedia.org/wiki/Petabyte

And this is how I finally realised why my new 4GB iPod only holds 3.73GB according to Windows… Apple describes it in base 10 (so 4 * 10^9 bytes) and Windows describes it in base 2 (about 3.73 * 2^30). I feel cheated :(

]]> By: Akshay Java http://ebiquity.umbc.edu/blogger/2008/01/09/how-google-processes-20-petabytes-of-data-each-day/#comment-16500 Akshay Java Wed, 09 Jan 2008 23:13:20 +0000 http://ebiquity.umbc.edu/blogger/2008/01/09/how-google-processes-20-petabytes-of-data-each-day/#comment-16500 I have been looking into MapReduce lately and I found Hadoop, which is an open source implementation lucene.apache.org/hadoop/ There is also an interesting article on how to run MapReduce using Amazon EC2 http://wiki.apache.org/lucene-hadoop/AmazonEC2 I have been looking into MapReduce lately and I found Hadoop, which is an open source implementation lucene.apache.org/hadoop/
There is also an interesting article on how to run MapReduce using Amazon EC2
http://wiki.apache.org/lucene-hadoop/AmazonEC2

]]> By: Simon Reinhardt http://ebiquity.umbc.edu/blogger/2008/01/09/how-google-processes-20-petabytes-of-data-each-day/#comment-16491 Simon Reinhardt Wed, 09 Jan 2008 17:21:53 +0000 http://ebiquity.umbc.edu/blogger/2008/01/09/how-google-processes-20-petabytes-of-data-each-day/#comment-16491 >> If big numbers numb your mind, 20 petabytes is 20,000,000,000,000,000 bytes (or 22,517,998,136,852,480 for the obsessive-compulsives among us) Why would it be 2^51 * 10? Peta means 10^15 or 1 000 000 000 000 000 so 20 000 000 000 000 000 is correct. Nonetheless, very impressive. :-) >> If big numbers numb your mind, 20 petabytes is 20,000,000,000,000,000 bytes (or 22,517,998,136,852,480 for the obsessive-compulsives among us)

Why would it be 2^51 * 10? Peta means 10^15 or 1 000 000 000 000 000 so 20 000 000 000 000 000 is correct.

Nonetheless, very impressive. :-)

]]>