How Many Library of Congresses Worth of Data Per Hour?

The data we can and do produce is accelerating at an astounding rate.  WalMart processes 167 Libraries of Congress worth of data per hour.  No longer are we thinking about terabyte or even petabyte storage.  For the biggest players in the data space, they are operating in a space far beyond that.

Cisco announced yesterday that it would feed our never-ending data-creating and consuming habits with a faster internet, at least some time in the future. Core IP routing functionality which will hum along at 322 terabytes of data, a measly one LOC (Library of Congress).

There’s room for should in what we process and store, however.  Is “Are you watching the Oscars?” really storage worthy data, regardless of context?  Is my 100th username and password combination really storage worthy (or for that matter even processing worthy).

Perhaps not. “The data-centered economy is just nascent,” admits Mr Mundie of Microsoft. “You can see the outlines of it, but the technical, infrastructural and even business-model implications are not well understood right now.” (The Economist, “Data, Data, Everywhere” March 5, 2010)  (have to love the ST Coleridge reference…)