Data Collection, Mining and Processing At Scale

The fastest growing entity today is information. Information is expanding ten times faster than the growth of any other manufactured or natural product on this planet [66%+ per year for many decades]. We see the expansion of information everywhere. Less visible, harder to track, but exploding the same is the expansion of knowledge. The number of scientific articles published each year has been increasing in a steady rise for more than 50 years. Over the last 150 years the number of patent applications has increased. By this rough metric, knowledge is growing exponentially…

 

…Yet the paradox of science is that every answer breeds at least two new questions. More answers, more questions. Thus even though our knowledge is expanding exponentially, our questions are expanding exponentially faster. And as mathematicians will tell you, the widening gap between two exponential curves is itself an exponential curve.

-Kevin Kelly, The Expansion of Ignorance

10 Posts From This Blog Discussing Data Mining & Information Filtering:

• Information Day Trading
• Sifting for Quality Information
• Why I Can’t Read Novels Anymore
• Exponential Times
• Should I Become a Sharing Snob?
• We Feel Fine: Searching Emotions In Real Time
• Collective Emotions on the Obama Win
• Use Del.icio.us Bookmarking for Competitive Advantage
• Hack: Outsource Business Intelligence to Del.icio.us
• Yahoo Could Blow Mahalo Out of the Water