A Big Year for Hadoop

Our CEO, John Schroeder was recently interviewed in the press and asked about his predictions for Hadoop in 2012. Simply put, he sees a Big year for Big Data. It’s not just the scale of data growth. John shared his view that the ability to process and analyze Big Data is changing the game for companies and it’s changing the game in every aspect of their business.

Many enterprise IT vendors are wrapping themselves in some sort of “Big Data” cloak. The storage vendors talk about how they’ve always understood Big Data and cite the petabytes of storage that customers manage with their technology. The database and data warehouse providers make similar claims. Virtually, every vendor has some sort of Big Data presentation and by virtually every vendor I am of course including the virtualization providers. When MapR refers to Big Data we are talking about the MapReduce framework. This is the game changing approach popularized by Google.

We tend to take Google’s dominance for granted but when the Google search beta debuted in 1998 it was the 19th search engine on the market. The market was already well served with Yahoo, Excite, Infoseek, AltaVista, and a host of others. Within two short years, Google was the dominant player. The reason? MapReduce enabled Google to index much more data, much more quickly, and much more cheaply than any other provider. MapReduce is a paradigm shift, a new architecture that trumps existing approaches and provides any organization with the same power of changing their respective competitive landscapes. Google published a white paper on MapReduce in 2003. A Yahoo engineer, named Doug Cutting read the paper and the result was Hadoop. We’ve seen Hadoop emerge as a robust ecosystem with innovations happening across the Hadoop stack.

This is unfolding to be a big year. John’s predictions for 2012 encompass five major developments in Big Data. These include:

• Hadoop emerges as the safe platform choice for Big Data. The deployed footprint, complementary products, and available technical resources all reinforce the adoption of Hadoop.

• Real-time analytics take-off. Analyzing streaming data from application logs to messages augments existing batch applications.

• Hadoop applications move from experimental to mission critical. The number and breadth of deployed Hadoop applications also expands.

• Consulting firms augment their offerings with Hadoop specific consulting services expanding the number of available services vendors. Organizations benefit from the large and growing education and consultancy services.

• Big Data is no longer limited to companies that can ‘roll their own’ as the application ecosystem expands rapidly. In addition to the rapid expansion of Hadoop applications, 2012 sees the emergence of applications and services that leverage an underlying Hadoop engine.

We’re looking forward to a big year. We hope you join us.

Streaming Data Architecture:

New Designs Using Apache Kafka and MapR Streams




Download for free