This week in Tokyo at the Hadoop Conference Japan 2013 Winter, Ted Dunning presented “The Power of Hadoop to Transform Business.” He talked about what he called “the new future of Hadoop” and how the future that is emerging isn’t the future we thought – it’s better.
At the heart of the change is the fact that Hadoop no longer stands apart from everything else. Integration of Hadoop with traditional IT makes a big difference in the way companies can use scalable computing. And MapR contributes to this change by making data stored in distributed files more easily accessible, using familiar NFS.
Use cases Ted included in the presentation showed how the newly emerging Hadoop ecosystem can transform business endeavors in powerful ways. For example, new capabilities in web technology make it possible to integrate the use of real-time analysis (such as using open-source Storm) with long-time analysis (using MapR’s MapReduce). Now the web-tier can be rooted directly on a Hadoop cluster such that data does not have to move. Another example of business process being transformed in spectacular fashion comes from a new approach to building a web-based recommendation engine that makes use of “future Hadoop”. The combination of Apache Solr for scalable indexing and Solr search with an Apache Mahout based recommender running on MapR’s Hadoop distribution was able to cut the deployment time for the recommender system from 8 hours to 3 minutes – with significant business implications.
The final example Ted presented is the new Apache project called Drill, under development with support from MapR and others. Drill will provide fast and flexible computation with an ease of use that makes it user friendly for data analysts as well as developers. Drill will fill the gap between batch processing and stream processing, providing interactive analysis based on query times in the range of milliseconds to minutes, at scale.
The audience for this Tokyo talk responded enthusiastically with lots of good questions. Ted has provided his slides here, and he invites your feedback, saying he will “see you some time in the future”.