Community Makes Push Toward Beta at Productive Apache Drill Hackathon

MapR recently hosted the first Apache Drill hackathon, with nearly forty people in attendance who helped push Drill toward its first beta release. It was great to see people from companies such as Visa, Cisco, LinkedIn and Hortonworks come together to harden and enhance the Apache Drill project. 

The hackathon participants worked on many different aspects of Apache Drill. Over the next few weeks, these features will be incorporated into mainline. Here’s a preview of what we worked on, coming soon to a master near you:

  • Enhanced the Optiq SQL parser to support the TIMESTAMPTZ data type to express date types that carry time zone information
  • Built a clustering setup guide on the Wiki for those who want to try out Drill in clustered environments
  • Enhanced the testing framework by adding a QueryResultAccessor and Validator interface to simplify writing new tests as well reduce memory consumption and increase execution performance of existing tests
  • Added array index references for use with repeated/array data types used in components such as the Text and JSON readers.  (Give it a try: select columns[4] from `file.csv`)
  • Built a new StoragePlugin that supports accessing Kafka for real-time analysis of incoming event streams.  (Enhanced optimizer pushdown will be coming soon.)
  • Worked on additional JSON extraction and manipulation functions
  • Enhanced Drill’s ability to run multiple clusters concurrently while using multicast
  • Enhanced the new C++ client to support more platforms and Nullable ValueVectors

These features will be added to the existing feature set of Apache Drill, which includes innovative features such as an advanced Volcano-inspired cost-based optimizer, powerful complex data types that don’t require schema or materialization, and an incredibly efficient and powerful memory design.

As always, it is a pleasure to be part of such an amazing community. Thank you to everyone who attended the hackathon and made this such a great event!  We look forward to having more Apache Drill hackathons in the future.

For More Information

You can read more about Drill in the detailed article Apache Drill:  Interactive Ad-Hoc Analysis at Scale.

Here are some useful links for those of you who are interested in using or contributing to Drill:

no

Streaming Data Architecture:

New Designs Using Apache Kafka and MapR Streams

 

 

 

Download for free