MapR recently hosted the first Apache Drill hackathon, with nearly forty people in attendance who helped push Drill toward its first beta release. It was great to see people from companies such as Visa, Cisco, LinkedIn and Hortonworks come together to harden and enhance the Apache Drill project.
The hackathon participants worked on many different aspects of Apache Drill. Over the next few weeks, these features will be incorporated into mainline. Here’s a preview of what we worked on, coming soon to a master near you:
- Enhanced the Optiq SQL parser to support the TIMESTAMPTZ data type to express date types that carry time zone information
- Built a clustering setup guide on the Wiki for those who want to try out Drill in clustered environments
- Enhanced the testing framework by adding a QueryResultAccessor and Validator interface to simplify writing new tests as well reduce memory consumption and increase execution performance of existing tests
- Added array index references for use with repeated/array data types used in components such as the Text and JSON readers. (Give it a try: select columns from `file.csv`)
- Built a new StoragePlugin that supports accessing Kafka for real-time analysis of incoming event streams. (Enhanced optimizer pushdown will be coming soon.)
- Worked on additional JSON extraction and manipulation functions
- Enhanced Drill’s ability to run multiple clusters concurrently while using multicast
- Enhanced the new C++ client to support more platforms and Nullable ValueVectors
These features will be added to the existing feature set of Apache Drill, which includes innovative features such as an advanced Volcano-inspired cost-based optimizer, powerful complex data types that don’t require schema or materialization, and an incredibly efficient and powerful memory design.
As always, it is a pleasure to be part of such an amazing community. Thank you to everyone who attended the hackathon and made this such a great event! We look forward to having more Apache Drill hackathons in the future.
For More Information
You can read more about Drill in the detailed article Apache Drill: Interactive Ad-Hoc Analysis at Scale.
Here are some useful links for those of you who are interested in using or contributing to Drill:
- Join the Apache Drill User Group: A great way to get started is to attend the Meetups, hear about what is happening with Drill, and share ideas. http://www.meetup.com/Bay-Area-Apache-Drill-User-Group/
- Check out the Apache Drill project: http://incubator.apache.org/drill/
- Other ways to get involved: http://incubator.apache.org/drill/index.html#get_involved
- Follow Apache Drill on Twitter: https://twitter.com/ApacheDrill
- Join the mailing list or the Google+ Hangout: If you’d like to find out more about how Drill works, join the mailing list or the weekly Google hangout.