There’s a reason the industry refers to Big Data as “Big” Data. According to IBM, we create 2.5 quintillion bytes of data. Here’s another eye-opening stat: 90 percent of the data in the world today has been created in the last two years alone. Mobile devices continue to drive that trend, now that we can all consume data whenever, wherever. In fact, according to Cisco, last year’s mobile data traffic was nearly 30 times the size of the entire global Internet in 2000.
There’s no doubt that data will continue to grow at an exponential rate. So the question many companies are faced with today is: how do I take advantage of this myriad data at my fingertips? When should I migrate my current architecture to a Big Data platform? And what does that process entail?
As head of technology and development at Harte Hanks, a company that just went through this journey, I’ll share some lessons learned and insights into our experience and our collaboration with MapR.
Harte Hanks’ Big Data Journey
For the past 80 years, Harte Hanks has been in the business of enabling smarter customer interactions. The world has changed quite a bit in that time – in 1923, we were still decades ahead of the world’s first computer, much less the smartphone.
One thing that hasn’t changed is that marketing services companies need to pace way ahead of their clients needs and anticipate market changes, like the shift to Big Data. Harte Hanks saw the changing patterns of purchase behavior as the catalyst for a technology shift.
Our clients, across financial services, retail and technology in particular, needed to access broader types of data and process larger volumes of information, generated by the way their customers were researching and interacting with their brands across a variety of channels and devices. We knew that all of that data should feed into our clients’ marketing process to help them drive smarter marketing programs and customer interactions – we just needed the right tools.
Choosing the Right Platform
With massive data storage, processing power and pricing have become major purchase considerations for our clients. Here were our criteria when looking at a Big Data platform:
- Multi-tenancy: We needed a model that worked for a multi-client platform so that we could cost-effectively scale for multiple clients. This may not be as big of a consideration for a smaller company, but we knew we’d be adding servers and needed a cost-effective solution.
- Flexibility and scalability: MapR makes it very easy to replicate information across clusters and scale servers depending on our clients’ needs, and we can do it quickly and cost effectively. The high availability fault tolerance and data mirroring were also important considerations.
- Ease of use: We needed to implement this solution quickly and easily, and we liked the plug-and-play functionality and simple UI MapR provided. We also were looking for a company that would provide end-to-end technical support and MapR delivers the services we need.
Re-Architecting Your Data Platform
We already had a large subset of clients using our data management capabilities at the time, so we needed our migration approach to the Big Data platform to be evolutionary. For existing customers, we used Splice Machine to implement a seamless migration path that involved the least upheaval to their current environment. With Splice Machine, our clients gain the flexibility and scalability of Hadoop while still being able to leverage their existing systems, tools, queries and reporting, as well as campaign tools. Splice Machine provides an integrated solution that allows connectivity into Hadoop while maintaining standard database connections via JDBC or ODBC for easy integration of their existing products that are used to manage their business. As their needs expand, we can offer a path to integrate unstructured data and larger volumes of data sets.
For new clients, we can start with the core functionality available in MapR and design capabilities that leverage the benefits of the Hadoop architecture.
Benefits of Big Data Architecture
The benefits of our new Big Data platform are numerous and we are already seeing the results. First and foremost, we are increasing customer satisfaction through faster time to value and more complete, enriched data sets. Data processing that used to take one to three days can now be accomplished in hours, if not minutes, and with larger, enriched data sets, the clients get deeper customer insights for higher efficiencies and real-time marketing effectiveness. With the MapR architecture, our roadmap is more robust and can evolve more rapidly with our clients’ changing needs, in a matter of days rather than weeks or months.
The time for Big Data is now, and to stay at the forefront of the evolving data marketplace, it’s important to take a comprehensive and realistic look at your company and client’s needs. We found MapR to be the solution that worked best for us to integrate the many types of data – digital data, survey data, reference points and more – all while maintaining the performance and ease of use we required. Good luck to you!