MapR and AWS solution is enabling IDEXX to flexibly scale its business at lower cost, and gain access to critical customer data instantly for rapid response times.
IDEXX Laboratories, Inc. provides diagnostics and information technology solutions
for animal health, and testing technologies to ensure water and milk quality. IDEXX
helps more than 50,000 veterinary practices worldwide advance medical care,
improve staff productivity and increase profitability through diagnostic tests, pointof-care
instruments, reference laboratory services, digital radiography, and practice
As the IDEXX businesses continued to grow, its traditional database technology
could not keep pace. “We are bringing in more business, and our customers want
data analyzed in different ways, including more historical data, which means having
to retain data longer,” explains Terry Schutte, IDEXX Senior Systems Administrator
for Software R&D.
The company’s primary data store is a relational database hosted on Amazon Web Services (AWS). The growing size of this database meant that daily jobs to aggregate and summarize data were taking too long to run and consuming database resources that were impacting online operations. “We needed a solution which could offload this processing, easily scale to support the growth of the existing product and could be leveraged for future data intensive projects,” says Schutte.
“We are at peak capacity and processing an ever-increasing amount of unstructured data. As we build the data business, our challenge is scaling it. We ran into limits of what we could afford to do,” he says. IDEXX aggregates and analyzes data from participating practices and offers a free industry benchmark report to the participating practices. The practices can easily see where they stand in comparison to the industry and can change their best practices to try and close any gaps. IDEXX also utilizes the aggregated and de-identified data to create syndicated reports for pharmaceutical and nutrition companies. The companies use these reports to identify trends and better understand the industry.
IDEXX was looking for a solution that could be easily scaled for processing and storage capabilities with minimal engineering or operations effort and integrate with their existing processes and workflows. It needed to leverage proven technology, and be cost effective as their business scales. The new solution had to be compatible with their existing systems, which included their AWS infrastructure and Java 7. “Our primary reason for choosing MapR M3 on Amazon Elastic Compute Cloud (Amazon EC2) was the ability to run Hadoop under Java 7 against Java 7 compiled applications,” says Schutte. “It was hard to find support for Java 7 in the MapReduce ecosystem. MapR’s performance and architectural improvements stood out.” In addition, providing support for developers running both Windows and Linux development environments was crucial. By providing native clients for both Windows and Linux, our developers were free to continue working with whichever operating system they were most comfortable with. “From an operational perspective, MapR is easier to use than Cloudera and higher performing, and we benefit from the optimizations made to Hadoop.” he says. “MapR has a solid product and is a strong company. We could see that MapR was a company that was going to thrive in the marketplace. They have a great support environment and ecosystem,” he adds.
IDEXX has realized multiple benefits from their MapR and AWS solution, including increased flexibility and control to scale their business, faster customer response times, the ability to retain all their data and ease of experimentation.
Increased flexibility to scale the business:The MapR and AWS solution gives IDEXX more flexibility and control to respond to changing customer demands. “By running MapR clusters on EC2, we retain full control over the configuration and operations of the MapR cluster, while continuing to have the benefits of EC2 hosting-which means no capital expenditures and the flexibility to scale our environment based on demand. We pay only for capacity that we use,” says Schutte. “The ability to easily grow and shrink a MapR cluster on EC2 via simple scripts provides quicker turnaround time for our developers and testers. Being able to flexibly scale our cluster to support whatever load is huge. It lets us easily scale to support growth of the business.”
Faster turnaround time for customer requests:The new MapR/AWS solution dramatically improves the company’s ability to respond to customer requests. For example, if a customer asks a specific question related to the marketplace or trends, IDEXX can respond immediately. “Before, we would have to schedule developer time to write a query to see if we even have data. The whole process could take months,” explains Schutte. “Being able to scale the environment quickly and cost effectively helps us turn around answers to our customers’ questions much faster. Today we can cut through our data and run queries using MapReduce and get the answer in a day, or even hours.”
Access to all the data, all the time:The
new system removes prior constraints
on how much data IDEXX can store
and process. “Now we have the ability
to store and process all the data, all the
time. Everything is at our fingertips. If
a customer has a question, we simply
write a quick job to answer the query,”
Increasing the speed and ability to process all the data provides IDEXX with a competitive advantage. “Having the capacity to store and process all the data at once means we can provide customers more answers to more questions. And the speed of processing doesn’t vary.
Ease of experimentation The MapR/ AWS solution makes the cycle of testing much easier. IDEXX can have multiple development environments running in parallel with less risk. “It’s very easy to experiment. I can snap on a test cluster, do some testing and then tear it all down,” says Schutte. “It helps me find the optimal way of doing things and make decisions that fit our environment and business. It helps us avoid mistakes in development. It’s a huge benefit.”