Goodgame Studios wanted to create an open platform, that allowed them to use the best tools for the job. They had a long list of platform requirements which included security mechanisms for protecting against unauthorized access, the ability to set up specific access criteria to support a multi-tenant Data Platform, sophisticated mechanisms to protect against data loss and provide 24/7 availability. Additionally they wanted to give their analysts easy access to various data sets without the need to learn multiple new technologies.
Goodgame has created an open platform where they can quickly and easily integrate new technologies. They can respond to gamer behavior in real time, allowing them to create a more compelling gaming experience. And they have reduced costs while maintaining performance with a new consolidated cluster that provides enhanced security and multi-tenancy.
Goodgame Studios is a leading free-to-play, online game development company based in Hamburg Germany. The company operates across web and mobile platforms and has over 270 million registered users worldwide. With more than 1200 employees from over 65 countries, they are the largest German employer in the gaming software industry. They have become Germany’s biggest game companies, with more than $220 million in revenues in 2014.
Goodgame Studio’s vision is to become one of the worldwide leading gaming companies. To realize their goal, the company follows the philosophy that data always beats opinion. Goodgame Studios needed to implement a solution that delivers high-quality and reliable data in order to make sound decisions.
Additionally, with their rapid international growth and new product development plans, Goodgame Studios knew they had to put a technology infrastructure in place that could keep up with the company’s growth. Their existing technology platform could not meet their changing needs. They needed to be able to respond to customer gaming behavior in real time and also have the flexibility to bring in emerging new technologies to keep their games on the cutting edge. Their focus here pertains to personalized recommendations, artificial intelligence, and fraud detection.
Their old system included a different Hadoop distribution, HP Vertica as the SQL analytics engine, and a self-written Java framework for data ingestion. The previous Hadoop distribution had a couple of significant limitations. First, it did not offer the openness and interoperability to add new tools and components to their technology stack, preventing them from leveraging new computational engines as they wanted. This was critical because the team at Goodgame is always exploring new technologies as a way to maintain their competitive advantage. Second, it could not support the consolidation of all their clusters into a co-located data platform, but rather it would require a complex data architecture with the overhead of maintaining distinct clusters.
After comparing the Hadoop distributions, MapR was chosen for their technology stack because of its interoperability, flexibility, performance, and multi-tenant capabilities. MapR is a central component to the Goodgame consolidated cluster with Apache Mesos as their general resource management system, Apache Spark as the computational and analytics engine, and a combination of HP Vertica and Apache Drill as their SQL query engines.
Apache Drill is used as the data exploration tool for newly generated information, where the design has not yet been finalized and continues to change during the deployment process. Drill also provides SQL querying on flat files for our financial department, and is used with MicroStrategy for reporting. All data not transferred to the analytics database engine is queried by Drill. MapR enables Goodgame to link all these systems together through open APIs, making the data flow smooth and giving them the ability to add components and data to the stack easily.
Open system provides flexibility tointegrate new technologies.
In order to stay ahead in the highly competitive gaming industry, Goodgame Studios needs to continue on the cutting edge of technology and be able to quickly integrate the latest technology. For example, they chose Apache Mesos instead of YARN for their resource management system in their platform. Since MapR provides a full read/write file system with POSX NFS support, a much wider range of tools can be used with it.
“We needed an open platform so we can implement any kind of software and introduce new technology components whenever we wanted to. We could not do this with other Hadoop distributions,” explains Andreas Scheel, Senior Data Warehouse Development, Goodgame Studios. “With MapR, there are no limits on opportunities. If something comes out that’s new, we can use it immediately, we don’t have to wait.”
Faster development-to-deployment cycle.
Goodgame Studios creates massively multiplayer online (MMO) games where the intercommunication between gamers is most important. The game should adapt based on user behavior. Goodgame collects anonymous data that describes how customers are playing the games. They can do A/B testing immediately to see the customers’ responses when they make changes in a game. Their prior implementation required a batch process for preparing the data for analysis. The shift to a real-time analytics environment, made possible by MapR and Drill, allowed them to be much more responsive. “We used to have to wait some time to get the results of these tests. Now we can do it live. If we change the logic within a game, we can immediately see user behavior and feature adoption in response to that,” says Scheel.
Goodgame Studios is also looking forward to the document database capability in MapR-DB. This will speed the application development cycle with the easy-to-use JSON paradigm. MapR-DB is in place at Goodgame Studios, and Scheel will look to implement MapR-DB as a powerful, reliable database engine for storing JSON data.
Increased performance at lower cost.
An important advantage to their new infrastructure is the ability to consolidate their clusters. The ability to run HP Vertica directly on MapR, a certified configuration, was especially critical. “We couldn’t co-locate HP Vertica with Hadoop until we switched to MapR. With MapR, we are able to integrate HP Vertica onto the same physical servers. Data ingestion can be done in parallel without touching the network. And our analysts see data quicker than before,” explains Scheel.
“With the MapR file system, we need less hardware to do the same computations,” he says. “We estimate that we need around 1/3 of the hardware than we needed before, and we still have the same speed. And since we maintain less hardware, it reduces costs and maintenance is easier.”
Multi-tenancy enhances security.
Goodgame Studios takes advantage of the powerful security and multitenancy capabilities built into the MapR Distribution. Their finance department uses this cluster for their analytics and reporting. “We’re storing very sensitive internal financial data. It needs to be highly secured. Before using MapR we could not be sure of that. Our previous Hadoop cluster was not secure,” he says.
“The MapR encryption and granular access control features made it easy for us to use the cluster in a multi-tenant environment and ensure protection against unauthorized data access,” he says. “To ensure no loss of data, we use MapR snapshot functionality and mirror volumes to a second cluster. In addition, with MapR we don’t have a single point of failure, which the other distributions actually still suffer from.”