“We receive massive amounts of data from a variety of different systems, covering all types of securities (equity, fixed income, FX, etc.) from around the world,” says Paul Cao, Director of Data Services for Wells Fargo’s Capital Markets business . “Many of our models reflect the interactions between these systems – it’s multi-layered. The analytic solutions we offer are not only driven by customers’ needs, but by regulatory considerations as well.
“We serve the company’s data needs across the entire banking business and so we work with a variety of data types including reference data, market data, structured and unstructured data, all under the same umbrella,” he continues. “Because of the broad scope of the data we are dealing with, we needed tools that could handle the volume, speed and variety of data as well as all the requirements that had to be met in order to process that data. Just one example is market tick data. For North American cash equities, we are dealing with up to three million ticks per second, a huge amount of data that includes all the different price points for the various equity stocks and the movement of those stocks.”
Cao says that given his experience with various Big Data solutions in the past and the recent revolution in the technology, he and his team were well aware of the limitations of more traditional relational databases. So they concentrated their attention on solutions that support NoSQL and Hadoop. They wanted to deal with vendors like MapR that could provide commercial support for the Hadoop distribution rather than relying on open source channels. The vendors had to meet criteria such as their ability to provide utmost in security, ease of ingest, ability to scale, high performance, and – particularly important for Wells Fargo – multi-tenancy.
IN THE NEWS