2013 was notoriously named as the “Year of the Mega Breach” with eight breaches exposing more than 10 million personal identities each. In Target’s data breach alone, hackers collected credit card details on about 40 million people who had shopped in person at the retail chain between Black Friday and December 15th. Despite significant investments in information security, data breaches continue to rise in 2014 and seem poised to significantly overtake the volume and cost of breaches in 2013. Home Depot’s data breach, disclosed in September 2014, resulted in hackers stealing 53 million stolen identities, quickly surpassing Target’s breach. Is the trend of mega breaches here to stay?
With the widespread usage of mobile and cloud computing, billions of smart devices with interconnected communications (also referred to as IoT) and the plethora of customer and end-user data being collected by organizations, data breaches that impact millions of individuals is the new norm. How can businesses make sure that the bad guys don’t stay unnoticed for months, extracting data every day?
The Challenges with Conventional Security Solutions
While the threats have evolved over the last ten years, security spending has not! Organizations continue to spend a lot of money in point security solutions that focus on blocking known vulnerabilities, attack vectors and intrusions. Attackers are more sophisticated, organized and industrialized, and use multiple threat vectors; their motivations range from profit to politics. It’s virtually impossible to prevent all intrusions.
While firewalls, IPS/IDS and malware defense solutions play an important role in securing organizations, it is vital to connect the dots from the variety of security-oriented data that is collected from these solutions to provide complete visibility into what is happening within the environment across networks, servers, users, applications and end points. Current security solutions either do not have the capabilities to aggregate, analyze and correlate information from multiple sources, or cannot scale and handle the volume of data generated by the activities over a period of time. The greatest area of unmet need with conventional security solutions is effective, targeted attack prevention and breach detection. Organizations are failing at early breach detection, with more than 92% of breaches detected and notified by a third party—this is what ultimately impacts the size and cost of the data breach.
Threat Intelligence and Fraud Prevention with Hadoop
The same big data platforms that form the foundational technology for storing and analyzing volumes of consumer data and their purchase behavior, IoT data, mobile data etc. can be leveraged for security analytics and fraud prevention.
By leveraging Hadoop and the ecosystem of dependent technologies, organizations can ingest large amounts of structured, semi-structured and unstructured log data from multiple sources in batch and real time in a cost-effective manner. Using machine learning and statistical analysis, organizations can provide the following actionable intelligence:
- Identify anomalies and correlate events indicating a threat or attack
- Detect suspicious user behaviors
- Identify unauthorized access from devices over a network
- Flag/block suspicious transactions
- Perform forensic analysis/investigations on suspicious activities, employees, and trades
Security Analytics and Fraud Prevention Representative Architecture