The Big Data Train’s Next BIG Destination – Cognitive Analytics

The advancement of analytics has paralleled the advancement of big data in many BIG (Business, Industry, Government) sectors!  Both analytics and big data are on unstoppable trajectories right now – moving toward destinations that are beyond the greatest dreams of the first database builders of the 1960’s and 1970’s (RDBMS). 

The business output from traditional data (and business intelligence) systems (databases, spreadsheets, electronic tables) is a report (i.e., information), containing a quantitative description of what was captured in the data records (i.e., historical transactions). This is extracted from the data via descriptive analytics, which provides hindsight (from past data) and oversight (from current data) of your domain’s activities.

Modern big data systems contain a much broader and deeper quantitative picture of your domain than a traditional RDBMS.  Big data systems combine data from multiple sources (inside and outside your organization), multiple channels (social media, web logs, purchase transactions, customer service interactions), and multiple viewpoints (context, content, sentiment, location, time). As a consequence, it becomes possible to build predictive models of the behavior of objects (customers, or machines, or business activities) within your business domain.  The business output from our big data systems can therefore become much more than a report (information) – the output could actually represent new knowledge (about the past, present, and future of your domain).  This is discovered from the data via predictive analytics, which provides foresight about what is likely to occur regarding the objects within your domain.

A recent development is the emergence of prescriptive analytics, which provides insight into how objects behave, going beyond predictive models (which are typically trained on historical patterns and behaviors, and which therefore cannot predict a behavior, or pattern, or outcome that has never been seen before).  Insight is needed in order to see beyond what has happened in the past – in order to understand objectively under what conditions a given object (customer, machine, business process, competitor, or whatever) will act (or react) in a certain way, perhaps even in a “new way” that was not seen in the historical training data.  

Armed with the insights and knowledge that prescriptive analytics models can provide, and tasked with specific business objectives to guide the models, it is then possible to prescribe a set of conditions (marketing campaigns, external triggers, offers, instrumental parameter settings, business decisions, etc.) that is most likely to yield an optimal outcome.  Consequently, optimization theory becomes an important and essential skill for the data science team that is reaching to achieve prescriptive analytics success. The business output from such projects will be optimized outcomes (maximized revenues, minimized costs, maximized sales, minimized down time, maximized ROI, happiest customers, maximized conversions, …).

The upsurge of interest in prescriptive analytics is not too surprising, since it is not really a new methodology.  The field of operations research (OR) has applied optimization techniques for many decades to these types of business problems.  The era of big data has now created a boon for OR in business – the vast volume and variety of big data can provide a rich, deep, and reliable source of fuel for prescriptive analytics models, while the increasing power and capabilities of analytics tools can be employed to turn prescriptive analytics into a useful, ubiquitous, and robust business modeling tool across the whole enterprise.

But, wait!  That’s not the end of the analytics story. The best is yet to come.

The next stage of development in analytics comes from the emerging field of cognitive computing.  The IBM Watson machine is the prototype of this type of computing – the machine is able to access a vast store of historical data, then applies machine learning algorithms to discover the connections and correlations across all of those information nuggets, and then uses that “knowledgebase” as the engine for discovery, decision support, and deep learning.  The result is cognitive analytics, which delivers what’s right in a given situation – i.e., the right answer at the right time in the right context.  Think of the game Jeopardy – if the information provided is “1984” and the context is “non-fiction authors”, then the correct response is “Who is George Orwell?”.  But if the information is “1984” and the context is “best movies”, then the correct response is “What is Amadeus?”  And if the context is “American presidents”, then the correct response to “1984” is “Who is Ronald Reagan?”

Cognitive analytics is the best paradigm for data-driven discovery and decision-making.  Machine learning algorithms applied to big data will mine the data for historical trends, real-time behaviors, predicted outcomes, and optimal responses. The cognitive machine (powered by cognitive analytics algorithms) can be deployed to operate autonomously (without human intervention) in appropriate settings.  Of course, not all applications should be left to a machine to make the decision, but it is not unreasonable to allow a machine to mine your massive data collections autonomously for new, surprising, unexpected, important, and influential discoveries.  It is also acceptable to allow autonomous operations in some environments, including deep space probes operating far from Earth, or supply chain monitoring and ordering processes, or common help desk support functions, or massive data center operations, and many more applications involving smart devices, smart metering systems, and smart data (i.e., self-aware data, whose metadata tags were self-generated through automated cognitive learning algorithms at the point when the data were ingested into the big data system). 

The sky is the limit with cognitive analytics.  Consequently, the big data analytics train is now moving ahead at hyper-speed and is taking us to places (of discovery, decision management, and deep learning) that we never imagined when the first databases and business intelligence tools appeared on the market.  So, what’s next, beyond cognitive analytics? The latest news about the strength of MapR in the big data marketplace should inspire even greater visionary applications of analytics to data. Maybe we need cognitive analytics, pushing the envelope with MapR, to find the answer to that question.

Streaming Data Architecture:

New Designs Using Apache Kafka and MapR Streams

 

 

 

Download for free