Evolution of Analytics


Dhritarastra said: O Sanjaya what did my sons desirous of battle and the sons of Pandu do after assembling at the holy land of righteousness Kuruksetra? Chapter 1, Verse 1, Bhagavad-Gita

The opening chapter of Bhagavad-Gita introduces the setting, the scene, the circumstances and the characters involved determining the reasons for the Gita’s revelation. Having the supernatural ability to witness the events directly within his mind as if he was there present, Sanjaya narrates the battlefield scenes and the characters involved in the Kuruksetra war to the blind king, Dhritarastra.

Analytics has been in existence since the inception of mankind in some form or another. In the past it was based on gut, intuition, personal beliefs, pre-conceived notions; in today’s internet world, we have data.

Just as Minister Sanjaya narrates the events of the war to his king, we, Analyst Ninjas, use reports and charts to describe what happens in the business world although not nearly as real-time as Sanjaya. Depending on whom we are talking to, the metrics have simply been substituted from weapons, chariots, horses, bowmen, and conchshells, to Sales, Costs, Margins, Customers, Customer acquisition cost, Customer Lifetime Value, %Repeat, % Retention, ROI, Customer satisfaction, growth rates, % target achievement, and so on.

As business commentators, armed with vast amount of historical information, it becomes important for us to direct the management’s focus towards those one or two key areas of improvement on a daily basis.

However, I presume the style of narrative was becoming alarmingly similar to that of Sanjaya’s and this led to the need for diagnostic analysis.


A man, Bob, nearly suffocates when his tongue suddenly starts swelling up in the mouth. He is admitted to Princeton Hospital, New Jersey, where a team of physicians led by a pill-popping genius doctor diagnose Bob with everything from ALS to a post-viral autoimmune reaction. After a series of heavy metal tests, it is finally discovered that Bob had been a victim of gold-poisoning.

I’m talking about an episode titled “Clueless” from the iconic medical show, “House, M.D.”. The show follows a Sherlock-esque doctor, House, who has an uncanny knack for sifting through a wide array of weird symptoms to diagnose and save patients that no other physician could. Typically, every episode begins with a patient going through an illness or severe pain because of unknown causes followed by a diagnosis process through a series of tests (MRI, CAT, blood, urine, biopsies) conducted by House’s team to figure out what was wrong.

In the business-analytics industry, once we report on the symptoms (sales is growing / declining), it is natural to follow up with the questions of how and why. Identifying and defining what the real problem is a problem half-solved. Our blood tests or biopsies are done through slicing and dicing of information and testing of various hypotheses.

A small sample Root-Cause-Analysis would be: “Why are our sales declining?”

For each hypothesis, we go back and validate with the data available. Of course, there will be instances where we will be clueless about the root cause; most often than not, due to lack of complete information. At those difficult times, we rely on our past experiences and creative juices to design the way forward.

Or, we resort to popping pills.


In 1878, William Henry Preece of the British Post Office dismissed one of the most incredible inventions of the 19th century. “The Americans have need of the telephone, but we do not.  We have plenty of messenger boys.”

In 1903, President of the Michigan Savings Bank advised Henry Ford’s lawyer, Horace Rackham, not to invest in the Ford Motor Company. “The horse is here to stay but the automobile is only a novelty – a fad.”

What you read above is considered to be two of the worst tech predictions of all time. The thing about a prediction is that there is always an ongoing debate about the actual wording or the context of the prediction or the method of attribution; at times, the debate turns out to be more interesting than the prediction itself.

Today, if your friends text you that they are going to reach your place in exactly 42 minutes, you know for sure that they are referring to Google Maps, perhaps the best predictive application available as of date. Via crowd sourcing, Google Maps predicts the traffic on all routes and gives us the time estimate to reach our destination with staggering accuracies.

In the business context, the foundation for predictive analytics is based on probabilities. Using a combination of statistical algorithms, machine learning and forecasting techniques, we can create predictive models that not only work towards satiating human curiosity and appeasing the fear of unknown to a certain extent, but add to the utility aspect as well. Few applications listed below:

·                     Predict credit score in banking industry to approve loans
·                     Forecast sales to optimize inventory
·                     Fraud detection
·                     Discover better quality leads
·                     Swype

While predictions can be done using just one variable, the accuracy generally increases by including more pieces of information. Internet of Things, smart home systems and other advanced technological devices have inbuilt sensors to detect and evaluate thousands of variables to continuously learn and predict human behavior. It is quite impossible for us to match the efficiency of an intelligent machine with such high processing capabilities.

However, some say, Linda Goodman had perfected the art of prediction way back in the ‘60s using just three variables – sun sign, gender, and age.


“What! You are seven years old and you don’t have a life plan yet?!”

Many wrongly define Prescriptive analytics as the fourth stage of evolution. To be honest, it is actually the end goal of all types of analytics which results in more effective, data driven decisions.

If I open any app on my mobile today, I instantly get flooded with recommendations. LinkedIn shows the jobs that are tailor-made for me; Amazon Prime displays drama movies to watch; Domino’s tells me what to eat; Quora suggests new topics to follow; Flipkart, well, Flipkart points out that my nephew can always hack through my phone’s screen-locks; it recommends me to buy Avenger toys.

Being in the analytics-consulting domain, we help brands to predict customers’ purchasing behavior and determine the relevant information to display to them. It need not be a study of purchases alone; it could be a click, a hover, a search, a word, a phrase, a thumbs-up, a subtle sign of interest. Companies are using offline and online data to optimize the customer experience in-store and on their websites and apps which eventually translates to their desired outcomes. However, advising million-dollar brands with the help of numbers and charts is not an easy task. It requires the intersection of business acumen, math, technology, and dare I say, an ability to perceive or understand human behavior. But, it is indeed an opportunity and an exciting challenge to understand the way the world works.

There are, however, instances when we do go beyond customer and could be placed under prescriptive analytics. A few such projects that I’ve encountered at work:

·                     Pricing
·                     Market mix modeling
·                     Inventory optimization
·                     Discount optimization
·                     Leads optimization
·                     Product recommender
·                     Menu rationalization
·                     Subject-line recommender (Email open rate predictor)
·                     Fraud prevention


Jimmy Fallon invites android Sophia to the famous The Tonight Show.

“Jimmy, would you like to play rock-paper-scissors, robot style?”


“Okay, let’s get this game going. Show me your hand to start. Rock…Paper…Scissors…shoot.”

Jimmy shows a rock; Sophia, paper.

“I won,” she says. “This is a good beginning to my plan of dominating human race.”

There is a round of nervous laughter amongst the audience.

“Just kidding,” she says, smiling.

Nervous laughter continues.

Although Dr. David Hanson’s humanoid robot, Sophia, may be programmed before every Television show, you cannot help but admire the Audrey Hepburn look-alike machine that interacts with others with such deep human-like expressions.

Cognitive computing encompasses platforms like Natural Language Processing, probabilistic reasoning, machine learning, and speech and vision recognition to make sense of vast array of unstructured data. Its application currently lies in building robots that can take up the repetitive labor work or access areas that are either dangerous or inaccessible to us; for instance, nuclear power plants. In medicine, if a machine could read thousands of journals within a few hours, a medical diagnosis could be completed in a matter of seconds. In business, we already see chat bots taking over the customer help centers’ functions and maybe in the near future, there could no longer be any marketing, finance, IT or operations teams; just a couple of individuals to press a few buttons and monitor the automated work.

However, this is far too futuristic (?). We are still in the early stages and nobody really knows how the future is going to unfold. Technology is changing so fast that it can be, at times, overwhelming. You begin to wonder whose side to take on during the ongoing debate between Elon Musk and Mark Zuckerberg over the potentially apocalyptic future of Artificial Intelligence. Are we training machines or are the machines training us?

For now, all I can say is that our future is certainly going to be an intoxicating mix of flying cars, drones, employee-less retail stores, robot-maids, West worlds and Black Mirrors.

No comments:

Post a Comment