Big Data: Exploring the difference between foresight and insight.

Recently, however, with the rapid advancement of Big Data data technologies, we are beginning to get some of the level of personal care and attention back. Tools like AI are digesting siloed, disconnected data sets and making sense of it. The petabytes of dark data are now starting to enrich customer profiles and enabling meaningful personalization, the essence of smarter customer experiences. The days of silently filing away information about every interaction between every user of a product or service, so the aggregated mass of metrics is so complex and massive that it has been almost entirely useless, are coming to an end.

Recently, in Forbes Magazine, Phani Nagarjuna, Sutherland’s Chief Analytics Officer, discussed the big challenges that Big Data creates. Among other aspects of this transition away from dark data, his focus is on the value of predictive and prescriptive insights, and using that insight in real time.

In advertising, for example, predictive tools are being used to deliver targeted messages at the time when a user is most likely to respond. In the telecoms industry, analytics is being deployed to predict periods of high usage so that additional equipment can be brought online to anticipate peak demand times. In retail, insights are used to predict when interest in a product is likely to peak so that stock levels can be maintained accordingly. And beyond those kinds of efficiency applications, we’re also seeing the tech used to increase revenues, like the Sutherland’s fashion retailer client who recently harnessed analytics and AI technology to provide a personal shopper service, tailored to its users particular style, driving a 67% increase in sales conversions within a year.

As more channels for data acquisition emerge (think about the connected home, the Internet of Things, and the spread of digital assistant hardware like Alexa in home environments) the volume of data that companies will be able to collect will continue to grow, at near exponential rates. In that scenario, advanced tools like predictive analytics become central to avoiding a return to the bad old days of dark data.

But, as Nagarjuna notes “You must be smarter than your data”. The limits of predictive analytics, much like the old days of web metrics and setting cookies to track user behavior for ad targeting, are found in the way humans engineer processes to make use of the data. Which means the rise of smarter data analytics is also driving the growth of design research and anthropology in service and product design. The numbers tell a story, but people write the script. Combining analytics and human empathy in design is the most likely way companies will harness the more holistic understanding of users’ behavior, interests and habits which is afforded by advances in analytics.

Combining analytics and human empathy in design is the most likely way companies will harness the more holistic understanding of users’ behavior, interests and habits which is afforded by advances in analytics.

This raises an interesting point. Big data provided the business world with trends and the ability to form a narrative to describe and predict the movement of the crowd. Advanced, predictive systems have evolved that big picture to include a means to predict the movement of individuals. But in both cases, it’s the very human skill of studying and relating to the lives of your customers that defines the difference between data that offers actionable insights and white noise. So when Mrs. Jenkins orders the low-carb gluten-free carrot cake, the system can prompt you to hold the sugar in her tea and reach for the organic Agave syrup instead. And to make sure she comes back, remember to tell her she’s looking like the new diet is working great for her, too…

Follow Phani Nagarjuna’s recommendations for tackling the 3 biggest big data challenges on our parent company blog.