Does big data herald the end of consumer unpredictability?

Sunrise (credit: mehul.antani)

Predicting consumer behaviour has always been an essential activity for organisations as part of their decision making processes. Now organisations can leverage the rich pools of data, particularly from social media, to predict behaviour, find new customers, and improve retention and satisfaction.

Will customers switch to our supermarket if we open one in this town? Will this customer fail to make repayments on our loan? Which of our customers will switch to a different network provider when their contract expires? To answer such questions organisations have tended to rely on traditional market research or looked within their own data sets: sales records, customer service enquiries, credit scores, etc.

All sound sensible approaches. But rather than using analytics to tell you what has happened in the past, most organisations would like to  predict what consumers are going to do in the future:

  • Systems that identify which customer acquisition targets are likely to be most valuable to you in the years to come.
  • Systems that zoom in and out on human behaviour, from the macro level future trends that will influence strategic decision making, down to the micro level and the specific behaviours of an individual consumer.
  • Systems that can be integrated with the levers within your organisation that can help you influence consumer behaviour for your benefit.

And all of this in real time.

This is a bold vision but the big data era that we find ourselves in is rapidly rushing us towards systems that do just that. But to get this benefit, we need to look beyond enterprise data and commercial segmentation models and embrace other sources of data:

  • sensor data from the smart meters and devices that are in our homes, vehicles, environment and even on our wrists or clothing
  • mobile data from our phones and apps
  • data on where and how we search for people, places and things
  • and, of course, social – that continuous stream of content through which we publish how we feel, where we are, what we think, what we like and don’t like, who we’re connected to, what we’re doing, what we are going to do…

Listening and social analytics tools can help you track and understand what’s being said out there but the leap in value comes when you combine this with enterprise and the other data sources above in a meaningful way. This allows you to pull the right signals out of the noise and use the outputs to drive real actions in your organisation.

An example of how this has been done

  • Wonga provide automated, small same day loans. They found that traditional credit scoring was completely non-predictive for its target consumers.
  • Instead they developed an algorithm that examines a broad range of data sources, including social media, to make real time assessments of a person’s credit worthiness. Wonga profits tripled in 2012.
  • The constant influx of new data from the loans that they do accept enables the algorithm to be continuously tuned and its predictive capabilities improved.

The technologies to do all this are available. What’s required are the smarts – knowing which questions you can ask of the data and how your organisation can use what you find.

You need to be able to understand the datasets and crucially how to match them up. How do you know that this person out on social is the same person who phoned your call centre last week? It’s also not just about the hard data, models and algorithms; the human factors are arguably even more important. How are you going to use social data respectfully and responsibly? How are you going to mitigate and compensate for the distortions that social data introduces? How are you going to establish trust with consumers so that they willingly share their data with you because of the value or benefits that they receive as a result?

What do you think? Can examining the increasing volume of data from ever more sources enable us to better predict consumer behaviour?

Read more insight and case studies about big data.

Enhanced by Zemanta