Jade ThirdEye Sat, Sep 5, '20 15 min read

Data is the new oil.

December 8 2019 Wuhan in the Chinese province Hubei a man is diagnosed with a coronavirus currently unknown to the world. Two weeks and a dozen or so known cases later an algorithm flags this new virus and its characteristics, which is highly similar to the SARS virus that swept across South East Asia about 15 years or so ago.

Chinese authorities responded by enacting the so-called containment protocol, putting drastic measures in place to isolate citizens in Wuhan from each other and working closely with national and international authorities to track down any potential carriers of this virus.

These are invasive measures but having the data to back it up was crucial to get buy-in from the international community. As a result, the virus was effectively eradicated by mid-January 2020. Now, this obviously did not happen. Could it have happened? Maybe, but only with the necessary data, which is trusted, that can be turned into actionable information.

COVID-19 is a pandemic that is still severely impacting a large part of the world – it has the potential to completely change society as we know it. Luckily the lack of available information, despite a trove of data, usually doesn't have the same consequences but it does have consequences.

Endless possibilities. A stark reality.

What we're currently witnessing in real-time, COVID-19 playing out on the world stage, often happens at a different scale in our businesses. Over the past ten years, we have frequently heard the promise of data or even big data being akin to a gold or oil rush – being able to harness the power of data to unlock incredible insights and use them to find opportunities, mitigate risk, and enrich the customer experience. Some of the biggest organisations in the world have managed to deliver on this promise, and they are based almost exclusively on a vast amount of data and the ability to turn data into information. Just think of Google, Facebook, Amazon. However, most organisations are drowning in data but it's rubbish.

In our line of work, we come across many businesses who are thinking about applying AI in the form of machine learning, who want to put their data to use by fuelling predominantly predictions and forecasts. Yet organisations struggle to grapple with the state, trust, or truth of their data. We see the hopes of new CRM, ERP, or marketing automation systems delivering a reliable set of data to work to utilise. And further down the track, we have witnessed how these systems have failed to deliver their desired outcomes. And this brings us to our view on data.

How to get more value out of your data

Data is a lot like crude oil. In its natural state, data is useless – it is sloppy, messy, and not very useful until it is refined. Only then can the data be used to power all sorts of personal and business ‘vehicles’. While the data processing pipeline can be viewed in many ways, we use a very simple, three-phase concept:

  • Generating and collecting data
  • Cleaning, refining, and consolidating data
  • Creating meaningful information from data

Our intention for the remainder of this article is to leave you with a new tactic for each of these phases, which will help you take your data capability to the next level.

Generating and collecting data

Data comes in many different ways - through websites, apps, analytic tools, customer conversations, business processes, and core systems to name a few. The volume is significant yet often it’s not fully clean and there are normally huge gaps. So even though there is a sea of data, you still need more data to make sure you have a complete(ish) data set – for some, this means a single view of your customer. So what can be done to fill the gaps?

Data strategy #1: Look for data in new places

Data is waiting to be collected. Consider the stockpiles of PDFs and documents filed away in your servers. Many of these files aren’t connected to your current customer or finance systems. By using text mining tools, it’s possible to extract data from this and populate it into your existing systems – filling more gaps. It’s also worth considering CCTV footage that you have been collecting for years. With computer vision, you can train an AI to identify all sorts of activity and data points, such as empty car parks, out-of-stock items, and customer movements. As mentioned, this type of data wasn’t readily available in the past yet it is using owned channels, so the investment is minimal.

Cleaning, refining, and consolidating data

Now if you do go down this path and generate new data, keep in mind that you will also have to think about the different ways on how to process and store this data. For these examples, the text mining generates and stores data at the same level that was used before. However, with computer vision, data collected can become very expensive very quickly, as video generates data incredibly quickly. Thankfully there are cloud storage options open to you, and there are also ways to store information as text, which drastically cuts down capacity requirements. Either way, make sure your IT team is across what you’re setting out to do.

Data strategy #2: Decentralise your data

If you think about the trends that have happened over the last five to ten years, SaaS is a classic example of platforms emerging where you no longer have a single central ERP system, which is your one source of truth of everything in your business. What you end up having is islands (or silos) where there is a system that looks after CRM. You might have a completely different system for finance, billing, inventory management and so forth.

Part of the reason why applications like Salesforce in CRM have become so popular is that people now try to do the same thing that was done 20 years ago, adding modules on top of the same system. On the most part, this doesn’t work because these systems have been built to do one thing very well. And yes they are extensible and configurable but they are typically not made for every single thing that you need to do. Businesses simply end up creating more data silos.

A very simple example of such a problem is a CRM and dispatch system. The CRM system will have your address information about your clients. The dispatch system will have operational information about a particular action that needs to occur. If the two systems don’t talk to each other, information needs to be duplicated. Now, if a client moves office or house, that information will need to be manually changed in both systems. This raises the vital issue around trust.

Decentralising data is a little bit like a running sushi restaurant where everyone has the chance to grab exactly the same thing. Instead of the historical method of sending data from one system to the other, you put data on a travelator (distribution mechanism) and other systems can take or use the data as they please. Each data set has a main (or a master) system responsible for maintaining the integrity of the records, so other systems can trust the data it receives.

Thanks to the ‘travelator’, it is relatively easy to add or remove other systems without risking the performance of the whole tech stack. This gives you greater confidence to trial various projects and increase the speed at which you can innovate.

Create meaningful information from data

It’s one thing to generate more data. It’s another thing to make sure other systems have access to the most up-to-date data across your business. The final piece of the data puzzle is to your data – make it really easy for people to understand it and make decisions from it.

Data strategy #3: Narrate your data

At a basic level, you could view this as creating an executive summary from a spreadsheet. Drilling into this more, consider someone who is comparing five insurance companies and have just entered their details into a comparative quote and apply tool. There are some very complex calculations that go under the hood of such a tool. Imagine if the customer was exposed to that and asked to run the sums themselves? Rather than overwhelming them, the customer can be presented with three options that best suit their needs, perhaps Good, Better, Best options.

From the multitudes of options available to that insurance customer, they were able to make a far more informed decision because their attention was focused on what really mattered.

The magic happens with data visualisation, which can take the form of dashboards, charts, and drill-downs. When so much is at stake, having a very clear and accurate understanding of what the data is illuminating can be the difference between a business becoming the next Apple or the next Blackberry.

Yes, data can be messy, but it also can live up to its reputation. If you want to boost the value you are getting from your data, and want to get that up and running in weeks rather than months or years, look for data in unusual places, make sure that data circulates well, and lastly, don't just think about pure visualization. Think also about other narration think about turning it into a paragraph or two that draws attention to the one thing you need to get across.

Talk to us about making data meaningful