Data science is a branch of information technology that deals with the analysis and processing of large volumes of data, which may be structured or unstructured or a mix of both, in order to find unseen patterns and derive meaningful information. Data science is useful to identify market opportunities, for process optimisation and cost reduction, and to identify abnormal financial transactions, among others.

Data science leverages several disciplines such as statistics, mathematics, software engineering, predictive analytics, data modelling, and machine-learning algorithms development. A typical project involves components that require expertise from several of these areas in combinations of varying proportions from one project, to another.

As technology finds its way into all our daily activities, so do the digital data trails we leave behind, be it at retail outlets, banks and many other places. Many organisations have realised they are sitting on a veritable gold mine of information that they can capitalise on and put to good use.

Many organisations have realised they are sitting on a veritable gold mine of information

The process to prepare data in such an application is quite intricate but in general involves the following key steps: a) identify and evaluate a business opportunity in data; b) collect and prepare data for analysis; c) evaluate alternative analytical models; d) select and test an analytical model on a test data set; e) present findings to decision-makers; and f) launch the system into service with real live data.

Once the system is up and running, what comes out may sometimes be surprising, as illustrated by the following examples:

In retail outlets such as supermarkets, planning managers put a lot of effort aimed at careful product placement on shelving, as they know that placing pasta and sauces close to each other, for instance, entices customers to buy more products. Thus, uncovering such placement opportunities clearly provides ample room to drive more sales and profits.

Based on this, a retail chain decided to launch a data-mining project on the retail data they possessed on several hundred stores and millions of retail transactions.

Surprisingly, among other insights, data revealed that there was a pattern in purchases that a statistically significant portion of receipts that included nappies for newborns also included packs of beer in the same purchase. It turned out that when fathers made the purchase following the birth of a child, they also included a treat for themselves! If managers can detect many such patterns and take steps to place products more strategically, this could lead to significant gains for this chain.

In another case, a car insurance company applied data-science techniques in order to curb insurance fraud cases while delivering excellent service to honest customers. This is not easy, as introducing lengthy customer checks can result in a significant workload for the company and also annoy customers that, in their vast majority, are legitimate. Thus the insurance firm sought to use data- science techniques to extract key indicators of fraudulent claims.

Among the key findings was that a high percentage of the fraudulent claims occurred in recently opened policies, and more surprisingly, such claims clustered around weeks of holidays. Armed with this knowledge, the company could make changes to internal procedures to be in a better position to stop such claims from being paid, while keeping the administrative overheads contained and legitimate customers happy.

There is no shortage of other examples, ranging from facilitating background checks for customers seeking credit from banks to applications in healthcare via data from wearable devices that monitor and prevent further health problems for patients with chronic illnesses.

Data-driven decisions pay back substantial dividends to organisations, so much so that many have dedicated teams entirely dedicated to this work. As demand for such teams grow, it follows that those who either possess or are able to acquire data-science skills are very much sought after.

Johann Mifsud is an executive at eSkills Malta Foundation.

This article was prepared by collating various publicly available online sources.