This article first appeared in Enterprise, The Edge Malaysia Weekly on February 12, 2018 - February 18, 2018

Here is a big question: How big is big data? Is it as big as the 2016 gross domestic product of Bahrain (US$31.8 billion)? Or Costa Rica (US$57.4 billion)? Or Qatar (US$152.4 billion)? Or Bangladesh (US$221.4 billion)?

Incidentally, Malaysia’s GDP was US$296.3 billion while Singapore’s was US$269.9 billion, as per the World Bank’s estimates.

So, the big answer: It depends who you ask.

According to Hamburg, Germany-based Statista, the global market for big data was worth just under US$34 billion last year, closer to the 2016 GDP of Bahrain. However, by 2026, the global big data market is likely to jump to US$92.2 billion, just below Ukraine’s 2016 GDP of US$93.2 billion.

Another agency, Wikibon, notes that the global big data market is on track to grow from US$18.3 billion in 2014 to a whopping US$92.2 billion by 2026, representing a compound annual growth rate (CAGR) of 14.4%. Wikibon is a global community of consultants who believe technology adoption can be improved through an open-source sharing of free advisory knowledge.

The most bullish outlook comes from International Data Corp. Last March, IDC updated its Worldwide Semiannual BDA (Big Data & Analytics) Spending Guide, which stated that the global BDA spend would reach US$150.8 billion in 2017 (equal to Qatar’s 2016 GDP), up 12.4% from the previous year. Commercial purchases of BDA-related hardware, software and services are set to grow at a 12% annual clip between now and 2020, with sales crossing the US$210 billion mark (Bangladesh’s 2016 GDP) by then.

The devil is in the details, or more precisely, in the definition. Gartner defines BDA or “advanced analytics” as the autonomous or semi-autonomous examination of data or content using sophisticated techniques and tools, typically beyond those of traditional business intelligence.

BDA helps one to discover deeper insights, make predictions or generate recommendations. Gartner’s definition includes data and text mining, machine learning, pattern matching, forecasting, visualisation, semantic analysis, sentiment analysis, network and cluster analysis, multivariate statistics, graph analysis, simulation, complex event processing and neural networks.

IDC defines BDA as a new generation of technologies and architectures designed to extract value economically from very large volumes of a wide variety of data by enabling high-velocity capture, discovery, and analysis. IDC’s definition also includes the hardware that powers BDA solutions.

“After years of traversing the adoption curve, big data and business analytics solutions have finally hit mainstream,” says Dan Vesset, IDC’s group vice-president for analytics and information management.

“BDA as an enabler of decision support and decision automation is now firmly on the radar screens of top executives. This category of solutions is also one of the key pillars of enabling digital transformation efforts across industries and business processes globally.”

Which sectors will lead in BDA? Four of them: BFSI — banking, financial services and insurance; manufacturing (discrete and process); government (especially taxation, internal/external security and law and order); and professional services (such as hospitality, accounting and auditing). These sectors would have spent US$72.4 billion on BDA solutions as at end-2017.

“They will also be the largest spenders in 2020 when their total investments on BDA exceed US$101 billion. The fastest growth in BDA spend will come from banking (CAGR: 13.3%), followed by healthcare, insurance, securities and investment services, and telecommunications, each with a CAGR of 12.8%,” says IDC.

Jessica Goepfert, IDC’s programme director for customer insights and analysis, says that outside of financial services, several other industries present compelling opportunities. “Within telecoms, for instance, BDA is applied to help retain and gain new customers as well as for network capacity planning and optimisation.”

How about the media? “The media industry has been plagued by massive disruption in recent years, thanks to the digitisation and massive consumption of content. BDA can help firms make sense of and monitor their readers’ habits, preferences and sentiment,” says Goepfert.

Malaysia’s moves

What’s Malaysia doing in the BDA space? “BDA is no longer uncommon in Malaysia and many organisations have begun their big data journey. The BDA market for software solutions alone is set to top RM595 million by 2021, up 10.9% from 2016. Between 2016 and 2021, Malaysia’s CAGR will be 8.8%, which is good,” says IDC.

But it can be better. “By using analytics capabilities, organisations can run their business operations better, improve the overall customer experience and expand their product and service offerings by increasing their competitive advantage. Organisations in Malaysia have started to prioritise analytics solutions. C-level executives are constantly seeking intelligent solutions to convert raw data into meaningful information to make strategic decisions in their businesses,” says Ng Quan Xiong, IDC software market analyst for Malaysia.

The early adopters are small and medium enterprises as well as those in traditional manufacturing. “I am amazed at the level of big data adoption by start-ups such as ServisHero and iFlix. Among large businesses, Top Glove Corp Bhd has taken the leap of faith by using BDA to enhance output,”

Malaysia Digital Economy Corporation (MDEC) CEO Yasmin Mahmood said at Big Data Week Asia in October last year.

Top Glove is listed on both Bursa Malaysia’s Main Market and the Singapore Exchange’s Mainboard. It is the world’s largest rubber glove manufacturer and has 2,000 customers in 195 countries. The company worked with a Malaysia-based big data firm on a BDA solution that uses sensors to monitor the mixture in rubber production, automating what was previously a manual process.

IDC thinks visual analytics is a rising BDA application in the country. “Visual analytics, which has great potential to impact business processes, is taking the lead in Malaysia because of its functionalities to support the analytic workflow, provide immediate access to readily consume analytics and enable non-IT staff to analyse, visualise, share and review data,” says IDC’s Ng.

The crunch

So where’s the crunch? Talent, or the lack of it. That is not a Malaysia problem per se, but a global one.

The McKinsey Global Institute quantified way back in 2011 that the US alone would be short of 140,000 to 190,000 data scientists — or people with “deep analytical skills” — by 2018. That forecast has been grossly underestimated. Now, McKinsey forecasts that millions of people will be needed to serve as translators of the results of the work of data scientists to the rest of the organisation.

“It may be easier for domain experts, with deep knowledge of the business in which they are engaged and the requisite interpersonal skills, to obtain sufficient knowledge about data analysis to act as the translator for data scientists. That is compared with data scientists trying to gain enough knowledge about the domain, especially the language of that domain,” Forbes magazine quoted The Sloan Management Review’s report as saying.

In other words, it will be easier for healthcare professionals, for example, to learn about data analytics than it is for data engineers or software developers to grasp the complex and intricate world of medicine or genetics.

What’s Malaysia doing to address the talent crunch? The government has set up a unique public-private partnership (PPP) initiative called the Data Star programme. This is a data science finishing school designed to fast-track the development of world-class data professionals. It is a joint effort of MDEC and Asean Data Analytics eXchange. The goal: help Malaysia churn out 20,000 data professionals by 2020.

The goal is ambitious, the demand is huge and the market stretches far beyond Malaysia. The US is — and will continue to be — the largest BDA market, with a market size of more than US$80 billion this year. The second largest region will be Western Europe, with a BDA spend of US$35 billion, followed by Asia-Pacific ex-Japan at US$14 billion. IDC estimates that Latin America and Asia-Pacific ex-Japan will see the fastest growth in BDA spend, with five-year CAGRs of 16.2% and 14.4% respectively.

From an end user perspective, very large businesses (those with more than 1,000 employees) will be the biggest BDA spenders. They will cumulatively spend US$100 billion on BDA this year, accounting for up to 60% of all BDA spend, says IDC. SMEs, especially those with fewer than 500 employees, will account for 25% of the global BDA spend this year. That is a huge opportunity.

Can Malaysia take advantage of this? For Malaysia to be a BDA hub, it needs not only to churn out trained talent as fast as possible but also kick-start a number of BDA proofs-of-concept. PoCs help companies and individuals experiment and fail, and in the process, learn to understand, clean, assimilate, analyse and deal with huge swathes of data, both from structured sources (such as databases) and unstructured sources (such as photographs, email, social media feeds, video streams and weather and traffic data).

The big BDA wars will be fought by countries such as China, India, Russia, Brazil and the Philippines. They are also out to groom talent, conduct PoCs and encourage local companies to innovate in the BDA space.

A few killer applications could potentially tip the scales in niche BDA spaces. Malaysia has more than 25 multinational corporations as well as innovative local players already working on key BDA applications.

The bottom line for Malaysia: More start-ups need to blossom, more opportunities need to open and more PoCs need to pop up. The PPP could be the perfect platform for piloting performance.

Raju Chellam is vice-president of new technologies at Fusionex International, Asia’s leading big data analytics company

Save by subscribing to us for your print and/or digital copy.

P/S: The Edge is also available on Apple's App Store and Android's Google Play.