Why companies need citizen data scientists
Data analysts need not be a specialised priesthood. The insatiable demand for actionable data calls for any qualified employee and leader to trawl "data lakes".
THE insatiable appetite for data created by an ever increasing need for services, information and response designed to meet the needs of the digital native is overwhelming enterprise resources to keep up.
Data-driven insights are enabling companies to improve their operations in many ways, from better customer insights to more efficient supply-chain logistics to enhanced product design.
However, the sheer volume of data, increasing exponentially day by day, has outpaced the availability of qualified talent able to analyse data and deliver meaningful, actionable insights for organisations.
This shortage of talent is a critical issue. According to a report in Nikkei Asian Review, the shortage of available talent to decipher data patterns and models is a pressing issue for South-east Asian start-ups' growth.
While many of these startups transform from small garage-based projects into substantial enterprises, the competition with international enterprises and better-funded companies for data scientists is crippling their potential for growth. The report states that "in South-east Asia, unlike in Silicon Valley, there is not a deep reservoir of executive experience".
An Apec report titled "Data Analytics and Skills Shortage" similarly warns that, although data science and analytics skills are in high demand in the Asia-Pacific, supply is critically low.
The pressure is felt in all markets and across all sectors, including manufacturing, healthcare, retail and finance. The public sector, too, is constrained from offering improvements in the efficiency, transparency and cost-effectiveness of citizen services due to the lack of talent.
Artificial Intelligence (AI) and machine-learning solutions are getting better all the time at untangling the giant webs of data that pour into their servers every day. While AI tools are certainly needed to perform tasks such as classifying data and identifying relationships, these solutions lack the creative flair that comes from human intuition.
Given the ever-increasing volumes of data and the need to understand the collected information, companies cannot rely on tools without the advantages of human talent. Translating insights derived from data into effective or even game-changing business decisions calls for highly trained digital-analytics experts.
But data scientists are hard to find, expensive and sometimes choosy. Finding and retaining the right digital-analytics talent is easier said than done; it's no wonder they are sometimes called "unicorns".
In recent years, demand for data analysts and scientists around the world - and particularly in South-east Asia - has steadily climbed, and is outstripping the supply. An AT Kearney study found that as many as two-thirds of leading companies have been unable to meet the demand for analytics talent. This is a staggering proportion.
So how should enterprises square this circle?
THE AGE OF THE CITIZEN DATA SCIENTIST (CDS)
It is time to welcome the age of the citizen data scientist or CDS.
Enterprises already have in place subject-matter experts and line-of-business leaders whose routine responsibilities find them facing everyday business issues that could benefit from big-data solutions. Organisations can leverage these internal skills to bring the basic data-science expertise in-house for advanced analytics, while minimising the burden on organisational resources.
The role of CDS can be filled for example by industrial engineers, who, once they are provided with the right tools, can leverage their background in maths, statistics and modelling.
Engineers can use these skills to extract more value from the analytics solutions they use, as well as dig deeper into data provided to them.
Analytics providers are working to build tools to support the CDS. For example, they are creating data visualisation tools, powered by an underlying analytics engine, so it is easier for the CDS to create more "ah ha" moments using data, algorithms and models.
What this approach will lead to in clear-minded organisations is an effective, standardised improvement in the complex process of analysing massive amounts of data implemented by smart employees trained to handle specific types of analysis and interpret the results for the benefit of their lines of business. This will be in addition to the extravagant promises of exponential productivity growth.
From a corporate point of view, the CDS is the key to using data to predict the future. Instead of focusing on what happened in the past, managers realise that predictive or prescriptive analytics can help them forecast what will happen in the future and what to do about it, enabling their employees to be action-oriented and base their decisions and actions on data.
Once analytics is viewed this way, it is no longer a mythical function for a highly specialised priesthood, but becomes democratised that most employees and their leader is thinking about at all times. And it is out of the need to feed this concept of analytics for the masses, in the face of a shortage of data-science talent that the CDS has arisen.
Unstructured data from sources like transactions, sensors, web logs and social media is already filling data warehouses to capacity. Called data lakes, this is estimated by Cisco to grow by 500 zettabytes per year by the end of 2019, hence the move by enterprises to extract hidden value in data lakes. (One zettabyte is equal to one trillion gigabytes.)
This is an ideal opportunity for CDS to prove their value by working with AI solutions to gather the insights that can inform business.
A data lake is home to unlimited quantities of both raw and refined data, and is scalable to huge volumes; the task is to find correlations across data elements and build entities that are of meaning and value to the enterprise.
BUILDING A SUPPORTIVE ENVIRONMENT
Given the growing importance of the CDS to the mission of extracting meaningful insights from data, what can enterprises do to build a supportive environment for them?
First, it is necessary to recognise that not everyone can be - or wants to be - a CDS. It takes some experience and a detailed understanding of aspects of the business, knowing what questions to ask and wanting to find the right answers.
Secondly, there is a huge number of tools available to the CDS, many with overlapping capabilities. For every tool the company makes available, there must be the necessary support, in terms of training, reading materials, web links and "experts" to help with particularly complex problems.
Third, make sure the CDS document their work - metadata combined with the ability to have a business glossary, and the ability to track the transformation elements created by the CDS, is critical to the process.
And finally, before the CDS can dive into the data lake, the enterprise needs to curate what goes into it and who can access it - governance is crucial.
What's more, protocols must be set to determine how and when results of the CDS' work, or solutions based on those results, are used to drive business decisions.
When these issues are recognised and addressed, CDS are sure to deliver significant ROI and prove their value as a complement to the expert data analysts who are in such short supply.
TRENDING NOW
South-east Asia has lost Nvidia’s best salesman
Demand, tech capabilities position Apac and AI as Skyscanner’s next growth frontiers: CEO Bryan Batista
Peter Kwee’s Ridley Park bungalow on the market with S$100 million asking price
Beyond the greenback, Vietnam’s big borrowers turn to Asian-currency bonds