How the expansion of analytics capabilities can help you stay ahead of the competition

The global finance industry is at a tipping point for data and deep learning. About half (45 per cent) of companies are pulling away from the others with more artificial intelligence and machine learning (AI/ML) deployments, while the remainder, often stuck in an experimental phase, is at risk of falling further behind.

A new study by London Stock Exchange Group (LSEG) found that the gap is growing between businesses capable of integrating high-quality data into production-grade AI/ML products and services, and those that aren't.

LSEG conducted telephone interviews with 482 data scientists, quantitative analysts, model governance professionals and C-level executives in the US, Europe and Asia-Pacific between May and June this year. The annual survey aimed to understand the level of AI/ML adoption in the companies, the data being used, how they hire talent, and which platforms and tools they utilise.

In Asia-Pacific, businesses are expanding their adoption of AI/ML, and are fast catching up with the US. Half of the respondents (51 per cent) indicated that their company deploys AI/ML in multiple areas of the business, up from 37 per cent last year. This year-on-year increase outstrips that of the US, where growth has levelled.

Companies now see how crucial AI/ML is to their business and are increasing their investment accordingly. About nine in 10 (94 per cent) of the respondents believe that AI/ML is a core component of their business strategy, compared to just 69 per cent last year. Almost the same proportion (87 per cent) is making significant investments in AI/ML this year, up by 10 percentage points from last year.

"The future of financial services is distributed and tech-enabled, with data at its core," says Mr Regan Tuck, Asia-Pacific Lead LSEG Labs.

GET BT IN YOUR INBOX DAILY

Start and end each day with the latest news stories and analyses delivered straight to your inbox.

Sign UpVIEW ALL

"Artificial intelligence and machine learning are quickly moving from emerging technologies to being critical tools of survival, and it's time for businesses to take the leap."

Growing appetite for data

Over the years, companies have increasingly diversified their data sources and integrated unstructured data into their models. Unstructured data refers to data that are often in text, image, or voice formats and are not readily processable.

Since 2018, the proportion of finance firms working exclusively with structured data, digitised and stored in relational databases, has steadily dropped from 50 per cent to 28 per cent.

The use of unstructured data has correspondingly gone up. This year 99 per cent of companies used such data, up from 70 per cent in 2018.

While financial data sets such as pricing information, company data and textual information from news or filings remain the key data sources for AI/ML models, firms are supplementing them with alternative data culled from non-traditional sources to build competitive advantage.

Alternative data may have been around for a decade but demand for it shot up in response to Covid-19. As markets remain volatile, finance companies turned to non-traditional data, generally perceived to be unique and timely, to interpret the impact of shutdowns on various industries.

The pandemic forced people to stay at home, accelerating the adoption of e-commerce and, in some cases, the loss of jobs. Reflecting these trends, the study found that the use of specialised data sets, such as app installs, Web traffic and job listings, in AI/ML models have doubled from the previous year.

The study also recorded a marked jump in the use of other alternative data such as supply chain, shipping, sensor data and satellite imagery. This is likely sparked by a surge of interest in trade flow data as finance firms wanted to know how ships were moving around and what commodity they were carrying in the wake of abrupt changes to mobility due to Covid-19.

Overcoming barriers to turn alternative data into insights

Despite the promise that unstructured data holds, embracing non-traditional data sets can lead to a spike in challenges around accessing, using and handling data.

Most companies do not have an in-house team large enough to ingest and analyse the massive amounts of alternative data sets that are now available to be collected or licensed. Compared to data from traditional sources, alternative data may be incomplete and difficult to integrate into existing data sets.

Across the board, the difficulties of working with new data have ratcheted up. The top two challenges identified by respondents in the survey revolved around their capacity to manage the size and frequency of the data as well as being able to link different data sets easily. Companies have to rethink how they manage and store data, as the use of AI/ML generates more data, resulting in bigger data sets that require bigger computers and bigger neural networks.

Since most firms lack processes to manage data assets, this leads to multiple issues around connecting and aligning different data sets, especially as companies use more data sets, and add non-standardised data to the mix.

Linking data is crucial to finding insights in new or alternative data sets, as they are unlikely to offer value in isolation. Open identifiers, like Refinitiv's PermID, can offer advanced search and discovery across company documents and data to tackle complex data management challenges. Linking should also be factored into the cost of onboarding new data, as data science teams often have to revisit traditional data sets to ensure they are easy to access and link.

Even data annotation, which used to be relatively straightforward with structured data, has become an obstacle as more firms use unstructured data sets and natural language processing. Some 39 per cent of the respondents saw it as a major challenge, up from 30 per cent last year.

The reliance on alternative data will continue to grow, fuelled by advances in cloud computing to process large amounts of data in real time; the rise of one-stop data marketplaces with a range of data suppliers; as well as technologies such as deep learning and natural language processing to filter unstructured data.

"Data quality and data availability, along with extracting actionable insights from data, will always be a challenge," says Mr Tuck.

"But financial institutions and fintech companies that have expanded AI/ML in their business strategies can create better data strategies, develop faster insights, and be more prepared for the unexpected."