Boosting data quality: How to make a plan for storage, usage and governance

DATA storage and consumption are growing rapidly as businesses undergo a digital transformation and embrace machine learning.

Businesses need to have high-quality, reliable data, which means data governance is an essential practice.

For those seeking to improve their data governance, here are some considerations.

1. Have a plan for your data

Collecting data is useless without having a plan for how you will use that data and how you will keep that data safe and complete.

Data needs to be treated consistently across all departments and areas of an organisation. That’s where data governance comes into play. No matter what industry you are in, you need a system of rules, processes, procedures and accountability for every stakeholder that interacts with data.

Data governance can vary by industry, but broadly defines:

• Who can take certain actions involving data;
• What data individuals can work with;
• When and where data is collected and processed;
• How data is handled.

Managing and securing high-quality data is a competitive advantage. According to Gartner, poor data governance can cost organisations an average of US$12.9 million a year.

Organisations that have a well-defined data governance process benefit from fast and accurate data availability, faster and better-informed decision-making, improved data security, cost savings and revenue growth.

2. Put all the data you have together

Despite the rapid adoption of new technology and the importance of data, many businesses are not embracing or accessing the full benefits it has to offer.

IBM estimates that up to 80 per cent of an organisation’s data is “dark data” – data that employees take offline or place in a silo that removes control and visibility from the rest of the business, such as in a spreadsheet or saved on a local desktop.

At Domo, we have found that 70 per cent of businesses are making decisions with siloed or insufficient data, and 47 per cent of new data records contain at least one critical error, preventing businesses from unlocking the true potential of their data, and maintaining data quality and governance.

To maintain data visibility and privacy, businesses need to consider data integration. This involves moving vast amounts of data into a centralised location, or synchronising trusted repositories – whether it be the cloud, data warehouses or legacy systems.

Democratising data – particularly dark data – and hosting it collectively, ensures it is accessible to everyone in the business, without the need to request it from IT teams.

The improved visibility and accessibility also allow more time to be dedicated to revenue-building activities.

3. Make data available, but also consider who gets access to what

While democratising data is crucial, datasets also need governance. Data integration into a system such as the cloud does not automatically limit control and governance over data.

Businesses must ensure each member of the organisation is assigned a role or attribute – meaning each has certain levels of access from the minute they log in.

An organisation may also wish to assign an individual or team to monitor the system, ensuring that the correct people have access to certain datasets, while limiting access for others. This further ensures data privacy and integrity.

4. Create a data governance framework

Data governance is a critical responsibility and no simple task. Having an organisational framework with roles for oversight, management and accountability is critical.

Any governance framework will include policies, processes, structures and technologies that make such control possible.

Each framework will be unique to an organisation, and align with the company’s mission statement with regard to data, regulatory obligations, goals, key performance indicators and methods of accountability.

When creating a framework, consider:

• Data architecture
• Data modelling and design
• Data storage and operations
• Data security
• Data integration and interoperability
• Documents and content
• Reference and master data
• Data warehousing and business intelligence
• Metadata
• Data quality

Cybersecurity remains a challenge in Singapore, with 84 per cent of organisations having experienced some form of cyberattack between March 2022 and March 2023 – ranking first in the world for attacks during this period.

With this in mind, a key to maintaining data governance and quality is education and data literacy. When integrating data into a new system, some organisations may still find data is being transferred back into silos.

To ensure users are not continuing to export data into other tools such as spreadsheets, which can quickly pose a security risk, organisations should consider conducting training workshops through their IT team or third-party specialists.

This will help speed up staff adoption of the new system while also encouraging correct use across the organisation.

With these points addressed, data can be compiled, analysed and distributed at much-greater speed and scale.

Data teams will be able to focus more on innovation and less on simple data delivery, while other departments and teams can improve decision-making and reporting, and, in turn, increase sales and revenue.

The writer is chief data officer at data platform Domo