Why every byte counts: Enabling growth with robust data governance
OF ALL the accelerated changes ushered in by the pandemic, perhaps the most impactful has been the emergence of data as an essential public good. With more and more businesses going digital overnight in the last few years, the sheer volume of data generated and processed has consistently exceeded previous limits with every passing minute. Like with any other resource that is widely consumed, data needs to be properly stored, processed, and managed, securely and efficiently in today’s digitalised reality.
However, data’s unique characteristics mean that its value is eroded when it is deemed untrustworthy. In the high stakes, highly competitive tech ecosystem that is driving the next generation of new ways to leverage data, market factors are driving up the cost for companies without a solid data governance apparatus in place. We have all witnessed the most successful company and corporate behemoths being fined hefty amounts and taking massive reputational hits – all stemming from failing to uphold the highest standards for data management and security. It has therefore become a strategic imperative for companies to ensure their data governance remains at the highest standards, yet is nimble enough to leverage data in response to market shifts.
What does good data governance look like?
At a recent Economist Impact webinar held with National Australia Bank, FWD Insurance, AEON and Databricks, it was highlighted that particularly for industries that are evolving rapidly amidst the data revolution, robust data governance must be derived from a deep understanding of privacy, security, and operational risks, and also driven in sync with ongoing changes across the compliance landscape. Strong data governance can be built when organisations approach it as an ongoing journey instead of an objective to be reached.
In addition, the lack of understanding between data management activities and operating models, and the disconnect between legislation and security policies, were brought up by the participants as the most critical challenges to good governance. While employing data experts as stewards for each business unit was proposed as a solution, the continued shortage of relevant talent across industries also meant that there will be prolonged issues around scaling agile and decentralised data-powered businesses while still ensuring proper data privacy and governance.
What can be helpful is leveraging the latest technologies and techniques to extract the most value from data. Artificial intelligence (AI) and machine learning (ML) are solutions that can embed privacy and security into data management models, process complex data volumes, and handle data at an increasing scale. New, emerging architecture such as the data lakehouse which combines cloud-native flexibility alongside AI and ML, also hold great potential in being able to assess different and new kinds of risks as the macroeconomy progresses and matures, analyse how these market risks impact compliance and regulations, and deliver the needed adjustments to enterprise data.
Cross-border considerations
Data governance has also become more complex for organisations operating across borders and dealing with various regulatory frameworks across the world. While the European Union’s General Data Protection Regulation (GDPR) is considered the global standard, markets in Asia Pacific prefer their own localised equivalents. For example, the Personal Data Protection Act in Singapore, the Philippines’ Data Privacy Act, and Australia’s Privacy Act.
For many organisations operating in multiple topologies, the increasing complexities of cross-border data regulation is a pain point that requires more granular management. Data stores and business applications will have different types of data models, complexities, and security apparatus of their own in each market, along with the mandatory compliance and regulatory components.
To leverage the most value out of cross-border enterprise data, organisations cannot rely on the usual practices for accessing and controlling all such data. There is an urgent need to unify the disparate data compliance needs of each market while also keeping an eye on the shifts across the industry and regulatory landscape.
Seek balance between governance and enablement
In an increasingly data-driven economy, data governance will only become more critical to any business model. Companies are under pressure today to continuously rethink their infrastructure but at the same time also need to manage emerging risks that come with rolling out transformation strategies. As a result, key initiatives like moving to the cloud and decommissioning legacy platforms are often approached with a sense of caution and dread. However, data management and governance must be perceived as key components of business growth, and open-source platforms exist as essential tools that can enable the right balance of governance versus accessibility and use.
Weighing the considerations from all these factors will be no easy task. It is important that organisations first look towards their data platform as the foundation for a balanced approach to data governance. Platform architecture that is built on open source and open standards maximises flexibility, streamlines existing data stacks and eliminates data silos. Having such a bedrock efficiently ensures the appropriate level of governance is always present, better manages risk appetites of shifts in data management and control, and allows democratised, flexible access, enabling the right stakeholders to move and use data correctly and intelligently.
The writer is global field chief technology officer at Databricks
