What’s in the Metadata?

In a recent blog post, the Data.NSW team make a compelling case for bringing together people across the government to promote smarter use of data.

Alex Peattie then builds on this by Uniting Data Champions in NSW Government and rightly points out that there is a wealth of data in government agencies sitting around, unloved and unutilised.

While the Data Champions Network tackles this problem by gathering the human capital associated with data, there is an ocean of value in maximising the utility of data itself, e.g/ recognising the ‘unloved and unutilised’ data as Data Assets. Promoting good Metadata practices takes us one step closer to achieving this.

So, what is Metadata (and what it’s not)?

The simplest way to define Metadata is ‘Data in context’ – business and technical context. All data is created by someone, for someone, for some reason, somewhere and somehow. Consequently, another way to summarise metadata would be the ‘who, why, what, where and how’ of data. It really is the human aspect of data.

The title, subject & author of a report, access permissions to a database, table dimensions, metric definitions, file types and size and anything that adds context to data – business, technical or otherwise – is Metadata. For any enterprise, it is everything that describes the data assets owned by the enterprise. It is not however, the data itself.

Think of it as a classical library card that lists information about a book – it’s title, author and location. While the library card lists a fair deal of information about the book, one must open the book itself to read its contents.

Metadata on (and at) Data.NSW

The NSW Government Open Data Policy recognises the value of metadata in ‘finding, understanding, controlling and managing’ datasets.

Agencies are required to include Metadata and Data Quality statements when publishing datasets on Data.NSW to help users navigate, access and consume data.

The Data.NSW team are therefore committed to supporting metadata capabilities – for enterprise data within government, and for open data on Data.NSW. One of the most viewed and downloaded datasets on Data.NSW – All Water Data - Groundwater construction – is a great example of open data accompanied by good metadata.

Internally, the governance team at Data.NSW are helping business users navigate enterprise data with the help of a data catalogue that contains crucial metadata.

Is this data sensitive? Where is it stored? Who has access to it today, and who accessed it in the past?

More importantly, how quickly can access be provisioned or restricted to it based on changing business requirements?

Questions like these and many more are front and centre in government enterprises where data and data sharing are central to policymaking.

Structured metadata helps address them. More importantly, when embedded within a robust data governance framework, it gives confidence to data owners, custodians and consumers alike.