Skip to main content

A NSW Government website

Data.NSW

Data Management Life Cycle


 

Back a page 
Data Management and Practice
Next page 
Data Quality

 

The IDMF incorporates the key principles of the IMF into a schema designed to be applicable to infrastructure data across the asset lifecycle:

IDMF Diagram 15
Figure 15: IDMF data management lifecycle

The two additional core features of the lifecycle that apply across each stage are governance and security.

The application of the data lifecycle as a standardised process across the life of an asset enhances communication between stages of the lifecycle, as different stakeholders inherit data from previous data custodians.

Create / Capture / Collect

The data management lifecycle begins with planning for the creation, collection, capture or acquisition of data. In the context of the IDMF, it is also the entry point for each stage of an asset’s lifecycle, where data has been shared or inherited from the previous stage of the asset lifecycle.

Organise / Store

In order to support Asset Management, organisations must organise and store large amounts of data in an appropriate and efficient manner. The choice between storage options depends on many factors and is never one-size-fits all.

Storage environments should meet organisational information requirements (OIR), be compliant with relevant legislation and policies, be interoperable across different information management systems, and allow for the data to be stored and managed for the life of the asset.

Storage selection will be impacted by specific considerations, including:

  • Intended use of the data (by whom and for what purpose)
  • Characteristics of the existing database or information systems
  • Type and volume of the data to be integrated
  • The frequency the data will be accessed
  • The frequency the data will be updated
  • The speed at which data will need to be accessed, particularly in emergency situations
  • Currently available technology

It is important to maintain a common environment for data storage. While data may be housed in a number of different source management/storage systems, a single portal can provide a single point of access that draws that data together for viewing and analysis. A single source of truth helps to avoid the duplication that results from storing the same data in different locations. Apart from the extra work needed to update data in multiple locations, duplication increases the risk that data will be amended in one location but not others, increasing the risk of incorrect or out-of-date information being used.

Analyse / Use

Data analytics is a key part of the infrastructure management process because it allows large amounts of data to be transformed into useful information. Analytics can help identify and solve problems or predict issues that have not happened yet. For example, analytics can be used to optimise infrastructure management by enabling:

  • more effective and efficient maintenance programs
  • monitoring asset condition and performance
  • identifying infrastructure gaps
  • meeting minimum reporting requirements prescribed in legislation and policy
  • ensuring effective capacity utilisation and planning.

Share

When working with data, it is important to remember that data can be closed, shared or open because of the sensitivity of the data, the level of risk associated with the data, and the permissions given on how it can be used and published. By understanding where data comes from, who can use it, and what can be done with it, the opportunities associated with sharing and using data can be optimised.

The Data Spectrum diagram developed by ODI illustrates the differences between closed, shared and open data.

Figure 17: The data spectrum in infrastructure
Figure 17: The data spectrum in infrastructure

Benefits and safeguards in sharing data

Data sharing is a fundamental requirement for the management of most infrastructure assets. This is because on most infrastructure projects, there are numerous stakeholders that provide specialised services across the asset lifecycle. The need to exchange data between stakeholders in a timely and efficient way is key to the success of the asset’s management. For example, the management of an asset relies on data exchange between a design team, a construction team, manufacturers and suppliers, as well as operation and maintenance teams.

Sharing of infrastructure data and other data across NSW Government is encouraged, provided appropriate protections are in place. The Data Sharing (Government Sector) Act 2015 aims to remove barriers to data sharing within NSW Government, and to facilitate and improve government data sharing.

However, while NSW Government encourages the release of non-sensitive infrastructure data, the release of data should always take full account of security and privacy considerations. Guidance is available from the Information and Privacy Commission’s guide on Data Sharing and Privacy.

Various arrangements can be made to between organisations to establish data sharing processes, including Memorandums of Understanding or Data Sharing Agreements. Further information is available at Data.NSW.

Closed or Secure Data

Closed or secure data is data that only people inside an organisation can see and use. National security, confidential business reports, and work emails are examples of data that organisations keep secure. There can be good reasons why data is closed and why closed data should not be available in the public domain.

From an infrastructure perspective, closed data is generally associated with sensitive or critical infrastructure or operations. The following references provide guidance on the ability to share (internally or externally to Government) infrastructure data:

  • Federal government requirements on critical infrastructure assets in the Security of Critical Infrastructure Act 2018
  • NSW critical infrastructure, including the ability to improve data sharing through the Trusted Information Sharing Network (TISN) for critical infrastructure resilience
Shared Data

Five SafesShared data is data that is shared with a specific organisation, or group of organisations or people, for a specific purpose. Data sharing is how NSW government agencies can provide authorised access to the data they hold in a controlled manner, to help deliver better outcomes to the people of NSW.

Guidance on sharing of data is provided by Data.NSW, including the Five Safes, also referred to as data sharing principles.

The Commonwealth Data Sharing Principles help agencies to think about all of these factors together and better manage any risks associated with data sharing.

The five Data Sharing Principles (‘The Principles’) provide a framework for government agencies to share data safely:

  • Share data for appropriate and authorised purposes
  • Share data only with authorised users
  • Use data in a safe and secure environment
  • Apply appropriate protections to the data
  • Ensure public outputs from data sharing projects do not identify the people or organisations in the data

If the joint protections offered by the Principles are not sufficient to protect against the risk of data breaches or data re-identification, then the data should not be shared.

Open Data

Data.NSW

Open data is data that anyone can access, use and share. Governments and organisations have opened up access to data such as weather records, train timetables and real-time running, allowing others to use this data and discover new solutions for the benefit of all. However, simply releasing data is not sufficient and for data to be considered truly open, the owners must clearly state that other organisations or people can use it in any way they like, as without express permission, the data cannot be considered as open.

The NSW Government Open Data Policy is clear about its objectives and defines open data as follows: “data is open to the extent that its management, release and characteristics meet the principles of openness”.

In accordance with the Open Data Principles of the policy, agencies must manage data as a strategic asset to be:

  • Open by default, protected where required
  • Prioritised, discoverable and usable
  • Primary and timely
  • Well managed, trusted and authoritative
  • Free where appropriate
  • Subject to public input

A key component of defining information requirements for infrastructure must include defining. where on the data spectrum from closed to open the relevant infrastructure data fits.

Re-use / Maintain

The re-use of existing infrastructure data assets for additional purposes will be one of the greatest sources of value for NSW government agencies, industry and the general public. Re-use of whole of government data assets such as those presented in the NSW Digital Twin will break down existing silos and artificial barriers to the use of information across cluster, administrative and jurisdictional boundaries. Re-use of data is facilitated by standards, interoperable data systems, common data formats and proactive sharing and release of open data.

Ongoing maintenance of asset data is a core process in infrastructure management. With the collection of large amounts of infrastructure data, it is essential to have processes in place to monitor, maintain and update the data to ensure the ongoing efficiency and improvement of the infrastructure asset. Specification of the frequency of update of data to align to organisational reporting requirements will reduce the friction associated with generation of ad hoc reports and data updates.

Archive / Destroy

When an infrastructure data asset reaches the end of its life, the asset data should be archived or destroyed. To avoid large storage costs (or to minimise the risk of premature data destruction), agencies should assess and identify considerations that apply to the retention and destruction of data such as:

  • Is there personal information in data assets? If so, under the Privacy and Personal Information Protection Act 1998, personal information should be destroyed as soon as the objective it was collected for is completed, in accordance with relevant requirements in the State Records Act. Note: Personal information may be required to be kept in some contexts, for instance, throughout an ongoing legal proceeding, and should be done so in accordance with the Privacy and Personal Information Protection Act 1998.
  • Is there health information in data assets? If so, health information should be managed under the requirements of the Health Records and Information Privacy Act 2002.
  • Data and records should be disposed of in accordance with NSW Information Classification, Labelling and Handling Guidelines. The Guidelines recommend records are disposed of with the same level of security that they are maintained. Guidance on de-identifying information is available from the Information and Privacy Commission.
  • Are internal or external services dependent on the data?
  • Are very large volumes of data involved? If so, it may not be economical to maintain the data for long periods of time. Approaches will be needed to routinely purge the data that is not needed for ongoing use.
  • Are there any audit or accountability requirements applying to the infrastructure asset management process?

The State Records Act 1998 sets the rules for how long all government information needs to be retained. Depending on the type of infrastructure, the data asset will have different legal retention and destruction requirements. Agencies should refer to the NSW State Archives and Records website for more information. Any decision to archive or destroy data must also be made in accordance with the organisation’s records and information management requirements.

All retention and destruction decisions need to be authorised and documented to achieve transparency and accountability over the destruction of infrastructure data assets. Governance and approvals must be defined in the agency’s data governance documentation to ensure compliance with the State Records Act 1998, as well as with consideration of Government Information (Public Access) Act 2009 (GIPA) requirements. If working with multiple service providers, agencies should make sure they can all support and deploy the data retention and destruction frameworks required.

Back a page 
Data Management and Practice
Next page 
Data Quality

 


Last updated 29 Jul 2024