State Records Home
Personal tools
You are here :: Home → Recordkeeping in the NSW public sector → Government Recordkeeping Manual → Rules → Standards → Digital recordkeeping
 

Standard on digital recordkeeping

Filed under:
Standard on digital recordkeeping

Printable version

There is a PDF version (PDF, 245kb) of the standard for printing.

Table of commentary

An account of the comments received during public consultation on this standard is available in the accompanying Table of Commentary (PDF, 114kb).

Compliance timetable

There is a compliance timetable (PDF, 55kb) for this standard, with requirements phased in beween 2008 and 2012.

Executive summary

This standard sets out the minimum compliance requirements for NSW public offices for defining their digital records, digital recordkeeping system functionality and the creation and management of recordkeeping metadata for digital records.

By implementing systems in accordance with this standard, public offices can maximise the business benefits that can be derived from effective management of digital information. As the Government Chief Information Office has noted:

"Information is the basis of most activity in the public sector. It is used for planning, decision making and customer service. The better information is managed, the more effective these activities will be." [1]

Benefits of effective digital recordkeeping include improved service delivery through faster access to accurate business information and more effective use of public resources by avoiding the over retention of unnecessary data. Compliant digital recordkeeping systems can also minimise the risks associated with poor recordkeeping in the digital business environment. These risks include an inability to account for past actions or decisions, a lack of corporate information on which to plan and make future decisions and a failure to meet legal requirements.

This standard differs from others in the suite of records management standards issued by State Records in that it is an enabling standard. By meeting the requirements of the Standard on Digital Recordkeeping, public offices will be better able to comply with the other records management standards, in particular the Standard on Full and Accurate Records and the Standard on Appraisal and Disposal of State Records.

Compliance with this standard will be monitored by State Records and reported on in its Annual Report. There is a compliance timetable indicating how the requirements of the standard are to be implemented and by what dates. Public offices will be advised in advance of any compliance monitoring activities.
This standard is supported by tools and guidance on digital recordkeeping, see Appendix A.

Introduction

Purpose

The purpose of this standard is to establish minimum requirements for digital recordkeeping in the New South Wales public sector to enable NSW public offices to implement digital recordkeeping systems that will support business efficiency and organisational accountability.

By meeting the requirements of this standard, public offices can achieve basic recordkeeping functionality in the digital information environment. This functionality can then form a sound basis on which to build more customised functionality to meet their specific business needs.

Authority

This standard will be issued under s.13(1) of the State Records Act 1998 which enables State Records to 'approve standards and codes of best practice for records management by public offices.'

Application

This standard will apply to all public offices as defined in section 3 of the State Records Act, to which Part 2 of the Act applies.

Background

What are digital records?

A digital record is digital information, captured at a specific point in time that is kept as evidence of business activity. Digital records means 'born digital' records such as emails, web pages, digital photographs, digital audio files, GIS files or database records, as well as scanned versions of paper records that have been digitised in business processes.

Public offices create, generate, send and receive digital information every day – in word processing and email applications, using web authoring tools, in databases and by other means. Some of this information must be retained by the public office in order to meet business, accountability-related and other requirements for evidence. These are digital State records, and are subject to all the same requirements under the State Records Act 1998 as paper based records and files.

The following list contains some typical examples of forms of digital information that are digital State records if they are made and kept, or received and kept, by any person in the course of the exercise of official functions in a public office:

  • an email complete with header information and attachments
  • a web page
  • a word-processed document
  • output from a line of business application
  • a digital audio recording
  • fields from a database that together represent a transaction or group of transactions
  • a text message
  • a digital photograph
  • a spreadsheet
  • a scanned image of a paper document, or
  • a computer aided drawing, map or plan.

What is digital recordkeeping?

Digital recordkeeping is routinely saving digital records into systems where they can be managed properly and made available as needed.

This standard does not require public offices to keep their records in digital form; for some smaller offices printing and filing may meet their needs. However, for business carried out using automated systems, capturing and keeping records in digital form is often the best method to ensure the complete record is captured. Keeping records in digital form also supports business efficiency and useability of the information in the records.

A good digital recordkeeping system can implement access, disposal and other rules so that digital records are managed as efficiently as possible. Information Asset Management Software (IAMS) tools such as Electronic Document and Records Management System (EDRMS) or Enterprise Content Management (ECM) products often form the core of digital recordkeeping systems. In some circumstances core business systems are adapted to keep records by interfacing with IAMS tools or, in a Service Oriented Environment, by creating recordkeeping services to interact with business systems. Another alternative is to carry out changes to core business systems (such as case management systems) to enable them to produce and keep records of transactions performed in the systems. It is also common in government today to see mixed or hybrid systems that are capable of managing paper and ‘born digital’ records and possibly also scanned / digitised copies of paper records.

There are a number of essential functionalities that any digital recordkeeping system used for keeping digital State records in the New South Wales public sector must possess. These are described in this standard, along with the essential recordkeeping metadata that the system must be capable of managing.

Another important aspect of digital recordkeeping is ensuring the authenticity and accessibility of digital records over time. Ability to reproduce digital information while preserving the fixity and essential characteristics that give it meaning as a record requires the adoption of a mix of strategies including planned and documented migration, creation and use of recordkeeping metadata and the adoption of stable, long term formats. Such formats include (but are not limited to):

  • PDF/A-1, the ‘archival’ format for PDF. PDF/A-1 has fewer "bells and whistles" than traditional PDF which minimises future migration requirements. PDF/A-1 is more open than traditional PDF because it is maintained by the International Standards Organisation, not one specific vendor [2]
  • Open Document Format (ODF), an open XML-based document file format for office applications to be used for documents containing text, spreadsheets, charts, and graphical elements
  • Hypertext Markup Language (HTML)
  • Extensible Hypertext Markup Language (XHTML), a version of HTML that conforms to the XML syntax and therefore allow automated processing to be performed using standard XML tools
  • Open Office versions of Microsoft PowerPoint, Microsoft Excel
  • Joint Photographic Experts Group File Interchange Format (JPEG or JFIF), Tagged Image File Format (TIFF), Portable Network Graphics (PNG) for digital images, and
    Free Lossless Audio Codec (FLAC) for digital audio files.

For more information refer to State Records’ Policy on digital records preservation and supporting guidance.

What is recordkeeping metadata?

Recordkeeping metadata is an essential part of any digital recordkeeping system. It is structured or semi-structured information that enables the creation, registration, classification, access, preservation and disposal of records. In digital recordkeeping, recordkeeping metadata is often in machine readable form, allowing for the automation of many of these tasks.

Some examples of recordkeeping metadata that are commonly implemented in digital recordkeeping systems are:

  • identifiers (to identify an entity in the system such as a record or a record author),
    dates (such as date registered or date destroyed), and
  • information on the business function that a record relates to.

Effective implementation of recordkeeping metadata requires the creation and management of rules to govern how the metadata is created, captured and maintained. The New South Wales Recordkeeping Metadata Technical Specification (Word, 2.76MB) provides rules relating to metadata that can be customised for organisational implementation.

Structure

This standard contains nine minimum compliance requirements, divided into three sections:

  • minimum requirements for digital recordkeeping systems
  • minimum requirements for recordkeeping metadata, and
  • minimum requirements for recordkeeping metadata management.

Definitions

Digital record
A digital record is digital information, captured at a specific point in time that is kept as evidence of business activity. Digital records means 'born digital' records such as emails, web pages, digital photographs, digital audio files, GIS files or database records, as well as scanned versions of paper records that have been digitised in business processes.

Digital State record
A digital State record that is made and kept, or received and kept, by any person in the course of the exercise of official functions in a public office, or for any purpose of a public office, or for the use of a public office.

Digital recordkeeping system
A system that captures, maintains and provides access to digital records over time. A digital recordkeeping system can be:

  • a business system with recordkeeping functionality, or
  • a business system linked with a dedicated records management / information asset management system, or
  • a dedicated records management / information asset management system.

Disposal
A range of processes associated with implementing appraisal decisions. These include the retention, deletion or destruction of records in or from recordkeeping systems. They may also include the migration or transmission of records between recordkeeping systems, and the transfer of custody or ownership of records. (AS 4390 Part 1 Clause 4.9)

Metadata
Data describing data and data systems. In records management, recordkeeping metadata is data that describes the context, content and structure of records and their management through time. (AS ISO 15489 Part 1 Clause 3.12)

Migration
The act of moving records from one system to another, while maintaining the records' authenticity, integrity, reliability and useability. (AS ISO 15489 Part 1 Clause 3.13)

Metadata schema
Logical plan showing the relationships between metadata elements, normally through establishing rules for the use and management of metadata specifically as regards the semantics, the syntax and the optionality (obligation level) of values. (AS ISO 23081 Part 1 Clause 3.3)

For further definitions of general recordkeeping terms used in this document, see State Records’ Glossary of Recordkeeping Terms .

Acknowledgement

State Records is grateful to Archives New Zealand for giving us permission to use information from their Electronic Recordkeeping Metadata Standard (2008) in the development of this standard.

Further information

For more information on this standard, please contact State Records .

Using the Standard

This section provides information on implementing this standard.

Knowing what digital records must be made and kept

For a public office to successfully implement digital recordkeeping, it must first specify the records that will be captured into digital recordkeeping systems.

Decisions about what records to keep of organisational business processes should be based on an analysis of the regulatory environment, business and accountability requirements and the risk of not capturing the records. A useful technique for defining the records is to analyse business processes and identify the transactions that comprise them. Transactions will either automatically produce records that may need to be kept (for example an email or an automated reports from a business system), or will indicate where a record should be created or captured and kept (for example a note made after a meeting).

The level of detail used to define what digital records are to be kept depends on how much information is needed for implementation and the level of risk associated with the records and the business they document.

Digital records might be defined at different levels and in different ways, for example:

  • organisational policy stating that all outgoing business related emails are saved by the sender into the organisation's EDRMS
  • specifications for a database system that identify the fields to be saved as a record each time an entry in the database is updated.

See Appendix A for links to more information on defining digital records and using a risk management based approach to ensuring the creation and capture of digital records.

Implementing digital recordkeeping systems

A digital recordkeeping system may be either:

  • a business system with recordkeeping functionality - that is a system that meets the recordkeeping functionality and metadata requirements specified in this standard, or
  • a business system linked with a dedicated records management / information asset management system, or
  • a dedicated records management / information asset management system.

A records management / information asset management system will usually have at its heart a software application such as an Electronic Record and Document Management System (EDRMS), a Records Management Application (RMA) or an Enterprise Content Management (ECM) system. For New South Wales Government agencies, there is a whole of government panel contract for Information Asset Management Software.

To be effective, a digital recordkeeping system must be implemented with policy, procedures and training to ensure that all users understand their role in meeting the organisation's recordkeeping requirements.

See Appendix A for links to more information on designing and implementing digital recordkeeping systems.

Managing recordkeeping metadata

Meeting the requirements of this standard is a starting point for New South Wales public offices in establishing metadata creation, capture and management rules for their own business environments.

In addition to documenting mappings from the minimum requirements of this standard to organisational digital recordkeeping systems, other elements that are needed to implement a more comprehensive recordkeeping metadata strategy are:

a customised recordkeeping metadata schema that describes the organisational recordkeeping entities (e.g. agents, records), their properties (e.g. title, identifier) and their relationships (e.g. authorised by, containing). NSW public offices can use the New South Wales Recordkeeping Metadata Technical Specification (Word, 2.76 MB) as a basis for the development of their own customised recordkeeping metadata schemas, and
business rules on how recordkeeping metadata will be captured and managed, from what sources and who will be responsible for monitoring the quality of the metadata.
See the Appendices for links to more information on defining structures and rules for recordkeeping metadata and managing recordkeeping metadata.

Requirements

1 Minimum requirements for digital recordkeeping system functionality

Requirement

Benefits of complying

Risks of not complying
 1.1

The public office must define the digital State records that it will make and keep.

Note:

The level of detail used by the public office to define the digital records to be made and kept should be adequate for implementation purposes and based on an assessment of the risk associated with the records and the business they document.
  • Meet business requirements for records
  • Digital record requirements are ready to be implemented in new or upgraded business systems
  • Meet legislative and regulatory requirements including archival, audit and oversight activities
  • Supports compliance with the Standard on Full and Accurate Records
  • Business failure as a result of a lack of reliable information documenting transactions
  • Inefficient use of resources while staff try to determine what records to make and keep
  • Digital records inadvertently destroyed in breach of disposal authorisation requirements of State Records Act
  • Inappropriate / unnecessary records made and kept
  • Failure to produce records required by accountability watchdogs
 1.2

The digital State records that the public office has defined must be captured into an official digital recordkeeping system.

Note:

A digital recordkeeping system can be:

  • a business system with recordkeeping functionality, or
  • a business system linked with a dedicated records management / information asset management system, or
  • a dedicated records management / information asset management system.
  • Records are managed in context making them more meaningful
  • Records are easily retrievable when needed / time saved searching for records
  • Records are routinely disposed of in accordance with requirements
  • Recordkeeping is integrated into business systems environments where necessary and appropriate
  • High costs of back capture projects when systems are decommissioned if records not captured contemporaneously
  • Records are left vulnerable to accidental or fraudulent alteration or deletion
  • Information security breaches if records not protected
  • Breaches of privacy if records not protected
 1.3

Any digital recordkeeping system used for keeping official records must possess the following functionalities:

  • capture read only versions of digital records
  • retrieve and present digital records in human readable form
  • restrict or permit access to records by specified individuals or groups
  • capture and manage the minimum required recordkeeping metadata as defined in this standard.
  • Records can be relied upon as trustworthy evidence of business
  • Supports compliance with the ISO 27000 (Information security) series of standards
  • Records are left vulnerable to accidental or fraudulent alteration or deletion
  • Information security breaches if records not protected
  • Breaches of privacy if records not protected

2 Minimum requirements for recordkeeping metadata

Requirement

Benefits of complying

Risks of not complying
2.1

Digital records must be captured into a digital recordkeeping system with:

  • unique identifier
  • title
  • date of creation
  • who/what created the record
  • the business function/process it relates to
  • the creating application
  • record type (e.g. letter / memo / report / contract / fax / schematic / blog, or locally defined types)
  • Information vital for successful migration to new hardware/software is recorded, reducing cost of migration projects
  • Supports long term accessibility and authenticity of records
  • Records disposal projects for digital records are easier and less costly to implement because records are more readily identified
  • Information is available to help protect records’ essential characteristics through migrations, reducing cost of migration projects
  • Additional metadata on role of records in business process makes records easier to manage and retrieve
  • Improved service delivery through faster access to accurate business information
  • Records may not be adequate as evidence for the courts or watchdog agencies owing to lack of contextual information
  • Records cannot be linked to people or workgroups that created them
  • Potential for costly retrospective projects adding vital metadata to records
  • Records could be subjected to migration or other change that removes essential characteristics and therefore meaning / useability and reliability as evidence
  • Over retention of records that cannot be disposed of because they are inadequately described.

 
2.2

 Any of the recordkeeping processes (listed below) that are performed on a record must be documented with:

  • the date of the action
  • identification of who/what undertook the action
  • what action was undertaken

The recordkeeping processes are:

  • registration into a recordkeeping system
  • apply or change access rules
  • transfer of control
  • destruction
  • migration
  • Recordkeeping actions can be automatically documented saving time and effort
  • The public office can account for past recordkeeping actions
  • Supports compliance with the ISO 27000 (Information security) series of standards
  • No audit trail of actions taken on records, leaving the organisation vulnerable to corrupt behaviour
2.3

The transfer of control or destruction of records must be documented with:

  • process metadata as above
  • an authorisation reference for the transfer or destruction (e.g. FA234 2.4.5; GA27 1.2.3; By court order etc.), and
  • in the case of transfer of the records, the name of the receiving organisation (e.g. Dept of X; State Records).
  •  Corruption prevention.
  • Records will be easier to track after administrative change / restructuring has occurred
  • Supports compliance with the Standard on the appraisal and disposal of State records
  • No audit trail of records destruction, leaving the organisation vulnerable to corrupt behaviour
  • Likelihood of breach of the State Records Act s.21(1), potentially incurring penalties
  • Inability to track records transferred to another agency in administrative change
 2.4

At least the minimum required recordkeeping metadata as specified in this standard must be persistently linked with digital records and aggregations of digital records, including when they are transferred out of their original creating environment and through subsequent migrations.
  • Records with adequate metadata easier and less costly to manage and migrate over time
  • Records are meaningful and useable at all stages of their existence
  • Records required as State archives have adequate descriptive information
  • Vital records are more easily identified and managed over time.
  • Records without adequate metadata are more time consuming to migrate, leading to higher costs for migration projects
  • Records that have been migrated without adequate metadata lack context and are less useable after migration

3 Minimum requirements for recordkeeping metadata management


RequirementsBenefits of complyingRisks of not complying
3.1

Recordkeeping metadata must be disposed of in accordance with the requirements of the State Records Act.
  • Less time spent reconstructing what happened to destroyed / transferred records
  • Information needed to support migration and systems development projects is available
  • More effective use of public resources by avoiding the over retention of unnecessary data
  • Breach of State Records Act s.21(1) potentially incurring penalties
  • Information about records that have been destroyed unavailable for scrutiny for example by auditors or watchdog agencies
3.2

Metadata mappings from the minimum requirements of this standard to organisational digital recordkeeping systems must be documented and maintained, including any changes to these.
  • Creating mappings make it easier to check effectiveness of recordkeeping and confirm compliance during system design and implementation projects
  • Easier and faster to upgrade / migrate systems with documentation of how fields have been implemented
  • Unable to retrieve records as local terminology is not incorporated in organisational recordkeeping metadata
  • More difficult to migrate records if metadata not defined
    Inadequate contextual and management information about records in place

Appendix A: Supporting guidance


RequirementGuidance
1 Minimum requirements for digital recordkeeping system functionality  
1.1

The public office must define the digital State records that it will make and keep.

Note:

The level of detail used by the public office to define the digital records to be made and kept should be adequate for implementation purposes and based on an assessment of the risk associated with the records and the business they document.
1.2

 The digital State records that the public office has defined must be captured into an official digital recordkeeping system.

Note:

A digital recordkeeping system can be:

  • a business system with recordkeeping functionality, or
  • a business system linked with a dedicated records management / information asset management system, or
  • a dedicated records management / information asset management system.
1.3

Any digital recordkeeping system used for keeping official records must possess the following functionalities:

  • capture read only versions of digital records
    retrieve and present digital records in human readable form
  • restrict or permit access to records by specified individuals or groups
  • capture and manage the minimum required recordkeeping metadata as defined in this standard.
2  Minimum requirements for recordkeeping metadata  
2.1

Digital records must be captured into a digital recordkeeping system with:

  • unique identifier
  • title
  • date of creation
  • who/what created the record
  • the business function/process it relates to
  • the creating application
  • record type (e.g. letter / memo / report / contract / fax / schematic / blog, or locally defined types)
2.2

Any of the recordkeeping processes (listed below) that are performed on a record must be documented with:

  • the date of the action
  • identification of who/what undertook the action
  • what action was undertaken

The recordkeeping processes are:

  • registration into a recordkeeping system
  • apply or change access rules
  • transfer of control
  • destruction
  • migration
 2.3

 The transfer of control or destruction of records must be documented with:

  • process metadata as listed in 2.2, above
    an authorisation reference for the transfer or destruction (e.g. FA234 2.4.5; GA27 1.2.3; By court order etc.), and
  • in the case of transfer of the records, the name of the receiving organisation (e.g. Dept of X; State Records).
 2.4  At least the minimum required recordkeeping metadata as specified in this standard must be persistently linked with digital records and aggregations of digital records, including when they are transferred out of their original creating environment and through subsequent migrations.
 3

Minimum requirements for recordkeeping metadata management

  
 3.1

Recordkeeping metadata must be disposed of in accordance with the requirements of the State Records Act.
 3.2

Metadata mappings from the minimum requirements of this standard to organisational digital recordkeeping systems must be documented and maintained, including any changes to these.
  • AS ISO 23081.1-2006 Information and documentation - Records management processes - Metadata for records Part 1: Principles
  • AS ISO 23081.2-2007 Information and documentation - Records management processes - Metadata for records Part 2: Conceptual and implementation issues

Appendix B: Sample metadata mapping 

"To achieve implementations that are robust and which will enable easy updating, formalized mappings between the metadata schema adopted and the specific metadata within business systems should be maintained separately as an organisational resource." (AS ISO 23081.2-2007 Information and documentation - Records management processes - Metadata for records Part 2: Conceptual and implementation issues, 11.5 Registration)

Please note:

The sample mappings below map elements from two fictitious organisational digital recordkeeping systems;

  • a dedicated records system using information asset management software [4] and
  • a custom built case management system

to the minimum required metadata elements in this standard.

They are provided as samples only - it is not required that mappings are completed in precisely this format, nor does the information provided relate to any particular Information Asset Management Software application.

Organisational digital recordkeeping system 1: Information Asset Management Software system

Required metadata

IAMS

Unique identifier

Record Number used for document and file level identification

Title

Record Title used for document and file level titling

Date of creation

Date Created

Who/what created the record

Author

The business function/process it relates to

Business Process

The creating application

Electronic Details

Record type (e.g. letter / memo / report / contract / fax)

Document Type = Project Report

Document Type = Client letter

Document Type = Document (default)

Process metadata: Registration into a recordkeeping system

The date of the action

Date Created

Identification of who/what undertook the action

Creator

What action was undertaken

Details

Process metadata: Changed access rules

The date of the action

Audit Events

Identification of who/what undertook the action

Audit Events

What action was undertaken

Audit Events

Process metadata: Transfer of control

The date of the action

Audit Events

Identification of who/what undertook the action

Audit Events

What action was undertaken

Audit Events

An authorisation reference for the transfer (e.g. FA234 2.4.5; GA27 1.2.3; By court order; NAP etc.)

Disposal Schedule

The name of the receiving organisation

Details

Process metadata: Destruction

The date of the action

Audit Events

Identification of who/what undertook the action

Audit Events

What action was undertaken

Audit Events

An authorisation reference for the destruction (e.g. FA234 2.4.5; GA27 1.2.3; by court order; NAP etc.)

Disposal Schedule

Organisational digital recordkeeping system 2: Customer Management Database

Required metadata

CMD system

Point of capture metadata

Unique identifier

Customer ID

Title

Ref

Date of creation

Date created

Who/what created the record

UserID (Interface with LDAP)

The business function/process it relates to

Classification (Encoding scheme: Department’s business classification scheme)

The creating application

System log (Default value: CMD)

Record type (e.g. letter / memo / report / contract / fax)

Contact Type (Values: Complaint, enquiry, update details)

Process metadata: Registration into a recordkeeping system

The date of the action

System log / Event date

Identification of who/what undertook the action

System log / UserID

What action was undertaken

Event Name

Process metadata: Changed access rules

The date of the action

Event Date

Identification of who/what undertook the action

UserID

What action was undertaken

Event Name

Process metadata: Transfer of control

The date of the action

Event Date

Identification of who/what undertook the action

User ID

What action was undertaken

Event Name

An authorisation reference for the transfer (e.g. FA234 2.4.5; GA27 1.2.3; By court order; NAP etc.)

Event Details

The name of the receiving organisation

Event Details

Process metadata: Destruction

The date of the action

Event Date

Identification of who/what undertook the action

User ID

What action was undertaken

Event Name

An authorisation reference for the destruction (e.g. FA234 2.4.5; GA27 1.2.3; by court order; NAP etc.)

Event Details

Footnotes

[1] 'Information as an asset', Government Chief Information Office website, accessed 12 August 2008 http://www.gcio.nsw.gov.au/ict-key-strategies/information-management-1/information-as-an-asset

[2] National Archives and Records Administration, Frequently Asked Questions (FAQs) about Transferring Permanent Records in PDF/A-1 to NARA, College Park,n.d.URL: http://www.archives.gov/records-mgmt/initiatives/pdf-faq.html (cited June 2008).

[3] Please note, the second edition of this Technical Specification is due in early 2009. However, public offices should feel confident that system built using either version will comply with the minimum requirements of this standard.

[4] A number of Information Asset Management Software (IAMS) applications are available for NSW Government agencies to purchase from a whole of government panel contract managed by the Department of Commerce.

© State of New South Wales through the State Records Authority of New South Wales, 2008.
First published September 2008
This work may be freely reproduced for personal, educational or government purposes. Permission must be received from State Records Authority for or all other uses. ISBN 978-0-9805148-5-8