3. Guidelines for use (GA33)

• 3.1 About migration
• 3.2 Conditions for the destruction of records
• 3.3 Records excluded from this authority
• 3.4 Relationship to other State Records general retention and disposal authorities
• 3.5 Destroying digital records

3.1 About migration

What is migration?

Migration involves a set of organised tasks designed to periodically transfer records from one hardware or software configuration to another, or from one generation of technology to another, while maintaining the records’ authenticity, integrity, reliability and useability. Data is generally inevitably changed by the migration process.

When migrating records, it is important that migration is performed on all aspects of a record – namely the record object and the metadata that supports it. Migration also can be performed on metadata only, for example, when the metadata that is used to manage hard copy records is moved from one system to another.

Migration is the key preservation strategy recommended in State Records' Policy on digital records preservation. It is a standard and often necessary practice, frequently driven by business needs to improve system functionality, that needs to be planned and managed appropriately to ensure the ongoing authenticity, accessibility and useability of digital records.

What is not migration?

For the purposes of this authority, the processes known as refreshment and replication are not regarded as forms of migration.

Refreshment is where data is transferred unchanged from one medium to another of the same type, for example, copying data from one CD-R to another. Replication is where data is transferred unchanged from one medium to another of a different type, for example, copying data from a hard drive to a CD-R.

Both these processes are forms of copying, not migration. While it is important to ensure that complete copies are made when refreshing or replicating, refreshment and replication are not subject to the stringent conditions outlined in this authority that apply to migrations. This is because refreshment and replication do not alter records in the same way that migration does. Migration as defined in this authority has the potential to have a significant impact on the accessibility and integrity of digital records and is therefore subject to the controls outlined in this authority.

Refreshment and replication should however always be performed with care and diligence. Appropriate checking should also be undertaken to ensure that the copying process was successful. Any superfluous copies of records that remain after refreshment or replication is complete may be disposed of in accordance with normal administrative practice.

Different types of migration covered by this authority

This authority authorises the destruction of the source records that remain following numerous different migration scenarios. These are:

Note that this authority does not authorise the disposal of records that have been transferred from hard copy, or paper, into a digital format. The disposal of paper records that have been scanned or digitised is covered by General Retention and Disposal Authority – Imaged records (GDA24).

3.2 Conditions for the destruction of records

Because of its capacity to significantly alter record content and structure, migration is a high risk activity. Migration actions need to be well planned and effectively implemented in order for them to produce accurate versions of the source records that can be regarded for business and legal purposes as true copies of the original.

As a result of the high risk nature of migration activities, this authority outlines six conditions that must be met before source records can be destroyed. Implementing these conditions will help public offices to perform migration operations that produce accurate and authentic records. The conditions also seek to ensure that the migrated records are appropriately managed and that the migration process is adequately documented.

To be able to destroy source records, a public office must ensure that the conditions for destruction described in this authority are met. These conditions are that:

1. The migration is planned, documented and managed

2. Pre and post migration testing proves that authentic, complete, accessible and useable records can and have been migrated

The testing to ensure that the migration process can deliver records that are authentic, complete, accessible and useable is divided into two phases: pre migration testing and post migration testing.

3. Source records are kept for a period of no less than six months following their successful migration. In many cases their retention period will be longer than the mandatory six months minimum. The specific retention period will be based on organisational risk assessment.

Following their successful migration, source records must be kept for at least six months.

A retention period of six months allows time for any unforeseen issues associated with the migration that may emerge following post migration testing to be identified and rectified. Retaining the source records for at least this period will enable the migration to be repeated if it is discovered that some or all of the migrated records do not meet quality control standards or business requirements.

The six months retention period should begin to be calculated from the conclusion of successful post migration testing, where the migration and all outstanding issues associated with it have been signed off by the Chief Information Officer, or designated equivalent within the organisation.

In high risk scenarios it is likely that source records will need to be kept for longer than the minimum six months retention period required by this authority.

To determine whether source records need to be kept for more than six months and then to determine the specific length of the required retention period, public offices will need to use risk assessments to determine an appropriate retention period. These risk assessments should at a minimum consider:

• the business purpose of the records
• the risks associated with this business
• the potential business, financial and legal implications of the loss of or damage to the migrated records
• the size and complexity of the migration and the likelihood of problems associated with it
• the complexity of the records being migrated
• the capacities of the target system and the possibility that all aspects of this system and its impact on the records are not fully understood at the time of migration, and, where relevant,
• the nature of the metadata that is not being migrated.

Public offices should always err on the side of caution and if any problems or concerns with the migrated or transferred records are noted, such as corruption of portions of the record, or loss of information or distortions in the records caused by the new capacities or functionality of the target system, then the identified retention period must be extended by at least six months. Again, this additional retention period should begin to be calculated from the conclusion of successful post migration testing, where the repeated migration and all outstanding issues associated with it have been signed off by the Chief Information Officer or appropriate officer.

Retaining source records

It should be noted that this authority is not a requirement to destroy source records. In some situations, such as migrations between public offices, it may facilitate business processes for the transferring organisation to retain a reference copy of the source records for the length of their retention period, as specified in the relevant functional retention and disposal authority. Additionally, in high risk business environments, risk may best be mitigated by retaining source records as a managed copy of the new official records. Cases, however, where public offices need to keep source records indefinitely should be very rare. Virtually all source records should be able to be destroyed after their retention period has expired.

4. For migrations within the public office, the target recordkeeping system meets the requirements of the State Records Standard on Digital Recordkeeping and preserves the records as the official records of organisational business

Under this authority, migrated versions of records are deemed to be official records. Procedures must be in place to ensure that the migrated form of the record is identified and managed as the official record of organisational business. All other versions of the record, including the source records, are copies and should not be regarded as the official record.

To preserve their authenticity, migrated records must immediately be captured and preserved in the most appropriate official recordkeeping system. It must be possible to demonstrate an unbroken chain of custody throughout the preservation process. This means that records should be taken from their parent system and migrated directly into their target system with as minimal intervention and time delay as possible. Any real or perceived lapse in the management of the record during the migration process may be seen as compromising its authenticity.

As official records, the migrated records should be managed in accordance with State Records' standards and requirements. To meet the conditions specified in this authority, migrated records must be managed in systems that meet the requirements of the State Records Standard on digital recordkeeping. These requirements are outlined in Appendix 1.

An official recordkeeping system can provide for the online or offline storage of migrated records, depending on the business requirements of the public office.

It can be difficult for a transferring organisation to ensure that this condition is met by the external organisation that is now responsible for the records. Therefore this is not a mandatory requirement for migrations between public offices. However it is critical in inter-agency migrations where records are transferred as a result of administrative change that both parties ensure that the target recordkeeping system meets the requirements of the State Records Digital Recordkeeping Standard. All parties involved in the migration should work to ensure that the records are migrated and maintained in ways that best meet business, legal and accountability requirements, including the requirements of the State Records Act.

5. For migrations to State Records, the migrations meet the conditions outlined in State Records' transfer requirements

Section 29 of the State Records Act allows State Records to issue mandatory guidelines to govern the process of transferring State archives to State Records. When records are migrated to State Records, public offices must ensure that all conditions outlined in State Records' transfer requirements are met by the records being migrated.

6. The disposal of source records is appropriately documented

To comply with this condition, care must be taken to document and preserve destruction and transfer information after the source records themselves have been destroyed. (For further information about the appropriate means to destroy records, see State Records' Destruction of records: A practical guide and Recordkeeping in Brief 51: Destroying digital records.)

For migrations within the public office, the documentation of the destruction of the source record can take a range of forms, including:

• metadata at the aggregate or system level which states that the previous version of this record group or system was destroyed on the specified date, by the identified authorised officer, in accordance with the conditions outlined in this authority
• metadata at a more granular level that documents the destruction of the previous version of specific files in accordance with necessary disposal requirements
• a final migration report that outlines the destruction of source records, providing enough detail to identify all records or groups of records that have been destroyed.

The situation is different for source records that remain following migrations between one organisation and another or between one organisation and State Records. To document both the destruction of these source records and the migration/transfer of the records, a more detailed record needs to be retained. In addition to documenting the destruction of the source record and the authority for this destruction, a record must also be kept which identifies:

• the identity of all the records that have been transferred
• where the records have been transferred to, and
• the date of the transfer.

Whatever its form, documentation about records' disposal must be retained for significant periods of time. For example, for records that have been transferred to a successor organisation as a result of administrative change, the General Retention and Disposal Authority - Administrative records (GA28), requires that information about this transfer must be retained for a minimum of twenty years after the transfer (GA28, entry number 12.11.6). It also requires that details of records transferred to State Records must be retained on an ongoing basis within the organisation (GA28, entry number 12.11.3). The General Retention and Disposal Authority - Administrative records also requires that any record documenting the implementation of disposal decisions should be maintained for a minimum of twenty years (GA28, entry number 12.11.1).

3.3 Records excluded from this authority

In addition to records that have been refreshed or replicated, the following record types are specifically not covered by this authority:

• records copied for convenience or reference only
• encrypted records
• computer back-up tapes
• transcribed records
• analogue records being migrated to digital formats
• paper source records where their informational content has been transferred to a digital format

Records copied for convenience or reference only

This authority cannot be used to destroy official records after copies of these records have been made for convenience or reference purposes. Increasingly records are copied to different formats to facilitate web based access, downloading, data sharing etc. This authority is not designed to cover the disposal of records that have been copied in this way.

Copied records of this type are generally authorised for destruction under the normal administrative practice provisions of the State Records Act (see State Records' guidelines on Normal Administrative Practice (Guideline 8, entry 3.1.5)) or may be referenced in relevant general and/or functional retention and disposal authorities. The original records that have been copied should be retained in accordance with the requirements of the relevant general or functional retention and disposal authority that applies to them.

Encrypted records

Encrypted records are usually created as a by-product of online authentication and security technologies such as Public Key Infrastructure. Encryption processes create a protected copy of a record that can be shared with other persons or organisations for information and business purposes. Records that have been encrypted are not covered by this authority.

The disposal of encrypted records is covered by the general and/or functional retention and disposal authorities that apply to the business documented within the encrypted record.

Computer backup tapes

Computer or system backup tapes are duplicate records, created as part of disaster management strategies on a daily or routine basis and overwritten frequently. Their purpose is to preserve organisational records in the event of system failure by creating a copy of all system data.

Disposal of back up tapes is not covered by this authority. Established routines for the destruction or overwriting of backups should be documented in disaster management or continuity planning policies and procedures.

Transcribed records

Transcription is used to copy the informational content of a record into a more accessible format. It creates a new record but both the original and transcribed record are generally kept as official records of the business. The source records for the transcription e.g. audio tapes, short hand notes, do not then fall within the parameters of this authority.

Disposal of the original source records after they have been transcribed should be covered by the relevant general or functional retention and disposal authority. Depending on the nature of the transcription process, and where this is supported by documented procedures and quality control processes, disposal of the original records may also be permitted under the normal administrative practice provisions of the Act (see State Records' guidelines on Normal Administrative Practice (Guideline 8, entry 3.1.4)).

Analogue records being migrated to digital formats

This authority is not intended to be used as a means to destroy analogue records that have been migrated to digital formats, such as analogue film or audio visual formats.

Analogue records involved in migrations of this type should be covered by relevant functional retention and disposal authorities.

Paper source records where their informational content has been transferred to a digital format

This authority does not apply to paper source records where the information contained in these records has been transferred into a digital format, for example information from an index card or data entry form that is input into a database.

The transfer of index card information in this way is covered by the General Retention and Disposal Authority - Administrative records (see GA28, entry 12.9.4). Depending on the nature of the business process, and where this is supported by documented procedures and quality control processes, disposal of certain data entry input forms may also be permitted under the normal administrative practice provisions of the Act (see State Records' guidelines on Normal Administrative Practice (Guideline 8, entry 3.1.6)). Other information transfers of this kind should be covered by relevant functional retention and disposal authorities.

3.4 Relationship to other State Records general retention and disposal authorities

This general retention and disposal authority has similar requirements to those outlined in the General Retention and Disposal Authority - Imaged records (GDA24).

These authorities however cover different types of records. GDA24 covers (primarily) paper source records that have been imaged or scanned to create a digital copy and this authority covers records that have been created and maintained in digital formats and then migrated to a different digital format.

3.5 Destroying digital records

For more information on practical considerations of destroying digital source records that are authorised for destruction under this authority, public offices should refer to State Records' publications Destruction of records: A practical guide and Recordkeeping in Brief 51: Destroying digital records.

Footnotes

[1] Essential characteristics are the key characteristics that are critical to a record's meaning, use or organisational value. Essential characteristics will differ according to record type and the business purpose served by the record. For example, a report contains a map where colours are used to signify different agricultural areas. These colours have meaning - the report could not be interpreted accurately if these colours were not preserved. Therefore the colours are an essential characteristic of the report and any migration performed on this report must ensure that this essential characteristic is maintained.

[2] During the migration, public offices can decide not to migrate all metadata components of a record. This must only be the case for those metadata elements that have no ongoing business or accountability relevance for the organisation. Decisions not to migrate certain metadata elements should be fully considered and comprehensively documented.