Recordkeeping In Brief 11 - Digital imaging and recordkeeping
About this guidance | Digital images of records and the law | Necessary qualities of digital images and imaging systems | Managing images as State records | Disposal of originals that have been imaged | Keeping documentation of imaging systems and processes | Day boxing / batching | Digitisation projects
About this guidance
The purpose of this guidance is to assist public offices in NSW to establish and maintain imaging or digitisation systems and processes that will result in authentic, complete and accessible image copies of records.
This guidance relates mainly to the imaging of current business records as they are received or created. The section 'Digitisation projects' addresses the needs of the digitisation or imaging of collections of records for access or preservation purposes.
Digital images of records and the law
Under the existing legal framework in New South Wales:
- there is no barrier to public sector orgnisations tendering digital images of records as evidence for legal proceedings, for FOI applications or for other evidentiary purposes
- under the Evidence Act 1995, an 'original' is not automatically regarded as better evidence than copies, including those made and kept by electronic means
- the value of a record as evidence will, however, depend on how it has been created and kept, and organisations need to be prepared to show how they have protected images from tampering or loss.
Necessary qualities of digital images and imaging systems
A good summary of the qualities that are necessary in image copies of records to ensure they can be relied upon as evidence both for current business or later on is that they are:
- authentic
- complete, and
- accessible.
| To be.. | ..an image copy must be.. |
|---|---|
| authentic | the product of routine, authorised copying and registration processes |
| complete | an accurate, legible reproduction of the original, without substantive changes or deletions of content |
| accessible | available and readable to all those with a right to access it, for as long as it is required |
The advice below describes ways in which the imaging process, quality control measure and technical specifications selected can support the creation of authentic, complete and accessible image copies.
Imaging process
Measures which are recommended to ensure that the imaging process captures authentic, complete and accessible images include:
- policy and procedures on imaging of records (may be part of broader records management policy) that are known by staff and authorised at a senior level
- systematic application of recordkeeping metadata to images to facilitate their use and management. This should include:
- adherence to standards set by organisational policy and procedures;
- procedures for accommodating images with incomplete metadata;
- relevancy and accuracy of metadata;
- grammar – check for correct grammar, spelling and punctuation, especially for manually-keyed data;
- consistency in the creation of metadata and in interpretation of metadata; and
- completeness of metadata - all mandatory fields should be complete. [1]
- where digitisation involves colour as an intrinsic part of the record, implementers should consider including a standard colour sheet with the image (such target sheets are commercially available from manufacturers such as Fuji or Kodak, and comply with the ISO 12641 standard)[2]
- the use of Write Once Read Many (WORM) optical media or 'read-only' controls in network servers used for storing images
- controls over access to the images by logins or other measures
- the maintenance of thorough system documentation, including description of any image enhancement techniques
- preservation measures and monitoring to protect the optical media on which the images are stored from deterioration, and
- the use of security controls such as access passwords and audit trails to prevent any alteration of the images.
Quality control
Industry standards for imaging recommend quality control measures including:
- the issue of certificates verifying the integrity of the copying process, particularly when it is done by an external bureau
- periodic testing and cleaning of scanning equipment[3]
- careful preparation of the documents to be scanned, for example, to ensure they are not folded, obscuring information
- random sampling of pages scanned to check quality. For large volumes, samples of 5-10% are acceptable. This quality checking to examine:
- smallest detail legibly captured (e.g. smallest type size for text; clarity of punctuation marks, including decimal points);
- completeness of detail (e.g. acceptability of broken characters, missing segments of lines);
- dimensional accuracy compared with the original;
- scanner-generated speckle (i.e. speckle not present on the original);
- completeness of overall image area (i.e. missing information at the edges of the image area);
- density of solid black areas; and
- colour fidelity.[4]
- the retention of the originals for quality control / disaster recovery purposes for a period of time defined in organisational policy/procedures are consistently implemented (see the section on 'Day boxing', below, for further advice on keeping originals after imaging).
Recommended technical specifications
It is recommended that standard formats are used for making digital image copies of records. The specifications below[5] are recommendations only for suitable standards for the imaging of records. Please note, some projects may warrant specifications producing higher quality image copies – for example, copying of historical collections for access online.
| Document Type | Resolution | Bit Depth | File Formats | Compression |
|---|---|---|---|---|
| Text only, black and white | Minimum 300ppi | 1 bit (bi-tonal) | - TIFF - PDF, or - PDF/A [6] |
Lossless compression |
| Documents with watermarks, grey shading, grey graphics | Minimum 600 ppi | 8 bit greyscale | - TIFF - JPEG, or - PDF/A |
Lossless compression |
| Documents with discrete colour used in text or diagrams | Minimum 600 ppi | Minimum: 8 bit colour | - TIFF - JPEG - PDF, or - PDF/A |
Lossless compression |
| Black and white photographs | Sufficient to provide >3000 pixels across long dimensions | 8 bit greyscale | - TIFF - JPEG - PDF, or - PDF/A |
Lossless compression |
| Colour photographs | Sufficient to provide >3000 pixels across long dimensions | 24 bit colour | - TIFF - JPEG - PDF, or - PDF/A |
Lossless compression |
| Black and white negatives | Sufficient to provide >3000 pixels across long dimensions | 8 bit greyscale or 24 bit colour | - TIFF - JPEG - PDF, or - PDF/A |
Lossless compression |
| Colour negatives and transparencies | Sufficient to provide >3000 pixels across long dimensions | 24 bit colour | - TIFF - JPEG - PDF or - PDF/A |
Lossless compression |
Managing images as State records
Image copies of State records are themselves State records. As for State records in other formats, a failure by an organisation to protect them from loss or deterioration would constitute a breach of Part 3 of the State Records Act, 1998.
Particularly where the originals are being routinely destroyed (using the General Retention and Disposal Authority: imaged records, see 'Disposal of originals..', below), some principles to follow when managing the image copies to ensure that they continue to be authentic, complete and accessible are:
- capture of the image copies as records into organisational recordkeeping systems
- management of the image copies under the usual framework of organisational recordkeeping rules e.g. policy and procedures
- keeping adequate recordkeeping metadata to facilitate the records' preservation, use and retrieval (See Recordkeeping in Brief 18 - Introducing recordkeeping metadata for more information), and
- documented planning for long term accessibility of the images (See Managing digital records).
Disposal of originals that have been imaged
General Retention and Disposal Authority: imaged records is the key authorisation for the disposal of original (hardcopy) records that have been imaged or microfilmed.
General Retention and Disposal Authority: imaged records operates together with other retention and disposal authorities in that the record’s retention period must be known before it can be dealt with after imaging. It is important to ensure that any records that are being considered for destruction under General Retention and Disposal Authority: imaged records are covered by a current, approved general or functional retention and disposal authority.
There are also a number of conditions for the destruction of originals and a list of exclusions to the Authority. The process below will assist you in sentencing records that have been imaged, using General Retention and Disposal Authority: imaged records.
Are there any special requirements to retain the originals?
One of the conditions for the destruction of originals is that you have assessed whether any special requirements exist to retain the records in their original format.
While the exclusions to General Retention and Disposal Authority: imaged records (section 1.4) identify common classes of records that may not be destroyed after copying, there may be other requirements to retain originals that are unique to your public office.
Examples of unique requirements to retain records in their original formats that may apply to a public office could include:
- a special legislative requirement for a record to be retained in its original format - for example, for display
- a business rule that relies on distribution of a hard copy record around the organisation for annotation
- the possible use in the future of graphic materials eg posters, designs, photographs, brochures for display, or
- customer expectations that they will be able to access certain records relating to them in their original formats.
Where there is concern about the destruction of originals and the retention of image copies in their place for particularly high risk areas of business, public offices should seek a legal opinion.
The assessment of requirements relating to record format should also be considered in making the determination of how long to retain originals for quality control purposes (see 'Day boxing/batching').
Officers responsible for imaging projects should document the results of their assessment of such requirements for retaining originals, and obtain approval to proceed from a senior manager before commencing routine destruction of records.
Process for sentencing originals of imaged records
| If.. | then.. | |
|---|---|---|
| 1 | you can identify an appropriate disposal class for the record in a current, approved general or functional retention and disposal authority | go to Step 2 |
| 2 | the record is not of a type listed in the 'Exclusions' in General Retention and Disposal Authority: imaged records | go to Step 3 |
| 3 | under the record's original disposal authorisation, the record is authorised for eventual destruction | the original may be destroyed after copying, provided the conditions listed in General Retention and Disposal Authority: imaged records are met |
| 4 | under the record's original disposal authorisation, the record is required as a State archive | determine whether the record was created or received before or after January 1, 2000:
|
Please note:
- Where originals are destroyed under General Retention and Disposal Authority: imaged records, the image copies must be retained for the full retention period specified for the originals.
- Details of records destroyed under General Retention and Disposal Authority: imaged records should be recorded, including what was destroyed, when, by whom and the disposal authority/ies used.
- Where originals are not authorised for destruction under General Retention and Disposal Authority: imaged records, they should be registered in a recordkeeping system for management and retrieval purposes and retained for their full retention periods.
What about records that were imaged before General Retention and Disposal Authority: imaged records was issued?
Provided that the originals are eligible for destruction under the rules in General Retention and Disposal Authority: imaged records, they may be destroyed. You should check the documentation that you have about the imaging process used and the quality control measures taken, so that if required, you will be able to demonstrate that these images are authentic, complete and accessible, and that the other conditions for destruction were met. If you do not have adequate documentation, it may be prudent to retain the originals. Remember, originals identified as State archives that were created or received prior to January 1, 2000 are not eligible for destruction after copying under General Retention and Disposal Authority: imaged records.
Keeping documentation of imaging systems and processes
Your organisation may be called upon to 'back up' its claims that its records copied as digital images may be relied upon as evidence, either by a court of law, an external investigative body or a member of the public.
One of the ways you can do this is to provide documentation detailing how the images were made and kept. Where originals have been destroyed using General Retention and Disposal Authority: imaged records, such documentation becomes even more crucial.
| Keep copies of... | ..to show.. |
|---|---|
| organisational policy and procedures that apply to the making and keeping of digital images of records, including superseded versions of the policies / procedures |
|
| documentation of the design of the imaging system |
|
| documentation of assessments carried out of any requirements to retain records in original format from legislative or business requirements |
|
| planning documents relating to ensuring the long term accessibility of the image copies |
|
| disposal documentation (e.g. metatada in records system indicating date and time of destruction and authorisation). |
|
Day boxing / batching
It is good practice and a condition for the destruction of originals under General Retention and Disposal Authority: imaged records to keep the same records for a period of time for 'quality control' purposes after scanning.
This may be done in a variety of ways, but many organisations choose to 'day box' or batch records that have been scanned. This is fine, provided that care is taken to ensure those originals not authorised for destruction after imaging are preserved.
A good strategy for managing originals when scanning is to:
- establish a set period for keeping originals for quality control purposes - e.g. 3 months, 6 months or 1 year - and apply it consistently
- ensure the person doing the scanning is aware of the exclusions from the General Retention and Disposal Authority: imaged records (e.g. records required for current legal proceedings, archives received before Jan 1 2000), so they can take these originals out to be retained and managed separately, and
- make sure the image copies are being registered in a recordkeeping system and properly sentenced using the appropriate retention and disposal authority.
Digitisation projects
In contrast to the digitisation of current business records, sometimes older records are digitised because of:
- a need to provide access to fragile or very old records in a way that will not damage them
- a desire to publish the records online for a particular group or on the World Wide Web, or
- an interest in using General Retention and Disposal Authority: imaged records to destroy originals of a collection of temporary value records that are required to be retained for a very long period of time.
In general the recommendations in the Imaging process’ and Quality control sections of this guide (above) apply for these kinds of digitisation projects. There are, however, some additional good practice recommendations for these projects, including:
- the creation of a very high resolution master file which is kept in secure storage, from which copies can be produced
- where access is the primary concern, greater emphasis on indexing and item level metadata, rather than seeking to register the records according to business activity classifications in a recordkeeping system, and
- use of resource discovery metadata for images that are to be published on the Web (see:
www.agls.gov.au ).
The National Library of Australia guidelines, Digital capture and image creation contain advice on the equipment to use to capture or create images, pixel depth at capture and storage and scanning resolution for the "master" images, based on their own experience digitising their collection. Available at: http://www.nla.gov.au/digital/capture.html
For more information
Archives New Zealand Digitisation Standard, 2007. Available at: http://www.archives.govt.nz/continuum/rkpublications.php
Australian Standard AS ISO 15801 - 2006 Electronic imaging - Information stored electronically - Recommendations for trustworthiness and reliability
British Standards Institution, PD0008 Code of practice for legal admissibility of information stored on electronic document management systems, 1996.
Evidence Act 1995 (NSW)
International Standardization Organisation, ISO/TR 12654-1997 Electronic imaging - Recommendations for the management of electronic recording systems for the recording of documents that may be required as evidence, on WORM optical disk
International Standardization Organisation, ISO/TS 12033-2001 Electronic imaging - Guidance for selection of document image compression methods
Queensland State Archives, Guideline for the Digitisation of Paper Records, 2005. Available at: http://www.archives.qld.gov.au
State Records Authority of New South Wales, General Retention and Disposal Authority - Imaged records, 2006.
Footnotes
[1] Content from this section taken from the Archives New Zealand Digitisation Standard, 2007, Available at: http://www.archives.govt.nz/continuum/rkpublications.php . State Records NSW wishes to thank Archives New Zealand for permitting the reproduction of this content for this guide.
[2] - [5] ibid
[6] PDF/A is a constrained version of PDF version 1.4 with various proprietary fonts and formats removed, issued as International Standard ISO 19005-1:2004. it is a suitable format for long term records at risk from technological obsolescence.
© State of New South Wales through the State Records Authority, 2009. First published March 2000 / Revised December 2003 / Revised February 2007 / Revised October 2009
This work may be freely reproduced and distributed for most purposes, however some restrictions apply.
ISSN 1440-3978
Previous:
Recordkeeping In Brief 4 - Using consultants for records management
