Sources for Step D assessment
- recordkeeping requirements, identified in Step C: Identification of recordkeeping requirements, and
- required recordkeeping functionality, as outlined in Introducing DIRKS, Characteristics and functionality of recordkeeping systems, and
- all relevant system components, including policies, procedures and training materials.
Once you have identified the system or systems that should be the focus of your assessment, you need to determine their adequacy.
To do this it is necessary to establish measures or benchmarks against which your systems can be assessed.
The benchmarks that should be used to assess your systems are:
- identified recordkeeping requirements and
- required recordkeeping functionality.
You should measure the systems you need to assess against both of these types of requirements.
Identified recordkeeping requirements
Recordkeeping requirements pertaining to your organisation's business activities can be identified using Step C of the DIRKS analysis. If you have not undertaken Step C but are aware of the recordkeeping requirements that relate to the area of your business requiring assessment, be sure to document these. You should have some documentation of recordkeeping requirements to ensure the consistency and comprehensiveness of your analysis.
It may be helpful to reframe the requirements for evidence as a series of questions. The answers to these questions should help to determine whether the requirement is satisfied or not.
For example, in doing its Step C: Identification of recordkeeping requirements assessment, one organisation identified that the following requirement applied to the management of its licence records: 'Licensees should not be given access to the records of other licensees.'
To determine whether the system they use for managing their licencing operations is meets this requirement, they could ask:
- is the system capable of restricting access to designated users?
- does system user training make it clear that access restrictions apply to licensee records?
- do the policies that are part of this system inform staff of these access rules?
If the response to each of these questions is yes, then it is likely that the recordkeeping requirement has been met.
Remember that recordkeeping requirements may have several components to them. Be sure to read your requirements thoroughly and measure whether all its requirements have been fulfilled.
For example, for a licensing activity, a recordkeeping requirement could be:
'When a licence is revoked, a record of the reasons for revoking the licence will be created and retained with the licence records.'
This requirement contains a number of parts. Firstly it states that records must be created. In your system assessment you would firstly need to ensure that records documenting the licensing process are in fact being created, including records of licence revocation. You would then need to determine that the system is capable of linking all related records in ways that meet your identified business requirement.
It is also important to realise that the sources that you used in step C contained implicit and explicit references regarding the form, content and quality of evidence your organisation should satisfy. It is important that the survey techniques that you use in step D are flexible enough to assess the variety of recordkeeping requirements identified in the earlier step.
Required recordkeeping functionality
Required recordkeeping functionality refers to the recordkeeping controls and business rules that are necessary to ensure your system operates effectively as a recordkeeping system.
The range of qualities a system should possess in order to be a recordkeeping system is outlined within the section of Introducing DIRKS, Characteristics and functionality of recordkeeping systems. Review this section to help identify the range of recordkeeping functionality your business systems may need to possess. You can turn these requirements into benchmarks for your system assessment.
In the NSW public sector, all records of government business must only be disposed of in accordance with authorised retention and disposal authorities. General recordkeeping rules (outlined in Introducing DIRKS, Characteristics and functionality of recordkeeping systems) also specify that records must reside in a system where they cannot be tampered with or altered. In your system assessment, you may decide to assess whether your systems are capable of providing this recordkeeping functionality.
Questions you may want to use to assess this functionality in a system could include:
- is the system capturing business records?
- is the system ensuring that all necessary metadata is captured?
- are records protected against alteration/deletion?
- is the system implementing disposal decisions?
- are strategies for the long term preservation of records in place?
- can records be easily accessed and retrieved?
- can the system track record use and management?
- is information duplicated?
Note, that depending on the business it documents, the system you are assessing may need to meet these and a range of other requirements.
If you undertook Step C of the methodology, you may have identified a range of recordkeeping requirements relating to numerous business activities your organisation performs. When undertaking specific system assessment in Step D, you need to make sure you identify the recordkeeping requirements that pertain specifically to the system or systems you are assessing. That is, you need to identify the recordkeeping requirements that are relevant to the area of business you are focussing on.
The same requirement applies to the recordkeeping functionality you are assessing. Make sure you assess a system only against the functionality you need it to have in order to meet your business requirements, and the recordkeeping functionality outlined in Introducing DIRKS, Characteristics and functionality of recordkeeping systems. Only compare a system to the recordkeeping requirements that relate to it.
Tip: Use others' experience
If you have access to the research of other organisations that have undertaken the DIRKS methodology and compared their recordkeeping requirements against the current functionality of their systems, this data should provide some useful models for you.
Be aware, however, that you cannot replicate the recommendations of another organisation's research in your own environment, even if the organisation performs very similar functions. Each business environment is unique and while you can certainly learn from the experience of others, it is very important to ensure that your DIRKS work is specific to and helps resolves the issues facing your own organisation.
To assist with you system assessment, you may want to examine a range of other sources to ensure you have a good understanding of the system's role, operation and structure and to ensure you are measuring the full functionality and capacity of the system against your recordkeeping requirements.
Given systems are comprised of people, policies, procedures, tools and technology, you will want to include all these components in your assessment.
There are a range of sources you can use to provide you with the information you will need, and to ensure you cover all relevant aspects of your system.
Interviews with system users and managers
The best means to understand how a system works is to talk to the people who use the system and the people who manage the system. These staff can summarise the role and functionality of the system for you and answer any specific questions you have.
If possible, try to talk to a range of staff. Include operational staff as well as managers to get a good understanding of the system's operation and desired functionality.
Tip: Realise that you may be invading someone else's turf
It is important to realise that when examining a business system in depth, you may upset the manager with responsibility for that system. They may see the system as being their's, and not a recordkeeping system that you should be concerned with. It is essential to get their support and advice for your assessment. Discuss your objectives with them and work with them to try and improve the system in ways that will meet both of your requirements.
Remember, DIRKS is intended to be a flexible methodology. Therefore you can include questions about system operation and functionality as a part of the interviews you may conduct in Step C to help identify recordkeeping requirements.
Tip: Explain your project clearly
Clearly explain to staff the types of information you are seeking in your system assessment. Clearly explaining your objectives will save time for all participants and will ensure you obtain the type of information you are seeking.
See for further guidance:
- Guide to interviews
- Sample interview questions
In the course of your system assessment you must examine the policies and procedures that support system maintenance and use. Remember that policy and procedure are a key component of any system and must be included in your assessment.
Example: Lack of clear and consistent procedure contributed to the Jamaican banking collapse
In the Jamaican banking collapse, lack of clear and consistent procedure was identified as a major system failing, and one of the significant contributing factors to the collapse:
While deliberate failure to create and keep accurate and complete records of financial transactions did contribute to the problems experienced by the banks, the failed banks' routine practices of records creation and recordkeeping were an equally, if not more, significant problem. As a result of these practices, managers and directors in these financial institutions and bank supervisory authorities lacked the trustworthy and timely and accounting and management information they needed to maintain effective control of the banks' operations, to assess and manage their financial positions and risk exposures, and to prevent fraud. 
This assessment also identified that:
The problems created by weak enforcement of any existing controls permitted the banks' officers to record transactions according to their own motivations, preferences, and personal standards. 
Consequently poor evidence and information was created. This significant problem could have been avoided if clear procedural documentation had been in place.
All systems that need to operate as recordkeeping systems must be supported by policies and procedures. If the systems you are examining are not sustained by this type of documentation, be sure to make note of this. Steps E, F and G of the methodology outline how you can develop policies and procedures as part of the process of transforming business systems into recordkeeping systems.
Tip: Check procedures for board management
If your organisation is regulated by a board, it is important to ensure that the minutes for this board are appropriately made and maintained. In your system assessment you could ensure that the procedures for board management state that:
* all board minutes should be signed by the chairperson
* an official record is retained
* an electronic copy is kept for ease of reference.
Recordkeeping tools operating within the system
If they exist, recordkeeping tools, such as disposal authorities and business classification schemes, are key system components. In your system analysis you will need to examine these, if you have not done so already, to determine their currency and effectiveness.
When looking at tools you will also want to assess whether and how they are implemented within the system. You may want to ask:
- Are they employed systematically?
- Is their use automatic?
- Could their use be made easier?
- Are they supporting or hindering the satisfaction of recordkeeping requirements?
Educational programs supporting the system
Given the crucial role that staff play in a system's effective operation, your Step D analysis may also include an assessment of the training or communication programs that explain to staff how to use the system. Your organisation may not have such programs in place, but if courses, brochures or other resources exist, you should examine these to get an understanding of how the system operates, what users are required to do and other training frameworks that may be required.
Talk to key staff about the training they have received to help them perform their roles. Document any training programs you identify, and also document when they do not exist.
Example: Lack of training contributed to the Jamaican banking collapse
Lack of training in recordkeeping system operation was a problem that contributed to the Jamaican financial collapse:
In many of the failed banks, responsibility for the management of records stores was assigned to low-level clerical staff. For example, one person in charge of the registry for the credit files was untrained in recordkeeping principles and techniques. Neither had this person received any training in the basics of credit administration. Thus, this clerical officer was ill-equipped to appreciate the information retrieval requirements of the bank’s credit administrators and risk analysts and had little, if any, understanding of the value of, and techniques for, creating indexes searching tools] to support managers' information requirements. 
Technical documentation describing the system
Obtaining a good, basic knowledge of how the technical components of your organisational systems work - how they process and manage information - may be crucial to this assessment. In implementing Step D it is crucial to know your recordkeeping requirements, but also to understand how your online transaction processing system or decision support system fares in terms of its recordkeeping functionality.  You may be able to gain a lot of information about your existing systems by reading relevant documentation on system functionality, requirements and operations. This documentation could include procedure manuals, disposal schedules, business rules specifying access rights, standards that need to be complied with, training materials etc.
If documentation is poor or non-existent, talk to staff that use the system and examine the system in operation to gain an understanding of its technical infrastructure.
You may also want to examine the business rules applied within the system - any rules built into it that specify how the system should function and manage data - and see how these correspond with or relate to the recordkeeping tools, such as disposal authorities, that should be employed within the system.
 Victoria L Lemieux, 'Let the Ghosts Speak: An Empirical Exploration of the 'Nature' of the Record, Archivaria, Number 51, Spring 2001, 86
 Ibid, 101-102
 Ibid, 105
 This strategy is outlined in Philip C. Bantin, Indiana University Electronic Records Project, Phase II, 2000-2002, Skills Required to be an Effective Manager of Electronic Records White Paper, BloomingtonIndiana, 2002. Accessed in January 2003 via the IndianaUniversity website at: http://www.libraries.iub.edu/index.php?pageId=3313