The Interview as an Audit Tool During an IT Audit

Conducting onsite interviews is a critical part of any IT audit and can lead to the gathering of information not readily apparent through reading documentation and examining physical evidence. A number of articles have been written on this topic, although they are primarily focused on financial audits.^{1, 2} This article outlines steps toward conducting successful interviews of IT staff about their processes, and discusses the use of diagramming techniques to help document and facilitate these interviews.

Interviews of key personnel during the course of an IT audit are no less important than interviews conducted during financial and business audits. Indeed, interviews and process walk-throughs during IT audits are necessary to gain a deep understanding of the actual procedures followed and identify possible control weaknesses in these procedures that may not be evident in a review of documentation and evidence.

Preparation

Defining and narrowing the scope of the discussion are also important, as these steps will facilitate the construction of questions to be asked during the interview. When preparing for an interview, a review of the organization and process to be discussed is critical. An understanding of the procedures that are documented, along with the identified controls associated with the processes and procedures, is necessary to be able to formulate questions and follow along with interviewees as they describe their job responsibilities. There may be occasions when existing controls are not well documented, if at all; it is in these cases where a process walk-through can be useful in identifying control points. If there is a previous audit of the area, a review of the report and possibly the related work papers can be helpful, but they should not influence the current audit as they were from a previous point in time.

Questions to be used during the interview can be developed through a combination of procedure documentation review, past experience with the area being audited and past experience on similar audits for other auditees. In general, it may be useful to have a general set of questions (topics) that outline what is to be covered during the course of the interview. These questions and topics are based on the audit test objective, and the degree of detail is dictated by the level of the person being interviewed—the higher the level, the less detail is expected during the course of the interview. Specific questions should not be provided prior to the interview, even when requested, as the auditee may choose to simply provide a written response to the questions without undergoing an interview. Providing an outline of the topics to be covered, rather than the specific questions, should provide the auditee the information needed prior to conducting the actual interview. The questions asked need to be open ended, resulting in a conversation and eliciting a detailed response, not a simple “yes” or “no” answer.

The identification of staff to be interviewed is also important. Because newly hired staff are likely to know less about actual operations and may be involved with less-critical operations, they should be avoided whenever possible. While it is more difficult to get time with more experienced staff (as they will be needed more by the organization), they will know more about the actual operations performed. This is another reason why it is critical to be very clear about the scope of the interview when scheduling the meeting.

When preparing for an interview of staff related to change management, it is important not to assume everyone agrees on the definition being used. Will the discussion include controls around source code changes, migration of application changes through development environments, changes to infrastructure components or emergency changes? Frequently, these items will be managed by different staff and, in larger organizations with many platforms, conducted differently depending on the platform in question.

The Interview

There are many opinions on where to conduct the actual interview. Some advocate having the interview conducted in the interviewee’s workspace for reasons that include having possibly needed information nearby.³ But to minimize distractions, it may be a good idea to conduct the interview away from the interviewee’s work area, unless an actual demonstration, walk-through or screen captures need to be included in the work paper documentation. The amount of on-hand information needed during an interview is minimal for process-related walk-throughs. Using a conference room with a whiteboard may help when mapping out processes.

It is helpful to begin the interview with a review of the purpose and objective of the interview to ensure that there is agreement on the purpose and the right people are present to accomplish the objective. Additionally, confirmation of the agreed-upon time constraints specified when scheduling the interview is likely to assure all participants that their time will be used wisely. It is useful to communicate to the interviewee that notes will be taken and there may be some follow-up items to complete after reviewing notes taken during the interview. It may be helpful to spend a bit of time talking about the auditor’s background to help start the conversation.

Depending on the size of the audit staff, it may be a good idea to have another audit staff member attend the interview as an observer, but prior to the audit that individual should be requested to refrain from asking questions unless he/she feels something critical is being missed. The addition of an extra observer helps clarify questions when reviewing notes taken postinterview and establishing follow-up items. Lead auditors may consider attending interviews that will be conducted by more junior audit staff or with audit staff who may not be familiar with the entity being audited. The goal is not to outnumber the auditee through a show of force, but to ensure accuracy (in the case of having more than one auditor present) and reduce any contention and the possibility of the interview getting off track (in the case of a junior auditor conducting the interview). The next item is applicable for any type of interview, be it a job interview, a TV/radio interview or any other speaking occasion intended to gather information. It is critical to listen to the full response to the questions asked, not cutting off the person responding or completing their response for them. Another important point is to avoid questioning the interviewee as to their skills or abilities; such questions are likely to cause the interviewee to close up and the interview will quickly end. Many IT auditors come from IT backgrounds, so the temptation is great to question others about their skills or prowess. This temptation should be resisted, but the interviewer should feel free to question areas in which controls may be weak or missing. As an auditor, this is where expertise should come into play and will not be taken as an affront. In the course of the discussion, topics may arise that are interesting to talk about, but not in line with the objective of the meeting or the audit overall. It is important not to get sidetracked into such discussions.

Those with an IT background are likely to be familiar with a number of diagramming techniques. If, during the course of the interview, auditors feel they have established rapport with the interviewee, diagramming specific processes can be an incredibly effective tool in identifying control weaknesses and establishing where actual procedures do not match those provided in written documents. Depending on what is being reviewed, there are a number of diagramming techniques available to use. This does not have to be very formal; it can function as a way to help map processes and identify control points. The Basics of Process Mapping provides a description of some very useful diagramming methods including relationship maps, cross-functional process maps (swim-lane diagrams) and flowcharts.⁴ Another excellent resource is Workflow Modeling: Tools for Process Improvement and Application Development.⁵ The older, out-of-print book Diagramming Techniques for Analysts and Programmers is also a useful resource for diagramming.⁶

Regardless of the type of diagram used, the auditor has a variety of tools with which to create these diagrams. Microsoft Visio is an excellent tool; however, some work paper systems may not allow the attachment of Visio files. MS Word (2007 and later) has a feature called “smart art,” which provides for some process-related charts. MS PowerPoint can also be used. That said, do not use these tools during the interview; too much time may be spent trying to use the tool rather than focusing on the process being discussed. Paper and pen or the aforementioned whiteboard is generally sufficient.

During the interview, auditors should be cognizant that as closely as they are watching interviewees’ reactions, the interviewees will be watching the auditors as well. Physical posture, facial expressions and hand gestures will be observed. In fact, in one case an auditor was conducting an interview and taking notes, and one of the people being interviewed remarked, with concern, “He is reaching for the red pen.” This was in response to the auditor’s habit of using a red pen to make notes for follow-up later, but it was clearly misinterpreted by the interviewee. Subsequent to that, the auditor made sure to use only one pen for note taking, replacing the red pen with an asterisk in the margin.

Closing the Interview

At the conclusion of the interview, it is helpful for the auditor to review the items covered and highlight any areas of potential concern. The auditor should take care not to worry the interviewee by stating that there are major breakdowns or issues; instead, the message should be that some areas will need additional follow-up or that there appear to be possible control gaps. If there were items discussed for which physical evidence/documentation will be needed (e.g., change tickets, project documentation), the auditor should review these and clarify the process that has been used for obtaining all audit evidence. The interview should conclude with a word of thanks to the interviewee for his/her time.

After the Interview

After the interview has concluded, the auditor must closely review any notes taken during the interview and construct a list of follow-up questions and physical evidence that may be needed. If the interview was conducted with another member of the audit team, the two auditors should review and compare notes together to ensure a common understanding of the discussion.

Using Diagrams

During a process-oriented interview, the use of a diagram can enhance the conversation by helping to clarify control points and ensure mutual understanding. Simple processes can benefit from the use of a flowchart. The intent is not to create a detailed model of the entire process but to break the process down into small pieces and focus on areas identified as possibly lacking controls, as determined during a review of the documentation or during the discussion.

For example, during a review of the change management process, the documentation mentions the use of a source code management system; however, it does not go into much detail. The system documentation that came with the product could be reviewed, and this would be a good time to diagram the process. A simplified diagram (figure 1) shows a Visio diagram of a simulated flow in which more than one person can check out the same source code, resulting in the existence of two versions of the source code that could be considered current: v3.2.3 and v3.3.1. The installation of the source code tool was performed by the technical support group of the organization. Knowing only that the tool works in the computing environment, the tool was installed using default settings. The development team did not specify or change the settings to include a control that would prevent the simultaneous checkout of code that results in multiple code streams. This may be a desired effect or may result in rework needed to merge the streams together later. By diagramming the process interactively with the auditee during the course of the interview (on a whiteboard or blank sheet of paper), the possible control weakness can be easily identified.

The previous example demonstrated the use of a simplified flowchart to help guide the conversation and map out a picture of the code management process. More complex processes that use more individuals or groups would benefit from a swim-lane diagram. Swim-lane diagrams provide a high-level flow and identify the people or groups involved with the process. Figure 2 shows a swim-lane diagram of the application change management process. This is a greatly simplified flow of the change management process; however, unlike the flowchart, the swim-lane diagram shows the various groups involved and how they interact. Each of the boxes on the swim-lane diagram could be further expanded using flowcharts.

The creation of a swim-lane diagram should be done after procedural documentation has been read and the necessary interview(s) have been conducted. It is a more complex diagram than a simple flowchart, and creating one during an interview will likely take valuable time away from the interview itself. In fact, when conducting the audit, any diagrams created are best done on a whiteboard or blank sheet of paper, and the use of a tool should be saved for later. Once created, the diagram can be reviewed with the auditee for accuracy and offered to the auditee to use in his/ her department’s documentation. Offering the diagram to the auditee may help demonstrate audit’s goal of being a partner rather than an adversary.

Conclusion

Many times, an interview can help auditors identify possible control breakdowns in an IT environment or people performing their job in a way that does not follow the documented procedures. While some things identified in an interview are easily countered (e.g., “No, I misspoke. We do it this way.”), it may lead an auditor to request evidence that would support something mentioned in the interview. While there is little formal training offered on this skill, it is definitely a skill worth acquiring. Learning how to develop and ask questions, establish rapport, and obtain information can only really be learned with practice. Confidence is built over time, and it can lead to more effective interviews. Using diagramming tools, such as flowcharts, can greatly enhance the quality of the conversation and understanding of key process flows and controls.

Endnotes

¹ Leincke, L.; J. Ostrosky; M. Rexroad; J. Baker; S. Beckman; “Interviewing as an Auditing Tool,” The CPA Journal, February 2005
² Seipp, E.; D. Lindberg; “A Guide to Effective Audit Interviews,” The CPA Journal, April 2012
³ Ibid.
⁴ Damelio, R.; The Basics of Process Mapping, 2^nd Edition, Productivity Press, USA, 2011
⁵ Sharp, A.; P. McDermott; Workflow Modeling: Tools for Process Improvement and Application Development, 2^nd Edition, Artech House, USA, 2009
⁶ Martin, J.; K. McClure.; Diagramming Techniques for Analysts and Programmers, Prentice Hall, USA, 1985