Interviews are valuable data-gathering tools in the internal auditor’s tool kit. However, they can be an expensive undertaking, especially in terms of time, for the large-scale collection of evidence. For example, imagine trying to understand the tone at the top in a 10,000-employee enterprise. Casting a wide net is desirable to get a broad understanding of employees’ perceptions of the enterprise’s ethical climate. To do so, it would be helpful to interview 500 employees, but the options available to auditors include:
- Physical interviews—Even with 10 internal auditors devoted to the task, it would take a minimum of 125 hours to conduct 15-minute interviews. At a rate of US$85 per hour, the costs of data collection alone would be more than US$10,500—plus the additional time to analyze and make sense of all the unstructured data.
- Surveys—To gather 500 responses, one option is to email 2,000 employees and promise each one a US$10 gift card for completion of the survey. To build, test and analyze the survey, it would probably cost approximately US$6,000. Although a survey would be efficient, the data are often bland and uninformative. Employees may be unmotivated to provide important details, and answering multiple-choice questions results in a limited amount of good data.
Given these two options, it is common for internal auditors to not even attempt to collect these valuable data or to collect only a small data sample that is not very meaningful. Either way, the internal auditor misses the opportunity to take the pulse of the enterprise and add value by identifying where improvements can be made.
New technology offers the potential to cost- effectively collect large interview data sets. Researchers have been working to automate the auditing interview process with virtual interviewers. The interviewer is human-like, but it is an autonomous agent (avatar) that can conduct the interview, record and transcribe the responses, and analyze the results.
Virtual Auditors vs. Human Auditors
This technology is intriguing, but the real question is whether it can produce valuable internal audit data. Researchers tested whether virtual auditors could outperform human auditors in three experiments involving more than 500 participants.1 Initial evidence suggests that in addition to being more efficient, this method can be far more effective than using human interviewers. The researchers found that employees were 21 to 32 percent more likely to disclose timekeeping violations to an avatar interviewer than to a human interviewer with significant interviewing experience.
One of the exciting aspects of this technology is the ability to customize interviews to the interviewer. In the first experiment with 290 individuals, researchers compared virtual interviewers with human interviewers. For half the participants, the virtual interviewer was digitally altered to resemble the participant, based on a photo taken of the participant. Prior research shows that individuals are more forthcoming when dealing with people who are similar to themselves, so participants’ photographs were morphed into the virtual interviewer’s face, subtly making the interviewer resemble the participant. Very few participants noticed the similarity, yet on average, they rated their interviewers more likable than did the participants questioned by an unmorphed virtual interviewer or a human interviewer. Because people tend to disclose more sensitive information to those they like, this is a potential benefit to using virtual interviewers.
In the second experiment, researchers collaborated with the internal audit group at a private university to interview 129 part-time employees about their timekeeping practices (e.g., working more hours than allowed, overreporting their hours or attending to personal business during work hours). The face-morphing technology was not used in this experiment, which compared the virtual interviewer with three human interviewers chosen to represent new internal auditors. Sixty-five percent of the employees interviewed by the virtual interviewer disclosed at least one violation of internal timekeeping controls, while on average, only 44 percent of the employees interviewed by a human being disclosed such a violation (figure 1).
The third experiment compared virtual interviewers with seasoned interviewing professionals (rather than entry-level interviewers). One interviewer had 32 years of Big 4 audit experience, one was a controller, one was an audit senior manager and one was part of the university’s internal audit group. The human interviewers were all instructed to do their best to elicit complete and accurate information from the employees. In addition, to address the feasibility of using a survey, an additional group of employees completed a survey instead of being interviewed.
The results? The virtual interviewer surpassed both the experienced interviewers and the questionnaire. Fifty-four percent of the employees interviewed by the virtual interviewer admitted to at least one internal control violation, compared with 33 percent in the survey and 22 percent, on average, to the human interviewers (figure 2). The virtual interviewer elicited more than twice the admissions compared with the humans and nearly two-thirds more than the survey.
Upsides to Virtual Interviewers
This study of virtual interviewers represents one piece of a larger puzzle. There are still many more questions that need to be addressed about automated interviewing. However, this emerging technology shows significant promise and noteworthy benefits. First, automated interviewing is scalable. Although the initial investment is substantial, the system can easily scale across time and space—a valuable benefit in today’s changed landscape due to the COVID-19 pandemic. A virtual interviewer can interview many people in different geographic locations at the same time and can be programmed to speak in various languages. Many enterprises are already utilizing virtual communication to conduct first-round hiring interviews using services such as Shortlister.com,2 myInterview.com3 and VidCruiter.com.4 The results of these interviews allow enterprises to identify the best candidates for more extensive interviewing.
At the present time, organizations interested in using this technology should conduct the virtual and human interviews in tandem to continue to fully validate the technology.
Another benefit of virtual interviewers is their appeal to younger individuals who prefer computer-mediated communication. Generation Z and late Millennials—who have never known life without the Internet and social media—are entering the workforce. On the surface, being interviewed by a virtual interviewer feels normal to them, but perhaps there is an underlying reason that young people, and others as well, prefer to communicate with virtual interviewers: They feel less judged. In one survey, college students reported a preference for speaking to an avatar rather than a human about sensitive topics.5 Perhaps they recognize that a computer-driven interviewer lacks the social, intellectual and moral capacity to judge them. Or perhaps the artificialness of the interaction relieves the tension of in-person communication. This phenomenon has been studied extensively. People feel comfortable doing and saying things on the Internet they would never do or say in person. Virtual interviewers may be able to capitalize on this feeling to enhance evidence collection.
It is interesting to note that in the third experiment, employees admitted more violations to the virtual interviewer than to the survey. A survey cannot judge the employees, so what accounts for the difference? One explanation is that human beings are social creatures, and sociality is heavily anchored in the face. The mere presence of a face triggers an instinctual desire to communicate with it. Perhaps this is why many text-based chatbots are depicted with faces. However, research demonstrates that when an avatar’s face is too human-looking, it can actually inhibit interviewees—they find it disturbing. Clearly, there is still a lot to learn about designing a virtual interview to maximize honesty and engagement.
The scalability of virtual interviews can also condense the interviewing window because multiple employees can be interviewed simultaneously. In some situations, this helps prevent biased responses, which may occur if gossip about an investigation or a project spreads among employees. When this occurs, employees may arrive at the interview already primed by conversations with their peers.
Downsides to Virtual Interviewers
Currently, the most significant drawback to virtual interviewers is that this technology has undergone initial tests only in mock laboratory accounting settings. It should be noted that self-disclosure to virtual humans has been thoroughly tested in many academic studies, but the type of self-disclosure tested is ground in relationship building, not business settings—they are different constructs. At the present time, organizations interested in using this technology should conduct the virtual and human interviews in tandem to continue to fully validate the technology. Once the technology has been fully validated, the next question will be whether it can be used in-house or whether it requires a third-party provider. Keeping the technology in-house means that internal audit groups will have to compete for already hard-to-hire technologists, adding animation and graphic expertise to the growing list of technological demands. Using a third-party provider means that internal audit groups will face cost issues, trust issues (e.g., trusting a third party with sensitive data) and the potential inability to customize the technology to provide maximum benefits to the internal audit group.
Other Uses of Avatars
Although virtual interviewers are new to internal auditing and accounting, they have been used successfully in some other areas. For example, the University of Arizona (Tucson, Arizona, USA) teamed with the US Department of Homeland Security to build an automated screening kiosk to interview individuals crossing the border into the United States.6 The virtual interviewer asks border crossers a series of questions while sensors in the kiosk (e.g., thermal camera, eye tracker, microphone) monitor the crosser for deception. When the system flags a potential deception, the individual is screened by a border agent.
Although this technology is still several years away from broad adoption, it strongly suggests that tasks previously considered safe from automation may not be safe in reality.
In another example, the US Army created an interrogation training system in which the avatar is the interviewee.7 This allows field soldiers to practice interrogating subjects and working with translators to collect intelligence.
In internal auditing, an automated virtual interviewer system could also be used to evaluate different interviewing techniques. For example, by adjusting the avatar’s tonality, phrasing and nonverbal cues and measuring the effectiveness of each approach, the interviewing training process could be improved. Similar to the US Army’s training system, new auditors could be trained to conduct better interviews by practicing with avatars. Such a system could train auditors to exercise professional skepticism and identify effective follow-up questions.
Conclusion
Virtual interviewers are just one part of the technology revolution that is changing the face (pun intended) of internal auditing. Although this technology is still several years away from broad adoption, it strongly suggests that tasks previously considered safe from automation may not be safe in reality. Internal auditors of the future will need to embrace technology and change to an even greater degree, whether it involves working with virtual interviewers or other technologies. Internal auditors who keep apprised of the new technological possibilities will be value-adding leaders in their enterprises.
Endnotes
1 Pickard, M. D.; R. Schuetzler; J. S. Valacich; D. A. Wood; “Innovative Accounting Interviewing: A Comparison of Real and Virtual Accounting Interviewers,” Accounting Review, vol. 95, iss. 6, 2020, p. 339–366, https://doi.org/10.2308/tar-2017-0235
2 Shortlister, https://shortlister.com/
3 Vid Cruiter, https://vidcruiter.com/
4 My Interview, https://www.myinterview.com/
5 Pickard, M. D.; C. A. Roster; Y. Chen; “Revealing Sensitive Information in Personal Interviews: Is Self-Disclosure Easier With Humans or Avatars and Under What Conditions?” Computers in Human Behavior, vol. 65, 2016, p. 23–30, https://doi.org/10.1016/j.chb.2016.08.004
6 The University of Arizona Eller College of Management, Tucson, Arizona, USA, “Avatar,” https://eller.arizona.edu/eller-news-topics/avatar
7 Dooley, K.; “PEO STRI Helps Soldiers Refine Intelligence Collection Techniques,” US Army, 11 June 2007, https://www.army.mil/article/3527/peo_stri_helps_soldiers_refine_intelligence_collection_techniques
MATTHEW D. PICKARD | PH.D.
Is a professor of accounting data and analytics at Northern Illinois University (Dekalb, Illinois, USA). He has published articles in a variety of journals, including The Accounting Review, Journal of Information Systems and Computers in Human Behavior. He researches artificial intelligence applications in accounting and teaches a variety of data and analytics courses.
DAVID A. WOOD | PH.D.
Is the Glenn D. Ardis professor of accounting at the School of Accountancy at Brigham Young University (Provo, Utah, USA). He has published more than 100 articles in a combination of respected academic and practitioner journals, monographs, books and cases. Wood is a coauthor on a market leading accounting information systems textbook. He works with the EY Academic Resource Center and has provided consulting services to various organizations.