“Weakest link” and “unpredictable”: To an information security practitioner, these descriptions almost certainly identify the human component of a layered security approach because humans are unpredictable animals who are susceptible to temptation, emotions and complacency. After all, the strongest firewall and the most sophisticated intrusion detection/prevention software will not prevent an individual from disclosing sensitive data via social engineering. In fact, the recent data breach at RSA, the security division of EMC Corp., is proof. In this breach, the attackers were able to gather information about the company’s SecurID two-factor authentication products through a phishing attack. The attackers sent a Microsoft Excel file via e-mail to employees, and at least one employee opened the file, thus opening the door to the attack. Therefore, the breach was not a technology problem, but a human problem.1 It is no wonder that, for many years, information security professionals have been touting the importance of providing security awareness training within an organization. However, no research has been conducted to determine the most effective components of a security awareness training program. Professionals say to do it, but they do not say how to do it.
To address this lack of research in the area of security awareness components, research was conducted for the doctoral program at the University of Fairfax (Vienna, Virginia, USA) to examine the relationship of user awareness training components and perceived security effectiveness. The research extended the work of Kenneth Knapp, who, in 2005, addressed the questions of the relationship of top management support on perceived security effectiveness and the constructs that mediate that relationship. Knapp examined four mediating variables: user training, security culture, policy relevance and policy enforcement.2 Based on the findings of his research, it was concluded that, of the four mediating variables that he identified, user training had the strongest relationship between the independent variable of top management support and the dependent variable of security effectiveness. The purpose of this research, then, was to focus on the relationship between user training and security effectiveness.
Project Objectives
The purpose of this study is to help organizations increase their chances of implementing effective security awareness training by identifying the best possible set of user awareness training variables. Using a survey, IT professionals’ perceptions of security effectiveness within their organizations were measured based on their attitudes toward:
- Frequency of user awareness training
- Method of user awareness training
- User awareness training compliance monitoring
Research Approach and Methodology
An anonymous survey was used as the data collection tool. The survey used a five-point Likert scale to measure the participants’ attitudes toward components of their organizations’ security awareness trainings and the security effectiveness within their organizations. The research was driven by three questions:
- What is the relationship between training frequency and perceived security effectiveness as measured by the survey?
- What is the relationship between training delivery method and perceived security effectiveness as measured by the survey?
- What is the relationship between training compliance monitoring and perceived security effectiveness as measured by the survey?
Validity of the survey was addressed by using a panel-of-experts approach. A peer review/field trial was conducted during the period of 31 July 2009 and 8 September 2009, using a 10-person expert panel that consisted of ISACA members from the St. Louis (Missouri, USA) and Illinois (USA) chapters. Surveys were sent to the expert panel via e-mail with a cover letter that explained the proposed research project and that asked the following questions:3
- Is the content of the questionnaire appropriate for the audience?
- Are the survey items clear, and do they make sense?
- Are any of the survey items intrusive, invasive, potentially embarrassing or of a sensitive nature?
Feedback was received via e-mail and incorporated into the survey as deemed appropriate; for the most part, changes were minor.
Reliability of the survey was measured using a test-retest sequence administered to a pilot study panel of nine ISACA members who did not participate in the expert panel. The pilot study occurred during the period of 24 November 2009 and 12 January 2010. The survey was loaded into an online survey web site, and members of the pilot study were randomly assigned a number from one to nine. The panel was then sent an e-mail that instructed the participants to complete the survey on the online survey web site. The first phase of the test-retest sequence occurred between 24 November 2009 and 3 December 2009. The content and wording of the questions were not altered for the retest phase, but the questions were randomly reordered. On 21 December 2009, another e-mail was sent to the panel, instructing participants to complete the survey again. The final phase of the pilot study was completed on 12 January 2010. The results were downloaded into Statistical Package for the Social Sciences (SPSS) software for analysis. The expected outcome of the test-retest sequence was that there would be little or no significant difference between the results of test and retest data. One question did show significant difference and was, therefore, removed from the final survey.
Upon completion of the pilot study, the final version of the survey was created for the online survey web site. The link to the survey was sent via e-mail to all ISACA chapter presidents for distribution and to the ISACA headquarters for posting on its web site. The link was also posted on ISACA social and professional networking sites. The survey remained accessible on the survey web site between 1 March 2010 and 31 March 2010. The data were then analyzed.
ISACA was chosen as the research site because the international professional association is comprised of a balanced mix of various levels of IT professionals working in multiple industries and in various capacities. IT professionals, rather than regular employees, were chosen for the research because they are more aware of IT security issues and are a more homogeneous group. Presenting the survey to the random public would have likely resulted in more inconclusive findings because of the heterogeneity of such a large group. By the same token, limiting the survey to one industry or organization would have limited the scope of the research.
As Knapp points out in his research, a debate exists within the IT community regarding the measurement of perceived security effectiveness. The elusive nature of the term “effectiveness,” coupled with the sensitive nature of asking an organization to measure its security, poses a challenge to developing a common industry definition.4 Richard A. Caralli elaborates on the challenging aspect of defining security effectiveness by pointing out that security is contextual and not an isolated discipline; it depends on the organization and its operations. Furthermore, effective security “must take into account the dynamically changing risk environment within which most organizations are expected to survive and thrive.”5 As such, Knapp did not attempt to establish a definition for security effectiveness for his research. Instead, the perceived effectiveness variable in his study was “based on the subjective judgment of security professionals.”6 As this research extended Knapp’s, it also based security effectiveness on the subjective judgment of security professionals.
Respondents
A total of 133 ISACA members, representing multiple industries, participated in the survey. The largest percentage (26 percent) of the respondents worked in the finance, banking and insurance industry. Government and professional services were the two second most popular industries. Nonprofit and industrial technology had the least representation at 1 percent each.
The participants represented multiple countries, albeit the majority of participants were from the United States (73 percent). Other countries represented included: India (17 percent), Costa Rica (5 percent), Australia (2 percent), Belgium (2 percent) and China (1 percent).
The majority of participants (60 percent) reported that information security is a secondary responsibility of their jobs. This is understandable considering that the majority of the participants (43 percent) identified themselves as audit professionals. “Information security professional” ranked third at 12 percent.
The majority of respondents (89 percent) held at least one professional certification. Certified Information Systems Auditor (CISA) and Certified Information Systems Security Professional (CISSP) were the most common. Other popular certifications included Certified Information Security Manager (CISM), Certified in the Governance of Enterprise IT (CGEIT), Certified Internal Auditor (CIA) and Certified Public Accountant (CPA).
Survey Results
Six key findings came out of this research, which can be categorized as:
- Overall security effectiveness
- Training frequency
- Training methodology
- Training compliance monitoring
- Training topics
- Relationships
Overall Security Effectiveness
The first finding is related to security effectiveness as indicated by how participants ranked their agreement to the survey statement: “My organization secures its data and information effectively.” Overall, the majority of participants agree that their organizations secure their data and information effectively.
However, the majority of those participants do not strongly agree. Only 22 percent strongly agreed that their organization secures its data and information effectively. Close behind are those respondents who were neutral on the subject (20 percent). However, the majority of respondents (42 percent) did agree that their organization secures its data and information effectively. While positive overall, these numbers also show that 36 percent of those surveyed believe that their organization is either not effectively securing its data or were neutral on the subject. Clearly, there is room for improvement.
Training Frequency
The majority of organizations (50 percent) represented in this research deliver user security awareness training once a year. However, only 24 percent of the organizations delivered training more than once a year, and 17 percent delivered the training during new employee orientation only. An even smaller portion of the organizations either delivered it on a voluntary basis only or never.
When training frequency was cross-tabulated with perceived security effectiveness, the “once a year” category had the highest rate of participants who strongly agreed that their organization secures its data and information effectively. The “more than once a year” category follows closely. However, the other categories have little to no instances of participants strongly agreeing. It would appear that the fewer times employees are exposed to user security awareness training, the less likely they will be to view their organizations as effectively securing data.
Training Methodology
According to the survey, the majority of organizations (69 percent) use some combination of methods to deliver training, vs. the 4 percent of participants who reported that their organizations used all of the methods listed in the survey. Those organizations that depend solely on policies and procedures as user security awareness training made up the next highest majority (8 percent). Only a small portion of the respondents answered that their organizations used only one training method (other than policies and procedures) to deliver user security awareness training.
When training method was cross-tabulated with perceived security effectiveness, the “combination of methods” category had the highest rate of participants who strongly agreed that their organizations secure their data and information effectively. The “computer-based training only” category was ranked second. However, the other categories have little to no instances of participants strongly agreeing. It would appear that employees exposed to only one type of user security awareness training methodology were less likely to view their organizations as effectively securing their data.
Training Compliance Monitoring
The concept of training compliance monitoring is that organizations verify that their employees are satisfying the security training requirements, retaining that knowledge and implementing it in the workplace. The research data showed that training compliance monitoring was the second best predicator for perceived security effectiveness. This variable was analyzed from two perspectives: how and how often the compliance monitoring was conducted.
The majority of respondents (38 percent) indicated that their organizations use electronic sign-off as the only means for training compliance monitoring. The next highest group of respondents (20 percent) stated that their organizations use a combination of methods for training compliance monitoring. Another 20 percent stated that their organizations use no training compliance monitoring methods.
Those respondents whose organizations use only electronic sign-off for training compliance monitoring are the largest group to strongly agree that their organizations effectively secure their data. Those respondents whose organizations use a combination of methods for training compliance monitoring are the second largest group to strongly agree that their organizations effectively secure their data. Clearly, participants from organizations that use other methods (such as verbal tests or monitoring tools) or no methods at all are the least likely to strongly agree that their organizations effectively secure their data.
The majority of respondents (34 percent) indicated that their organizations conduct training compliance monitoring once a year. The next highest group of respondents (22 percent) stated that their organizations conduct training compliance monitoring more than once a year. Another 22 percent stated that their organizations never conduct training compliance monitoring. It is interesting that 16 percent of the respondents stated that they did not know how often training compliance monitoring was conducted. This raises the question: Do the organizations conduct the monitoring in such a way that employees do not know about it, or is it that the respondents were confused by the question on the survey?
Those respondents whose organizations conduct training compliance monitoring once a year are the largest group to strongly agree that their organizations effectively secure their data. Those respondents whose organizations conduct training compliance monitoring more than once a year are the second largest group to strongly agree that their organizations effectively secure their data. Clearly, participants from organizations that conduct training compliance monitoring only when new access is granted to a system, or that do not conduct training compliance monitoring at all, are the least likely to strongly agree that their organizations effectively secure their data.
Training Topics
The next area of findings concerns the material that organizations are teaching their employees regarding security awareness. As seen in figure 1, the most popular security awareness training topic pertains to e-mail. Passwords and Internet usage are close behind. However, topics such as social engineering and data encryption appear closer to the bottom of the list.
Security awareness training topics were cross-tabulated with perceived security effectiveness. Those respondents whose organizations covered all of the topics included in the survey are the largest group to strongly agree that their organizations effectively secure their data. The participants whose organizations included some combination of the topics listed in the survey are the second most likely to strongly agree that their organizations effectively secure their data. The respondents whose organizations covered only one topic in their training were the least likely to strongly agree that their organizations effectively secure their data.
Relationships
Finally, this research showed that there is a strong correlation between perceived security effectiveness and the components of training method and training compliance monitoring. However, the relationship between training frequency and perceived security effectiveness was inconclusive. Therefore, it can be concluded that training method and training compliance monitoring are the strongest predictors for security effectiveness.
Implications
Results of this research can provide practical guidance to information security practitioners and those setting the policies within organizations. Four main implications can be surmised based on this study.
Provide Training at Least Once a Year
Findings showed that those respondents whose organizations provided training once a year had the highest rate of strongly agreeing that their organizations secure their data effectively. Participants from organizations that provided training more than once a year did not greatly improve their numbers.
Employ Multiple Training Methods
In addition, based on the findings of this research, training method appears to have the biggest impact on perceived security effectiveness; furthermore, the use of multiple methods of training produced the highest correlations to perceived security effectiveness. As such, organizations should consider focusing resources on training methodology. They should strive to combine various tools, including:
- Computer-based training
- Policies and procedures
- Newsletters
- Leader-led training
- Video
- Posters
- Brochures
Ensure Compliance
This research also showed that training compliance monitoring had a strong relationship with perceived security effectiveness. It is not enough for organizations to merely implement a security awareness training program. Policy makers within organizations should strive to better monitor the training for compliance; equally important is that they ensure consequences are in place for noncompliance. As one survey respondent stated, “Training sessions are performed annually, and attendance and understanding are monitored. However, no action is taken that I know if someone does not participate in the training. It is just reported to management.”
Respondents from a recent Enterprise Strategy Group survey stated that training users on confidential data security policies was the most important measure for protecting proprietary information.7 Yet, only 36 percent of government workers are held accountable for knowing information security policies and procedures via their annual performance evaluation. In addition, only 48 percent were tested throughout the year on what they learned in awareness training.8 Therefore, while it is important for leadership to monitor employees for adherence to policies, it should not be performed only once a year. Such monitoring should be an ongoing occurrence, especially considering that some public-sector studies have shown that more than 80 percent of breaches occur not because of malicious intent, but because employees claim not to know about a policy or because they simply ignored it.9
Teach Relevant Topics
This research revealed that topics such as social engineering and data encryption appeared at the bottom half of the list of the most popular training awareness topics within the organizations of the respondents. However, current data breach information is proving these to be critical areas. For example, social engineering attacks continue to grow. In fact, phishing, a social engineering technique, is included in the 2009 Verizon Business Supplemental Data Breach Report’s top 15 most common security attacks.10 Furthermore, encryption is also playing a more critical role when one considers that lost/stolen laptops, smartphones and removable storage devices are contributing factors to data breaches. In fact, according to a 2009 Ponemon Institute study, 36 percent of all data breach cases examined involved lost or stolen laptop computers or other mobile devices. These types of data breaches tend to be more expensive than other incidents, costing approximately US $225 per victim.11 As such, practitioners should develop security awareness training that places more emphasis on these relevant topics.
Conclusion
Security awareness training alone will not secure an organization, just as technical solutions alone will not secure an organization. Although individuals are taught the secure way to interact with data, they may not always follow that training. Clearly, information security must involve a layered approach that includes both technical and nontechnical solutions. Security awareness training is a vital nontechnical component to information security. As such, it is in the interest of the public and private sectors to continue to research this component that directly impacts security’s weakest link: humans.
Endnotes
1 Westervelt, Robert; “RSA SecurID Breach Began With Spear Phishing Attack,” SearchSecurity.com, 4 April 2011, http://searchsecurity.techtarget.com/news/article/0,289142,sid14_gci1529523,00.html?track=NL-102&ad=824622&asrc=EM_NLN_13603105&uid=10604598
2 Knapp, Kenneth Joseph; “A Model of Managerial Effectiveness in Information Security: From Grounded Theory to Empirical Test,” dissertation, Auburn University, USA, 2005
3 Lease, D.R.; “Factors Influencing the Adoption of Biometric Security Technologies by Decision Making Information Technology and Security Managers,” dissertation, Capella University, USA, 2005
4 Op cit, Knapp
5 Caralli, Richard A.; Managing for Enterprise Security, Carnegie Mellon University, USA, 2004, www.sei.cmu.edu/reports/04tn046.pdf
6 Op cit, Knapp
7 Berrong, Stephanie; “Creative Approaches to Security Awareness Training,” Security Management, July 2009
8 SecureInfo Corp., Information Security Awareness Report: The Government Workers’ Perspective, USA, 2007, www.secureinfo.com/downloads/reports/SecureInfo-InfoSec-Report-Dec-2007.pdf
9 Government Security, “Study Shows Fed Workers in Dark About Security,” 31 May 2007, http://preview.govtsecurity.com/news/fed-workers-in-dark/
10 Verizon Business, “Verizon Business Issues 2009 Supplemental Data Breach Report Profiling 15 Most Common Attacks,” PR Newswire, 9 December 2009, www.prnewswire.com/news-releases/verizon-businessissues-2009-supplemental-data-breach-report-profiling-15-most-common-attacks-78840502.html
11 Ponemon Institute, 2009 Annual Study: Cost of a Data Breach, PGP Corp., USA, January 2010
Karen Quagliata, Ph.D., PMP
has worked in the IT field for more than 10 years in diverse capacities. Quagliata currently works within the financial services industry as an information security analyst, specializing in risk management. In addition, she is a published author in various industry publications.