Many organizations, particularly those highly regulated by government entities, such as banking and insurance, employ a Three Lines of Defense model for risk management. In this model, originally conceived by The Institute of Internal Auditors (The IIA) for the financial services industry to better manage risk, the first line is operational management, the second line is risk monitoring and oversight, and the third line is audit.1 Although different organizations may delineate the roles for each line differently, the essential construct is widely applied through financial institutions and technology domains such as cybersecurity. Typically, the model is embedded within the organization’s documented enterprise risk management framework, which governs the risk management approach. The framework itself is required by regulators, such as the US Office of the Comptroller of the Currency (OCC) for financial institutions within the United States, and recommended by international standards, such as International Organization for Standardization (ISO)/International Electrotechnical Commission (IEC) 27001.2 In the case of information security within US financial institutions, the applicable standard is 12 CFR Appendix B to Part 30 Interagency Guidelines of Establishing Information Security Standards, which states that financial institutions must have a comprehensive written security program to ensure security and confidentiality of customer information.3 More explicitly, each institution must have procedures to manage and control risk. Correspondingly, the Sarbanes-Oxley Act of 2002 (SOX) Section 404 requires organizations to assess internal controls related to their financial systems.4
To achieve these and other regulatory requirements, organizations have introduced risk assessment and control testing processes to ensure that adequate controls exist in line with their established risk management framework. Most have adopted industry-standard control frameworks, such as the US National Institute of Standards and Technology (NIST) Risk Management Framework (RMF), 5 the Risk Management Association’s Enterprise Risk Management Framework,6 COBIT® 20197 or ISO 27001.8
A typical risk and control process (e.g., NIST RMF) provides guidance for several steps to categorize, identify, implement, assess and monitor controls. To accomplish the “assess” portion of the process, first line of defense elements within an organization evaluate or test the control to see if it is achieving the intended objective. The assess function is arguably the most critical element of the process. If the system is tested properly, it will be fundamentally perly, it will be fundamentally secure.9
Control Testing Overview
Technology control testing involves four basic steps: gather evidence, analyze the evidence, assess effectiveness and substantiate the results. The first three steps can range in time and complexity; well-designed and automated controls can quickly be tested, while those more complex and manual in nature may require more time and resources to thoroughly evaluate. The risk framework also dictates when controls will be tested—usually annually, but sometimes quarterly. Typically, organizations use a binary effective or ineffective rating in their assessment, though some may have a three or five scale option.
Control testing has three fundamental challenges:
- Time and investment cost—Meeting the equired standards for control assurance requires people and time, which are cost factors to any organization. In addition, thelarger the control portfolio, the more people required to conduct testing.
- Continuous monitoring—Control testing is done on a set frequency (e.g., annually or quarterly), which can be inefficient. Alternatively, a continuous monitoring capability can provide cost reductions through improved efficiency and effectiveness, reduce risk velocity, reduce remediation cost, and reduce impact on second- and third-line validation activities.10
- Scope coverage—Technology controls are typically implemented over a large scope of systems and processes; control testing frameworks dictate sampling methods to extrapolate overall effectiveness. However, sampling has limitations, including the potential for flawed population selection, nadequate coverage of the true population and inability to attain assurance about risk reduction holistically throughout the technology environment.
The Benefits of Automation
The automation of controls and the testing process can substantially reduce the impact of control testing challenges. Specifically, automation can reduce the manual overhead associated with evidence gathering and sample analysis, provide continuous monitoring capabilities to enable remediation prior to formal testing, and increase scope coverage to 100 percent of systems, eliminating sampling. The degree to which automation provides the maximum benefit depends on:
- Whether the control itself is automated or manual (i.e., control owners manually generate reports on a periodic basis for control testing evidence)
- The degree to which the control life cycle process is manual (i.e., the organization uses a technology system or application to manage the testing life cycle))
The greatest benefit is achieved when controls are automated and the associated systems, including the risk management/control system of record, are in place.
THE AUTOMATION OF CONTROLS AND THE TESTING PROCESS CAN SUBSTANTIALLY REDUCE THE IMPACT OF CONTROL TESTING CHALLENGES.
Case Study: Technology Change Management
A US Midwest regional banking institution has a technology footprint of approximately 300 systems and 250 active technology controls. The organization has a mature risk management framework with an established system of record for managing the risk and control portfolio. In addition, the first line of defense technology risk the organization uses is the ServiceNow governance, risk and compliance (GRC) module to feed RSA Archer. ServiceNow is an international vendor product that provides digital workflows to increase productivity, including technology, customer and employee workflows.11 RSA Archer is an international vendor product that provides integrated risk management capabilities.12 The GRC module permits technology-specific control workflow management and other technology specific capabilities (i.e., support for continuous monitoring). The key aspect of this infrastructure is that ServiceNow GRC is also used by the organization for technology service management such as incident management, change management, access provisioning and configuration management. This integration point is significant because ServiceNow GRC has capabilities internal to the integration that support automation without requiring external techniques such as remote process automation (RPA), a generic capability that automates manually intensive processes such as keystroke date entry.
The first line of defense risk team (the risk team aligned with the operational technology teams responsible for the controls) evaluated the COBIT® - based control inventory to determine which existing controls offered the maximum automation benefit (e.g., reduction in testing life cycle time, maximum coverage and continuous monitoring potential) with minimal cost (e.g., effort to redesign controls, development time and impact to the ecosystem). The team identified the change management controls that optimally met this objective. The change management control domain is referenced in COBIT 2019’s Build, Acquire, and Implement (BAI) processes of BAI105 Manage organizational change enablement, BAI106 Manage changes and BAI107 Manage change acceptance and transitioning.13 The fundamental objectives of change management controls include ensuring that a defined change process exists for technology systems, ensuring that the changes are appropriately managed and approved to avoid incidents and outages, and ensuring that appropriate governance is in place to effectively manage system changes.
The control inventory analysis revealed the organization’s 15 change management controls (a mix of manual and automated evidence, but primarily system reports) could be redesigned into two controls with four attributes. The 15 controls relied on sampling 25 of the approximately 300 applications, so the existing construct was limited in coverage. No continuous monitoring existed. All controls were tested on a semi-annual basis and typically took approximately 810 hours (.38 full-time employee [FTE]; this is found by dividing the hours savings by 1750, the annual equivalent hours of an FTE).
Working with control designers, risk officers, control owners and ServiceNow GRC internal developers, the team streamlined (redesigned) the existing 15 manual controls into one control with four attributes (approval types required), developing more than 2,000 indicators (ServiceNow automated trigger queries) to automate the control testing life cycle, increase coverage to 100 percent of nearly 300 applications and establish continuous monitoring to ensure full effectiveness for the forthcoming test cycle.
THE CONTINUOUS MONITORING ASPECT ALLOWED THE CHANGE MANAGEMENT TEAM TO CORRECT POTENTIAL TESTING EXCEPTIONS PRIOR TO THE ACTUAL TEST, THUS ENSURING 100 PERCENT EFFECTIVENESS WHEN THE TEST EVENT DOES OCCUR.
The team first redesigned the 15 manual controls into a single control with four attributes (all types of change management approvals). This effort streamlined the disparate controls into a manageable, cohesive element against which logical software queries could be most effectively applied. The 2,000 indicators were required to run against the larger population of 270 applications and the four designed attributes, thus allowing full coverage of the target population rather than using a sampling of 25 applications in the prior state. The corresponding time savings in labor resulted from the automation itself coupled with the need to only evaluate exceptions versus individual change records in the sampled population. The effort achieved continuous monitoring by establishing a system-generated advisory to the change management team when change records were out of compliance. The continuous monitoring aspect allowed the change management team to correct potential testing exceptions prior to the actual test, thus ensuring 100 percent effectiveness when the test event does occur. A summary of the effort is outlined in figure 1.
The net investment of the effort included approximately 80 hours of resource time (control design and development/user acceptance/implementation) with a duration of approximately 45 days (executed in parallel with other duties). The team expects to further reduce the .11 FTE current state requirement once SOX requirements are further analyzed, refined and accepted to comply with the higher rigor SOX necessitates.
Conclusion
In this case study, the organization realized an optimal solution using automation for change management, recognizing significant resource time savings, attaining 100 percent scope coverage, and implementing on-demand and weekly continuous monitoring. The continuous monitoring process enables the control owner to monitor and remediate exceptions prior to testing.
The control analysis inventory review identified a possible 60 percent of the remaining portfolio could similarly be automated, particularly those domains using ServiceNow (most notably access management and incident management). Although the investment level of effort remains a constant, the potential value depends heavily on the control area—the team does not expect to realize a 70 percent reduction in control testing times across that 60 percent population. For the 40 percent remaining population, the team will identify longerterm automation opportunities and explore external techniques such as RPA.
THE VALUE PROPOSITION OF AUTOMATION CAN BE REALIZED BY ANY ORGANIZATION WITH THE APPROPRIATE RISK MANAGEMENT FRAMEWORK AND ASSOCIATED SYSTEMS.
Automation will permit the first-line team to expand control testing capacity without adding significant resources, expand the coverage for system controls to reduce the risk profile and improve control effectiveness through continuous monitoring. The second and third lines of defense will realize a corresponding benefit in their oversight responsibilities. The value proposition of automation can be realized by any organization with the appropriate risk management framework and associated systems.
Author’s Note
The author wishes to acknowledge the contribution of Keith Mangine, Matt Foos, Ralph Baisden, Anusha Akarapu and Aaron Kramer in the execution of this use case and their ongoing partnership.
Endnotes
1 Mehta, ; “Three Lines of Defense for Cyber Security Professionals,” Governance,Risk, and Compliance, 3 October 2019, https://grcmusings.com/3-lines-of-defense-for-cyber-security-professionals/
2 International Organization for Standardization (ISO)/International Electrotechnical Commission (IEC), ISO/IEC 27001 Information Security Management, Switzerland, https://www.iso.org/isoiec-27001-information-security.html
3 Office of the Comptroller of the Currency, United States Code of Federal Regulations, 12 CFR Appendix B to Part 30 Interagency Guidelines of Establishing Information Security Standards, USA, 1995, https://www.ecfr.gov/current/title-12/chapter-I/part-30
4 US Congress, H.R.3763 107th Congress Sarbanes-Oxley Act of 2002, 30 July 2002, https://www.congress.gov/bill/107th-congress/house-bill/3763/text
5 National Institute of Standards and Technology (NIST), “Risk Management Framework (RMF) Overview,” FISMA Implementation Project: CSRC, 30 November 2016, https://csrc.nist.gov/projects/risk-management/about-rmf
6 The Risk Management Association (RMA), Enterprise Risk Management Framework, https://www.rmahq.org/erm-framework
7 ISACA®, COBIT® 2019, USA, 2018, https://www.isaca.org/resources/cobit
8 Op cit International Organization for Standardization (ISO)/International Electrotechnical Commission (IEC)
9 Dubsky, L.; “Assessing Security Controls:Keystone of the Risk Management Framework,” ISACA®Journal, vol. 6, 2016, https://www.isaca.org/archives
10 Vohradsky, D.; “A Practical Approach to Continuous Control Monitoring,” ISACA Journal, 2, 2015, https://www.isaca.org/archives
11 ServiceNow, “About ServiceNow,” https://www.servicenow.com/company.html
12 Archer Integrated Risk Management, https://www.archerirm.com/
13 Op cit ISACA
Michael Powers, Ph.D., CRISC
Is an IT risk director at a US Midwest regional banking institution and adjunct professor in quantitative statistics, project management and cybersecurity for three universities. He can be reached at mpowersphd@gmail.com.