Any unforeseen event that does not cause apparent damage while perhaps having the potential to do so is particularly interesting for providing an idea on how to build a risk indicator,1 which warns of the approach of an unwanted situation and should be analyzed as such. In the field of information security, having timely alerts on the quality of the information protection system is of vital importance. Performance and risk indicators are essential sources for measuring the effectiveness of the protection measures adopted. The difficulty is finding the appropriate information to develop indicators fit for purpose.
Attention is mainly focused on the performance indicator because it provides an immediate and continuous measurement of the evolution of an action. Risk indicators, on the other hand, are linked to signaling the occurrence of a specific event and focused on the prevention or containment of potential consequences. In other words, performance indicators are a showcase of the results achieved, while risk indicators are a magnifying glass that can be used to observe what is useful for the organization to achieve its objectives.
Differentiating Between Indicators
Performance indicators are often easier to identify than risk indicators. In the project requirements, the parameters for the control have already been established (i.e., the measurements of the extent to which the functioning of the activity corresponds to what is expected). For example, consider a car; its purpose is to transport people from one place to another more rapidly and efficiently than previous forms of transportation. In this case, speed is an important parameter, and the speedometer is a typical performance indicator. The speedometer instantly indicates the speed, and it does so for the entire period of vehicle activity. Ideally, there should also be an indicator of the maximum speed limit and an audible signal should be emitted in the event of reaching the preestablished speed threshold. This is a risk indicator. It signals when the speed reaches the threshold beyond which it is no longer acceptable to accelerate because there is a potential for unintended consequences (e.g., fines, damage to people or things).
Due to the ability of the risk indicator to predict the elevating risk of a given situation, it is also called an early warning (i.e., it highlights the entry into a potentially dangerous condition). However, there is an objective difficulty in identifying risk indicators because they are not part of the original idea of what is to be achieved, and often, the elements to determine them are discovered only in the operational phase.
Performance is a required result, and the way to evaluate it is always well defined in the project requirements, while the risk event is a consequence of unexpected vulnerabilities (despite the risk analysis, the exact moment remains undetermined) that could emerge in the development, implementation or operational phases or because over time new threats can arise and change the risk scenario. The ability to signal the approaching risk cannot be based only on exceeding the operating limits, but it must be consistent with the risk scenario and its assessment.
Determining Risk Indicators
An empirical way to determine a risk indicator is to start from the risk scenario, considering all the vulnerabilities that have emerged in operations. Then, instead of continuing with the risk analysis, a cause-and-effect analysis should be implemented to identify the factors underlying the identified weaknesses. This step highlights the dynamics of the relationships between the operational actions, the elements of weakness, and the identified threats to establish measures to estimate the extent of the phenomenon and use them to determine an alarm activation threshold in the event of a risk level variation. Within the set of identified causes, the organizations must choose the one that will produce the most serious impact, and this will be the basis of the key risk indicator (KRI).
The path to establishing the risk indicator is quite easy, but only if the analysis has suitable sources of data on vulnerabilities. In addition, it can be helpful to focus on a category of situations that are potentially harmful but without obvious consequences—near miss incidents. Not all of them are significant, but they are a source of potential vulnerabilities that cannot be overlooked in the risk scenario.
Understanding Near Miss Incidents
A near miss incident is an unplanned event that has the potential to cause an impact but does not actually cause significant consequences. Although it may not produce obvious effects on the protection system, it is, in any case, an alert to the presence of a real vulnerability, and, as such, it must be addressed through a preliminary analysis that determines its severity. Due to the similarity of this concept with that of the risk indicator, near miss incidents can be considered ideal candidates for creating risk indicators.
The first consideration is the type of risk indicator that can be built. It is lagging because it is based on events that have already occurred. Furthermore, it has the possibility of satisfying the most significant characteristics that make the indicator effective:
- Impact—The analysis carried out within the risk scenario ensures that only reports relevant to the organization are considered.
- Effort—The indicator is a warning already present in the control environment and only needs to be evaluated in terms of severity.
- Reliability—The indicator is a real occurrence of an unexpected event where the vulnerability that facilitated it must be assessed.
- Sensitivity—The evidence gathered guarantees the accuracy of the risk analysis because it is derived from concrete facts.
- Repeatability—Phenomena are constantly monitored, and the analysis can take advantage of past events stored in the risk register.
Due to the similarity of this concept with that of the risk indicator, near miss incidents can be considered ideal candidates for creating risk indicators.
Further consideration has a beneficial impact on the organization’s risk culture. Having a mature risk culture means also analyzing events that have not caused damage. Formalizing the systematic analysis of near miss incidents is a way to give attention to apparently insignificant events and is an indication of maturity in risk management. In an immature risk culture, these incidents would be considered lucky events that did not require specific actions. Instead, completing this analysis would create time to understand the event and take action.
A near miss incident is a risk indicator that could activate an appropriate event management procedure if it should occur in a foreseen situation or could lead to the intervention of a risk practitioner to determine the cause and assess the severity of the risk. There are two types of near miss incidents that can be used to clarify how to generate risk indicators: the results of an internal audit and the detection of computer viruses.
Near Miss Incidents and Internal Audit
The concept of a near miss incident has various meanings, including the unexpected violation of policies, guidelines, standards, regulations or procedures or even deficiencies in the definition of rules. The internal audit process ensures alignment between the correct functioning of the organization’s processes described in its organizational documents and the business objectives. Near miss incidents can function in similar ways to internal audit findings in that they alert to potential violations or vulnerabilities that need to be addressed. When the auditor detects deficiencies in the organization’s control environment, the findings should not only be filed in the audit opinion or remediation plan, but also should be communicated to the risk manager to analyze the implications for the objectives. The results of the audit recall the way risk indicators act (i.e., they warn of anomalous situations). Investigation is always needed.
Internal audit, by definition, must demonstrate the effectiveness of the internal control system. Detected weaknesses such as deficiencies in the assignment of roles or responsibilities, deadlines not met in updating documents or operational processes, or controls not performed provide an indication of organizational risk. Prior to the audit, these situations may be deemed fully functional, but any noncompliance is a potential cause of an incident (i.e., a near miss incident). Therefore, the finding is a risk indicator and its causes must be analyzed, the severity of its consequences with respect to the business objectives must be determined, and effective treatment must be undertaken.
Near Miss Incidents and Viruses
In addition to organizational weaknesses, risk indicators can be identified in the technological field. For example, antivirus reports allow the creation of interesting indicators despite the fact that antivirus systems have lost much of their effectiveness over time. The antivirus system needs to be reevaluated. It is perceived as a normal basic function to protect against attempts to compromise personal computers, while the protection of critical systems—rightly considered of greater value—is entrusted to sophisticated anti-intrusion tools capable of interpolating information from various network and system sources. The attention paid to the reports of these tools is very high. On the other hand, the notification of a virus detected on a personal computer almost never receives the classification of an incident if it does not affect critical systems, despite the fact that it has overcome some sort of defensive barrier and, therefore, has the potential to do damage. Generally, only the automatic remediation of the infected computer is foreseen, and it is recorded as a statistically irrelevant event certifying the success of the antivirus system.
Near miss incidents can function in similar ways to internal audit findings in that they alert to potential violations or vulnerabilities that need to be addressed.
Instead, from a risk point of view, warning of the presence of a virus has a different meaning. It is an opportunity to analyze the intrusion ability demonstrated by the virus and, consequently, the effectiveness of the in-depth defense system. This is why it is important to analyze two aspects of the infection. The first point is the intrusion technique used by the virus to evade defenses. It is helpful to understand what went wrong and which layers of protection were found to be vulnerable. In addition, the technique used by the virus could bring out a new threat. The second point is to understand the value of the potentially compromised resources at the depth of the defense system where the infection took place, and possibly reevaluate the level of risk.
If a virus has no significant consequences, it is often classified as having no impact, or being a near miss incident. Its relevance lies in the opportunity to investigate causes and to collect information to understand where there is a flaw in the protection system, and to prevent major incidents. It should be noted that not all viruses can be used as lessons learned. Only those related to particular attack methods that bring out vulnerabilities are helpful, for example, the intruding, moving or masking properties of the virus. For the analysis, it is necessary to separate the infection techniques from the effect of the infection itself. The first are risk indicators, as they identify the activity carried out to cause the incident, while the second are performance indicators (of the virus), as they show the extent of the consequences. All reports of viruses that, due to their particular typology or specific characteristics, bring out some vulnerability in the organization’s protection system should be included as risk indicators.
Conclusion
A near miss incident is an unplanned event that can potentially develop unintended consequences but does not actually develop them. From a risk perspective, it is an indicator of an anomalous situation and, as such, must be investigated to understand the potential impact on an organization’s objectives. It provides an opportunity to analyze causes, identify solutions, and strengthen the protection system.
A near miss incident signals the presence of some type of vulnerability that must be addressed within an adequate time to resolve it. It is certainly not a zero-day vulnerability that would require an immediate remedy to mitigate the risk of unintended consequences. Being a potential incident means there is time to act, and the risk analysis tells how much time. Regardless, that amount of time is short because there is evidence of a real defect. The vulnerabilities associated with the near miss incident risk indicator could be called one-day vulnerabilities. The risk indicator signals the need to place the light of the analysis before the possible darkness of the risk.
The vulnerabilities associated with the near miss incident risk indicator could be called one-day vulnerabilities.
Endnotes
1 ISACA®, Glossary, https://www.isaca.org/ resources/glossary
LUIGI SBRIZ | CISM, CRISC, CDPSE, ISO/IEC 27001 L A, ITIL V4, NIST CSF, UNI 11697:2017 DPO
Is a lead auditor and a senior consultant on risk management, cybersecurity and privacy issues. He has been the risk monitoring manager at a multinational automotive company for more than seven years. Previously, he was responsible for information and communication operations and resources in the Asia and Pacific Countries (APAC) region (China, Japan and Malaysia) and was the worldwide information security officer for more than seven years. He developed an original methodology for internal risk monitoring, merging an operational risk analysis with a consequent risk assessment driven by the maturity level of the controls. He also designed a cybermonitoring tool and an integrated system involving risk monitoring, maturity model and internal audit. Sbriz was a consultant for business intelligence systems for several years. He can be contacted on LinkedIn at https://it.linkedin.com/in/luigisbriz or at http://sbriz.tel.