A new hybrid risk assessment process for cyber security design of smart grids using fuzzy analytic hierarchy processes

List of abbreviations

AHP

Analytic Hierarchy Process.

CVE

Common Vulnerabilities and Exposures.

CVSS

Common Vulnerability Scoring System.

DoS

Denial-of-Service.

FAHP

Fuzzy Analytic Hierarchy Process.

HRA

Hybrid Risk Assessment.

IED

Intelligent Electronic Device.

IT

Information Technology.

MCDM

Multiple Criteria Decision-Making.

MF

Membership Function.

OT

Operational Technology.

PLC

Programmable Logic Controller.

RA

Risk Assessment.

RE

Risk Event.

RL

Risk Level.

SG

Smart Grid.

SIEM

Security Information and Event Management.

TCP

Transmission Control Protocol.

WF

Weight Factor.

1 Introduction

Smart Grids (SGs) offer a promising technology to improve the reliability, resilience, quality of service and smart management of the future energy systems. They support generation, transmission, distribution and control of electricity in real-time. For instance, they play a vital role in managing the power balance across different voltage levels and providing energy effectively to the society. New concerns regarding security threats have emerged with the implementation of SGs. It is of utmost importance to secure SGs by carefully considering potential threats, vulnerabilities and risks. Enhancing cybersecurity measures to protect against Risk Events (REs) is crucial, especially given the reliance on electricity for the day-to-day operations of various facilities, including homes, businesses, hospitals, schools and more. For example, the interconnected nature of SGs through networks with a large number of access and control points can make them vulnerable to threats leading to cyber-attacks. Hence, a Risk Assessment (RA) process for supporting the overall grid stability and security should be conducted. The RA process contains five steps; context establishment, risk identification, risk analysis, risk evaluation and risk treatment [1]. We include the first four steps of the RA process in this paper, as shown in Figure 1, where the dashed rectangle presents the RA steps and the circle indicates the expected outcomes. The risk treatment step is out of the scope of the present paper.

Figure 1:

An overview of the risk assessment (RA) process in the presented paper.

Carrying out an effective RA process is essential to provide a secure decision-making process in energy systems considering critical infrastructure and operations of Operational Technology (OT) aspects. In addition to data protection and cyber security risks in Information Technology (IT), OT related RAs have a broader scope, covering physical safety risks, operational disruptions and the potential impact on the related infrastructure. In the context of OT, there is a requirement for RA extension consequently. Our main contribution in the presented paper is to show the adaptation of fuzzy logic and Fuzzy Analytic Hierarchy Process (FAHP) in view of cyber security design for SGs. Moreover, the aim of our research is to understand the sources of risk, increase risk awareness, investigate which part of the network is under risk and at which level and assist engineers and/or operators in determining which risk should be considered first. Our Hybrid Risk Assessment (HRA) process enables an accurate representation of the levels and risk scores with respect to the REs.

The remainder of the paper is organized as follows. First, we begin with a brief statement of RA criticality in Section 2 presenting common approaches. Section 3 entails the main contribution of the present paper considering the HRA for SGs. This includes a representation of the HRA process with the Membership Functions (MFs) and FAHP. Moreover, in Section 4, we present a use case for a small-scale power grid to provide an application of the HRA process. Thereafter, a discussion regarding open research questions and also possible future work are addressed in Section 5. Finally, the conclusion is presented in Section 6.

2 Related work

Context establishment is the first step, which covers the scope, target and focus of the RA [1, 2]. It provides a basis for the definition of critical assets. The following step, risk identification, clarifies threats and vulnerabilities that may initiate unwanted events with respect to the predefined assets. This step entails determining potential events that could lead to harm or damage to assets [1]. Here, a threat is described as an initiator of events that may affect the system under assessment, while vulnerability is defined as a weakness or deficiency that triggers a threat scenario [1, 3]. Given the complex, interconnected nature of SGs and the rapid technological developments in this field, risk identification based on expert knowledge is critical. The third step, risk analysis, is an essential step in the RA process as it supports determining the Risk Level (RL). The final step of the present study, risk evaluation, is necessary to associate the results of the risk analysis with the risk evaluation criteria to consider if any action or treatment is required for the cyber-risks [1].

Risk analysis is one of the features of Security Information and Event Management (SIEM) systems [4] that play a crucial role in SGs, industrial automation and control systems by providing security threat monitoring and management particularly. These systems gather and examine data from different sources, such as network devices, servers and endpoints to detect and react to potential security incidents such as equipment damage, production downtime, or environmental harm. In a recent study [4], the risk analysis feature of the basic configuration of current SIEM solutions are evaluated as either low/basic or average in terms of implementation, but not as high/advanced. High/advanced, average and low/basic mean respectively the features are fully functioning, partially implemented and either poorly implemented or not implemented at all [4]. Given this situation, there is a need for improving risk analysis feature of current SIEM solutions [4, 5].

RA steps can be applied as qualitatively, quantitatively or hybrid to decide the likelihood, consequence and/or prioritization. While qualitative scale uses natural language expressions, quantitative scale is presented in numerical form, absolute scales and ratios. It can be also possible to use qualitative and quantitative RAs together [1]. In a nutshell, HRA supports investigation of which part of the grid is under risk and at which level.

To conduct an RA process, various approaches such as attack tree-based [6, 7], Bayesian attack graph-based [8], CORAS diagrams based [3], game theory [9] based and/or deep neural network-based [10] techniques are proposed in the literature. In an RA process, a risk matrix [1] is typically used to assign an RL based on the combination of the likelihood and impact of potential risks. Fuzzy logic [11] is a promising approach to conduct RA process. It is a mathematical modelling technique that enables the handling of uncertainties and imprecision in data. By allowing for the use of linguistic variables and fuzzy sets, fuzzy logic can capture the imprecision and uncertainty inherent in many RA. These linguistic values such as ‘Low’, ‘Medium’ or ‘High’ are allowed to be communicated more effectively with operators/experts. This can be useful for incorporating expert knowledge into the RA model. Fuzzy logic is a promising approach in SCADA RA process, but their current usage is limited [12]. The idea behind the Analytic Hierarchy Process (AHP) [13] is to deal with Multiple Criteria Decision-Making (MCDM) issues to reduce biases. An extension of fuzzy logic with AHP known as FAHP [14] can overcome uncertainty and subjectivity challenges. Due to the imprecise and uncertain nature of human decision-making, the FAHP is often utilized to tackle problems involving MCDM [15]. It has found effective adaptation in several domains for the RA process [16, 17]. Undoubtedly, when considering all relevant studies in the field, valuable information and useful methodologies have been provided. Nevertheless, there is still need for further development to support objectivity in the selection of parameters, considering more REs, accounting for variability of inputs, extending simplified risk analysis equations, presenting RL and prioritization. SG is still an open domain to demonstrate the applicability of RA process.

3 Proposed hybrid risk assessment process

In the proposed HRA, the consideration of past occurrences of similar events is incorporated as a factor referred to as likelihood. The evaluation of the impact of RE is conducted, considering their implications for power subscribers and the wider society. This factor is referred to as consequence. The scales that are utilized to assess likelihood, consequences and RL in the present study are based on the study [18]. Our solution is more suitable to power grids in terms of inputs (likelihood and consequence) and the evaluation of the MFs. This is due to the fact that RA process may vary depending on the unique characteristics and requirements of each specific domain. Novelty lies in showing the application of fuzzy logic and FAHP in the domain of SGs. In the following part, we explain the flow of fuzzy logic and FAHP applications as a basis for the HRA process. We illustrate our HRA process in Figure 2.

Figure 2:

Flowchart of the hybrid risk assessment (HRA) process.

Compared to [16], our approach is based on fuzzy logic and FAHP is adapted for SGs in terms of specific fuzzy logic’s variables and MFs. In the grid related RA process, one of the challenges is to evaluate the precise values of risks. Due to the specific interconnected and complex nature of SGs, data used to calculate risk scores and levels can be uncertain, incomplete and subject to change over time. Fuzzy logic and FAHP are methods that can be used to overcome this difficulty. Fuzzy logic allows for the modeling of imprecise or uncertain information by using fuzzy sets to represent concepts or variables that have a degree of membership in a set, rather than being a fixed true or false value. Additionally, FAHP is a method that allows for the prioritization and the rank of risk factors based on their degree of importance. It involves the use of pairwise comparison matrices and a weighting system to determine the relative importance of different risk factors and the overall risk score. HRA starts with the context establishment step. It can be defined as substations in the transmission and the distribution domains. After evaluating the first outcomes within the present paper, the context can be extended in the future. For the risk identification, experts’ opinion [18], [19], [20], MITRE ATT&CK [21] and NIST National Vulnerabilty database [22] be taken into account. These provide supportive definitions for our domain specific risks. After identifying risks, fuzzy logic provides the calculated risk scores and levels according to likelihood and consequences. Lastly based on these risk scores from fuzzy logic’s outcome and pairwise comparison from FAHP, the overall risk score and RL at the grid are obtained for usage of decision-makers. While RL is the linguistic representation, risk score is the numerical representation of the risk involved. Fuzzy membership values are placed in the interval [0,1]. The MFs can be directly related to the linguistic terms used to describe the RL, such as ‘High’, ‘Medium High’, ‘Medium Low’ and ‘Low’. While Gaussian MFs are used likelihood and consequence representations, the triangular and trapezoidal MFs are used for the RL representation to conduct fuzzy logic. To apply FAHP, a triangular fuzzy MF, A ̃ = (a,b,c), is used in the presented research. The identified risks will be the inputs to build the qualitative rules that are associated with the MFs. The mathematical formulation of a triangular MF is given by Equation (1):

In our study, each risk is evaluated according to two factors, namely likelihood and consequence [1]. The columns of qualitative expressions and description in Tables 1 –3 are based on a previous study [18]. These descriptions are extended with fuzzy logic MFs. A similar logic for generating the MFs’ parameters is defined in the study [23].

Since Gaussian MFs can be useful for smooth and continuously differentiable fuzzy model [24], they can be applicable for likelihood and also for consequence variables. Here Gaussian MFs are used for likelihood and consequences in Tables 1 and 2. These Gaussian MFs were created with the help of tune fuzzy inference system using fuzzy logic designer from MATLAB. Triangular and trapezoidal MFs are applied for RL in Table 3. The MFs’ parameters presented via the help of Fuzzy Logic tool box of MATLAB.

As depicted in Figure 2, the step following the decision about the fuzzy MFs, is to clarify the inference process to set conclusions from a set of fuzzy rules. The number of fuzzy rules in the fuzzy rule base depends on the number of qualitative expressions adopted for representing likelihood and consequence. For example, in the current research, there are 4 qualitative expressions for likelihood and 4 qualitative expressions for consequences which make up 16 rules in total. These rules are listed in Table 4 considering likelihood and consequence. In the present fuzzy rules, a total number of 16 if–then rules were created based on the size of the risk matrix of 4 × 4.

After the step of inference, the defuzzification process is completed which leads to a risk score of a given index. This process is fundamental for converting the output of a fuzzy system from a fuzzy set into a crisp value that can be used to support the decision-making process. The centroid method is applied for defuzzification in the case study. The evaluated risk scores of REs feed the FAHP process. It starts with pairwise comparison to determine the relative importance of the risk factors. Then the process continues with usage of triangular MFs to represent the degree of membership of the risk factors in different sets. Table 5 presents the importance with the help of the triangular fuzzy numbers to create the pairwise comparison matrix in the FAHP.

As depicted in Figure 2, to construct fuzzy pairwise comparison matrix that will lead to FAHP, the arithmetic operations for two triangular fuzzy numbers a ̃ p t p l , t p m , t p u and a ̃ q t q l , t q m , t q u are taken into account. The operations for fuzzy logic addition (⊕), multiplication (⊗), division (∅) and exponential (exp) can be found in [16]. Geometric mean can be used to calculate Weight Factor (WF) [16]. D F w ̃ i is defuzzified mean value of fuzzy WF [16].

The total RL of the SGs is calculated by Equation (6) [16]. In the last step, we aggregate the results of pairwise comparison according to the equations from 2 to 6 to present the risk outcome as an overall risk score and RL.

4 Use case: HRA process of small-scale power grid

Conducting HRA process for SGs can be helpful to support stability and security as introduced in [25]. The proposed HRA process in Figure 2 is applied and tested considering a case study with five REs in a small-scale SG. Due to the safety concerns associated with using large-scale and in-action SGs for experimentation, the case study incorporates smaller-scale version of an SG, referred to as test-beds or demonstrators. Our chosen subject of study is a small-scale SG. A prototypical configuration of such a grid includes both Programmable Logic Controllers (PLCs) and Intelligent Electronic Devices (IEDs). PLCs can be programmed to perform diverse tasks such as monitoring equipment conditions, regulating the flow of electricity and retrieving data from sensors. IEDs are specialized devices designed specifically for power systems. They are used to detect and isolate electrical faults, manage power system parameters and monitor power quality. Simulating these devices allows researchers and engineers to experiment with novel control techniques, communication protocols and protection measures in order to enhance the dependability and efficiency of power systems. In Figure 3, an overview of a small-scale SG from the “KASTEL Security Lab Energy” is illustrated. The different PLCs are responsible for the control of a simulated physical system including a wind turbine, photovoltaic generator and battery system. As depicted in Figure 3, the simulated models are based on MATLAB. By comparing the power generated and the power required, the PLC determines whether to charge or discharge the batteries. The IEDs receive and process data from sensors and other equipment to issue protection and control commands such as tripping circuit breakers to identify and isolate electrical faults.

Figure 3:

Small-scale smart grid (SG) architecture at the “KASTEL Security Lab Energy”.

Five REs are defined in Table 6. The Common Vulnerability Scoring System (CVSS) and Common Vulnerabilities and Exposures (CVE) in the NIST National Vulnerabilty database [22] were taken into account. Moreover, the presented likelihood and consequence are interpreted considering the literature studies [18], [19], [20]. The results of fuzzy logic are shown in Table 7. After the fuzzy logic is proceeded, FAHP is conducted. To start with FAHP, pairwise comparison is shared in Table 8. It serves as an analysis for evaluating the relative significance of various risks during the decision-making process by contrasting each risk with all the others in a matrix format.

These calculations from Tables 7 and 8 are performed with the help of the Fuzzy Logic tool box of MATLAB. According to these calculations, RE4 has the highest risk with the level of ‘80 % High’. Considering these outcomes and Table 3, RE2 and RE4 must be treated initially. The overall RL is calculated for the grid by the arithmetical operation 6 and it is 11.28 with ‘24 % Medium High’ and ‘76 % High’. The risk outcome is evaluated on the basis of fuzzy logic and arithmetical operations equations from 2 to 6. Considering the risk outcome, it can be understood that the treatment actions RE4, RE2, RE5, RE3 and RE1 will be carried out sequentially.

5 Discussion and future work

In this section, the result of the proposed HRA process is discussed with possible enhancements to be addressed in the future work. Regarding IT domain, there are various existing RA approaches to calculate the risk outcome. Most of them estimate the impact and likelihood with the help of risk matrix [1]. Risk scores of five REs are represented at Table 7 in Section 4 by using a risk matrix with multiplication of the likelihood and the consequence. According to the findings obtained from Table 7, it is observed that the risk scores calculated using the fuzzy logic and risk matrix are close to each other.

The RLs obtained from fuzzy logic can belong to more than one linguistic set such as the REs from 2 to 5. Since the linguistic scales such as ‘Medium High’ and ‘High’ are shared with the decision-makers, an outcome from fuzzy logic can provide more detailed information about the risk than a risk matrix. For instance, as indicated in Table 7, the RLs of RE2 and RE5 are obtained from the risk matrix as ‘Medium High’, while from fuzzy logic they are ‘97 % Medium High’, ‘3 % High’ for RE2 and ‘53 % Medium Low’, ‘47 % Medium High’ for RE5. According to the presented study, fuzzy logic provides more detailed information than the risk matrix to decide on the treatment sequence. These detailed information can be helpful to the decision-makers in implementing mitigation strategies that are tailored to the unique characteristics of each risks. This outcome can increase the likelihood of successfully reducing the impact of risks. Risk matrices may encounter challenges in adequately representing the multidimensional aspects of risks and their interdependencies in the dynamic and evolving nature of SG technologies. From this perspective, fuzzy logic is capable of effectively addressing the uncertainties, complexities and interdependencies inherent in RAs specific to SGs. Based on these insights, the case study supports that the suggested HRA process is a promising approach. Our proposed approach can be applicable in different RA processes for SGs use cases when expert knowledge is available. Expert knowledge can provide insights into potential uncertainties of the assessment process for example identify possible attacker intentions or threat consequences. A comparison in terms of quality is suggested for future work. This contains the inclusion of more REs that could help qualify and quantify limitations across the different approaches. As a future work, we may consider the potential of expanding the fuzzy logic by incorporating fuzzy trees to accommodate a broader range of input variables.

Our study presents the applicability of fuzzy logic and FAHP based RA for SG. Circumstances may change or new threats may emerge, requiring adjustments to the RA process. Automating the RA process with effective techniques and tools like machine learning algorithms and simulation models can be a beneficial way to support the expert knowledge in the HRA process in a further step. To have a better understanding for risk sources and naturally enhance a secure decision-making process, future work will focus on increasing the number of REs and conducting the HRA process in a larger scale and heterogeneous substation. Additionally, this will enhance our understanding of the risks arising from communication protocols utilized by various manufacturers. Future work will focus on integrating expert knowledge with data-driven insights gathered from cyber-attacks launched against the IEC 61850 substation at “KASTEL Security Lab Energy” to improve decision-making process. Further endeavors are required to focus on the collection of real-world data from IEC 61850 Substations during cyber-attacks, with the aim of validating and enhancing calculated risk values under realistic scenarios. Moreover, potential future applications are explored for integrating this approach into SIEM systems to enhance their risk analysis feature.

The RA findings play a crucial role in effectively communicating information to relevant stakeholders, which can be valuable in demonstrating aspects such as policy adherence or compliance with directives and regulations [1]. For instance, we presume that through gaining an understanding of the risk outcomes generated by the HRA process and taking into consideration the identified risks and their potential impact, organizations and policymakers can appropriately align their compliance measures to effectively address the most significant threats. Moreover, they can establish more targeted measures to address cyber-security challenges. Therefore, future efforts will be dedicated to expand IEC 61850 substation related REs. Overall, this will offer valuable support to decision-makers to design a list of action based prioritized risks and evaluating their acceptability.

6 Conclusions

The RA process is influenced by human decisions that can be ambiguous, blurred and hard to express with absolute numerical values. In the presented work, we propose an HRA process by using fuzzy logic and FAHP. Our main contribution is to provide a framework, based on fuzzy logic and FAHP to support comprehensive HRA process for the domain of SG. Using the presented approach, risk scores and levels are investigated. The outcomes will be beneficial for demonstrating a policy adherence with cyber-security recommendations in future energy systems since they provide a detailed understanding of an SG’s security posture. This information can be used to demonstrate that one has conducted a thorough RA and has implemented appropriate security controls to address the identified risks. The RA process can help the decision-makers implement appropriate measures to address these issues.