People often want to know how qualitative and quantitative risk assessments differ from each other. This is a good question in today’s world of digitalization where there are many different RBI (Risk-Based Inspection) software packages available.
Let’s rewind a little bit and first have a look at the risk assessment standards that are out there.
What standards are there?
There are several International engineering standards and recommended practices that outline requirements, methodologies and the implementation of RBI. Examples are ASME PCC-3, RIMAP, DNV-RP G101, API 580, API 581, API 571, etc.
The different standards are often applicable to specific sections of the industry. For example: ASME is an American standard specifically developed for fixed pressure containing equipment; API is also an American standard that has been specifically developed for the oil and gas sector; and RIMPA is a European standard that’s more applicable to power plants.
People sometimes confuse the different types of recommended practices. For examples, API 580 outlines requirements (e.g. conceptual approaches and necessary elements to be included in an RBI assessment), whereas API 581 outlines a methodology aligned with API 580. RBI software packages can thus be aligned with a best practice that outlines requirements (e.g. API 580), without implementing an associated methodology (e.g. API 581).
What do the standards recommend?
The standards typically do not recommend just one (e.g. only quantitative) approach. For example, API 580 gives guidance for RBI implementations, using either Level 1 (qualitative) or Level 2 (semi-quantitative) or Level 3 (quantitative) methods. (API 581 falls under the Level 3 RBI methods.)
However, what is typically recommended is that the RBI methodology and RBI team study method must be defendable, user-friendly, detailed, documented, transparent and auditable. For example, to be aligned with API 580, software should implement a user-friendly RBI methodology that the responsible plant inspection engineers and operations engineers fully understand. Otherwise it can lead to an increase in equipment risk rather than a risk reduction.
The standards also emphasise that the RBI technology method (whether it is Level 1, Level 2 or Level 3) must be robust. The selected methodology must reliably assess the Probability of Failure and the Risk Profiles of each of the DMs / FMs (Degradation Methods / Failure Modes) applicable, otherwise there cannot be confidence in the optimum inspection interval. Furthermore, the team study method must ensure identification of all FMs, operating limits, maintenance activities, and other risk mitigation actions.
What is qualitative, semi-quantitative and quantitative risk assessments?
So, now we know that qualitative, semi-quantitative and quantitative risk assessment software can all be acceptable according to the standards, but what’s the difference then?
Let’s start by looking at the definitions. Quantitative data is designed to collect cold, hard facts. Numbers. Quantitative data is structured and statistical. Qualitative data collects information that seeks to describe a topic more than measure it. Think of impressions, opinions, and views. Semi-quantitative data has a bit of both. Some parts of the data are qualitative and other parts are quantitative
Thus, quantitative risk assessment methodologies give quantitative estimates of risks, given the parameters defining them. In contrast, in a qualitative assessment, probability and consequence are not numerically estimated, but are evaluated verbally using qualifiers like high likelihood, low likelihood, etc.
If we now assume that reliable data is readily available, a full quantitative risk estimate should give the most precise and accurate results. Here, we must however note the following. The type of data to perform a good quantitative assessment is hard and time demanding to get, which often leads to data of less quality, which means less accurate results.
Accuracy is a function of analysis methodology, data quality and consistency of execution. Precision is a function of the selected metrics and computational methods. So, we need to be careful when looking at risk assessments, since the result could be very precise, but if there is still a lot of uncertainty inherent within the probabilities and consequences, then the result is still not accurate.
What are the benefits and limitations of qualitative and quantitative RBI software?
So, which is better then? There is no easy answer, since qualitative, semi-quantitative and quantitative risk assessments can all be successful. Let’s first try to compare typical qualitative and quantitative risk assessments:
- User-friendliness: This is probably the greatest benefit of qualitative These are typically easier to make user-friendly, since they are less complex.
- Transparency: For the same reason a qualitative methodology is typically much easier to fully understand. Due to the complexity of the quantitative calculations, quantitative methods tend to be implemented as black boxes.
- Precision and Accuracy: If good data can be obtained quantitative methodologies should win this one, since they involve rigorous quantitative assessments of the PoF (Probability of Failure) and the CoF (Consequence of Failure) associated with each equipment item. We must just remember that the accuracy will depend on the inherent uncertainty in the probabilities and consequences.
- Data Dependency: A qualitative analysis requires less data.
- Speed: Since a qualitative RBI analysis require less data, it is typically much faster. For a quantitative RBI analysis can be hard and time demanding to gather all the data.
- Objectivity: Results of a qualitative RBI analysis are heavily dependent on the team and their expertise in performing the analysis and thus more subjective. A quantitative analysis is more objective. However, one should not be fooled that a quantitative method is fool proof, for good results experienced RBI and inspection personnel are still needed.
- Automation: Since quantitative methods require less team input, they are easier to automate.
Best of both?
It is obvious that neither the qualitative nor the quantitative methodology is perfect. To increase the benefits and reduce the limits, one needs to combine these two methods.
Hence, semi-quantitative risk assessment! This methodology tends to be easy to understand and user-friendly, but also more accurate. Off course it must be supported by an experienced multi-discipline team study to ensure confidence in the results. One would then do a qualitative analysis in some sections of the risk assessment and in other sections (selected based on a confidence / sensitivity analysis) one would do a quantitative risk assessment.
Another approach to get the best of both, would be to first do a (faster) high-level qualitative RBI analysis to select the high-risk facilities in one’s plant. Then one can do quantitative RBI analyses only on these high-risk facilities. Remember, for accurate results, good data and an experienced multi-discipline team study remains a requirement.
What RBI methodology does IMS PEI implement?
So, you may ask, what does IMS PEI implement? The IMS PEI software implements S-RBI, a Shell-developed Risk-Based Approach, compliant to API 580, API 581 and to API 571’s damage mechanisms. The software implements qualitative, semi-quantitative, and quantitative methodologies. For the semi-quantitative and quantitative methodologies, it uses specific calculators (e.g. liquid release, CUI, SSC and other corrosion prediction models) and detailed questionnaires, to calculate StF (Susceptibility to Failure) and CoF (Consequence of Failure), based on the most relevant failure modes.
Based on its configuration, IMS PEI allows users to swap between methodologies for each RBI Analysis. The default is a semi-quantitative methodology, since this is preferred by most customers. However, with some customization to the software, the quantitative approach can be setup to comply fully with API 581.
Also, IMS PEI is not a standalone RBI tool. It integrates the RBI results with inspection results, wall thickness measurements / calculations and, schedules. The RBI results can thus be used to define next inspection dates that feed into the IDMS part of the tool, which can, in turn, interface with the site’s CMMS (Computerized Maintenance Management Software) (e.g. SAP).
Powerful equipment, flammable chemicals and high-pressure processes can all easily lead to hazardous or even deadly incidents. The process industry, especially the oil, gas and chemical sector, is hazardous and it is crucial to ensure the safety of our assets and of our workers. Therefore, it is essential to identify and prevent potential hazards on site.
“Successful engineering is all about understanding how things break or fail.” ~ Henry Petroski, America’s failure expert
Many plants rely on Safety Instrumented Systems (SIS) to help address these potential failures to, in turn, prevent the hazards from occurring. (A hazard is a potential source of harm or adverse health effect on a person or persons.)
Safety Instrumented System (SIS)
To understands what a Safety Instrumented System (SIS) is and how it helps to prevent Hazards, we first need to understand the different Lines of Defence (LOD) / Layers of Protection (LOP) and where Safety Instrumented Systems (SIS) fit in. These Lines of Defence (LOD) / Layers of Protection (LOP) are independent layers that serve to either prevent an initiating event (e.g. loss of cooling) from developing into an incident (e.g. a release of a dangerous substance), or to mitigate the consequences of an incident once it occurs.
Figure 1: Lines of Defence (LOD) / Layers of Protection (LOP)
The first layer is the Basic Process Control System (BPCS). The Basic Process Control System (BPCS) controls pressure, level, temperature, flow, etc. However, the problem is, Process Control Systems (BPCS) can fail! Designers and engineers cannot foresee every possible hazard and design control systems to prevent all of them. If so, we would not need alarm systems, relief valves, flares systems, etc. But since it is not so, process facilities need multiple layers of protection…
When a BPCS fail, the next layer of protection, after operator intervention, is the Safety Instrumented System (SIS), independent from the BPCS. A Safety Instrumented System (SIS) does not control anything. It monitors many of the same variables as the BPCS, but only takes action when a variable is outside its normal range, which generally means the Process Control System (BPCS) has failed.
Each SIS performs one or more Safety Instrumented Functions (SIF).
Safety Instrumented Function (SIF)
Safety Instrumented Functions (SIF) comprise out of three elements: sensors (e.g. a flowmeter) and logic solvers (e.g. a safety PLC) that detect dangerous conditions, and final control elements (e.g. a valve) that are manipulated to achieve a safe state.
Safety Instrumented Functions (SIF) respond to specific, defined hazards, by implementing specific actions to put the equipment into (or maintain) a safe state to provide a defined degree of risk reduction. The risk reduction required from a Safety Instrumented Function (SIF) is characterized by the Safety Integrity Level (SIL). This is related to the probability that the Safety Instrumented Function (SIF) will NOT work when required.
Safety Instrumented Level (SIL)
Safety Integrity Level (SIL) indicates the degree of risk reduction, provided by an Instrumented Safety Function (SIF), implemented by a Safety Instrumented System (SIS), within a given process. In other words, SIL is a measure of the SIF’s performance, in terms of Probability of Failure on Demand (PFD). When designing a SIF, the appropriate SIL is crucial for achieving the required level of safety.
IEC 61508 defines four SIL levels, with SIL 4 providing the highest level of safety performance. For example, SIL 1 corresponds to a Risk Reduction Factor (RRF) of at least 10, and SIL 4 to a Risk Reduction Factor (RRF) of at least 10,000. The table below shows the associated Average Probability of Failure on Demand (PFDAvg) and Average Risk Reduction Factors (RRFAvg) for each SIL.
0.1 – 0.01
10 – 100
0.01 – 0.001
100 – 1000
0.001 – 0.0001
1000 – 10000
0.0001 – 0.00001
10000 – 100000
So, the higher the SIL level, the higher the associated safety level, and the lower probability that a system will fail to perform. Normally, a higher SIL level means a more complex system and higher installation and maintenance costs. Process plants typically only require SIL 1 and SIL 2 SIFs. SIL 3 and SIL 4 SIFs are very rare and normally not economically beneficial to implement since they require a high degree of duplication. In most of these cases, one should reconsider the fundamental design of the process.
Here it is also important to mention that SIL levels only apply to SIFs. Individual products or components do not have SIL ratings. However, they can be marked suitable for use within a given SIL environment.
Functional safety standards: IEC 61508/61511
Let’s just talk a bit about the IEC 61508 and the other standards out there. In 1998 the International Electrotechnical Commission (IEC) published IEC 61508, the first international standard to quantify the safety performance of an electrical control system and introduce the concept of lifecycle. The main goal of this standard is to minimize the failures in all electrical/electronic/programmable electronic safety-related systems.
The IEC 61511 standard was developed as a process sector implementation of IEC 61508 and gives requirements for the specification, design, installation, operation and maintenance of Safety Instrumented Systems (SIS). In the U.S. ANSI/ISA-84.00.01 is used. This is the same as the international standard IEC 61511, with the addition of a grandfather clause to accommodate existing SIS installations.
These standards represent current good practice in the management of SIFs at process plants across the world. Adopting these standards will ensure suitable management of risk. Thus, SIL Assessment Software / SIL Calculation Software should also be aligned with these standards.
Calculating the Safety Instrumented Level (SIL)
To determine a SIF’s SIL level, the SIF’s overall PFD must be calculated. This SIL calculation basically combines the Failure Rate data for each of the individual SIF components (i.e. the sensors, logic solvers and control elements) and accounts for test frequency, redundancy, voting arrangements, etc.
Failure Rate data for each component can be obtained from the equipment manufacturers. But, even with this available, the calculation is quite sophisticated. Therefore, it is recommended to use good SIL assessment software / SIL calculation software to determine the SIL. Also, user competency and experience are essential, and the input of many disciplines are required.
But how does the SIL calculation fit into the bigger picture? For that we need to understand the Safety Life Cycle.
Safety Life Cycle
The IEC standards define a concept known as the Safety Life Cycle. This is a cyclic process where all hazards are identified and analysed to understand which hazards require a SIS. The Safety Life Cycle can be outlined in a few steps to show where the SIL calculations fit in:
- First identify the hazard and its frequency.
- Determine if this frequency is acceptable (with no SIS). If so, no SIS is needed, else:
- Determine the hazard’s SIL level by calculating the target RRF of each SIF.
Determine the SIF’s minimum RRF. This is the hazard’s frequency (without SIS) divided by the acceptable frequency. When the minimum RRF is known, the SIF’s target SIL level can be obtained from the SIL table. SIFs may have different target SIL levels.
- Design an SIS so that each SIF has a PFD corresponding to the target SIL level.
The SIF’s overall PFD is determined with SIL calculations. The SIF’s RRF can then be compared to the minimum required RRF (remember RRF = 1/PFD.). If greater than the minimum required RRF, the SIF is sufficient.
SIL determination requires care. Thus, one should also take care when choosing SIL assessment software / SIL calculation software. Ideally the software should not only focus on the SIL calculation but assist with the whole Safety Life Cycle analysis.
SIFpro software: An effective SIL tool
SIFpro is a powerful, yet easy to use, comprehensive software tool that offers facilities to create and edit Hazard & Operability (HAZOP) studies and Layers of Protection Analyses (LOPA) with risk reduction factors (RRFs). It also facilitates the design of SIFs with SIL verification, and proof test interval scheduling and optimization. In truth it supports the whole Safety Life Cycle analysis to reduces the risk to ALARP (As Low as Reasonably Practical), while ensuring good engineering practices by complying with IEC 61508/61511.
Furthermore, SIFpro includes a failure rate database and an extensive library for initiating event safeguards. The solution also documents the system’s design, logic and history to provide a consistent and defensible approach to safety system designs. Data analysis and reporting features allow users to make informed decisions.
Safety Integrity Level (SIL), Safety Instrumented System (SIS), Layers of Protection (LOP), Layers of Protection Analysis (LOPA), Lines of Defence (LOD), Hazard & Operability (HAZOP), ISA 84, IEC 61511, IEC 61508, SIL Assessment Software, SIL Calculation Software, Risk Reduction Factor (RRF), Probability of Failure on Demand (PFD), Safety Life Cycle, Basic Process Control System (BPCS)
Elsa Tolsma-de Klerk, Masters in Electronic Engineering
Niveta Rathore, B.Tech in Electronics, Instrumentation & Control
Elsa and Niveta work for Cenosco. For over 20 years Cenosco has been involved in many large energy, oil and gas projects worldwide, developing world class tools in the field of asset integrity risk management, health, safety, environment and quality (HSEQ), geomatics engineering and statistical analysis.
26 September 2019
- “Functional Safety of Electrical, Electronic and Programmable Electronic Systems,” IEC 61508, Intl. Electrotechnical Comm., Geneva, Switzerland, 2000.
- “Functional Safety: Safety Instrumented Systems for the Process Industry Sector,” IEC 61511, Intl. Electrotechnical Comm., Geneva, Switzerland, 2003.
- “Functional Safety: Safety Instrumented Systems for the Process Industry Sector,” ANSI/ISA-84.00.01-2004 (IEC 61511 MOD), Intl. Soc. of Automation, Research Triangle Park, N.C., 2004.
- Generowicz, “Functional safety: the next edition of IEC 61511”, 6th Safety Control Systems Conference, Melbourne, 2016.
- King, “Do You Really Need SIL 3?”, Chemical Processing, 2010
- Health and Safety Executive, “Lines of Defence/Layers of Protection Analysis in the COMAH Context”, Vectra 300-2017-r02, 2017.
- Gruhn, S. Lucchini, ”Safety Instrumented Systems: A Life-Cycle Approach”, International Society of Automation, USA, 2018.