After determining the scope of the product risk management activities and documenting them in the Risk Management Plan, now the actual risk assessment can begin. In this article, we look at the various elements of what goes into a risk assessment for a product and how to determine the overall acceptability of the risks for the product in question.
First, some definitions:
- Risk Assessment is the overall process of analyzing and evaluating the risks. When the risk management process is completed, this is the final outcome.
- Risk Evaluation is comparing the risk of the product against acceptability criteria, to determine whether the risk is acceptable or if more controls are needed to reduce the risk. As discussed in an earlier article Risk Management Plan, the acceptability criteria are defined in the Risk Management Plan before the Risk Analysis begins.
- Risk Analysis is the meat of the Risk Assessment process. This is where the hazards are identified, along with the sequences of events that lead to hazardous situations, and the possible harms to the user that accompany them. Risk Estimation, or the assigning of values to the probability of occurrence of harm and the severity of that harm, is part of the Risk Analysis process. The vast majority of the discussion and decision-making in risk management involves analyzing the risks.
How to Perform Risk Analysis
Effective risk analysis in compliance with ISO 14971 and other relevant regulations requires careful consideration of how data is collected, analyzed, and utilized. By leveraging the company’s data and resources efficiently, it is possible to develop a comprehensive risk analysis process that satisfies regulatory requirements and supports informed decision-making. This article will describe a general method that complies closely with the standard and can be modified to accommodate any unique characteristics of a company’s quality system while still executing risk management in an acceptable manner.
Safety Characteristics
Before risk can be analyzed, and before hazardous situations can be determined, the hazards inherent to the product must be identified. ISO 14971 requires that the characteristics of the medical device that could affect the safety of the device be identified. Good examples of safety characteristics include: if the device is implanted, electro-mechanical function, use of radiation or chemicals, or dependency on software for functionality, among many others. The standard includes a list of questions to help guide a company in identifying these safety characteristics. While many manufacturers utilize that list of questions as the sole checklist of safety characteristics, the standard cautions against doing so, and encourages the manufacturer to think carefully about the device and how it will be used during this exercise.
Traceability
No matter how exactly the various risk assessments are recorded, each unique risk analysis should include a unique identifier for ease of traceability and reference throughout the product life cycle. If a risk analysis is ever retired from the risk management file, for example, if a design change renders the hazard no longer foreseeable, retire the identifier as well; many problems in post-market surveillance tracking and metrics can be avoided by keeping one identifier related to a single risk analysis.
Hazard
Hazards, the potential source of harm, are determined from the analysis of the safety characteristics as noted above. Hazards fall into a number of categories such as electrical, mechanical, material, and the like. Hazards are scoped in relation to the nature of the device: A simple bandage will have far different inherent hazards than an MRI machine, for example. The ISO 14971 standard instructs the manufacturer to identify hazards based on the intended use and reasonably-foreseeable misuse of the device, as well as the safety characteristics.
There are several possible approaches to identifying hazards. A company may utilize high-level categories such as pneumatic or radiation (most useful if your risk analysis is solely a high-level, top-down type of assessment), or the hazards may be derived from the effects of the failure modes of the device if a bottom-up approach is most useful for the analysis.
Regardless of the approach, hazards from both normal and fault conditions should be considered and recorded. Contrary to a somewhat popular belief, risk is not the sole province of product failure, and a Failure Modes and Effects Analysis (FMEA) is rarely the most optimal tool for identifying all product risks. To illustrate this point, consider an implantable device: the very existence of a foreign object in the human body brings about various hazards and risks, even if said device is operating exactly as intended. All such hazards should be identified as part of the risk analysis. It may be helpful to identify each hazard as derived from a normal or a fault condition, for ease of classification during risk control or post-market activities.
Sequence of Events
Just as a failure mode of the product is not automatically a hazard, a hazard does not necessarily lead to harm. Indeed, harm can only occur if the user is exposed to a hazardous situation. The hazardous situation derives from the hazard, if and only if a specific sequence of events occurs. For each hazard, record the sequence of events where the user could be exposed to a harmful event; the sequence generally starts with the hazard (or effect of failure, for a fault condition) and includes each distinct event that must occur in order for the hazardous situation to manifest itself. The events can include everything from single or multiple failures, environmental effects, and/or additional human actions (whether in error or in response (or lack thereof) to the unfolding sequence. In the simplest terms: use the sequence of events to walk the reader from the hazard to the exposure to harm.
It is very common that the same hazard may lead to more than one hazardous situation, and they may or may not be unrelated to each other. If a single different event in the sequence of events can end up with a different type of exposure, record that sequence of events separately, with its differing hazardous situation. There is no limit to how many sequences of events may branch out from a single hazard, other than the scope of the design and its use environment.
When brainstorming sequences of events, start with either the hazard and work forward, asking “what happens next?”; or, start with the potential harm and ask “how can this happen?”. Knowledge of issues with similar products and the company’s experience will dictate which approach works best (or try them both).
Hazardous Situation
This is generally the final event in the sequence of events, where the user (patient, caregiver, clinician, etc.) is exposed to the hazard in question. Good practice is to write each hazardous situation as “User exposed to…”, to emphasize that this situation will immediately lead to harm. If this sentence structure seems awkward, or the exposure would not immediately lead to harm, then it’s very likely that a hazardous situation is not actually identified yet, and one or more events must be added to the sequence to get there.
Again, there may be more than one hazardous situation for each hazard; usually, each sequence of events is unique to the particular hazardous situation (even if only one of the events in that sequence is different).
Harm
The injury to the user as a direct result of exposure to the hazardous situation. This is generally expressed as the actual clinical damage to the user, e.g. “burn”, “pain”, “infection”, and the like.
There may be multiple harms that could arise from exposure to a given hazardous situation. Indeed, the ISO 14971 standard asks the user to identify all known and foreseeable harms.
It is not a good idea to identify only the most frequent harm that stems from a hazardous situation, or the most serious harm. Not only does this go against the principles of the standards and regulations, but it’s easy to demonstrate that the overall risk of lesser harms is at times greater than the risk of the most serious harm (because only a small fraction of harms that actually occur due to devices in the field are fatal, for example). At the same time, regulators tend to frown on companies that don’t appear to recognize that very serious or catastrophic harms are foreseeably possible, and take appropriate steps to control those risks.
If there appears to be no such harmful injuries, then it’s possible that the hazardous situation is really not that serious, and that a different sequence of events may be needed to show how the hazard can cause harm.
Risk Estimation
Once the hazardous situation and harm are identified, the risk can be estimated, generally by applying a predetermined rating system as follows:
Probability of Occurrence of Harm – the chance that a hazardous situation will actually lead to harm.
Severity of Harm – once the harm occurs, the seriousness of the consequence of that harm to the patient’s well-being
Risk Evaluation
Risk is defined as the combination of the Probability of Occurrence of Harm and the Severity of that harm. Once these factors are known, the risk can be evaluated against pre-approved rating scales to determine the acceptability of that risk.
Note that some regulatory requirements contradict this part of the ISO 14971 standard: for products marketed in the EU, for example, no risk can be rated as acceptable without some sort of benefit-risk analysis, regardless of how low that risk may be. Depending on where the manufacturer will market their products, the process for determining acceptable risk should be set up accordingly.
Risk Controls
Once the risk is evaluated, what should the company do about it? Make every effort to reduce that risk, of course. Those design specifications and/or requirements that help reduce the risk of using the product are known as risk controls.
Risk can be controlled by any of the following:
- Eliminating the hazard altogether
- Reducing the probability that the hazard will lead to harm
- This is usually accomplished by reducing the probability of the hazardous situation occurring, but in some rare cases the likelihood of harm could also be controlled
- Reducing the possible severity of the harm once it occurs
- While this is technically possible, companies should take extra care when declaring a reduction in severity. Remember that the severity rating is the amount of injury to the patient or user, NOT the probability that the harm will occur. Keeping these two concepts separate in the risk analysis leads to better accuracy and a more focused risk control strategy.
There are three types of risk controls as outlined in the 14971 standard:
- Inherent safe design: modifying the design of the product so that a certain hazard is eliminated or the probability of occurrence is reduced. One example could be a custom connection point to prevent unwanted line connections in a hospital setting.
- Protective measures: modifying the design of the product such that a certain hazard still exists, but the user is kept separate from the hazard and thus cannot be exposed to a hazardous situation. Shielding of electronic circuitry is an example of a protective measure.
- Information for safety: The least effective method of risk control is providing warning labels to the product, which caution the user to avoid certain situations that could expose them to a hazard. This information should be prominently displayed on the product itself to maximize its effectiveness as an informational risk control.
- Posted maintenance intervals and manufacturer-provided training could also be considered information for safety according to TR 24971.
Verification of Implementation and Effectiveness of Risk Controls
It is not enough to simply state that risk controls are in place for the product; in the medical device world, everything needs to be verified to be true, and risk controls are no exception. The product risk file should demonstrate the following:
- Implementation of the risk control; that is, proof that the control is actually included in the product. References to the product requirements or specifications are usually sufficient, as those are verified as part of design controls anyway.
- Effectiveness of the risk control, or evidence that the control element is actually reducing the risk as documented. Design-based control effectiveness can be shown based on design verification/validation testing; information-based controls (information for safety) may be demonstrated through product usability testing in many cases. Since risk controls can be considered “user-dependent” it is a good idea to identify these types and consider integrating them into user design validation or human factors testing. Non user-dependent risk controls can usually be verified for effectiveness within design verification.
Post-Mitigation Risk Evaluation
After the risk controls are identified, the risk evaluation is repeated. Document the various probability and severity values for each hazardous situation that will be realized after the risk controls are applied to the product.
- Documenting risk both before and after the risk controls are applied is a good way to demonstrate just how necessary the particular controls are, just in case the company decides to trim any features from the design in a future change initiative.
Risk Control Cause of Hazard
For some products, the implementation of a risk control can lead to exposure to a different hazard. If this occurs, analyze the new hazard using the same methodology as all the original hazards. Document which controls lead to different hazards in the risk management file.
- A common example of this would be for an infusion pump which automatically shuts off if it detects that the infusion rate is higher than what was programmed. While this action prevents an overdose of medication for the user, it also results in a delay of the intended therapy, which is a potential source of harm (or, a hazard).
As long as the resulting hazard has a lower risk than the hazard that is being controlled, the manufacturer can accept that this is a suitable trade-off for safe operation.
Benefit-Risk Analysis
After all the risk controls have been applied, the remaining risk in the product is known as residual risk. Before the product can be introduced to the market, the residual risk should be determined to be acceptable to the user population. In most cases, this is accomplished through use of a benefit-risk analysis: the comparison of the benefits of the device’s presence on the market against the potential residual risk. In short, if the benefits outweigh the risk, the residual risk is acceptable and the product can be launched to market.
Conclusion
In conclusion, conducting a comprehensive risk assessment is a crucial step in ensuring the safety and effectiveness of a medical device. By adhering to the guidelines set forth by ISO 14971 and other relevant regulations, manufacturers can develop a thorough understanding of the risks associated with their products and implement appropriate risk controls to mitigate those risks.
Throughout the risk assessment process, manufacturers must identify hazards, analyze sequences of events leading to hazardous situations, evaluate the probability and severity of potential harms, and implement risk controls to reduce these risks to an acceptable level. Furthermore, it is essential to verify the implementation and effectiveness of these risk controls and, when necessary, conduct a benefit-risk analysis to weigh the benefits of the device against its residual risks.
By diligently following these steps, medical device manufacturers can not only ensure compliance with regulatory requirements but also contribute to the overall safety and well-being of their users.
In subsequent posts, we will explore more components of a successful risk management file to keep you moving on your risk management journey. Stay tuned and join our newsletter to get the latest updates!
