During the product development pipeline, a lot has to go smoothly. Everything from the planning, design, tweaking, and especially implementation hinges on the quality and safety of your product. Whether your industry includes health science, app development, or even space, nuclear, or automotive sectors, understanding how quality control versus quality assurance works, and how they are different, is crucial. Quality planning is part of quality management. Organizations should set quality objectives and then specify the necessary requirements for operational processes, including related resources to achieve those quality objectives.

An organization focused on quality promotes a culture that results in the behavior, attitudes, activities, and processes that deliver value through fulfilling the needs and expectations of customers and other

relevant interested parties. Delivering a quality product refers to a product’s ability to meet consumer demand, expectations, and requirements while satisfying certain criteria like safety and consistent functionality. These standards not only ensure that your product or services satisfy consumers, but that it remains consistent throughout development and launch. Assurance and control are how quality is determined and kept. The quality of products and services includes not only their intended function and performance but also their perceived value and benefit to the customer. 

Related: What Is The Difference Between Clinical Evaluation and Clinical Investigation?

What is Quality Control?

Quality control, sometimes abbreviated as QC, is a process by which manufacturers identify and correct defects in a product. Its focus is on the product or service itself. Quality control focuses on fulfilling quality requirements. The quality control pipeline ensures that the manner in which a product or service is developed, maintained, and implemented are followed closely and accurately. Quality control is also an important step for verifying that deliverables meet standards set by your company. 

Quality control is a mainly reactive process since it is done in response to a known failure or defect. You are, in essence, controlling for the mistake or defect in order to prevent it from happening again. This is crucial since no product can survive development and launch if it is riddled with problems that remain unsolved. 

Consult with an expert to learn strategies to identify and solve problems to get your product launched.

What is Quality Assurance?

Quality assurance, also known as QA, is the cousin of quality control, in that both are needed in order to make doubly sure a product works as promised. The main goal of quality assurance is to prevent a problem or a defect before it even happens. Its focus is on processes. It works alongside quality control procedures in order to correct and plan ahead for issues that could lead to problems with the product. Quality assurance protocols monitor the activities, procedures, and processes that create the product are being followed appropriately. Quality assurance focuses on providing confidence that quality requirements will be fulfilled. Quality assurance departments are required to manage the compliance of government regulations and maintain production costs to ensure quality and patient safety. QA professionals oversee operations, so products and services meet current Good Manufacturing Practices (cGMP), for example, and internal company standards. They are also responsible for training, audits, documentation, and communication to leadership.

Quality assurance, due to planning for defects rather than reacting to them as quality control does, is a proactive manner of correcting mistakes. It prevents the defects from happening, recognizing potential failures that could arise in the pipeline, and may include contingency instructions for personnel involved.

Related: NEW Podcast Episode: 6 ways to collect Clinical Data with David Rutledge

Quality Assurance vs. Quality Control

Now that we have reviewed what quality control and quality assurance are in principle, it’s time to go over how they differ exactly between each other. Both have the same end goal of correcting problems and creating a better product. However, how and when quality control or quality assurance is implemented is what sets them both apart. 

Learn how our quality management service can play a significant role in helping you get your products approved.

Proactive and Reactive

Quality assurance is what is known as a proactive process, meaning that it aims to prevent problems before they happen through the design and construction of a product. Quality control is a reactive process, in that it responds to and corrects problems that have arisen already. Both are needed for quality to be maintained effectively. 

Quality assurance includes the design of documentation procedures like standard operating procedure (SOP) that all personnel must follow. These are preventative measures that keep things running smoothly. Quality control usually includes consistent and constant testing of products in order to not only find faults and areas for improvement but to confirm the product is performing as intended. The general order of things is that a quality control test sparks a quality assurance inquiry that corrects the documentation to include the results of the testing. 

Process and Product

The other most obvious difference when explaining quality assurance vs. quality control is that quality assurance procedures are focused more so on the processes that lead to a product’s quality, while quality control is concerned with the issues of the product itself. In other words, quality assurance concerns itself with the process, while quality control cares more about the product. 

Quality assurance can include detailed documentation, auditing, training, and implementations of engineering of administrative controls. Quality control involves a lot of product and laboratory testing in order to elucidate common problems that may occur. 

System and Parts

Quality assurance is concerned with the systems and procedures in place that establish quality standards. This process then tailors each pipeline towards creating better, more unified products. Quality control protocols are more focused on the individual parts that may fail within a final product; for instance, during testing of medical device systems that contain many components to support access, delivery, deployment, and then withdrawal.  Finding areas of a component or part failure is an element of quality control. QA procedures will usually lead to these instances of QC by modifying the system to include different parts.

Creation and Verification

Quality assurance activities spark changes in the pipeline for creating better-quality products. Quality control verifies that this created system works as intended. It is used to build better products in the future. QC is done post-manufacture for verification, while QA occurs early on prior to manufacture during the planning process.

Team and Personnel

Quality assurance protocols usually involve the efforts of the entire team; every member of the group works together to develop and adhere to QA documentation and SOPs. Quality control is usually the responsibility of individual personnel by testing products and following the SOPs for product verification. QA’s training across the entire workforce is part of this machinery.

Related: Mythical Medical and the Unbuilt Team: A hiring process that works even when regulatory requirements change, or global health pandemics occur

What is The Role of an eQMS?

An effective Quality Management System marries QA and QC together to ensure that they are followed appropriately. An enterprise quality management system (eQMS) allows leaders to oversee and document the process by way of cloud-based oversight. This system will ideally integrate all aspects of QA and QC to track changes.

If you do choose to use electronic systems, which I support regardless of the size of the company, make sure you use them consistently. Do not make exceptions for certain individuals, e.g., selected healthcare employees or senior-level management. It creates new levels of inefficiency and risks. This requires documents to be signed electronically or manually, scanned, and attached to records. Emails are required to document notification times or escalation of issues. It is inefficient and consumes the use of additional personnel resources. It sets a very poor example for employees that are being informed by management about the importance of quality.

Take-Home Message

Quality planning is part of quality management which focuses on setting quality objectives and specifying necessary operational processes, and related resources to achieve the quality objectives. Quality control  and quality assurance are essential parts of unifying the process for product creation and streamlining testing for better products. Both are critical for implementing new and exciting products for consumers.

Contact us today to learn how you can get your products approved with the proper quality assurance and quality control techniques.

Highly regulated industries like biomedical engineering, life science, and pharmaceutical development are always at risk of an audit by a regulatory agency like the United States Food and Drug Administration (FDA). The last thing you want to worry about is the FDA or related regulatory body finding an issue with your compliance with regulations. Documentation, moreover, detailed, accurate documentation is critical to avoid these pitfalls common to these industries. 

Keeping an organized and updated system that keeps your documents in order and ready when you need them can be challenging. This is especially true if you are a small startup and perhaps lack the capability to keep regulatory documentation paperless. Most of all, keeping these documents in order while also adhering to regulations with stellar compliance is a massive concern.

That’s why an effective document control system can make all the difference. 

The best way to reduce a risk of an audit or inspection by a regulatory agency is to obtain a consultation by a regulatory expert to ensure that you are doing everything correctly.

What is the Document Control System?

A document control system allows for easy arrangement and organization of files that comply with the quality assurance and regulatory standards set by governing bodies like the FDA. An effective document control system will typically be paperless, and their graphical user interface should make navigation and reporting compliance easy. 

Document control signals to regulatory bodies that your group follows good practice in the manufacturing and management of products. It assures that your group can access all documentation and track changes made to crucial compliance in standard operating procedures. 

Types of Documents Required for Creating Regulated Products

Clinical trials, for example, have massive documentation needs as they produce a lot of case report forms (CRFs) as well as protocol management copy that must be organized properly for the FDA and sponsors. Things like standard operating procedures (SOPs), CRFs, investigational review board (IRB) correspondence, plans and reports, checklists, adverse event logging, and protocol deviations all need to be handled carefully, and this can be challenging. 

Document control involves tracking different information related to the document, like the name of the person who created it, who approved the document, its status (valid, draft, in revision, awaiting approval), and authorization status (controlled/registered copy or authorized printout with signatures). Controlling for this information is usually a standard requirement for reporting to most agencies. With so many moving parts, a paper-based document control system is prone to errors and thus, nonconformances. A digitized document control system can eliminate these errors if implemented consistently across the organization.

Related: Ways to Collect Clinical Data

Document Control Is Not Just About Checking Boxes

The process of documenting information, and keeping that documentation controlled, is not just a system of checking boxes. While running through a basic checklist is how you meet minimum standards, you need to think about how best to organize things in relation to your company’s needs and structure. Document control is not a one size fits all process, and tailoring the process to your individual pipeline is critical for success. 

For example, hybrid paper and paperless models of document control technically meet minimum requirements for most agencies. However, doing so can hurt your company if it is not employed properly. Mismatched information between electronic forms and hard copies are common in this model and can spell disaster for the life cycle of your product. 

Related: Mythical Medical and The Unsolvable Equation: Benefit-to-risk analyses

Why You Need a Document Control System

If you are in the market for a better document control system than the one you are currently using (if you have one, at all), there are a few things you need to consider. Compliance will always be a primary concern. A good document control system will not only reduce the regulatory risks of compliance, but also increase transparency, improve security, and reduce the dissonance between hard copies of documents. There are other potential benefits.

Better Access Control

With electronic documentation controls, you can set user permissions. This ensures that only the correct people have access to the right documents. Say, for instance, you are an administrator, and you only want to share documents with either other admins like yourself or approvers. A quality document control system will allow you to do this, share and distribute and modify relevant documents without looping in the wrong team members. 

Improve Compliance

Electronic document control simplifies meeting compliance, and a particularly great control system will make obtaining authorization signatures or sharing passwords securely a breeze. These systems will minimize the risks of incomplete authorizations, incorrect forms, or drop-off in the audit process.

Learn ways that you can improve compliance with our services and solutions.

Transparency

Document control systems allow for seamless access to documents needed across many teams within your company. Increasing transparency improves data flow and prevents chokes in the pipeline of your product or service. When your team is taking too long to retrieve archived documents, you have an issue, especially during an audit or inspection. A good document control system will be able to handle many different file types, possess an accessible keyword query system, and control and modify user permissions accordingly.

Global Collaboration

If your workforce is spread across many different facilities over the nation, or even the world, a cloud-based electronic document control system will improve the speed at which collaboration can happen. A common holdup for large operations is the slow courier system across these office locations; electronic documentation controls eliminate this lag. 

Enhance Quality Management

Document control will make modifications to quality assurance protocols like SOPs, as well as updating in accordance with quality control testing far faster. It also ensures adherence to regulations and internal company procedures.

Create a Document Control Governance Board

A best practice would be to create a cross-functional document control governance board. It would include those within quality systems, information technology, and process owners. Their collaboration facilitates creating, implementing, monitoring, and adjusting the infrastructure. They can take a risk-based approach as they actively oversee and evaluate the current state and then develop strategies for improvement and growth as business needs change. Finally, the board and create metrics and present to important internal cross-functional stakeholders (reviewers, approvers, process owners) and senior leaders.

Recover from Disaster

When problems arise, such as a data breach, cloud-based document control systems can ensure your group will be able to land on its feet. Losing data can be disastrous for many business sectors, but good control systems can mitigate this risk effectively. 

Related: Metrics for Notified Bodies, Manufacturers, and Regulators

Bottom Line

Document controls are some of the most crucial components of running a company in a highly regulated field. The tools you need to adhere to compliance standards while also maintaining the flow of operations can be accessed through these document control systems.

Having trouble with the compliance standards? Contact us today for more information!

Clinical evaluations and clinical investigations are two of the most critical parts of a medical device’s journey to mass marketing. Despite this being the case, there are frequent, widespread misconceptions of what each means.

In this article, clinical evaluations, clinical investigations, and the processes, along with the requirements for each, will be discussed.

What is Clinical Evaluation?

When discussing clinical evaluation, we are not referring to the type where a patient is evaluated for medical reasons, though the two references do often get mixed up. Instead, we are referring to the clinical evaluation of a medical device. Clinical Evaluations are mandatory and were required starting with the early Active Implantable Medical Device Directive (90/385/EEC 20 JUN1990) and the Medical Device Directive (93/42/EEC 14 JUN1993). It was later incorporated into the first ISO 13485 quality management standard (see the current standard ISO 13485:2016 – Medical Device Quality Management System clause 7.3.7).  Now it appears in the European Union (EU) Medical Device Regulation (MDR) 2017/745 May 2017.

Clinical evaluation is the term used to describe a process plan where all of the relevant data regarding a device is collected and then assessed. The review determines whether there’s enough evidence to show that the device conforms to the necessary regulatory requirements and when a clinical investigation might be needed. The EU MDR 2017/745 Article 2 (Definitions) defines clinical evaluation as a systematic and planned process to continuously generate, collect, analyze and assess the clinical data pertaining to a device to verify the safety and performance, including clinical benefits, of the device when used as intended by the manufacturer. In other words, the clinical evaluation is a methodologically sound, ongoing (living) planned procedure to collect, appraise, and analyze ‘clinical data’ relating to a medical device. The manufacturer must answer the question, “Is there sufficient clinical evidence to confirm compliance with the relevant requirements for safety and performance when using the device according to the Instructions For Use (IFU)?”  

A document called a Clinical Evaluation Report is where the results of the process plan and analysis are recorded and documented. 

Related: Mythical Medical and the Unbuilt Team: A hiring process that works even when regulatory requirements change, or global health pandemics occur

How to obtain the clinical data?

There is more than one way to obtain the clinical data needed for the initial clinical evaluation. First, is the relevant scientific literature relating the design, intended purpose, safety, and performance. Here the manufacturer can either use the equivalence pathway or collect new clinical data on their device. If the equivalence route is chosen, the data must demonstrate compliance to or the spirit of the relevant regulatory requirements, e.g., Essential Requirements under the medical device directives or the General Safety and Performance Requirements (Annex I) under the newer medical device regulations. If equivalence is not demonstrated or established, other methods have to be used to collect new clinical data, e.g., surveys, clinical investigations, or if approved in other countries, large, “big data” hospital networks or registries. Some manufacturers may choose a combination of both equivalence and new clinical data for their initial submission.

The whole point of the clinical evaluation is to assess the clinical data to ensure that it conforms to the national requirements and standards. Once this data has been acquired, the clinical evaluation report can written and finalized. The clinical evaluation report is not just a paper exercise to meet requirements. It is an ongoing process of becoming compliant and then staying up-to-date on products being placed on the market.

It is mandated that the clinical evaluation and all of its documentation be updated continuously with any post-market surveillance information that becomes available. If for any reason, the post-market surveillance is found not to be necessary regarding the medical device follow up, there must be an acceptable justification and it must be documented. Current thinking, unlike in the past, among Notified Bodies and regulators is that those justifications for not performing surveillance would be rare. In regards to high-risk medical devices, the final report must include a clinical follow-up report.

Get expert guidance through the medical device approval process with Global Strategic Solutions. Learn more here!

What is Clinical Investigation?

Not to be confused with clinical evaluations, clinical investigations are clinical trials (studies) in at least one human subject, which are performed to assess the performance and, most importantly, the safety of a specific medical device. Recently, an international standard, ISO 14155:2020, was updated. This standard addresses the good clinical practice that is expected for the design, conduct, recording, and reporting of clinical investigations carried out in human subjects to assess the clinical performance or effectiveness as well as the safety of medical devices. Another international standard that also has relevance to performing a clinical investigation is ISO 14971:2019. All medical devices have risks. This standard is intended to help manufacturers of medical devices to:

• Identify the hazards associated with medical devices
• Estimate and evaluate the associated risks
• Control those risks
• Monitor the effectiveness of those controls

As clinical investigations are usually performed on higher risk devices, ISO 14971:2019 highlights three principles that should guide the manufacturer to mitigate or alleviate risks along the approval (and post-approval) pathway. First, they should design medical devices that are inherently safe. Next, protective measures or controls in the actual medical and/or the manufacturing process should be identified and implemented. Finally, safety information, such as found on the product labeling or within the IFU, should be communicated to either:

• those using the device
• coming in contact with the device, or 
• caregivers of those requiring the device.
  

These two international standards, along with the EU MDR, provide confidence that manufacturers are designing safe and effective medical products. Regulators and Notified Bodies will independently confirm this as well.

Clinical Investigation for Class III and Implantable Devices

Clinical investigations were first required in 1990 for all active implantable medical devices, virtually the highest risk medical devices at that time. That thinking has evolved, and authorities are taking a risk-based approach EU MDR 2017.745, Article 61(4). The current thinking is that for implantable devices and class III devices, clinical investigations shall be performed. There are some exceptions, three are described here although others exist:

• The device has been designed by modifications of a device already marketed by the same manufacturer, 
• The modified device has been demonstrated by the manufacturer to be equivalent to the marketed device, in accordance with Section 3 of Annex XIV and this demonstration has been endorsed by the notified body, and
• The clinical evaluation of the marketed device is sufficient to demonstrate conformity of the modified device with the relevant safety and performance requirements. 

In these cases, the notified body shall check that the Post-Market Clinical Follow-up plan is appropriate and includes post-market studies to demonstrate or confirm the safety and performance of the device.

Decisions to perform a clinical investigation under the EU MDR can be complicated and complex, but distill down to an active interaction between data obtained during the risk management and clinical evaluation processes and whether that data allows conformity to Annex I, “General Safety and Performance Requirements” (GSPR). The figure below illustrates this interaction.

In general, a medical device requiring a clinical investigation is ready to enter the market once a successful clinical investigation has taken place. It’s the results from the clinical investigation that will be included within the clinical evaluation which will be used, in part, for product approval.  The results (assuming satisfactory) show that the medical device complies with the regulatory requirements of performance and safety. Thus, clinical investigations hold considerable weight.

Related: Mythical Medical and the Unbuilt Team: A hiring process that works even when regulatory requirements change, or global health pandemics occur 

What is The Clinical Investigation plan?

There are many plans created to bring a medical device to the market. There is even a plan for clinical investigations. The clinical investigation plan involves, among other things:   

• Statistical power

• Patient population

• Clear and measurable objectives

• Criteria for stopping the trial

• Ethical consideration and informed consent   

‘What is The Manufacturer’s Responsibility?

In a situation where the sponsor of a clinical investigation is outside the EU or is not Union-established, it’s the manufacturer’s or sponsor’s responsibility to make sure that a legal or natural person is Union-established and acting as their legal representative.

In any regard, this said sponsor/manufacturer is responsible for taking care of a large part of this operation. They are responsible for:

• Insurance
• Design of the study
• Principal investigator designation
• Applications with the ethical committee
• Keeping documentation available and up-to-date. Also, incorporating the clinical investigation reports (data) into the technical documentation that’s legally mandated to be kept available by the manufacturer
• Incident reporting

What is The Role of The Competent Authority?

Once clinical investigation registration has taken place, competent authorities and saddled with the responsibility of providing the authorization for a clinical investigation to begin. According to the Medical Device Regulation (MDR), approval could take less than a month. The commission may engage in delegated acts to fine-tune the procedure. Each clinical investigation receives one single registration number or SRN for short. This is to maintain a more efficient and reliable exchange of data between member-states, the Commission, the public, and others.

Related:  Mythical Medical and The Unsolvable Equation: Benefit-to-risk analyses   

What is The Role of The Notified Body?

The Notified Body is a conformity assessment organization. They review the clinical evaluation and, if required, the clinical investigation. These documents, and others, are part of the technical documentation review that occurs before granting a CE certificate. There are many processes, reports, and personnel that undergo inspection or scrutiny prior to approval which includes, in part: 

• A benefit-to-risk analysis
• An inspection or discussion of how the device fits into current clinical practice by evaluating state-of-the-art information
• An assessment of the clinical safety and performance
• Auditing the manufacturer and its personnel
• Evaluation of post-marketing or post-approval processes and plans

Conclusion

Due to the massively impactful nature of medical devices, it is of the utmost ethical and business importance to carefully consider which aspects of safety and performance can be supported by not only pre-clinical evidence, but also clinical evidence from evaluation in humans. It is only through a thorough clinical evaluation of all the available data that regulators and Notified Bodies approve medical devices. There are other requirements, such as post-market surveillance, once a device is on the market. That will be the topic of a future article.

The EU MDR is basically a fascinating and highly intriguing story about redemption within the medical device sector or business. It’s about having a final chance to make a change before it’s too late. If done correctly, clinical evaluations, along with clinical investigations, are steps being taken now to ensure a bright, innovative future for medical device development that meets unmet medical needs. 

Get expert guidance through the medical device approval process with Global Strategic Solutions. Learn more here!

“Wait a minute. You want me to do what?”

That was the first response from Kai Lin, an employee at Mythical Medical, after being asked by senior management to set up a system to proactively oversee “significant change” processes affected by MDR Article 120, Transitional Provision. She was convinced this would be a dead-end, career-limiting, tedious project, and wanted nothing to do with it. Little did she know it would not only be a project that would have enormous, positive implications to millions of patients, but it would become very personal.

In this fictional story, I’ll guide Kai Lin through an active process that provides a framework to address significant change issues that notified bodies and regulators expect.

I will be discussing relevant topics and providing simple, practical clarifications and solutions that can be implemented globally.

Medical Device Coordinating Group and Significant Change

The European Union (EU), Medical Device Regulation (MDR) Article 120(3) Transitional Provision states:

…a device with a certificate that was issued in accordance with Directive 90/385/EEC or Directive 93/42/EEC and which is valid by virtue of paragraph 2 of this Article may only be placed on the market or put into service provided that from the date of application of this Regulation it continues to comply with either of those Directives, and provided there are no significant changes in the design and intended purpose.

In March 2020, the Medical Device Coordination Group (MDCG) published useful guidance to help manufacturers, notified bodies, and competent authorities. It is known as MDRG 2020-3 Guidance on significant changes regarding the transitional provision under Article 120 of the MDR with regard to devices covered by certificates according to MDD or AIMDD. See the link below.
https://ec.europa.eu/docsroom/documents/40301

I found the document useful as a framework to have these types of discussions. It outlines situations that would be considered significant changes that would cause medical device certificates to expire with immediate consequences of shifting to MDR re-licensure.

Could This Be A New Opportunity?

After Kai Lin contacted me, I was curious to learn what had caused this fascinating initial response from a credible, known leader with so many previous successes. Unveiling that could begin a journey that could influence hundreds of staff along millions of patients and healthcare professionals for years.

From one perspective, the MDR Article 120 guidance document is a simple, dull, straightforward flowchart outlining what could be a significant change. From another perspective, it opens a fascinating, extensive journey that could launch opportunities for career development, public health protection, improvements in quality of life, and personal growth.

A Brief Discussion During An International Conference

Kai Lin had just come back from eating a large meal with her team earlier this evening. During our first discussion after a long day at this international medical device conference, I discovered that Kai Lin was still bothered by her most recent assignment. “This is a waste of my time. It’s not helping me. It’s not going to be part of my destiny. I don’t care about what I’m doing now. It just doesn’t matter.”
I learned she had not considered the fundamental opportunity within EU MDR Article 120. “Significant change” implies substantial, major, considerable, and sizeable change, both in terms of medical devices generally and behaviors within organizations specifically, e.g., regulatory bodies, notified bodies, manufacturers, patient advocacy groups, and consultants. It was time to dive a little deeper, highlighting opportunities for leadership. Although a calmness settled on her spirit, a carefully appraising eye was evident.

As I went through each section of the guidance document pointing out the human, social cost, or burden if it was not implemented, it gradually but reluctantly became apparent to her. There would be life versus death implications and, in some cases, improvements in quality of life.

Suddenly, Kai Lin started complaining of stomach pains in her upper right part of her abdomen.

She had trouble just sitting, and it did not seem to worsen or change when she moved in her chair.

She started feeling nauseous. Eventually, she was taken to a local emergency or urgent care clinic.

Her destiny tonight would be within a surgical suite.

Significant Change: Related to Intended Purpose

During the transitional period, there are triggers that will directly influence medical device certificates under the current directives. The most obvious one is extending the device’s intended purpose, involving new users or patient populations. This includes changes in clinical use, such as anatomical sites, changes in access sites or deployment methods. It also includes opportunities to generate additional claims linked to broader intended use or medical indications. If significant clinical evidence supports expanding intended purpose, there are patients and healthcare professionals out there today that are waiting for other diagnostic, therapeutic, or monitoring options.
It is important to note that limitations or reductions in intended purpose or intended use due to Post-Market Surveillance (PMS) signals or changes in state-of-the-art is not considered a significant change. Regulators and notified bodies expect manufacturers to monitor PMS data, both passive and active, and make changes to ensure continued safety and performance of devices when used as intended.

Significant Change: Related to Design or Performance Specification

There is renewed focus to make decisions based on evidence and not just the “hope or promise” that medical devices are performing as intended. This means that design changes will require further clinical or usability data to support safety and performance.

For now, see MEDDEV 2.7/1 Rev 4 and MDCG 2020-6 for further guidance on clinical evidence. Ensure senior management knows that new risks will require control measures, including those existing risks being negatively affected by the change. This guidance also highlights the following: change of built-in control mechanism(s), operating principles, source of energy, or alarms.

Significant Change: Related to Software Changes

This is an exciting area for future professional growth. The following are highlighted: new or major changes of software operating systems or any component; new or modified software architecture or database structure, change of an algorithm; a required user input replaced by a software closed-loop algorithm; and new software diagnostic or therapeutic features.

Another feature that is discussed is a new channel of interoperability. Examples are the ability to safely, securely, and effectively exchange and use information among one or more devices, products, technologies, or systems. This includes displaying, storing, interpreting, analyzing, and automatically acting on or controlling another product. It also addresses a new user interface or presentation of data, such as a new format or new dimension or measuring units. There may also be other software changes that impact diagnosis or treatment delivered and that are NOT considered minor by the manufacturer, notified body, or regulatory agency.

This section of the guidance document again gets personal. If there are opportunities for using newer technologies, imaging displays, or decreasing user input to improve safety, then millions of people’s lives could be saved or certainly have their quality of life improved.

Significant Change: Related to Change of a Material

Many manufacturing staff are out of the spotlight yet are performing tasks that potentially have a ripple effect on the safety and performance of products. It is important to note that when decisions are made to change a material of human/animal origin, including the addition of new materials, this now is a significant change.

Furthermore, changes to a material containing a medicinal substance, changes to the medicinal substance itself, changes that may impact the quality, safety, or efficacy of the medicinal substance are also considered significant changes. Moreover, changes in ingredient(s) or material(s) from either existing or new suppliers that do not meet existing specifications are now significant changes.

If your organization is focusing on developing and overseeing quality management systems affecting these processes, you need to be reminded that your job matters. It’s important and has the potential to affect patients and healthcare practitioners worldwide.

Significant Change: Related to Change of Terminal Sterilization Method of Device or Packaging Design with Impact on the Sterilization

Although one of the last steps in a process, choosing a sterilization method is one of the essential steps in developing and putting into service a medical device. At best, using an insufficient or inappropriate sterilization method can prevent your medical device from being approved. At worst, inadequate sterilization can lead to the transmission of infectious diseases causing patient illness and even death. If you think your sterilization work is unimportant, it is certainly not. Your career could mean a magnificent, brilliant medical device may never get the chance to make a difference in the lives of millions.

These are costly processes, and technology is moving faster than expected. The guidance highlights that changes in terminal sterilization methods or design changes that affect or modify the sterility assurance are now considered significant changes.

Processes need to be developed and monitored to assess change in (i) packaging design, which affects functionality, safety, the stability of seal integrity, or (ii) shelf-life change validated by protocols NOT approved by a notified body.

Being involved in sterilization activities positions your staff to be responsible for producing compliant devices that will never face the consequences of an unsterilized product causing preventable injury or death. There may also be substantial financial penalties, lost reputations, and even the ruin of a business. At a minimum, there could be product brand-damaging effects due to recalls from contamination. Yes, your job is significant and critical.

Changes that are Potentially NOT Significant

Many changes can be made that are not considered “significant changes.” Modifications and their implementation will be verified by a notified body either as part of its surveillance activities or following a manufacturer’s submission for prior approval. Some that may not be considered significant changes are shown here. Please note that manufacturers should always remain responsible for providing evidence that all changes do indeed neither affect the design nor the intended purpose. Also, know that guidance is not law. You can deviate, just explain.

Significant Behavior Change

Months later, Kai Lin was well into transforming Mythical Medical processes to set up systems to proactively oversee “significant change” processes identified by EU MDR Article 120. In a follow-up meeting, being curious, I asked her what brought about her significant change in behavior.

Kai Lin said while in the hospital, dawn had broken in a mesmerizing, hypnotic way. She noticed all the devices that were either being considered for her or used on her during her hospital stay. Many of them were made by Mythical Medical.

There were all sorts of imaging devices such as ultrasounds, CT scans, X-rays, and MRIs. There were those devices used for laparoscopy, along with a variety of monitors. That was when it happened. It was suddenly right in front of her. Instantly, she no longer saw only the device names, catalog numbers, SKUs, or part numbers. She saw something more significant.

She continued “I started imagining seeing patient names etched on them too. Actual names of patients that would eventually be benefiting from these and other devices worldwide. Over there I saw Kunal, John, Jim, Tina, Elena, Stella, Robert, Ye, Olive, Haisley, Dewei, Keanu, Weiying, Santiago, Yuan, Jasmine, Violet, Sarah, Astrid, Charli, Henry, Jackson, Sebastian, Laura, Summer, Matthew, Vivian, Sky, Ezra, Azalea, Selena, Julian, Mateo, Anthony, Jaxon Marilyn, Amy, Ingrid, Aaliyah, Gwyneth, Chung, Avanti, Luna, Evelyn, Karen, Adele, Eliza, Kylie, Jake, Wyatt, John, Jack, Luke, Lynn, Calais, and Jae. There were so many, many more names. They slowly became familiar and endearing.

“Each device was no longer a cold, inanimate manufactured object, but a living system designed to either save lives or improve the quality of lives. The devices needed to be nurtured, developed, fashioned, maintained, repaired, and eventually put to sleep when their lifetime had passed. I felt ashamed that I had been so petty and shortsighted.

“Then, at my bedside, there was a very silent, unassuming, ordinary device doing extraordinary work. Its imaginary name, Kai Lin, was attached to my bed rail steadily, reliably monitoring my vital signs. Suddenly, I had imagined in its own quiet, unpretentious way, it began gently whispering, ‘You are alive.’ I think my medication had something to do with this, but I am not sure.

“When I got back to the office, the first group of devices I chose to begin to work with were those within the gastrointestinal group. It became personal. I realized right then that I am not wasting my time. This project was helping me to become a better person and leader. I was thrilled now that monitoring significant changes had become part of my destiny too. I care about what I am doing again. It matters, really it does. The team feels it also.”

Closing Thoughts

Now, more than ever, leaders need to be continuously monitoring not only how devices are performing in the marketplace, but the changes that should occur in the natural course of a device’s lifetime (or life cycle). It directly affects product development decisions, approvals, and future budget investments.

There also exists a need to lead efforts to re-train personnel to pay closer attention to product performance during all phases of product development. Tightening up systems around these activities could become a new opportunity for many.
The significant change law and guidance can be dull and straightforward. Still, it could become a fascinating journey that could launch further career development, public health protection, improvement in the quality of life, and personal growth. Kai Lin knew now that the project would have enormous, positive implications to millions of patients and health care personnel around the world. It would become more than just personal as extraordinary, exceptional times were ahead when her leadership would be questioned.

David R Rutledge, Pharm.D., FCCP, FAHA, President & CEO, Global Strategic Solutions, LLC, Silicon Valley in California. david.rutledge@globalstrategicsolutions.com.

+1 (630) 846-0350 cell. The homepage is www.globalstrategicsolutions.com.

Disclaimer: This is just a bit of fun. The purpose of articles like these is for the free exchange of questions, ideas, or comments regarding issues of global interest involving medical regulations and is not to be considered legal advice. These are just opinions and not advice; opinions may change over time, too. Use it at your own risk. This disclaimer will occasionally be posted or updated throughout

Sharing information and insights internally from either regulatory agencies or Notified Bodies is healthy behavior. Knowledge is power, several believe. It can change the world, your company, and your personal leadership behavior, but only when it is communicated in a timely manner and shared in the right ways. Break out of your self-imposed jail today.

Medical device and in vitro diagnostic manufacturers are struggling to create viable post-market surveillance (PMS) plans. In this fictional story, I’ll present a process that provides a framework to organize thinking to craft a robust PMS plan. I will be addressing relevant topics and providing simple, practical solutions that can be implemented globally.

The Product Approval Email

There was a thrill of anticipation in the air. Kai-Lin, an employee and manager at Mythical Medical (see previous articles), received the approval email from their recent product submission. Corporate announcements went out, informing that there were desserts in the conference room, but her efforts were not acknowledged. In the whole grand scheme of things, that’s fine. But to her, today, it wasn’t. She would learn that life is cruel. It gives you the test first and the lesson later.

There was that delicious “eight treasure rice pudding” (also known as “ba bao fan”), chocolate sugar cookies, and assorted ice creams awaiting the swarm of bodies heading that way. She was disappointed and tried to resist being seduced to celebrate with her colleagues. The announcement hurt mainly because the final issue preventing approval was addressing a weak PMS plan. It needed to be overhauled. Jae, the hero of the benefit-to-risk section, moved to another therapeutic area. The rest of the team was inexperienced. It fell on Kai-Lin’s shoulders to immediately re-write that section to meet an internal product approval company milestone, which she did.

It’s Time for a Break

The product approval celebration event was almost over. However, before Kai-Lin went back to work, she needed to get away and go to lunch. She needed “Kai-Lin” time. Grabbing some of her friends, they headed toward several of the local, popular lunch spots. As they walked toward a favored restaurant, En Thai Sing, known for their quick buffet, she froze as she saw another “colleague.” Trouble gathered on her forehead. Instead, they went next door to the Peking Inn café and joined several other co-workers. After casual introductions, they ordered.

Little did she know, amazing, remarkable things were evolving that would eventually become part of her heritage and legacy. That timeline, however, was not in her control. Hard work goes unrecognized, but not today during lunch.

Post-Market Surveillance Expectations

Wrapped in a sudden intensity of reflection and inward contemplation, Kai-Lin hesitated to participate in the lunch conversation as it turned to pain points they were experiencing on their projects. Then, the piercing question came directed like a laser to her heart, “How did you address the PMS issue? We are all getting that question.” A fiery exclamation of anger and contempt almost came out; however, a calmness settled on her spirit. She had to choose, “Do I want to make a point, or do I want to make a difference?” Sharing clinical, regulatory, and quality insights did not come naturally.

With the quiet inner strength of an emerging leader, she methodically went through the importance of ensuring they mapped out potential sources of data. Flipping over a napkin, she drew a circle with arrows pointing to several key sources. She kept using the words “living system,” “change in culture,” and “increasing expectations.”

She wrote on another napkin several PMS components that required internal investigation and further analyses. Then, without recognizing she now had their laser interest, and even the curiosity of another restaurant customer’s at the next table, she listed potential action items and communications that needed to be considered. In her mind, it was all so obvious. A post-market clinical follow-up (PMCF) plan was typically expected for all their products.

Clinical Investigation Process

Since some at the table were not experienced in running clinical trials, they asked her to walk them through that process. Some moderate to higher-risk products might require a post-approval investigation (or clinical trial or clinical registry) that would need to be documented in the PMCF plan. Local geographic clinical investigation regulations need to be considered, as well, along with personal data protection and data integrity laws. Before any official work occurs on structuring the investigation, there needs to be a clear, agreed-upon clinical question(s) of what is being asked about either the device or patient population and then justify why this approach is necessary.

A Detailed Clinical Investigation Surveillance Approach

A more detailed approach is shown below. The following is a method to use initially to develop a clinical surveillance trial that would answer the question(s). This process leverages experience with approval-based clinical investigations and could be modified depending on the specific device. A 3-phase framework is presented to consider (a) pre-site activation, (b) first subject (or patient) enrolled until the scheduled interim analysis or primary endpoint is reached, and (c) follow-up to eventual study close-out. Depending on the nature of the clinical surveillance investigation (or clinical trial or clinical registry), each phase could be modified, but not at the risk of inhibiting the proactive collecting and evaluating clinical data. This should be shared and discussed with regulatory agencies or notified bodies before pre-site activation formally begins.

As you modify this first template, ensure that the end result continues to (a) confirm the safety and performance throughout its product lifetime, (b) identify previously unknown side-effects and monitor for them, (c) identify and analyze emerging risks based on factual evidence, (d) ensure the continued acceptability of the benefit-to-risk analysis, and (e) identify possible systematic misuse, errors, or off-label use, with a view of verifying that the intended purpose (or use) is correct. Then, move to modify the next template.

Not all clinical surveillance trials are built the same. Factors affecting this phase (and budgets) would be the (a) question(s) being asked, (b) risk class of the device, (c) numbers of subjects (or patients), (d) number of sites involved, (e) countries where the sites are located, and (f) degree of monitoring. Depending on the inclusion criteria for subject (or patient) selection, it is valuable to regularly review site screen failure logs, if available, and their site protocol deviations. Insights into those two activities could be used, in part, to modify the protocol and downstream processes to ensure successful enrollment.

As the data is being analyzed, confidence intervals and standard error bars are an excellent way to show uncertainties with the clinical data. These data could be used, in part, to update the PMS plan strategy and subsequent PMCF plan, for example. These results are also beneficial because they can be used, in part, to develop future questions to be answered and serve as the basis to test new hypotheses in the future. It’s okay to say, “This is what we thought initially. This is what we know now.”

The decision to close out a clinical investigation (clinical trial or clinical study), should be discussed with the regulatory body or notified body. Be prepared to justify the rationale with evidence (data). Four potential justifications are provided below; however, others exist, too.

A Curious Customer Peeks In

Several at lunch today wanted Kay-Lin to attend their product team meetings and share her PMS Plan experience, having worked with several regulatory agencies and notified bodies. After briefly exchanging business cards, they paid and left the restaurant. “We need to do this more often. I appreciate you, Kai-Lin.” A few moments later, that previous curious customer walked out imperceptibly, too, momentarily pausing to survey, peeking in on the scribbled napkins left at their table. A business card also was discarded and left behind. “Hmm.” The customer quietly left the Peking Inn café.

Conclusion

Kai-Lin learned how to accept disappointment without being defeated. She did attend several product team meetings and assisted others in writing up robust PMS plans, most requiring PMCF plans, many of which required a clinical surveillance investigation, too.

The world does not end at the city limits of an email. Others have this same challenge, opportunity, or dilemma. It is not the time to be known as a hoarder of knowledge. Instead, embrace transparency and share. There naturally will come additional opportunities to learn and grow your experience that would have escaped you had you not been open to communicating. This behavior will inspire positive change in both your company and your leadership style. It has the potential to have a positive ripple effect that, in some cases, is unimaginable.

Sometimes, all you need to hear is, “I appreciate you.” Three straightforward, unsophisticated words that motivate, replenish, and encourage, even when your name doesn’t make the approval announcement email. Given enough time, wounds would diminish as your reputation and influence grow in unforeseen, astonishing ways. Hmm.

David R Rutledge, Pharm.D., FCCP, FAHA, President & CEO, Global Strategic Solutions, LLC, Silicon Valley in California. david.rutledge@globalstrategicsolutions.com . +1 (630) 846-0350 cell. The homepage is www.globalstrategicsolutions.com.

If you need help creating PMS or PMCF plans, contact me today.

Because the landscape is evolving, finding those treasured candidates is challenging partly because competent authorities, notified bodies, manufacturers, and consulting groups are all seeking similarly qualified candidates. The talent pool is shrinking. Organizations are all transforming, either searching outside their ranks to find those hidden, veiled gems, or shifting those employees within their organizations into new roles or positions. The hope is that they will transform to be (i) competitive, (ii) highly effective, and (iii) aligned with changing business strategies. Thus, it becomes crucial to reassess current teams to identify specific qualifications that are needed for upcoming challenges and opportunities.

The purpose of this article is to highlight the importance of visualizing where your team is now, making future decisions based on robust, easy to understand information and then linking their qualifications to quality system documents that become targets during audits or inspections. How do you currently answer this simple question, “What qualifies that person to do their job?” A straightforward, fun, and clever solution is presented.

The ‘To-Do’ List Grows

The message from her manager said, “I need your help. Can you stop by?” Kai-Lin, an employee at Mythical Medical (see previous articles), had just started working on her revised ‘to-do’ list, and she felt overwhelmed.

The last thing she wanted today was another ‘to-do.’ Little did she know that additional help was on its way. She eventually learned that her manager had just received approval to hire several employees who would be added to her team. In this fictional story, I’ll guide her through an active, proven process that provides a framework to organize thinking around hiring employees. The initial focus will begin by agreeing on qualifications that are needed to meet upcoming expectations, then performing an assessment of the current team and finally to highlight those qualifications and skills she needs to be looking for as she hires for the future.

Qualifications of the Team

During our first meeting, I learned that Kai-Lin did not want to repeat the mistake that happened last time, hiring the candidate that had the best face-to-face interview. This time would be different. She wanted to take a new approach. Her eventual desire was to build a talented team to meet increasing clinical, regulatory, and quality requirements. I asked her to put together a list of skills and competencies of her ideal future team and then to create three levels of criteria for each that could be used as a color code “heat map” of red, yellow, and green.

Using the potential skills list above, she selected several skills and developed relevant three-level criteria. The first one she picked was clinical investigation design, conduct, and analysis because she expects there will be more clinical investigations in the future. The criteria she created were (i) red for no direct experience, (ii) yellow for experience in < three clinical studies, and (iii) green for > than three clinical investigations. The design control process was chosen next. The criteria were (i) red for no experience, (ii) yellow for attending design control meetings, and (iii) green for presenting at design control meetings. Biostatistics was next with (i) red if only coursework experience, (ii) yellow if experience included interpreting stat tables for a report, and (iii) green if they participated in the development of statistical analysis plans. After going through the rest of her list, a ‘heat map’ was developed and applied to the six current employees and their administrative assistant. This map immediately and clearly showed the capabilities of the current team and where gaps exist. These results helped identify competencies and skills to target during the upcoming hiring process. It even provides insights into the cross-functional team.

Skills Assessment Visual

Organizations are uniquely built and require serious reflection to identify the qualifications and skills needed to perform the job as a team. This tool is flexible to create more relevant skills if these do not apply. There is value in having this information, in part, because you discover immediately where strengths are. For example, who is the strongest employee on the current team? What criteria did you use? If you based it off of core skills you get one answer but using another lens or perspective, you will gain insights into another employee. What two core skills do the team need to focus on during this hiring process? What three skills are the weakest on the current team? What are the overall strengths and vulnerabilities of the current team? How can the administrative assistant be leveraged to improve the team? Are there opportunities for professional development?

How do internal stakeholders complement the team? Which stakeholder might you consider interviewing? There may be value in promoting employees internally within the organization. Sometimes an internal candidate gets considered when an employee is deserving of a promotion that supports their professional development. Using this tool, you can see how they might affect the overall qualifications of the team. It’s often more cost-effective to train internal talent.

There is also something to be said for the institutional knowledge that a person brings to the role, not to mention established relationships with other stakeholders within the organization that might make for a much more seamless transition. One of the great strengths of many organizations is the depth and breadth of talent. Companies sometimes aim to develop their leaders internally, so it is not only appropriate that Mythical Medical be able to fill these new positions from within, but it is also proof of a successful human resources function.

What to Present to the Auditor (or Inspector)?

Qualifications should be documented in their curriculum vitae (CV) and should align with their job description (which might need to be updated), work instructions, and training records. It needs to be clear how this is all linked together. When employees earn a certificate for CAPA training or design control, for example, ensure training records are updated and added to their CV, too. The goal would be to use their education, training, skills, and experience to build the story about their competency. Now you need to link processes with people.

Process documents (SOPs) will need to be revised to incorporate a new section that addresses the skills or competencies required within the process. List the function responsible and skills required using a table format. Then, when you write the report that will become part of the technical documentation, state employees’ names along with the skills or competencies they will need to perform the job, ensuring it aligns with process documents. Sometimes more than one person will be listed to fulfill the qualifications of the role. The use of tables works great here, too.

Finally, it is common for auditors/inspectors to interview employees performing the work and their manager. As you prepare them, ensure they storyboard around education, training, skills, and experience. Highlight how their processes align with their qualifications. This framework or structure allows for easy, brilliant, and factual storyboarding when managers are asked the question, “What qualifies your employees to do their job?”

Closing Thoughts

Although there are hiring frustrations in general, assessing potential candidates during a health crisis like the coronavirus pandemic only raises those frustrations. Adding to this is that auditors (or inspectors) have renewed their efforts to assess employee qualifications.

Some have said that hiring employees is one of the most important things leaders do. Yet, your fulltime job remains while another “to-do” is added to your list. When Kai-Lin first contacted me, she informed me her initial goal was to hire them as quickly as possible. There were audits to prepare for, new submissions to be created, and responses to be written from previous questions received from their Notified Body. She was also expecting to hear back from a Competent Authority next month, and those responses would also need to be turned around quickly to meet an internal company milestone. Plus, she had two trainings due by the end of the month. Nevertheless, using this tool helped her to screen candidates efficiently and then target questions focusing on skills and competencies to do the job. Although one candidate had travel restrictions due to the health crisis and could not participate in a face-to-face interview, Kai-Lin and the interview team confidently conducted a virtual interview, focusing, in part, on the competencies and skills she had previously identified. Using this method, she found it was a more effective process, and it prevented someone from being hired simply because they “interviewed well.”

Renewing a focus on candidate qualifications can assist hiring managers to be more efficient and will also support online, virtual interviewing during a health crisis without jeopardizing your team. It also sets you up for success during audits (or inspections) so you can leave those closing meetings with smiles on your faces. Hiring the right person is hard work, but understanding the skills you need within your team just got easier.

David R Rutledge, Pharm.D., FCCP, FAHA, President & CEO, Global Strategic Solutions, LLC, Silicon Valley in California.

david.rutledge@globalstrategicsolutions.com +1 (630) 846-0350 cell. The homepage is www.globalstrategicsolutions.com.

Please contact me if you are facing hiring challenges too. As the talent pool is shrinking, it is critical to act fast even during times of global health pandemics.

Notified Bodies, manufacturers, and regulators have leaders. Communication among these entities can be tough and complicated. Great leaders within these organizations sometimes only desire three things: (i) predictability, (ii) approvals on time, and (iii) quality submissions.

These represent shared common aspirations that provide a platform for mutual collaboration. Yet, there must be a way to structure a program to facilitate communications using metrics. At its heart will be the need for agreed-upon data. Product submission metrics did not exist at Mythical Medical. People should be brought together to take a deep dive into these processes and then agree on the future, immediate next steps.

Ideas Are Needed

Ideas are the natural-born enemy of the way things are. Kai-Lin, an employee at Mythical Medical, is unaware she is on a journey to greatness (see previous articles). What she is keenly aware of is that serious communication challenges exist between Mythical Medical and regulators as well as their Notified Body. She and her team are in unfamiliar territory.

Improvement in working relationships with their Notified Body will take priority now due, in part, to the increased number of planned product submissions.

In this fictional story, I’ll guide Kai-Lin through a process to generate ideas that can be used to create and implement a communication strategy. I will be addressing relevant topics and providing simple, practical solutions. These principles can be implemented globally with regulatory agencies, too.

 A Benchmark Is Needed

Previously we learned Kai-Lin was all about making decisions based on evidence. She demonstrated that by using surveys to acquire specific sufficient clinical evidence (safety and performance data, in this case, to fill in a clinical evidence gap) and later overseeing the creation of a framework to institute a benefit-to-risk process. Her reputation continues to grow with regulators and Notified Bodies all over the world. She and her team are respected. However now, she is lacking data again.

During our first meeting, I asked Kai-Lin, “How are discussions going between Mythical Medical and your Notified Body?” She replied, “Some describe it as challenging.” We discussed why that was, and it became clear that pieces of information, data that could be available to assist with communication were not known. Without data, there was no benchmark to measure it against. Without a benchmark, key performance indicators (KPIs) could not be developed. Thus, the first initiative or proposal would be to gather data using metrics, data that had to be agreed upon by her team, and their Notified Body. Kai-Lin, like many others, needed to discover those little-known knowns.

Agreeing on the Data to Collect

Most of the world make decisions by either guessing or using their gut. Having data to establish an initial benchmark is not only a useful tool for communication, but it is also a competitive advantage. The data collected initially will change or shift over time. Some call this data swirl.

Carve out time to bring important stakeholders together for a robust discussion. Agree on what the current pain-points are. Use this information, in part, to agree on the final data to collect. A key here is to filter out ideas of what could be measured to what should be measured. Although this will take some time and a little more effort, the desire to gain efficient product approvals will numb the pain it takes to get you there.

1. Little-Known Knowns: Predictability

Mythical Medical should work with their Notified Body on agreed-upon metrics related to predictability. Here are four metrics to better understand factors that affect predictability: (i) Number of pre-notifications provided versus the total numbers of submissions (percentage), (ii) Percentage of pre-notifications provided shorter than eight weeks before submission, (iii) Number of project cancellations versus total projects (percentage), and (iv) Number of postponed projects versus total projects (percentage). The first two metrics imply that early communication is essential. It can make or break a medical device’s successful journey to the market.

Depending on the software or analytical capabilities of the Notified Body, these four metrics could be modified, or others could be added. The point is having this type of information can be useful both to the Notified Body and the manufacturer. It makes for easy graphing, and it can be tracked over time.

2. Little-Known Knowns: Approval Timelines

In business, approvals coming ahead of or later than schedule cause both upstream and downstream manufacturing challenges for people, processes, production, and eventual implementation of launch strategies. Thus, the best-case scenario is to plan thoroughly and have your product approvals hit the targeted date.

These three metrics should be tracked: (i) Number of delayed projects (not postponed projects), (ii) Within project milestone dates, e.g., submission date, 1st Notified Body review round, 1st responses by the manufacturer, 2nd Notified Body review round, 2nd responses by the manufacturer, number of additional review rounds, certificate date related to review closing data, and (iii) Overall project timelines or planned versus actual achieved date.

Data within these little-known knowns can be used to understand better and then characterize specific issues related to submission processes at both organizations. It is okay that you might find embarrassing or frustrating elements. Those are better known than unknown.

Project managers will find information-rich data here. Evidence from these metrics should be segmented to analyze details within individual projects. Find trends in what works and what doesn’t work. Action items can be developed that will have positive effects on all future submissions.

3. Little-Known Knowns: Quality

If you are struggling with quality metrics during the submission and approval process with your Notified Body, here are five simple metrics to start with: (i) Number of questions per pre-notification, (ii) Number of screening deficiencies raised during administrative quality checks, (iii) Number (average/median) of review rounds per project, (iv) Number (average/median) of deficiencies per project or review module, (v) Number of post-approval activities, follow-up items, and certificate limitations.

You get what you measure. What each organization finds will eventually change who they will become in the future.

These metrics will create an initial understanding of the quality of your submissions. Otherwise, how would you know where challenges exist and who should be accountable?

A nice outcome here is that you can update your submission checklist and train employees to minimize these delays. You can look at these metrics per project team or product class.

Closing Thoughts

When communicating with Notified Bodies, manufacturers, and regulators, the single biggest conundrum is the illusion that communication has taken place. If you want to examine this conundrum, you must have metrics like these. Without them, your competitors will eat your mistakes. Therefore, it’s time to discover those little-known knowns.

Become a great leader within your organization and implement metrics that address: (i) predictability, (ii) approvals on time, and (iii) quality submissions. If you and your Notified Body are struggling to create these metrics, work together to agree to begin somewhere. Metrics will evolve or even swirl over time. That is the natural evolution as data matures or is analyzed. With this evidence, benchmarks can be created. Then, KPIs can be developed to report on progress against the desired result.

This approach seems simple on the surface. However, to manage Kai-Lin’s expectations, I jokingly informed her, “Half the people will love the data, half will hate the data, and half won’t understand it.” However, in time, jewels of truth and pearls of understanding will emerge. Someone has said, “Listen to the experienced people in your life–not because they’re always right, but because they have a better understanding of being wrong.”

To pursue excellence, it takes relentless effort. If you want to go beyond what’s expected, you must evolve and grow too. Little did she know (and those like her), that implementing these metrics would take her down a path to become a fantastic leader. She is on a journey to greatness.

Metrics like these are relatively new for most organizations, but with changes in product approval and post-approval regulations around the world, deadlines are rapidly approaching. It’s time to collect this data and have discussions about future submissions. Some will say it’s too early until, unfortunately, it’s suddenly too late.

The author would like to thank Bassil Akra of TÜV SÜD.

David R Rutledge, Pharm.D., FCCP, FAHA, President & CEO, Global Strategic Solutions, LLC, Silicon Valley in California. david.rutledge@globalstrategicsolutions.com +1 (630) 846-0350 cell. The homepage is www.globalstrategicsolutions.com. 

Benefit-to-Risk Analyses

Some things make sense. They are easy, balanced, symmetrical, and predictable. That’s why Jae, an employee at Mythical Medical, liked mathematics, engineering, chemical reactions, and physics. Rules or laws are in place, and, for the most part, you follow them. Equations made sense, and he loved them.

He also joined a debate club because structuring logical arguments seemed to come naturally, too. That skill would come in handy at work. Regulations and laws, he would discover, were his friends too. They were so “black and white,” he thought.

Regulatory Affairs seemed to be the perfect fit to begin his professional career in the industry. Little did he know that a simple request would soon introduce him to a new world of “grey” within Regulatory Affairs.

In this fictional story, I’ll guide Jae through an effective process that provides a framework to organize thinking to craft a benefit-to-risk assessment. I will be addressing relevant topics and providing simple, practical solutions. These principles can be implemented globally.

The Simple Request

Kai-Lin, also an employee at Mythical Medical, asked Jae for help in creating a benefit-to-risk analysis (evaluation or assessment) for the company’s next product submission.

Previously (see previous articles) we learned Kai-Lin was all about making decisions based on evidence. Her reputation precedes her now because of her early work using surveys to create safety and performance data. Regulators and Notified Bodies all over the world respect her.

Her simple request today would soon change Jae’s Regulatory Affairs’ world forever. Things are not so simple as thinking in extremes like “black and white.” This perspective is dichotomous or polarized thinking.

Potential Benefit Versus Potential Risk

Jae initially set up two columns to guide his work on creating a benefit-to-risk analysis. This approach or method had worked so many times in the past. After a few hours, working alone, he had a wonderful, but small, limited number of items in the benefit column, but a massive, extensive list under the risk column. The two columns were not going to be easy to use this time.

After Jae contacted me, I learned that he was right. The exercise was not going to be like solving a math equation. I asked him to record the units of measurement out for every item listed under both columns. That’s when Jae realized that the units of measurement made the benefit-to-risk equation unsolvable. This really was like comparing apples to oranges. It is not possible today to use math to balance the benefits against risks objectively. There were other reasons for his frustrations, too, such as the frequency of events, severity, probability of having the event, the potential of having multiple events, and their duration.

Bring A Team Together

Jae did a fine job with his initial list. However, I advised him that to perform this analysis, others need to be involved. He should include those with clinical expertise such as seen within clinical research (clinical science), clinical evaluation, medical affairs, quality, risk management including product/clinical/software/cybersecurity representation, regulatory affairs, engineering, biostatistics, complaint handling, and safety.

This team should use all available clinical data, including results from clinical investigations or registries, hospital databases, usability, servicing records, CAPAs, customer/patient feedback, literature, etc.

Discuss Potential Benefits: Address these 12 issues

ISO14971:2019 defines benefit as having a positive impact or desired outcome of the use of a medical device on the health of an individual, or a positive impact on patient management or public health. Benefits can include a positive impact on clinical outcomes, the patient’s quality of life, outcomes related to diagnosis, positive impact from diagnostic devices on clinical outcomes, or positive impact on public health.

Identify Potential Medical Device Benefits

To begin the discussion of the potential benefits with Jae’s team, I suggested using a framework that includes addressing these 12 issues if they apply: (1) patient perspective, such as quality of life from a validated tool, (2) healthcare professionals and caregivers perspectives, (3) medical necessity, e.g., What are the other choices to patients/health care practitioners that would make this product a benefit?, (4) types of benefits (remember that claims should automatically be considered types of benefits), (5) magnitude of benefits in terms of time or scale so a metric can be used here, (6) likelihood or probability of experiencing one or more benefits, (7) reduction in the probability of death, (8) aiding improvement in patient function such as an organ system, (9) reducing the probability of loss of a function, (10) relief of symptoms, (11) duration of benefits in terms of curative, short-term, or long-term, and (12) benefits when compared to alternative therapies or state-of-the-art. This framework to organize thinking to craft a benefit-to-risk assessment allows Jae’s team to assemble evidence from one side of the equation, potential benefits that need to be considered in this analysis.

Now, with this wealth of information, Jae’s team should use these seven, broad benefit assessment criteria and then create narratives for each: (1) Types of benefits, (2) Magnitude of benefits, (3) Likelihood of experiencing one or more benefits, (4) Duration of effects, (5) Patients’ perspective on benefit, (6) Benefit factors for health care professionals or caregivers, and (7) Medical necessity compared to what is currently available, for example. If assumptions are being made, make them clear to the team.

Thus, benefits can range from how the patient feels or functions to how they survive. In some cases, there could be an acceptable surrogate outcome parameter that improves. Additionally, benefits could be related to the convenience of managing or diagnosing a disease or condition. Although manufacturers, in general, do not have a lot of experience documenting benefits, using this framework, becomes a fun activity.

Finally, uncertainties in benefits need to be addressed and might be related to sensitivity, specificity, accuracy, precision, reproducibility, wide confidence intervals, underpowered studies, missing data, or the impact of confounding interventions or physiological or pathophysiological factors.

Discuss Potential Risks: Address these 12 issues

According to ISO14971:2019, it is accepted that the concept of risk has two key components: (1) the probability of occurrence of harm and (2) the consequences of that harm, such as how severe it might be.

To begin the discussion of the potential risks with Jae’s team, I suggested using a framework that includes addressing these 12 issues, if they apply: (1) It is the patient who takes the risk for the promise of the potential benefit. How well does the patient understand the risk from the product?, (2) patient tolerance (or intolerance) to the product over time, (3) risks to healthcare professionals and caregivers, (4) severity of risks, (5) type of risks, such as product, clinical, software, and cybersecurity, (6) likelihood of experiencing one or more risks, (7) duration of exposure, (8) mitigation potential such as quantitative and qualitative risk control methods, (9) procedure-related versus device-related risks, (10) disease characteristics that could affect risks, (11) quantitative or qualitative residual risk estimates, and (12) risks from false-positive or false-negative results. The discussion should also include how risks might be mitigated or what measures can be in place to lower risks. The key here is to have the team agree on identifying residual risks. This framework to organize thinking to craft a benefit-to-risk assessment allows Jae’s team to assemble evidence from the other side of the equation, potential risks that will be considered in this analysis.

Now, with this information, use these six, broad risk assessment criteria and, once again, create narratives for each: (1) Types of harms or risks, (2) Magnitude or severity of harms or risks, (3) Likelihood of experiencing one or more harms or risks, (4) Duration of exposure to the population, (5) Patients’ perspective or tolerance (or intolerance) to harms or risks, and (6) False-positive or false-negative results. Again, if assumptions are being made, make them clear to the team.

Finally, uncertainties in risks need to be addressed and might be related to the product, procedure, subsequent treatments or tests, insufficient number of patients, differences in definitions, percentage of subjects that were lost-to-follow-up during a clinical investigation, protocol deviations, and user experience such as being inconsistent or not representative of likely real-world users.

Tell the Story

Is the overall residual risk acceptable in relation to potential benefits? Use the information gathered to identify potential benefits and potential risks and tell the story. When you put your mind to it, there are a lot of tools or potential sources of evidence in the benefit-to-risk toolbox that should be used depending on the risk class of the product. The crucial next step is to pull this all together to create a factual and evidence-based story that reveals how the potential benefits of the product outweigh potential risks. Knowing the uncertainties mentioned above could provide the manufacturer with information to develop and conduct prospective, post-market clinical (or performance) follow-up studies.

Regulatory bodies, as well as Notified Bodies, expect all relevant models, sizes, and device variants, to be covered in the benefit-to-risk analysis.

If the evidence does not support the conclusion that potential benefits outweigh residual risks, then the manufacturer should consider modifying the medical device, its intended use, or the target population. Perhaps other controls need to be in place.

Benefit-to-risk assessments are living analyses. They begin with justifying first-in-human studies. Next, they are revised to obtain approval to conduct pivotal investigations. Finally, assessments are continuously updated post-approval throughout the life cycle or lifetime of the device.

Medical therapies are not static and state-of-the-art changes. Benefit and risk perceptions change over time. Thus, benefit-to-risk analyses should be viewed as living systems that are to be revised regularly when new data or devices become available.

There are recent guidance documents and an ISO standard that you will find helpful as you apply this framework to organize thinking to craft benefit-to-risk assessments for your product:

(1)    ISO14971:2019 Medical devices – Application of risk management to medical devices. https://www.iso.org/obp/ui#iso:std:iso:14971:ed-3:v1:en https://www.iso.org/standard/72704.html

(2)    FDA Guidance “Factors to Consider When Making Benefit-Risk Determinations in Medical Device Premarket Approval and De Novo Classifications” issued August 30, 2019. It contains worksheets and examples. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/factors-consider-when-making-benefit-risk-determinations-medical-device-premarket-approval-and-de

(3)    FDA Guidance “Consideration of Uncertainty in Making Benefit-Risk Determinations in Medical Device Premarket Approvals, De Novo Classifications, and Humanitarian Device Exceptions” issued August 30, 2019. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/consideration-uncertainty-making-benefit-risk-determinations-medical-device-premarket-approvals-de

A textbook I would recommend is Safety Risk Management for Medical Devices by Bijan Elahi ISBN 978-0-12-813098-8 by Elsevier Publishing under the label of Academic Press 2018. https://www.elsevier.com/books/safety-risk-management-for-medical-devices/elahi/978-0-12-813098-8

Podcast on ways to collect Sufficient Clinical Evidence: https://www.youtube.com/watch?v=anQRz2weaqo

Podcast on the Benefit to Risk Analysis: https://www.youtube.com/watch?v=lwkTgNVnt9A

Closing Thoughts

Jae did create a benefit-to-risk assessment for Mythical Medical’s next product submission. He pulled a broad team together and led meetings using a logical, transparent framework. The results were presented for internal sign-off and were based on evidence that Kai-Lin required and not only on the belief that the product’s benefits outweighed its risks. Jae learned that benefit-to-risk-thinking is not always easy, balanced, symmetrical, or predictable.

Until manufacturers can mathematize benefit-to-risk analyses, the framework presented here can organize your thinking and allows immediate actions that your team can take and use now. It does require the use of “grey matter” (or your brain), but the outcomes are rewarding. It is flexible to use for all product risk classes.

David R Rutledge, Pharm.D., FCCP, FAHA, President & CEO, Global Strategic Solutions, LLC, Silicon Valley in California. +1 (630) 846-0350 cell.

Disclaimer

This is just a bit of fun. The purpose of articles like these is for the free exchange of questions, ideas, or comments regarding issues of global interest involving medical regulations and is not to be considered legal advice. These are just opinions and not advice; opinions may change over time, too. Use it at your own risk. This disclaimer will occasionally be posted or updated throughout the year.