The medical device industry is undergoing a paradigm shift as Artificial Intelligence (AI) and Machine Learning (ML) transition from novelties into heavily regulated realities. The turning point arrived when the FDA integrated its own internal AI tool, Elsa, into its scientific review and inspection targeting processes. With regulators actively leveraging the technology, MedTech companies can no longer treat AI as a buzzword; it demands a deep understanding of concrete regulatory frameworks and actual engineering rules.
To properly understand this evolution, the traditional internet analogy must be cast aside in favor of a more accurate comparison: electricity. Just as the adoption of electricity brought a wave of safety infrastructure, inspectors, and the National Electrical Code, AI is bringing an imminent mountain of new standards to the medical device landscape. Winning device companies will not be those that market themselves as "AI companies," but rather those whose devices simply perform better because of the technology and whose quality systems can explicitly prove that enhanced performance to regulators.
Navigating this terrain requires mastering fundamental regulatory concepts, beginning with Software as a Medical Device (SaMD) pathways and the distinction between locked and adaptive algorithms. Because adaptive algorithms continuously change in the field, they present a unique regulatory challenge that requires a total product lifecycle approach. By utilizing a Predetermined Change Control Plan (PCCP) and integrating proactive post-market surveillance directly into the Quality Management System (QMS), manufacturers can successfully clear these checkpoints and avoid costly deficiency letters.
Love this episode? Leave a review on iTunes!
Have suggestions or topics you’d like to hear about? Email us at podcast@greenlight.guru.
SaMD is software designed to perform medical functions—such as diagnosing, treating, or monitoring diseases—without being part of physical medical device hardware.
A Locked Algorithm is an AI model that remains completely unchanged after it is cleared and deployed. It performs its function exactly the same way every time until the manufacturer manually pushes a controlled update. An Adaptive Algorithm is an AI model that continues to learn, retrain, and evolve on its own based on new real-world patient data after it is deployed.
"The device companies that are going to win aren't the ones making the biggest deal out of having AI. They're the ones whose devices actually work better because of it and whose quality systems can prove that to the FDA." - Etienne Nichols
"With AI, clearance is more of a checkpoint. You're going to have multiple of these checkpoints along the way." - Etienne Nichols
We want to hear from you. Did this episode change how you look at your company's AI pipeline? Do you have questions about implementing a PCCP or structuring your design controls for machine learning?
We read every single message and love delivering personalized responses to our community. Send your thoughts, feedback, reviews, or topic suggestions for future episodes directly to our team at podcast@greenlight.guru.
This episode of the Global Medical Device Podcast is brought to you by Greenlight Guru.
Navigating the complex landscape of AI/ML regulations requires an airtight quality foundation. Greenlight Guru provides specialized Medical Device Success Platforms that unify your team’s efforts. By utilizing their dedicated QMS (Quality Management System) solutions, you can seamlessly build AI-specific design controls and map out risk management strategies under ISO 14971. Furthermore, their integrated EDC (Electronic Data Capture) solutions allow you to execute rigorous clinical data collection, helping you gather the high-quality, traceable real-world performance data required to monitor algorithm drift and satisfy total product lifecycle demands.
Discover how to scale your AI enabled innovation safely by visiting www.greenlight.guru.
Etienne Nichols: Hey guys, a few years ago when I first started paying attention to AI, it was really interesting. It was a novelty. I was fascinated by it, but also mostly amused by it.
It was very interesting. It was fun. You can make all kinds of different videos or pictures mostly. I think I first made my first AI video in 2020, late 2022, maybe early 2023. And it was hilarious. It was a picture of my face and my mouth moved to right read a script that I provided with it, provided it well.
Pretty quickly after that, people started feeling kind of AI fatigue. They weren't really.
I mean, it was pretty much dismissed as a toy. After a while people were so sick about hearing about AI that just mentioning the word and a headline was enough to make everybody scroll past it.
We've probably gotten back to that point again as well. I got pretty sick of hearing about it myself as well. But something has happened that really we can't afford to ignore.
When the FDA started incorporating AI into their own scientific review process, they started using it internally. If you remember, it's called Elsa. The moment that happened, the moment the regulators started using that, that binge worthy word, the industry couldn't pretend that it wasn't real anymore. Suddenly AI was everywhere in MedTech. Companies are building it into their devices.
QMS platforms are adding AI features, regulatory submissions are all being built by AI. People are talking about this in all different ways. And I kept hearing the same questions from people in the industry.
What does this actually mean for us? What does the FDA actually think about AI in medical devices? What are the rules?
And that's what today's about. Not the hype, not the things that you know.
But I want to talk about the actual rules related to AI in MedTech.
Etienne Nichols: Before we get into the actual guidance documents, I want to give you an idea or a framework or a way to think about this because I think most people are reaching for the wrong comparison when they try to understand what AI is or what AI is doing to our industry.
The medical device industry people say AI is like the Internet and I get why the Internet changed everything about how business worked, but I don't think that's the right analogy. In fact, if you think about AI, from maybe if you're in marketing or something like that, it does feel very similar with the Internet and how business changed But I think it's deeper than that.
I think AI is more like electricity because. Well, I'll just tell you what I mean. Before electricity, people used candles and lanterns to light their homes. And those people understood the light sources completely.
If your flame was weak, you maybe you trimmed the wick.
If the lantern was running low, maybe you needed to add some kerosene or fuel. You could diagnose every problem yourself. You were the expert.
Now I've built an off-grid house that I live in. I wired it all myself.
So that's why this analogy came to me is so poignant for me personally, because after the, the candles and the lanterns and that era of life, electricity came along and it was better in almost every way.
It was easier, faster, cleaner, brighter. There was way less risk of a fire.
You didn't have to trim those wicks before you went to bed or when you were waking up from bed. But here's the thing.
Etienne Nichols: Most people had no idea what was going on inside their walls. Those wires, the breaker box outside the transformer down the street, the whole network that had to work exactly right before that switch in your bedroom did anything. And if something went wrong, you didn't troubleshoot it yourself. You didn't trim the wick on your wires. You called an electrician. You had the National Electrical Code, the NEC, which I've actually now read quite a bit of in wiring my own house.
But most people are not doing that. Most people are hiring that electrician. So now we have inspectors, we have permits, we have certifications, we have journeymen who have studied under other people for years before they start doing this for other people's house.
But we didn't give, we didn't give up electricity because it was dangerous.
We did it because electricity, we stuck with it because electricity was powerful enough to matter.
And when something matters that much, you build standards around it. And so I would submit to you that that is where AI is right now. We're on the precipice of a mountain of standards and maybe a centralizing that information into that subset of people, the electricians of AI if you will.
Here's the other part of the analogy that I think gets missed.
Etienne Nichols: Think about it this way. If you were a toaster company in the early days of electricity, you used to make toast by heating it over a grill, perhaps some kind of direct flame situation. And, and then you switch to electric coils. You didn't go around telling everyone, we're an electric company, now we're an electric toaster company.
That wasn't the point. The point was that you can make better toast. You can make it faster; you can make it more consistent. You could plug it in and go.
The electricity was just the thing that made it work.
Your customers didn't care about the electricity. They only cared about the toast.
So that's the other thing you need to think about AI, artificial intelligence in medical devices is the same.
The device companies that are going to win aren't the ones making the biggest deal out of having AI. They're the ones whose devices actually work better because of it and whose quality systems can prove that to the FDA.
So that brings us to what the FDA is actually asking for. That's what we care about. Right?
Well, let's start at the foundation, because if you don't understand the framework the FDA is thinking in, the specific guidance documents, they're not going to make a lot of sense.
So, first thing I to understand is that the FDA has a category called software as a medical device. Samd, not SAMDA or SAMD. Let's just call it SAMD. Okay, if you have a different way of pronouncing it, hit me up.
And I know this is a little bit simplistic for some of you. Stay with me here for a minute. But if your AI is doing anything that falls under the definition of a medical device, whether it's analyzing patient data, supporting clinical decisions, and I know we have clinical support decisions or clinical decision support software, and a recent guidance about that, we talk about that at another time.
But detecting conditions, a lot of these are probably SAMD. That matters because SAMD has a full regulatory pathway. It's not an accessory; it's not a feature.
It is a device.
Etienne Nichols: All right, let's talk about locked versus adaptive algorithms.
This is another distinction that's. I think it's important for everything else we're going to talk about today.
A locked algorithm doesn't change once it's deployed. You trained it, you validated it, it performs a defined function, and it stays that way until you deliberately update it through a controlled process.
You decide how it's going to be able to change.
An adaptive algorithm continues to learn from new data after deployment. It changes on its own, which makes total sense. And it's. It's really where the power of AI can actually be harnessed.
The FDA has been pretty clear. Adaptive algorithms are fundamentally they are a different regulatory challenge because the thing you validated is not going to be the thing that's running in the field six months later. So, if you're thinking, well, wait, that means adaptive AI is impossible to clear.
Stay with me because that's where the next thing comes in. That's where PCCP, Pre-determined Change Control Planning comes in.
Etienne Nichols: All right, so let's think about the total product lifecycle approach. FDA's approach to AI isn't submit it once and you're done. It's more of a life cycle model. They want to see that you're monitoring the algorithm's performance after-market release, that you have a plan for what happens when it drifts, if it drifts, that your quality system connects the dots from the development all the way through to the post market surveillance.
And this is the biggest mental shift for companies that are used to thinking about FDA clearance as a finish line with AI clearance is more of a checkpoint. You're going to have multiple of these checkpoints along the way.
Okay, so what are some of the guidances that are important to read? We get to walk through some of those actual documents. The 2021 AI ML Action Plan is one we're going to consider.
Now this is a little bit older, but the FDA published an action plan in 2021 that laid out five specific areas they were going to develop guidance on. It's worth reading even now because it tells you, it tells you where the FDA's head was and it explains why the guidance that came after it looks the way that it does. So, the five areas were software modification, the predetermined change control plan, good machine learning practices, patient centered approaches, transparency, and real-world performance monitoring. Notice that most of these are about the process, not the algorithm itself, which I think is intentional. The FDA wasn't trying to become an AI engineer or a source of AI engineering.
They're trying to make sure your engineering process is sound.
Now if you remember in 2023 there, there was a guidance on Pre-determined Change Control Plans, PCCP. It was originally just for AI ML.
Later on, it came out just for a more of a generic PCCP.
And if you, if you the reason this came out the way that it did.
Etienne Nichols: Well, I don't want to get into the Omnibus Act necessarily, but it was, there's is actually in the code of federal regulations. So, there was an update in the Omnibus Act.
I think it was December of 2022.
But this is one that most companies haven't really.
I would, I would submit to you that most companies probably have not fully leveraged the PCCP. And it's one that really is important for AI ML. So, a Pre-determined Change Control Plan is exactly what it sounds like it's a document you submit to the FDA that says here are the types of changes we anticipate making to this algorithm after clearance.
Here's our protocol for making these changes.
Here's how we'll know if the performance is still acceptable. So, the FDA's guidance on the PCCPs cover three main required components. One is the description of the, of the planned modifications.
What kind of changes do you actually expect?
Is it going to be more training data? Is it an expanded intended use?
Are there going to be algorithm architecture updates? You have to be specific about the type of changes you're, you're planning.
Not, not every possible change in the universe, but the actual categories of modifications your development team is likely to make.
Etienne Nichols: Second is the modification protocol. How are these changes going to be implicit implemented? What are the steps the validation is going to require? Who's going to approve those?
This is basically a change control procedure specifically for your algorithm.
And then third is the impact assessment for each plan modification type. What's the expected impact on device performance and safety? What testing would you do to confirm it's still safe and effective?
What is the impact? I mean it's as straightforward as that.
And the failure mode that you'll probably see more often than not is that companies are going to treat that PCCP as a submission document, something you write right before you file. But the PCCP actually has to reflect your actual development process. If your development process is built around it from the beginning, the FDA is going to be able to tell you almost need to treat this like its own standalone submission that is a part of the submission.
So, you need a full understanding of all these different processes.
This is the 21-2020-21 draft guidance that are. It's called the 510(k) considerations for AI ML based SAMD. Few things worth highlighting. Training, validation and testing data the FDA wanted to understand where's your data coming from?
Etienne Nichols: What population does it represent?
How was it split between training and validation and testing? Is it representative of your intended use population? Whether it's geographically, demographically, clinically?
The last one's important as well. There have been additional guidance documents that have come out about the demographics and the clinical importance of your medical device. It's a safety issue.
If your algorithm performs well on one demographic and doesn't on another and you don't know that you don't have any, somebody else is going to have to tell you that that's a problem.
Also, there's a something called algorithm transparency. The FDA wants enough detail to evaluate the algorithm's performance and understand its failure modes.
You can't have a proprietary blend, all right? They're not asking for your source code, but they are asking for you to explain how it works well enough so that a reviewer can assess whether or not that is safe.
Etienne Nichols: Here's where I want to bring back the electricity analogy. So, when electricity became the standard, every house needed a certain kind of wiring, right? A breaker box, grounded outlets. You know, now you actually have to have tamper proof outlets where they're very difficult to stick a the, the what, what is the thing called the.
At the end of the wire, it's hard to stick into the outlet. Now I've learned this the hard way.
The infrastructure has to be there before any of the appliances could do what they're actually supposed to do. Right. It's all have to, has to be a part of your house.
So, your quality system is that infrastructure for AI.
The companies that are going to sail through an FDA review on an AI enabled device, those are the ones that are, they've built their QMS with AI in mind from the start.
They're not the ones trying to retrofit their existing system where when it's time to file specifically, there are a few things your QMS has to be able to handle that most traditional QMS frameworks aren't going to be built for.
Okay.
Number one, design controls that account for that algorithm change before it happens, not just after the fact in your change management process.
Risk management under ISO 4971 applied to AI specific failure modes. What does a false positive mean for patient safety? What about a false negative?
What if your algorithm performs well in aggregate but fails in a specific subpopulation?
Etienne Nichols: Those are some failure modes you're going to be documenting under 14971 and they're different from a mechanical component failure. And if you're struggling with ISO 14971 and how to do all of those, I believe it's 34971.
Let me look that up actually really quick. ISO 34971 that talks about applying this to an AI or ML device.
Yes. Application of ISO 14971 to machine learning and artificial intelligence.
So, this is going to be incredibly important as you're building your QMS.
Third thing that I'll say is the post market surveillance that's actually connected to algorithm performance, not just complaint handling, which historically is what the medical device industry used. Post market surveillance or that's where they got, they waited for people to complain about it.
That's not going to be acceptable if you have an adaptive algorithm, active monitoring of how that algorithm is performing in the real world with a defined process for what happens when, not if it drifts outside those acceptable bounds.
So that's going to be important. Post market surveillance is going to be incredibly important.
Not reactive but being proactive.
And it's a different kind of quality system than most companies have built. And the earlier you build it, the less painful that whole process is going to be.
All right, here's what I want you to leave and walk away with today. AI is not going away for medical devices. It does not matter if you have AI fatigue.
Etienne Nichols: You and I both were in the same boat there.
FDI FDA is not trying to stop this. They're trying to figure out how to let it happen safely. And so the guidance documents we talked about today and there, I'm sure there are going to be more that are coming.
Those are their attempt to build the National Electrical Code for AI.
Your job as a QARA professional or as a device company or product development engineer is to understand those rules well enough that you build your device and your quality system around that national, that NEC - National Electrical Code for AI, whatever you want to call it before you're in the middle of a submission wondering why you're getting deficiency letters.
When I was building my house, this off-grid house that I've built, I have, I don't know, I think 20 or 30 breakers in the breaker box. I had to have an inspector come and inspect it.
Had I not done a little bit of research and recognize what kind of breaker I had; he would have made me change every single one of those. And he actually told me some.
He said this latest national Electrical code change has actually increased the cost of homes by 15 to 2, $1500 to $2000 just in breakers and wiring alone.
It's important to stay up to date on these things.
So next week I want to talk about the mistakes a lot of medical device companies are coming or are incurring. I suppose the patterns I see in AI submissions that trip companies up, that things that I've heard not because they have bad AI, but because they didn't build the right foundation around it.
So hopefully we'll see you then. If you found this useful, send it to one person on your team who's working on an AI enabled device right now. Send this to someone who's interested in understanding AI in the medical device industry.
Because there's more than just the AI enabled devices we're going to get to talking in the third part of this series about the processes you can use when you're using AI.
Not necessarily for an AI enabled device, but for AI in your company itself.
So, if you're interested in learning more about that, hit me up on LinkedIn. Etienne Nichols I'm very active there. I'd love to hear any thoughts or questions you might have but stay tuned for the next two episodes.
We'll see you then.
Etienne Nichols: Thanks for tuning in to the Global Medical Device Podcast. If you found value in today's conversation, please take a moment to rate, review and subscribe on your favorite podcast platform. If you've got thoughts or questions, we'd love to hear from you.
Email us at podcast@greenlight.guru.
Stay Connected for more insights into the future of MedTech innovation and if you're ready to take your product development to the next level. Visit us at www.greenlight.guru. Until next time, keep innovating and improving the quality of life.
The Global Medical Device Podcast powered by Greenlight Guru is where today's brightest minds in the medical device industry go to get their most useful and actionable insider knowledge, direct from some of the world's leading medical device experts and companies.