The FDA is actively shaping the regulatory landscape for Artificial Intelligence (AI) and Machine Learning (ML) in real time. As the agency expands its internal expertise through the Digital Health Center of Excellence, FDA reviewers are becoming highly sophisticated. The era of submitting vague algorithm descriptions is over, paving the way for a more level playing field that rewards companies executing documentation correctly.
Navigating this evolving space requires a dual-front approach for global medical device companies. Manufacturers must balance the FDA's framework with the EU AI Act, which classifies AI medical devices as high-risk systems demanding rigorous conformity assessments and human oversight. Fortunately, a robust quality management system designed around proactive frameworks, such as the Predetermined Change Control Plan (PCCP), can bridge the gap between US and international expectations.
For Quality Assurance and Regulatory Affairs (QA/RA) professionals, this shift represents an unprecedented career opportunity. The future belongs to those who combine regulatory fluency with AI literacy. Success in the MedTech industry will not belong solely to the most complex algorithm, but to the companies and professionals who build compliant, disciplined systems around their AI technologies.
Love this episode? Leave a review on iTunes!
Have suggestions or topics you’d like to hear about? Email us at podcast@greenlight.guru.
Think of overfitting like a student who memorizes the exact questions and answers on a practice exam instead of learning the underlying concepts. When they take the real test with slightly altered questions, they fail. In AI, overfitting happens when an algorithm learns the training data too perfectly, making it excellent at analyzing that specific dataset but unable to make accurate predictions on new patient data.
Imagine a GPS map app that was programmed perfectly five years ago. Over time, new roads are built, traffic patterns change, and old exits close. If the app is never updated, its navigation becomes less accurate. Algorithm drift occurs when an AI medical device becomes less effective over time because the real-world clinical environment or patient demographics shift away from the original data it was trained on.
"The companies getting in trouble aren't the ones with bad AI, they're the ones with incomplete quality systems." - Etienne Nichols
"Your job in a regulatory submission is not to demonstrate that your AI is sophisticated. Your job is to demonstrate that it's safe and effective in its intended use." - Etienne Nichols
This episode is brought to you by Greenlight Guru. Navigating the fast-moving compliance landscape for AI-enabled medical devices requires software that keeps pace with innovation. Greenlight Guru offers comprehensive Quality Management System (QMS) and Electronic Data Capture (EDC) solutions designed specifically for MedTech. By streamlining your documentation, tracking design history, and capturing robust clinical data, Greenlight Guru helps you build the rigorous quality systems required to clear regulatory hurdles globally. Learn more at www.greenlight.guru.
We want to hear from you! What are your thoughts on the future of AI regulation? Are you implementing PCCPs in your current workflows? Send your thoughts, feedback, and topic suggestions to podcast@greenlight.guru. Etienne reads and responds to emails personally, and your ideas could shape our next episode!
Etienne Nichols: Hey, everyone, it's Etienne Nichols again. Welcome back to the third part of this series on AI, this one being where FDA's AI regulation is heading, how to stay ahead of it.
So, here's the thing about AI regulation and medical devices. This is going to sound really obvious.
It's not finished.
The FDA is figuring this out in real time.
In some ways they're ahead of the industry, I think. In other ways, I think the industry is ahead of them.
And you know, there's a lot of different uncertainty right now, even at the FDA over the last several years. If you think about the different layoffs. And then we have this person in Marty Makary.
I don't, I hear different ways of pronouncing his last name. I actually read his book several years ago, so I was surprised when he was put in. I was excited.
Now he's resigned and he was put pushing for a lot of different things, whether right or wrong. You know, it's to be, to be seen. But in some ways, I think this is an opportunity if you understand, or at least maybe if you can't understand what's happening, at least understand some of the technology behind what's happening.
Etienne Nichols: So, the question isn't whether your company is going to have to deal with AI, because it's you already are in some way, on some level. Most likely, if you're listening to this podcast especially, you're probably already dealing with it in some way or some form or fashion.
So, I started this series with what the FDA says, some of their guidance documents, and last week we talked about what not to do.
And today we're going to be talking about where this is all going and what it means for your career and your company if you want to stay ahead of it instead of just constantly chasing it.
So, the FDA stood up the Digital Health Center of Excellence.
I don't know what the right acronym for that one is at the moment, but as a dedicated resource for digital health, it includes AI, ML based devices. And that's, that's an interesting signal. It means the FDA is building internal expertise specifically for this space.
Etienne Nichols: Now, what does this mean for you? Well, part of it means that the FDA reviewers assigned to AI enabled devices are getting more sophisticated. The days when a company could get away with a vague algorithm description because no one on the Review team understood it well enough to push back.
Those days are over and most of us probably understood that.
And that's good news for companies who are doing things correctly. It puts everyone on a more level playing field.
Hopefully it will help safer, more effective devices get to market.
If your device is sold outside the US and for most established device companies, most likely those are.
There's a couple different regulatory challenges because you're really navigating two fronts.
There's the EU AI act which classifies AI systems used in medical devices as high risk by definition. High risk under the EU AI act means conformity assessments, transparency requirements, human oversight provisions and post market monitoring.
And that's on top of whatever the EU MDR requirements already apply and now granted the EOMD, the EU MDR and the EU AI act, they overlapped in ways that. That's not fully sorted out yet, certainly not in my mind and I don't think in anyone's mind. But the good news is that a strong quality system built around FDA's AI recovery require will cover most of what EU regulators are asking for.
Etienne Nichols: So now obviously most of is not a very good phrase in the medical device industry because that's actually bad news. Okay. Most of is an all right. And you need to know the difference if you're filling both markets.
In 2022, FDA, Health Canada and the UK's MHRA, they published five guiding principles for AIML based SAMD. The joint statements. Those are worth reading because it shows where the global alignment is heading.
When three major regulatory bodies agree on a framework, the rest typically are likely to follow. At least that seems like a reasonable expectation, right?
And those five principles, if you think about them specifically, they're supposed to be focused and bounded. That's number one. Plan must explicitly describe the specific plan modifications in regard to PCCPs.
They must be risk based, they must be evidence based, they must be transparent and they must have a perspective of the total product lifecycle.
They have to be. The updates have to be evaluated across the entire lifespan of the product from initial design to post market monitoring.
Etienne Nichols: Okay. The IMDRF, the International Medical Device Regulators Forum, has published guidance on AIML SaMD. That's also worth knowing. Even if you think of yourself as a US only company.
Standards tend to converge towards the more rigorous standard, not the less rigorous one. So, keep that in mind.
The FDA's issued warning letters for AI enabled devices. I think we've mentioned that in, in one of the previous ones when we talked about Purolea.
Those letters are public. They're worth reading, not for the drama, but because they tell you exactly what FDA considers non-negotiable. Now it sounds like probably, that probably sounds like I'm speaking out of both sides of my mouth when I talk about this.
I don't necessarily consider that first warning letter to be AI specific, although it did mention AI.
But I do think more thing more and more is going to come out. So, it's something to consider.
One of the patterns that you can see in enforcement is it's documentation first, algorithm performance second.
The companies getting in trouble aren't the ones with bad AI, they're the ones with incomplete quality systems.
Etienne Nichols: So undocumented algorithm modifications, post market surveillance plans that exist on paper but not in practice, claims and labeling that aren't really in keeping with their clinical evidence and that pattern is useful information. It means the path isn't necessarily build better AI, it's build better systems around your AI.
Right?
All right, so I'm going to say something that, you know, most of us are probably dancing around a little bit and that is FDA reviewers are not AI engineers. And that's not a criticism. I'm not an AI engineer.
I've just been steeped in it for the last three years. It feels like it's true. It has practical implications on how you write your submissions. If you consider how the FDA reviews, what their background is, what they know about these different things, the most technically impressive AI submission in the world isn't going to help you if the reviewer can't evaluate it. Your job in a regulatory submission is not to demonstrate that your AI is sophisticated.
Your job is to demonstrate that it's safe and effective in its intended use.
And that's those two different things. Those are two different things.
If your submission is so technically dense that a reviewer has to work hard just to understand what your algorithm even does, you're creating roadblocks against yourself.
Not because the FDA reviewers aren't brilliant. Most of them probably are very, very qualified and very, very good at what they do. But the submission review is not the place to educate someone on your algorithm architecture. What works better is to explain your AI in plain English that maps directly to FDA's own framework.
Etienne Nichols: So, use their vocabulary structure, your documentation around their guidance. Make it easy for the reviewer to find every piece of information they need in order in exactly the format that they would expect it.
And the ones the people who figure this out, that language barrier issue, those are the people who are going to move faster through the review process than companies with the most sophisticated technology. But less disciplined documentation.
This is also a career observation.
The skill that is going to separate QAR professionals in the next 10 years is the combination of both AIMLs, fluency and regulatory expertise.
I don't think it's one or the other. I actually think it's both.
You're not going to need a computer science degree. You don't need that technical training necessarily. But you do need to know what a training set is. You need to know what overfitting means.
You need to know why demographic representation and data matters for patient safety and how to document an algorithm's performance in language the FDA can evaluate.
And that's a rare combination. It's not going to be rare in five years.
Etienne Nichols: But the people who are building it now, they're the ones who are going to be a completely different position than everyone else when demand catches up with supply.
So, the FDA's guidance documents, obviously they're free. They're regulatory documents that you can read. So read them.
Read the 2021 AIML Action Plan. You might say, well that's old. Well, it demonstrates how we got to where we are today.
Read the 2023 PCCP guidance. Read the 510(k) considerations for AIML based SAMD. You'll be one of the few people in every room that you walk into of a people, of a person who's actually read that document.
Read those documents and read those regulations.
Etienne Nichols: So, what does it look like to stay ahead?
If you build your PCCP before you build your algorithm, that's going to be a cut above.
Don't build at the same time. Build it before you know where you're going. Most likely you know where you're going.
So, build the PCCP, that predetermined change control plan while you're building your algorithm.
The types of changes that you're going to want to make post clearance, those should inform how you design your development process, not the other way around.
Train your quality team on AI specific concepts. Now don't wait for a 483 to get that education.
That's 483 is a great motivator, but it shouldn't be the motivator.
Etienne Nichols: So, a quality engineer who understands what a validation data set is and why demographic representation matters, that's worth a lot more on an AI device program than one who doesn't.
And that knowledge gap, you can close that with a few hours of reading.
So read those things and get ready for that.
Treat your post market surveillance plan for an AI product as a product itself.
It needs version control.
It needs ownership. It needs define thresholds and a documented escalation. Escalation process for when those things drift. When you. And how do you detect that?
Don't just monitor and respond as needed. As has been the old, you know, that's been our old adage in medical device. Whether we say that out loud or not, we just wait for problems to roll in. You need to go find those things and figure that out.
Etienne Nichols: So, design that program like a product.
Audit your design history. File your DHF against the FDA's current AI guidance before your next submission.
Don't do it during.
Do it before your next submission.
That gap between your existing documentation and what the FDA is going to ask for is much easier to close when you're not on a submission timeline.
So, for the QARA professionals who are listening out there, the career ladder in this space and medical devices, it's never been less crowded at the top, meaning the people at the top.
Etienne Nichols: You don't have this diversification of knowledge, the combination of regulatory fluency and AI literacy, it's, it's actually rare. I mean you see all of this stuff on LinkedIn, you see a lot of these things, but when you actually talk to people in person, they know less than we're expecting that, that I would expect. As far as what some of these things are at the front of the…
Are at the very cutting edge. If you're a QARA professional who can sit across from an AI development team and ask the right questions about their training data, their validation meth and their PCCP structure and then you translate that into documentation the FDA can actually use, you are valuable in a way that is hard to overstate right now.
The recent warning letter with Purolea has again reinforced the value of the QA R professional and that role.
It holds a lot of responsibility.
Etienne Nichols: The reading list is still pretty short right now, so don't wait until it gets huge to try to catch up. But read those AIML action plans from the FDA. Read the PCCP guidance, the 510 considerations, the IMDRF guidance, read all of that.
Even read the EU AI act, at least the sections that apply to high-risk AI because that's what will be applied to the medical devices.
And I think that might be like something like 200 pages total. Most people in the industry haven't read any of it.
If you read all of it, you're immediately in a different category. Okay, so here's what I want to leave you with across all these three episodes that hopefully, you know, if you listen to all of them.
The FDA's approach to AI and medical devices.
Etienne Nichols: Think about it like the national electrical code for a technology most people don't fully understand yet. It's just like that. Just like that electrical code, it's not trying to stop the technology, it's trying to make it safe enough to trust so that it can go into every single house, every single company across the entire United States, across the entire world.
The companies that win in the AI enabled device space over the next decade are going to win because not because of the cleverest algorithm or the most useful AI.
Not necessarily, but they're going to in the medical device industry, they're going to win because their quality systems were built for this. Their teams understood what was required before it was urgent.
And the same is true for the QARA professionals in this space. The ones who thrive aren't going to be the ones who wait for someone to explain it to them.
Although maybe this is your wake-up call to go to go learn these things.
Etienne Nichols: I think the people who are going to be winning in this career role as a quality and regulatory professional, even some product development, project management, are going to be the ones who go and read those documents and understand those documents and can translate those into ways that are applicable for their companies.
Thanks for being willing to be with me and to listen to all three of these episodes.
It means more than you know.
If you ever want to talk more about this, or if you want to get my thoughts on other things, I'd love to share those thoughts. I'd love to discuss more.
Feel free to reach out to me on LinkedIn. You should know where to find me by now.
One more thing. If you did get value out of this series, share it with someone on your team who's working on an AI enabled device or on trying to implement AI into their workflow and how they can do this. That's the whole point of doing this, really sharing that knowledge and helping people out in their careers.
All right, everybody, reach out. Let me know what else you'd like to hear from or hear about, and we'll see you next time.
Etienne Nichols: Thanks for tuning in to the Global Medical Device Podcast. If you found value in today's conversation, please take a moment to rate, review and subscribe on your favorite podcast platform. If you've got thoughts or questions, we'd love to hear from you.
Email us at podcast@greenlight.guru. Stay connected for more insights into the future of medtech innovation and if you're ready to take your product development to the next level and visit us at www.greenlight guru. Until next time, keep innovating and improving the quality of life.
The Global Medical Device Podcast powered by Greenlight Guru is where today's brightest minds in the medical device industry go to get their most useful and actionable insider knowledge, direct from some of the world's leading medical device experts and companies.