In this insightful episode of the Global Medical Device Podcast, host Etienne Nichols engages with Carmen Brown, the regulatory and quality affairs manager at Proxima Clinical Research.
They delve into the specifics of In Vitro Diagnostics (IVDs) and explore the challenges and considerations faced by medical device manufacturers in the IVD pathway. Whether you're an industry veteran or new to the field of medical diagnostics, this conversation will enlighten you about the nuances of IVD regulatory compliance, risk assessment, and FDA interactions.
Interested in sponsoring an episode? Click here to learn more!
Listen now:
Like this episode? Subscribe today on iTunes or Spotify.
Some of the highlights of this episode include:
- IVDs are a Specialized Medical Device: Learn how IVDs, which include reagents, instruments, or systems for diagnosis, differ from other medical devices and what specific regulatory pathways they follow.
- Risk Levels in IVDs: Understand the significance of classifying IVDs into class one, two, or three based on the risk of false or inaccurate results, and how this impacts regulatory strategy.
- Pitfalls in Performance Characteristic Evaluation: Discover the complexities involved in evaluating the performance of multiband diagnostics and why traditional measures like sensitivity and specificity may not suffice.
- Likelihood Ratios Over Sensitivity and Specificity: Grasp why likelihood ratios are a more appropriate statistical measure for multiband diagnostics and how they relate to pretest and posttest probabilities.
- Clinical Relevance is Key: Recognize the importance of ensuring that the markers detected by IVDs are clinically meaningful and relevant to the condition being diagnosed.
- FDA's Expectations for IVDs: Gain insights into the specific data and performance characteristics the FDA looks for in IVDs and the necessity of clinical data in regulatory submissions.
Links:
Memorable quote:
"With IVDs, you're looking at it from the perspective of what's the risk of a false result or an inaccurate result. So you're looking at who's interpreting the results and the type of condition." - Carmen Brown
Transcript
Etienne Nichols: Hey everyone, welcome back to the Global Medical Device Podcast. My name is Etienne Nichols, and today with me is Carmen Brown, the regulatory and quality affairs manager at Proxima Clinical Research. Carmen, glad to have you on the show today. How are you doing?
Carmen Brown: Good. It's great to be here.
Etienne Nichols: Good to have you. I'll let you maybe introduce yourself, talk a little bit about your origin story, and then we can get into the overarching topic that we'd like to discuss today.
Carmen Brown: Yeah, absolutely. So, my background actually originally is in chemistry. I have a Bachelor of arts in chemistry from a long time ago, and I started as a chemist in a lab in a pharmaceutical company and kind of made my way through various different quality roles in that area and then ended up in regulatory at Proxima. And then I actually also have a master's in public health. And kind of, through my time at Proxima, developed an interest in diagnostics because they kind of overlap with my chemistry background and my interest in public health in that space and so kind of focused my expertise in that area.
Etienne Nichols: And that's a great segue into maybe the topic that we're going to be talking about today, which is IVDs. And specifically, considerations if you think you're an IVD or are pursuing an IVD pathway. Let's talk about that a little bit. Maybe we could define some terms as well. So, we say IVD. A lot of times, medical device manufacturers, they might say, well, okay, you're not a medical device, when in fact, they are considered a medical device. But what are some of the differences that we think about, or we should be thinking about when we think medical device and IVDs?
Carmen Brown: So, an IVD is a specific type of medical device. Specifically, it's defined as the reagents, instruments, or systems that are intended for diagnosis of a condition. But that also can mean the same thing that a medical device can do in terms of treatment, cures, or mitigation strategies. So, for example, an IVD can be used to detect a specific biomarker that may predict your response to a therapeutic, to a drug, or it may detect a genetic marker that suggests a predisposition to a certain type of cancer, those types of things. Or it may strictly diagnose you with, say, Covid.
Etienne Nichols: Okay, that makes sense. And when I look at the IVD page, just looking at the FDA, and maybe we can talk. I don't know if you want to go global or not, but if we just focus on the IVD or the FDA, all these acronyms. I know there's the class one, class two, class three, which is not coming from the IVD world, is interesting to think about the different levels of risk, and I wonder if you could walk us through, I don't know if you have specific examples for each class of IVD, but I'd be interested to hear your take on the different classes and the different risk levels.
Carmen Brown: Yeah. So, the different risk levels for IVDs function very similar to the different risk levels for a non-IVD medical device and looking at the differences in what the test is detecting, and that intended use is really what's driving it. With IVDs, you're looking at it from the perspective of what's the risk of a false result or an inaccurate result. So, you're looking at who's interpreting the results and the type of condition. So, for example, a class one IVD would be like something like a glucose monitor, where it's relatively low risk if the results are inaccurate or false to some extent, versus a class two, something like a sepsis diagnostic or class three, something like cancer, where that risk level is increasing as the class increases.
Etienne Nichols: Okay. And is there a positive or a negative when you come to that? Maybe class three, if you're saying you don't have cancer versus, you're saying you do, and maybe we're getting into the specifics of how an IVD works. I don't know. That's not even a question, really, but how are some of the things, or what are some of the things we have to think about when we're evaluating the risk for one of these devices?
Carmen Brown: So, I guess the biggest consideration that I look at, or that we see a lot at Proxima is the difference between a true diagnostic and an AIDS diagnostic. So, is the result providing additional information for the clinician to then make a conclusion that's going to reduce your risk level versus something that is intended to be solely a diagnostic? The physician can rely completely on that information, and there's no additional thought process that's around that. I have to say the former is probably more common of trying to walk that line of the aid to diagnosis because it tends to be a little bit easier of a pathway. There are certain conditions, such as like HIV, HPV, those types of things, that it really doesn't matter as much whether you're walking that aid to diagnosis or not. You're going to be class three regardless, just because of the risk associated with those particular conditions.
Etienne Nichols: And that risk, if we define it a little bit more specifically, is it due to the type of treatment that will follow, if it's a positive, for example, or a negative, if it says negative and you do have it, the lack of treatment, that all of the risks associated with that?
Carmen Brown: Yeah, absolutely. In the case of cancer, we've talked about that a lot. The risk of a negative result when the patient is actually positive in a life-threatening situation or a potentially life-threatening situation, or like with HIV, the potential for spreading the condition to others if the results were inaccurately negative?
Etienne Nichols: Okay, one other question I might have about that then, would be that aid to diagnosis or the one that actually gives you a solid positive. Solid negative. And that is the decision. Both are IVDs. One, maybe a few more questions about the aid to diagnosis. Is it an Aid? What are the other pieces of information that the doctor is looking at or the physician is looking at? Are there other tests that work in conjunction with it, or is it the doctor's own experience, or what are some of the things that you can determine whether or not you're an aid to diagnosis?
Carmen Brown: So, I guess the other things the doctors would be looking at would be patient history, family history, patient current condition. Oftentimes, I think of sepsis as an example. There are a number of biomarkers that indicate a possible infection. In vitro diagnostic might only look at one. And so there may be other tests that are used in conjunction to make that diagnosis or provide that information to the clinician. Or in the case of cancer, it may look at a specific type of cell in the body that's indicative of a tumor or something like that. So, it's just one piece of the overall puzzle that they're looking at. So that may be their clinical judgment. And then the other things that I mentioned.
Etienne Nichols: Okay, now, you mentioned that IVD is kind of a subset of medical device. It is a medical device, but a very specific type of medical device. I assume then, when you go to your pre sub or you want to discuss with the FDA, they're looking for very specific things. What are some of those specific things, and how should medical devices companies be preparing for those conversations?
Carmen Brown: Yeah, so I think it can be looked at as a positive in the sense of the performance characteristics for an in vitro diagnostic device, in terms of what the update is looking to see, are a lot more standardized. So, they're looking at your specific performance characteristics, your clinical relevance, clinical utility, and then how you developed those. So, for analytical performance, they're looking at sensitivity, specificity, accuracy, and those types of things.
Etienne Nichols: Okay. I guess it seems very straightforward, but I'm sure there are things that companies just really mess up with, or maybe they neglect. What are some of the pitfalls that you see companies getting into?
Carmen Brown: Oh, that's a great question. I think the biggest pitfall that I see is there are two, I guess, is with the outcomes of IVD. So, you can have both a binary outcome like we've previously been talking about, where it's, yes, no, normal, abnormal, there's only one, two possible results, really. But then there's also a push in the space to have these things called multiband diagnosis, where you're splitting patients into multiple risk categories. So, for example, the device is predicting a low, medium, or high risk of developing a certain condition or having a certain condition, and that's where those performance characteristics can get really tricky, and things like sensitivity and specificity start to break down. So, if we back up to maybe even the definition of sensitivity, specificity, sensitivity is how good is the device, or how accurate, or how likely is the device to correctly identify a positive patient as positive. And the opposite for specificity is how likely is the test to identify a negative patient as negative. And those two characteristics are thrown around a lot in terms of when you're talking about performance of a device. Obviously, the higher sensitivity specificity, the better the device. But in a multiband diagnostic, doesn't really work because it ignores all of the other bands at one time. So, you would have a sensitivity specificity for the low risk, assuming that those other bands don't exist at all, which isn't really the case and not the purpose of those multibands. So, we get into something called likelihood ratios, which is a statistical determination that is really better in multiband diagnostics in determining the performance of the test.
Etienne Nichols: Okay, tell me a little bit more about the different bands. I don't know if you have any specific examples, but I'm trying to wrap my head around this a little bit more because for the binary approach, the sensitivity and specificity. I love that example. Specificity. Let me see if I get it right. Sensitivity would be the likelihood of a positive patient showing positive, specificity being a negative. Showing a negative. What about those multiple bands? Could you go a little bit deeper? Just trying to wrap my head around this a little more.
Carmen Brown: Yeah. It gets really complicated when you start adding multiple results into a diagnostic. So, think about it this way. A patient submits a blood sample for the diagnosis or the likelihood of developing sepsis in the next 24 hours, just as an example, and the device outputs a risk category of low, medium, high. Okay, so low, meaning a patient is unlikely to develop the condition in the time frame. Medium somewhat likely. The doctor is then using their clinical judgment to determine what the treatment pathway is. Do they want to prescribe antibiotics? Do they want to admit patient? What's the next step? The high-risk category would be patient is highly likely. You should probably take some action. Miss patient, put them on pressers. Use your clinical judgment to determine those treatment pathways, and that's common with multiband diagnostics.
Etienne Nichols: Okay, now, likelihood, that's really interesting because I assume it's measuring the presence of something in the blood to determine whether or not it's going to get even worse. But how do you do that without a trend, like one point to the second point and an extrapolation of some sort? Can you expound upon that?
Carmen Brown: Yeah. So that actually is a really big common misconception, and I think was really hard to wrap my head around when I first started learning about likelihood ratios. So, likelihood ratios actually have nothing to do with the patient and more to do with the test. So, it relates the pretest probability to the post test probability and has to do with the test ability to determine the result of the patient. So, for example, if you had a likelihood ratio of one, the pretest probability is the same as the post-test probability. So that diagnostic test is essentially useless because it's not going to actually increase the ODS or decrease the ODS of the patient testing positive or negative, and it's not going to tell you anything versus a test that has a likelihood ratio of two, the post-test probability. So, the likelihood of the test determining what the patient is and providing information to the clinician is higher.
Etienne Nichols: Okay, so the pretest probability. I'm still trying to understand that. I'm sorry, I'm a little slow today.
Carmen Brown: I guess, but this pretest, it's hard.
Etienne Nichols: Yeah. So, what is it actually measuring? And I know this is just one example, but it sounds like this is something that people maybe run up against relatively often. It's measuring. I wonder if you could explain it one more time. Apologies, no.
Carmen Brown: So, I think it kind of brings us into like a threshold kind of concept, or talking about a specific threshold of how do you define what an outcome actually is in a diagnostic test? And that kind of breaks down, goes back to that sensitivity specificity is only able to be determined if you've defined a threshold. So, in a multiband diagnostic, you essentially have four outcomes, and you'd need a threshold for each of those outcomes. What we typically see is a single sensitivity specificity reported for the device as a whole, which is inaccurate because theoretically they could pick the threshold from the low band and the threshold from the high band to determine their sensitivity and specificity, and you end up getting perfect results, 100% either way, which isn't true and isn't really reflective of what's actually going on, especially in the middle bands. So, the likelihood ratio picks those thresholds for you, or you pick the thresholds and then determines the performance of the test within each band.
Etienne Nichols: Okay, so you're going to look at the sensitivity and specificity of the band, the upper and lower limit for the low risk, the upper and lower limit of the medium risk, and the upper and lower limit of the high risk. Is that accurate?
Carmen Brown: That's kind of how it's reported, but it really has to do with businesspeople and businesses wanting to see a really good sensitivity and specificity for marketing purposes.
Etienne Nichols: Sure.
Carmen Brown: But it fudges the numbers a little bit, because that sensitivity specificity of the medium risk band may not be as good as the low-risk band. Or you might have a test that's really good at determining that low risk, but they're not as good at determining those high-risk patients, and that's a problem that we run into. So, it kind of results in some inaccurate representation of the performance of the test. Likelihood ratios are also better at providing the clinician actual information because sensitivity and specificity and those positive predictive values are really driven by the study population and the pretest probability of the patients within that study population, whereas the likelihood ratio allows a clinician to evaluate the usefulness of the test for their specific patient because they can look at and determine, okay, I know the characteristics of this patient at this time. I can kind of guess at the pretest probability and determine whether or not that test will be useful for them. So, it's a more individualized approach, which is helpful.
Etienne Nichols: Okay, that makes sense. So, when they're starting to utilize this, is there a lot of training that goes on, or are you utilizing things for your testing that physicians are already familiar with as far as levels of different things in the blood and so forth?
Carmen Brown: Yeah. So that kind of gets us into one of the biggest considerations that FDA is looking for is clinical. The, I guess what the test is detecting has to be clinically meaningful. So, for example, a test that detects white blood cell count is only going to be clinically relevant for things that are looking at infection. It's not going to be important for, say, the diagnosis of diabetes.
Etienne Nichols: Okay.
Carmen Brown: So, it's really got to be clinically meaningful and something that the doctor is familiar with and providing that information.
Etienne Nichols: Okay. So, moving away from the actual, the way the IVD works, perhaps, and I appreciate your patience and going with me through that. What are some of the other things that as you go and you have these conversations with the FDA, they're specifically going to be asking that typical IVD companies are going to be faced with.
Carmen Brown: So, they're really looking at your analytical mean. It’s kind of, unfortunately, goes back to that sensitivity, specificity. And how did you determine; how did you run those tests? So, I think in the non IVD world, especially in the class one, class two medical devices, there's kind of a push to not use clinical data. And clients often get really freaked out when they're like, oh, we have to do a clinical study. But with IVDs, it's almost standard, because if you think about it, how do you determine a test actually works and can detect the biomarker? You don't have a clinical sample to actually detect that biomarker.
Etienne Nichols: Yeah.
Carmen Brown: So, there are some cases where you can use, like, a surrogate marker and things like that, but that's rare. And if you're trying to go that pathway, that's definitely something to bring up in a pre-sub with the FDA very early on.
Etienne Nichols: Okay. And tell me a little bit more about how that conversation would go. You're talking about something that there's not another mark or not a typical test on the market that you could use as a control. Explain that to me one more time, if you don't mind.
Carmen Brown: So, if you're trying to use, say, like a surrogate for human saliva, okay, and you're trying to detect something and to demonstrate performance of the device, but you don't want to use real saliva samples for whatever reason, sometimes that's financial, sometimes that's time based, things like that. Saliva is pretty easy. Usually, blood is one that's typically more expensive. But you could use, say, a mock saliva sample where someone has put together something that mirrors human saliva and use that in your analytical performance testing to demonstrate that the device can detect the marker in that surrogate. But the justification for how that surrogate saliva or mock saliva is the same as human saliva is something that you would have to discuss with the FDA. The conversation would go probably like you would provide that justification in your pre-submission packet and then discuss the pros and cons rationale, really what you're hoping to see, and maybe even some preliminary data suggesting performance with that surrogate.
Etienne Nichols: Okay.
What are some other thoughts or considerations that we need to be having as IVD companies? I know there's other topics we could go into, and feel free to do that if you wanted to. I know before on the call, we were talking a little bit about LDTs, laboratory developed tests, as well as the EUA and the ending of the EUA. I don't know if we want to go there or not, but what are some other considerations or thoughts that you have that you'd like to share?
Carmen Brown: Yeah. So, kind of with the laboratory developed tests, we get into the concept of labeling. So labeling is a big consideration with in vitro diagnostics. And we talked about particularly, like, the aid to diagnosis versus the true diagnostic. That's really important to have on the labeling. You'll see a lot of cautions or warnings of this device should not be used solely to diagnose. It should be used in conjunction with other clinical information. Those types of warnings must be on the labeling. When we get into laboratory developed tests, we're looking at a whole different device, really, because laboratory developed tests aren't covered under the FDA. They're under something called enforcement discretion, where the FDA essentially chooses not to regulate them, and they're regulated by CLIA, which is an entirely different entity.
Etienne Nichols: Yeah, we talked about that a little bit on the podcast in the past, clinical laboratory improvement Amendment, but that's a completely different subject.
So. Yeah, that's interesting.
Okay. And you mentioned a little bit about the EUA. I didn't know if you wanted to talk a little bit about companies that are going through that or trying to move away from. Really, you're kind of starting at the beginning, I guess, if you're an EUA diagnostic.
Carmen Brown: Yeah. So, I mean, a lot of the preliminary data that we've talked about in terms of the performance testing and the sensitivity and specificity, you had to show that in your EUA application. But now you have to go through the whole IVD submission process and demonstrate that there's continued results and continued performance, and you're meeting those same standards now that that's ending. So, I think there's kind of a common misconception that there's a difference in the data requirements or the burden for testing from an EUA versus an IVD submission. And there is, to an extent, they accept preliminary data. But now we're getting into the point of the EUA is ending, and you really have to demonstrate that that performance has continued and that you have the manufacturing capabilities to truly be a widespread, distributed diagnostic.
Etienne Nichols: Okay.
One of the things that I guess I'd ask, I mean, we can move away from e-way and LDT. I know that's kind of a fringe conversation. We could always come back to that, maybe later. But we talked about some of the misconceptions, some of the issues or pitfalls companies get into. But when you look at, you've worked with a lot of different companies that are IVD companies, what do you see the very best companies just doing really well when it comes to that submission. Any advice that you have based on those experiences?
Carmen Brown: Oh, that's a great question. I think the most common pathway that I see, I guess not even answering that question, the most common failure, I would say, is companies trying to pursue the LDT pathway first and then move into an IVD. So, the primary difference between an LDT and an IVD is an LDT is developed within a single laboratory and tested within that single laboratory. So, I gave the example of, like, the lab develops a specific test in Chicago, you're in California as a patient, your sample is sent to Chicago for that test, and the results are back versus an IVD is widely distributed and manufactured elsewhere. So oftentimes companies will try to open a Cleo laboratory to get some initial data on their eventual IVD and pursue that pathway later. That can get really tricky in terms of labeling. FDA looks at them as distinctly different devices. It can get really complicated with your quality management system, and it often doesn't actually produce the data that you want. So, it can be helpful in class one, lower risk class two devices. But in those truly class two, and even bordering class three, it often makes more sense to just collect the IVD data up front and pursue that pathway regardless.
Etienne Nichols: Yeah, that's fair.
Yeah. Always begin with the end in mind. That makes sense. That is interesting to think about the possible negatives of going that other or trying to create an LDT first and then move into IVD.
Carmen Brown: Yeah, I think that the true negative I think I see in that pathway is what happens when FDA decides to exercise their enforcement. So, we look at examples of like, ancestry, DNA, and 23andMe that were trying to walk the line of they weren't actually diagnosing anything, they were suggesting genetic predisposition to things like eye color, hair color. But then they were also including genetic markers for things like IBS, celiac disease, cancers. And those are more in the diagnostic space. And the risk level has increased to now, what's the risk of the user misinterpreting that? And FDA came a knocking and said, no, this is an IVD, regardless of the single laboratory developed test and forced them to submit an IVT application. So, you can run the risk of FDA issuing a warning letter, and then you don't have the data to support an IVT application because you've been working under an LDT and have to do this before you're ready.
Etienne Nichols: Yeah. Okay.
That's a really good example. I appreciate you sharing that.
Very cool. Any other thoughts or considerations we need to have for IVDs? I know you covered a lot of ground, but I'll give you one more chance to dredge something else up. No, I know there's a lot to this conversation.
Carmen Brown: Yeah. I think the other thing to look at is who's going to be regulating you. I think that's something we haven't really talked about and touched on. Typically, IVDs are covered under CDRH. They're considered devices, but there are cases where they're regulated by Seber, and you're considered a biologic. So, if it's a human tissue, blood to use something to detect some sort of tissue type, things like that, there is a possibility that you can be regulated by Seber. So, you do have to look at who's going to be your reviewer. Is it going to be CDRH? Is it going to be Seber? The other thing to consider is co-development, and that's another scenario we could get into as well. But that could be a whole another podcast on the coal. Development of in vitro diagnostics and drugs and combination products.
Etienne Nichols: Yes. So that would be pairing with a drug or maybe a biologic to be a combination product.
Carmen Brown: Yeah, I think I gave the example in the beginning of an IVD used for the detection of a biomarker that would suggest a response to a drug. So, if you're looking at, say, a drug for chemotherapy, you're predicting your response of positive or negative. You're likely to respond to this drug very well. Let's use it.
Etienne Nichols: Okay. Yeah, that is interesting. I don't know if you're able to weigh in on this, but whether or not you're typically. Obviously with the combination product, you have that PMOA, the primary mode of action. But those almost seem like two different things. Is typically the drug the sponsor in that scenario, or is there a typical situation?
Carmen Brown: I don't know that there is a typical situation because it depends on the development when the partnership is established. So, I think of a scenario where like a drug trial is in phase two and they bring in an IVD that's already on the market, that research use or investigational use would likely be covered under the IND of the sponsor of the drug versus the CDRH of the device. Whereas maybe there is a device that's not yet on the market and still in investigations, it may be covered under their ide to bring the drug into that process.
Etienne Nichols: Okay.
Carmen Brown: There's a number of different guidance on the co-development of IVDs and therapeutics, or even just medical devices and drugs simultaneously.
Etienne Nichols: Yeah. Either way, I suppose all of the sensitivity and the specificity and so forth would still be required, even if you're under the sponsor of a drug and so forth. All those things are still.
Carmen Brown: Absolutely. Absolutely they are. I think the only difference there may be who you're providing that information to. So, it may be the device manufacturer is providing that information to the drug manufacturer, who's then providing it to the FDA, or vice versa, or you're providing it directly to the agency itself.
Etienne Nichols: Okay.
Carmen Brown: It's definitely still required, still should be considered, and that's where the helpful standardization.
Etienne Nichols: Kind of comes in, I guess. You mentioned that example of the IVD that's already on the market, and then a new drug is coming along maybe phase two clinical trials and pairs with this already on the market, IVD. Would that potentially be a new indication for that? I guess it would still be very case by case.
Carmen Brown: It's definitely case by case. It can be a new indication. It cannot be. I've seen it where they're using a specific biomarker that is already on the market and the drug decides, well, let's evaluate if that biomarker is a predictor of the response to the drug.
Etienne Nichols: That makes sense. Very cool.
Etienne Nichols: Well, I really appreciate being on the show. You mentioned a couple of other topics. Maybe we'll have to explore some of the other topics at some point. But where can people find you and learn more about you or get a hold of you if that's something you're interested in people being able to do?
Carmen Brown: Yeah, absolutely. So, people can find me on LinkedIn, or they can email me directly. My email is carmenbrown@proximacro.com you can reach out to me there. Happy to connect. And if people have any other questions, fantastic.
Etienne Nichols: We'll put the links to her email in the show notes and her LinkedIn so that you can easily find her. And yeah, really appreciate everybody who's listened in today. And thank you, Carmen, for the conversation on IVD and the different considerations you need to have when developing your IVD and taking it to the FDA. We'll let you all get back to the rest of your day. Thank you so much.
Take care.
Thank you so much for listening. If you enjoyed this episode, can I ask a special favor from you? Can you leave us a review on iTunes? I know most of us have never done that before, but if you're listening on the phone, look at the iTunes app. Scroll down to the bottom where it says leave a review. It's actually really easy. Same thing with computer. Just look for that. Leave a review button. This helps others find us, and it lets us know how we're doing. Also, I'd personally love to hear from you on LinkedIn. Reach out to me. I read and respond to every message because hearing your feedback is the only way I'm going to get better. Thanks again for listening, and we'll see you next time.
About the Global Medical Device Podcast:
.png)
The Global Medical Device Podcast powered by Greenlight Guru is where today's brightest minds in the medical device industry go to get their most useful and actionable insider knowledge, direct from some of the world's leading medical device experts and companies.
