Digital Therapeutics Explained with Richard ‘RJ’ Kedziora

RJ Kedziora: 0:00

As we move forward and it's really all about how do we enable more effective transfer of that data in a meaningful fashion, you can go to any ATM and

Harsh Thakkar: 0:12

what's up everybody? This is Harsh from Qualtivate.com. And you're listening to the life sciences 360 podcast. On this show, I chat with industry experts and thought leaders to learn about their stories, ideas and insights, and how their role helps bring new therapies to patients. Thanks for joining us, let's dive in. All right, welcome to another episode of Life Sciences 360. My guest today is RJ Kedziora he is the Co founder of Estenda Solutions, a company that specializes in custom software and data analysts for healthcare and medical companies. Welcome to the show, RJ.

RJ Kedziora: 0:48

Thanks for having me.

Harsh Thakkar: 0:49

So I'm really excited to about this podcast because a lot of the topics that I was researching on your profile, around digital therapeutics, the intersection of healthcare and technology, these are all areas that I'm really curious to learn more about. Because I don't get to learn about these topics, being in pharma and biotech space, and having somebody with your expertise to come and share. I'm really excited for this. And I'm sure the audience will also find it useful.

RJ Kedziora: 1:16

Looking forward to it.

Harsh Thakkar: 1:18

So the first question I have for you in layman's terms, which even I don't know the answer to it, I hear like 50 variations of it. What is a digital therapeutic and what is not

RJ Kedziora: 1:31

digital therapeutic is at its heart, a digital health application. It's a software application, and you're taking out the medical professional. So it's really targeting the patient on their own as opposed to be part of the healthcare practice. The difference between a digital health application and a digital therapeutic is really, you think about supplements, vitamins, minerals, that you might buy verse, a medication, a drug, something that's prescribed, digital health applications are the supplements, the vitamins, the minerals, there is some evidence that they work, but not extensively, it has probably hasn't gone through randomized controlled trials probably doesn't have a lot of clinical research around them. When you get into a digital therapeutic, that's the key, extensive clinical research has been done around the use of that solution to be able to provide evidence that it does do what it says it does. And then it goes through a review by the FDA, to be able to say, yes, we've reviewed the evidence, this digital therapeutic solution, or much like a drug does provide a benefit does do what you claim it does. So if you look at those supplements and things like that, a lot of times they should if not all, as I have a little disclaimer like not meant to treat a particular issue, but with the drugs, they are the research is there to prove that it works. And then the secondary benefit, which is really interesting, from a business perspective, from a startup or even large enterprises perspective, yeah, d of digital therapeutic is that when a patient uses it, to reimburse the producer of that is reimbursed by the insurance company. So just like a drug is prescribed by a doctor, a digital therapeutic will be prescribed by a doctor to that patient, and the person in the developing company, then gets reimbursed by the insurance company. So you can imagine as a patient, if I'm going to pay for a digital health application, you know, I'm not paying 1000s of dollars for it, I'm gonna, if I'm lucky to pay $1 a month for it something like that. But when it comes through to the insurance, you're gonna get higher reimbursement rates. The

Harsh Thakkar: 3:56

So from your experience working with clients, what would you say are like the top three mistakes that companies make when they are creating a digital therapeutic product?

RJ Kedziora: 4:07

It's really early on as we talk to various companies not understanding what that FDA level of validation mean. Because there's a lot of people that can do software development, you can even document software development. But if you don't follow the processes mandated by the FDA and go through that review cycle, or doing that clinical trial, you're not going to get there. And you know, we've talked to a couple startups and they're like, oh, yeah, we you know, we're talking to a couple students, they developed this little thing in the garage. That's not going to pass FDA muster. Yep. So and then the second part that's on the software development side and the testing validation on that you're probably familiar with, yep, from your world. But on the second side is really what does that mean from a reimbursement per spective here in the US, it is particularly challenging. There are not a lot of guidelines today around those reimbursements. So it gets challenging various countries in the EU are making big strides in that area in tying approval from a clinical regulatory perspective to also reimbursement here in the US, you are getting approved by the FDA. But that doesn't mean you're going to be reimbursed by any insurance companies. So it's understanding that journey, how do you get from product to commercialization?

Harsh Thakkar: 5:32

How did you how did you get into this space? I was looking at your profile. And I wanted to understand when was like that moment where you're like, This is what I'm going to build the depth of my career, so to speak.

RJ Kedziora: 5:45

Yeah. Someone asked before, did data find you? Or did you find Yeah, yeah, I think I found data. It's always interesting when I can talk about, I've been doing this for decades, but it has been decades now. And I started out doing accounting systems and railroad cars, scheduling systems, and they're all great, they're very beneficial, and they're necessary pieces of software. But a lot of those challenges has been solved, very driven. And the people that work for us are very driven by around solving problems. And in healthcare, digital health. It's the use of data in the healthcare system is really early on. There's a lot of technology in healthcare around diagnosis, MRIs, X rays, things, things of that nature, drugs, you know, lots of technology, but use of data is really early on these days. So still lots of challenges around how do you use that data? How do you develop solutions. And then as I went on that journey through various different industries, healthcare gives you that extra benefit of feeling a little extra good at night, as we've developed systems, whether they're patient facing or clinician facing, it's that we hear those anecdotal stories of like, hey, using stuff you've developed, we were able to help patient with x, which is really rewarded. And then even one thing for us to develop a solution for our company, you then have to prove it. So we do clinical trials to prove out that the software actually works. So

Harsh Thakkar: 7:21

do you I you know, without sharing any confidential information, this is some question that I asked everyone. But I this is a disclaimer, like we're not sharing any confidential details, but I wanted to hear from you if you worked on any interesting projects with clients, or if you have a case study off the problem statement when they hired you or your team. And then how did you help them through the healthcare data analysts as part of the project? Do you have any examples you can share with

RJ Kedziora: 7:48

us? Yeah, you're right. It is very interesting because as as consultants, find lots of and the agreements, but we are part of a project with the Joslin Diabetes Center out of out of Boston in we started with this 20 years ago, when we got started. It's called the Johnson vision network. And it's managed within the Indian Health Services here in the United States, which and that's the government organization that's responsible for the Native American Alaskan populations. And it's a digital camera that takes a photo of the retina of your eye for detection of diabetic retinopathy, wow. And it's deployed at about 100 sites throughout the United States right now. And over those 20 years, we gather millions of images kind of thing. And they are owned by the Indian Health Services. So when we do partner with other institutions to make use of of that data, and that's where you address your question, you know, that problem statement and things like that. We help various companies with AI solutions. And what drives AI is that training dataset. So in terms of diabetic retinopathy, which is a disease that can develop if you have diabetes over time, it impacts the retina in the back of your mind, it's the leading cause of preventable blindness. So if we can find it, it's it is treatable. And that's the goal of this program within the IHS to be able to find that incidences of the disease and treat it when you get started. Those images are looked at by people take some time. So we're working with various people to use those images that we have to train AI systems to first do what are called refer or no refer type of grading on the images. Where is there an incidence of disease do you see something that means this person, this patient should go to a specialist, but then as we work with various things and think about this, there is the idea of grading a level of disease, mild, moderate, severe. So trying to go to that next level of like, okay, not only As a referral, what is that level of disease of incidents that we find that are now in its the data driven by those images. So first you have to anonymize the images, you have to take out any identifying information in and that can be behind the scenes or even embedded in the actual image. So you have to work on that. But then developing solutions such that humans looking at these images, the experts that are trained on how to look at the images can identify that things of interest. And then if you have two different experts, looking at the same image, they might have different impressions. So how do you adjudicate between those two people or even three people to do okay, here is what the you know, the AI needs to learn to understand to be able to do this in the future.

Harsh Thakkar: 10:47

That's, that's very interesting that you mentioned that your team is using or helping clients use AI and in different capacities, you give a really good example. It's also a little bit of an area where my company specializes in and we're also working on, we don't, we're not more into, like the example you mentioned about looking at the scans or the images and training. We, my company specializes in more in terms of data from like quality management or project management data set. So it's more like spreadsheets and things of that nature. We don't I don't do the images part. But that is very interesting. Because you know, with AI, you can recognize image tags, Weiss, and pretty much anything nowadays. So

RJ Kedziora: 11:34

yeah, absolutely. And it's also how do you bring together that images with the clinical information? So we know this person has diabetes, this person has high blood pressure, but then you know, numerous projects over the years have also brought in that personal monitoring information. So like glucose, blood pressure, so how is that impacting the patient? And how do you manage all of that?

Harsh Thakkar: 11:57

Yeah, I have a selfish question for you. And I'm going to ask it, because this is a challenge I'm facing right now in in the project I'm working on. So when you first started this project, what was like the accuracy percentage that you got just with the data set as is versus how much fine tuning did you have to do to get it, like, let's say, from 90 to 95, or 95 to 99? dB, you have anything you can comment on?

RJ Kedziora: 12:24

Not unfortunately, off the top of my head that I can comment on, I just don't have those numbers in my head. But I know definitely over time, that was part of that AI learning experience that journey working with these other companies. It's like, okay, you start out doing a training data set of 100. Okay, this is going to be enough to get to that level. Okay. And let's get it to honor let's go to 1000 Is that going to get us to the level of specificity that we want and in the accuracy and the confidence that this is going to be reliable? We do have a couple published papers out there on the subject, and I've done some industry presentations. you the last one was a conference called Arvo. So I can I can share those, you can drop them in the show. Sure.

Harsh Thakkar: 13:12

Sure. Definitely. Yeah, it's it's it's a very interesting topic, because the regulations from FDA and other agencies or international bodies like ISO and others, in this area of training the AI or using datasets, or what sample do you use, how to how to get the confidence in the percentage level? There's a lot of different information from different agencies and great regulatory bodies. But there isn't like a one stop shop, like, here's how you do it. So which is why companies like you, or even companies like mine, I get lots of these questions, where we have to be honest with the clients, like, Hey, I'm, I'm not gonna give you a magic pill, if that's what you're looking for I can, I can be with you. And I can look at your data and I can try to solve this for you. But this is still a very evolving phase, both in healthcare and pharma and bio.

RJ Kedziora: 14:09

Yet, in terms of regulations in AI, the EU is working on new legislation, which I like because it's really taking a risk based approach to depending on how much risk is involved. And the FDA in the US here has the same thing. When you look at medical devices, what is the impact if something goes wrong with these devices? But yeah, AI is changing morphing very quickly. Day after day. I recently saw a report that was just released in the EU on the AI in health care kind of thing and didn't even mention chat GPT at all. Which just shows you how fast yep, I was exploring Vegas. Like this is all anything anybody's talking about now. Yep. And their big 100 page report didn't even mention it.

Harsh Thakkar: 14:53

Yeah, it's a definitely interesting. I haven't I can't even keep up with all of them. I've read the there was a Stanford An AI report, which was also like 200, some pages that I skimmed through, I read the NIST risk management framework that came out I think, last year or early this year. So that was really good too, because it like you touched upon the EU and how they are writing the risk management for AI. It goes into a lot of details. So yeah, but definitely a lot to lot of information coming out in this field that it's, that's what makes it exciting to be working. In one project.

RJ Kedziora: 15:29

A lot of it is about transparency to so when companies for profit, commercial companies are developing these algorithms how how do they work, right? What is behind the scenes? What are the rules? What are the training datasets that you used? And there's incidences in the past? And I don't want to call out specific companies, but incidents is in the past in the industry, like, Okay, I'm training this algorithm, and it's only on white males from 40 to 50 years old. Yep. Well, when you then take that AI algorithm and apply it to teenage women, you're not going to get the same result exact. So you really have to be cautious of that. And there's even over over the years notes of training data locally, how are you coding data in your system? Is it in a way that the AI is going to be recognize it as it been trained with data of this nature? So it really for me, it all comes back to date?

Harsh Thakkar: 16:28

Yep, yep. Yeah. And I've heard the same sentiment from another guest that I had on the show, and he was experienced in clinical research. And he also had the same talk about AI in clinical research. And he also mentioned that taking care of bias, right, because as you said, if if diversity and inclusion is, is a problem in traditional clinical research, then it's also going to people or companies might introduce that bias into the training data set, like you said, about example of a male 40 year old versus a teenager. And you know, if you're training that data is not possible. How do you design your training? Right? That's basically what it comes down to, because there is opportunity for some bias to be introduced there. And companies may or may not share that information willingly.

RJ Kedziora: 17:20

Right? It's all something interests, everybody talks about chat to beauty these days. But I saw something interesting that was talking about a bias and a different perspective, that it biases his answers to confirm what you were asking it. It's just so you have to be really careful on how you phrase what you're asking it because it's like, it's not really intelligent. It's doing word prediction sense. Prediction is like, what is the next best word? And it's, it's has this confirmation bias, right? Oh, you got to be careful. Yep. Yep.

Harsh Thakkar: 17:55

Yeah, you mentioned that it's all it's all data. And somebody else in the past I had on the show also said, every company is a data company, whether they want to admit it or not. So that's, that's really interesting. And from that day on, I've kind of looked at every project I work on to say, Okay, this is your most critical data that you want to have some controls, or you want to be able to defend in front of an auditor or agency how you're using this data. So that's really interesting that you're the second person that has said, you know, it all comes down to data. So I want to fast forward. And because you've been in this space for so long, when you started in healthcare and technology like the intersection, what did you see that like the past? And then where do you see this going? Like maybe 10 years from now,

RJ Kedziora: 18:44

there has been an interesting journey, because when we started, it was pre EMR. So when you Mars existed in our first few projects, as a corporation were with military healthcare with the VA, which did have EMR systems. But then as you progress, you know, that was 2003 that we got started. And then EMR became mandated. And so now, everybody, really, you need an EMR. And in easiness, you have one version of epic or one version of next gen, and you move to another entity. There are lots of differences in how those things are implemented. But you know, and I'll go back to data again, that's really been the biggest part of the journey. So we started out in the world of diabetes. And in the early 2000s, it was really about finger sticks. And you'd be lucky if you get a person with diabetes to do a finger stick once, you know, twice, three times a day to get accurate sets of information. And then how do you get that into your data environment? How do you have that exists back then. So how did you get it to your server? So we did a lot of individual integrations around that How do we get data from the EMR? How do we get the data from this patient were in today, fast forward 20 years, everybody has an EMR, if you're a person with diabetes, particular type one, you're gonna have a continuous glucose meter, which takes a read reading every five minutes, every one minute, on some cases, these devices, if you're taking insulin as a treatment, we can now get that data electronic like, so there's this proliferation of data over over the last 20 years. But there's still challenges today, in accessing that, and understanding it to the industry moves from HL seven messaging as a means to move data around the healthcare system. And it was worked that functional, and not a lot of people understood HL savate. And today, we've moved to what's called the Fire API based standard FHI are fast healthcare in a standard. And the key, it's not magic, but the key to it is it's API driven. So there's just a much larger audience of software developers that understand how to use it. And while there is a standard NHL seven, there's a standard in fire of how you code data. But there are different ways of coding the same information. So when we're asked to integrate with the, you know, hospital or healthcare system, it's like, okay, what version of you know, the EMR Do you have, what features and capabilities or you have turned on the industry has moved from not really sharing data? Because it was difficult to now it's very easy to share data, but what data are you sharing? How are you sharing? What are the what are the data governance areas around that? And then it's always interesting as time goes on, and as people in they're like, oh, this standard is not that good. Let's create a new state. Yeah, you'd see that and healthcare, it's like, oh, let's build a new standard to address this. So today, now, there's just this proliferation of data. And if you, you go to your provider, and you're using a particular glucose meter, or blood pressure meter, or medication reminder at that you track what medications you're taking. And when you take them, and share that with their provider, you you're lucky if you get 10 minutes with that provider, they're constrained in terms of time, their time challenge. And they may never see the data from that particular device. What's interesting, what's pertinent? What's relevant. So, there are many projects in this area, how do you surface the important the relevant data? How do you provide actionable feedback to the patient? And then the idea that digital therapeutic? How do you provide actionable information to the doctor to the entire care team? So as we look forward and project forward, there's just going to be more and more data, sleep tracking, stress, I run triathlon, run, swim, bike run triathlon a lot. They're getting into now lactose monitoring and sweat and sweat, monetary, how much potassium? Are you losing in your sweat, so you know how to recover effectively? And so if you go into your doctor now and dump all this data on them, what are they supposed to do with it? So as we move forward, it is providing actionable intelligence to the patient to the provider, the care team, such that they can very easily make sense of this information and make good recommendations for that patient. Okay, how should we adjust your medications? Or you've been smashed? cribes a certain dose of insulin? While is the patient really doing what they set? It's like, okay, you're not in good control. And we've prescribed this level in the amount of insulin, these drugs that you're supposed to take on a daily basis. are you actually doing that while you're not in good control is at the medication or that you are not compliant? In people have many reasons for not being compliant side effects and being used are just daily stressors of life. So how do you address those? And as we move forward, it's an it's really all about? How do we enable more effective transfer of that data in a meaningful fashion? You can go to any ATM bank in the world and get your money out these days. And yes, you know, banking and money is simpler than the healthcare data in my opinion, but so we can get there.

Harsh Thakkar: 24:31

Don't Don't say that to a lot of the crypto folks Gus. Yeah. Yeah, but, but yeah, I get your point. And it is, as more and more technologies and digital therapeutics evolve in the space, there is obviously going to be more data. And not just more data, but different types of data that we don't know how to read or analyze. I mean, there are many tools that can, you know, analyze a spreadsheet or a CSV file or what other types of data data sets that people have been using for decades. But as the technology evolves, and there's new types of data, capturing that data from whether it's like a wearable device or something, you know, that records or listens to your, that you put on like a wristband or something that monitors, how do you get that data? And then like you said, how do you make sense of it? Right? Because otherwise, it's just pointless to have that data?

RJ Kedziora: 25:30

It's also, how do you do the analysis of that data? We've run into these occurrences or there's trigger and you're talking about glucose information and your blood pressure information sleep, you're generating volumes of volumes of data? How do you track that and analyze it in over the years, we always bring different experts to the table to help learn from them. And very often, in those early stages, it's like, oh, well, you just summarize the data, like when you worked in finance, I can take all of the transactions of the last month for this individual and summon up to a balance, you've lost all the value of that date of understanding a patient's journey. Over the course of that month, you need to be able to look at that data from a large longitudinal perspective. Yes, there are ways and averages and, and image calculations that that you can apply to understand that data at a higher level. But it is truly about that data and understanding the journey.

Harsh Thakkar: 26:30

So on that topic of understanding the data and the journey, if any data or any digital health or healthcare startup that's listening to this episode, who should they have on their team? You know, as they're building a product to make sure that they have the right expertise on their team? Do they hire somebody? Or do they work with an external partner? If they're listening to this episode, what do you what would you want to say to them, like you got to have these three people on your team, and here's why you need them. Yeah, particularly

RJ Kedziora: 27:02

in the healthcare world, you need someone that understands healthcare, they don't need to necessarily be a doctor. But someone that's medically trained that understands how the healthcare system works from a different a bunch of perspectives, you always you always want to consider your user, you want to understand that scenario. But you also have to consider the payment models and healthcare, who's going to buy your product, who's going to use it, it could be two different people. So the the insurance company may pay for the product, but the doctors using it for my hospital, there's somewhat of a disconnect there. And it makes it more challenging when you're out there selling it to someone that's really understands the industry. And that expertise is definitely valuable. Whether you bring that person in house or a consultant that can apply that to it, you definitely need quality assurance, folks, the FDA is there to talk to. So there are consultants out there that can help you. But you can also talk to the FDA, here's what we're trying to do. Here's what we're thinking we're going to claim it can do. And they can provide advice. There's also lots of information available on their website to help you at a very basic level. But there's lots of nuances there that you do need to talk to an expert. And there's new things coming out all all the time. So for you as the business owner, the startup to try and absorb and keep track of all that information, you're going to be drowning in paperwork, SOPs to read and procedures to read. And so having someone in the QA side will be really helpful. And then definitely someone that knows how to do software development, first and foremost, design it and then development such that they're first and foremost, probably these days addressing cybersecurity. Yeah, it is a noted challenge in the industry, all of these digital health applications. There was a lack of security that people have been finding as they do analysis. That's where we start today on our on our processes of cybersecurity doing those risk assessments. Okay, how do we secure the software doing secure by design, secure coding practices, to make sure if you're not familiar with OWASP, or W SP, go out there and read it?

Harsh Thakkar: 29:19

Yeah, yeah. And that's also a challenge, you know, cybersecurity and data. Security is also a challenge with a lot of the AI tools. Chad GPD, open AI, announced, I think it was this week or last week, the enterprise version of Chad GPD, which is going to basically have a lot of security measures and they're going to be they're not going to be using that data for any other reasons. It's not like HIPAA and like extremely like compliant but it has more compliance than the standard GPD that you can download news. Then Microsoft, I think is also testing out copilot which is their version Jump off AI that integrates with all office 365 apps. So they're, they're going and also at all the enterprise companies who are using Office 365 products to embed the AI there. So, yeah, security is also it's a very big concern. Most of the clients that I work with also are hesitant to use AI tools because of not enough information around the security data security piece. So

RJ Kedziora: 30:29

yeah, if you'd like chatty Beauty Bar, these other solutions, Facebook came out with one, you know, and some of them were even specifically designed more for coding than, than general use it for problem solving. But you can't rely on them. And you still need to know the question to ask. Yep, you're not coming at it with no training in computer science, software development, and are examples people out there. Oh, I created the I don't know anything. I question this because like, you have to know the first question to ask, Am I deploying this on a Linux system or halitosis, there's differences? What programming language do I use, let alone addressing all the issues around cybersecurity? Yeah,

Harsh Thakkar: 31:13

it's definitely an interesting area to be working. I know we're coming up on time. And I really want to thank you for coming to the show and talking about these topics. It was really great. Hearing your expertise in this area. I know you you also do a lot of presentation and events. Do you want to shout out any upcoming event that you're speaking at to the audience so they can tune in?

RJ Kedziora: 31:38

You don't have anything scheduled yet? But we will be at the AHL th conference coming up and October. I'll be there walking around. We want to track you down. More than more than willing to always have a conversation. Yep. Love to hear what you know people are doing and always sign you know, if you want I could sign an NDA. But yeah, grab a cup of coffee. Love to have a conversation.

Harsh Thakkar: 32:00

Where's that conference at?

RJ Kedziora: 32:02

Las Vegas? Las Vegas. Okay. Okay.

Harsh Thakkar: 32:05

Yeah, I just too many conferences that I have getting messages from LinkedIn. Like, oh, come to this one come to that one. Where are we? It's interesting. And where can people find you or connect with you after this episode?

RJ Kedziora: 32:19

LinkedIn is always good. Or just our corporate website. stena.com e s t e n d a.com. All right.

Harsh Thakkar: 32:29

Thank you, RJ. It was really fun talking to you. And I will let you know the feedback once this episode is released. But I'm sure the audience is going to find this valuable.

RJ Kedziora: 32:37

Yeah. Thanks for your time. Appreciate it. All right.

Harsh Thakkar: 32:40

Thank you. Thank you so much for listening. I hope you enjoyed today's episode. Check out the show notes in the description for a full episode summary with all the important links. Share this with a friend on social media and leave us a review on Apple podcasts, Spotify, or wherever you listen to your favorite podcast.