Happify Health's VP of Quality and Regulatory Affairs speaks on Reshaping the Future of Clinical Trials for Digital Therapeutics.

 

The therapeutic industry has relied on clinical trials as its tried and true method for evaluating efficacy for decades. However, in the new age of digital health, these trials are proving to be less efficient. How do we, as an industry, transform the model for digital interventions?

Listen to this recent DTx East 2021 panel discussion on Reshaping the Future of Clinical Trials for Digital Therapeutics with Kimberly Tableman, Castor’s Chief Clinical Development Officer, Athena Robinson, Woebot Health’s Chief Clinical Officer, and Mona Dean, Happify Health’s VP of Quality and Regulatory Affairs. They discuss:

  • The challenges we face with traditional clinical trials and digital therapeutics
  • Changes that can be made within the clinic to develop more efficient DTx trials
  • Insights into decentralized clinical trials and why many DTx companies are adopting this methodology
  • How decentralized clinical trials compare to traditional studies
  • The future of clinical trials for digital health interventions

Video Transcript

DUE TO THE NATURE OF THE PANEL DISCUSSION THIS TRANSCRIPT HAS A NUMBER OF ERRORS DUE TO SPEAKER OVERLAP. HAPPIFY WILL CONTINUE TO WORK TO CLARIFY SPEAKER ROLES AND COMMENTS IN REFINING TO BETTER ARTICULATE THE POINTS MADE IN THIS TRANSCRIPT

Moderator: Alright, our final panel discussion for the day. Reshaping the future of clinical trials for digital therapeutics. So if I can ask panelists to come up and I'll ask you to introduce yourselves.

Introductions: Alright. Hi. Thank you. Thanks for everybody for staying. We know that we're standing between you and dinner, so we appreciate, we appreciate your attendance. My name is Kimberly Tableman. I'm the chief clinical development officer at caster. I am Moana Dean, vice president of regulatory affairs and quality at Happify and I'm with Athena Robinson.

Wonderful. No we're efficient guys. All right. So let's start off talking a little bit about the challenges with current clinical trials as they apply to digital therapeutic. What ideas do you guys have about that? So I'll start. I think the digital therapeutics in general was just fairly new concept. And it's taking a while.

When you were designing these clinical trials, you're trying to appease multiple audience. You have physicians, you've got payers. You have patients and then regulators. So most clinical trials, in my opinion, in the ward, you know, from my background, working on various ones, you're trying to generate clinical evidence in safety and efficacy data.

That might be sufficient for FDA or some of the notified bodies or some of the regulators, but it turns out that it's not the data the pairs are looking for when they're trying to figure out is, is value at it to their ecosystem. And I think some of the odd burden approved is sort of unreasonable when it comes to clinical, you know, digital therapeutics and that sense you, we don't expect drugs approvals their initial approvers to go through and be compared to an existing.

We want them to be then compared to a placebo. And that's not the case here. Right? We're asking people to or digital therapeutics or compare it to a sham device to placebo, to standard of care. And it just makes it very difficult. And I think there needs to be consensus of like, what is expected?

What kind of data are we looking at? And it could be cost-effective and you know, again, digital therapeutics is not raising money like pharma does where they can run long-term clinical trials for a long time and, you know, do ups. And that's just not the case here for digital therapeutics.

So I think that's been most pain point for the companies that I work for. And I would add one of those are great points. I think another thing that hasn't been raised too much today, but I think is really important to touch on is some of the limitations in recruiting diverse and minoritized populations.

When you look at traditional clinical trials and decentralized clinical trials can be. Kim has sites, but they can also be site free. When we start to broaden the way we think about clinical trials and think about things being site free, we kind of abandon this traditional notion of being tied to a particular provider or a particular site.

We finally get to enroll. Diverse patient populations that allow us to generalize the findings of our study to people who are real need. And that can also be people who are geographically not near any providers. These are rural areas where folks don't get to be really part of clinical trial care.

And if you put two things together, maybe someone from a low SES background, as well as a rural area or someone who's working like as a shifts worker, the advantages of decentralized clinical trials really. As to being available. To do things like clinical assessments, et cetera, when someone doesn't really have to go into a site.

So I think there are some limitations with current clinical trials and thinking about being like geographically anchored. Yeah. And certainly the reaching on a set of populations also placed your crazy. Yeah. And I would just build on what Athena is, is talking about in the sense that I really feel like a digital therapeutics and decentralized clinical trials are a perfect marriage.

Yeah. That's what I call it. So that, you know, decentralized trials really allow us to reach those diverse patient populations and to reach patients through digital marketing efforts that we just talked about and then have a seamless patient experience from the point. Patient finds out about clinical research to the point of actually going through a pre-screening survey that they can complete online.

They then can opt to enroll into the clinical trial and even we, we enable at casts or, you know, So you're, you know, you are, you're literally just going through the entirety of that process and you're now in the trial. So I think that it's a real opportunity to, to leverage the decentralized clinical trial approach with Derek digital therapeutics.

Absolutely. I guess if we think about WIC, we make changes in terms of how clinical trials. To make that marriage one made in heaven. How can we really optimize and, and kind of think differently about how clinical trials are done in order to optimize their fit for digital therapeutics? Yeah, I'll take a stab at that one.

But I would just say that it's this idea of being a little bit more light touch. So if you think about how we traditionally run clinical trials, and I do have a background at Pfizer and GSK in clinical development. And so historically. We've looked for sites. You're kind of as the patient at the mercy of going into the right site, that that specific doctor actually knows about a clinical trial that's going on, that then tells you about the clinical trial.

And then you go through the process of enrolling in the clinical trial and typically. Historically, you have to go into the site on a regular basis, your, your visit schedule to have your procedures done. So again, I think that what we're able to do now that is really different and overcome some of those, those barriers is allowing people to have visits at their.

Right. Be able to be in the comfort of their home and be able to do data collection remotely. And then even talking about monitoring and looking at the data remotely. So rather than having to send a CRA in and doing the, the site monitoring on site you, we can actually be looking at that data real time.

So again, I think there's a lot of changes that we're seeing with the decentralized clinical trial model. And I think it's really. Yeah, I think you, I think you nailed it. And I, one thing that has come up a lot today is this theme around like patient centricity which is fantastic. And that's, I think that's why we're all here in the same room.

Anyways. We're really thinking about the patient and, and making meaningful change. And so whether or not that's through leveraging enrollment methodologies, creatively that are fully digital that, you know, don't rely on. Kind of pamphlets in someone's clinic, office space or that sort of traditional methodology of referrals that allows us to meet the patients where they're at, meet them on their social media platforms.

We have a study right now that is live, that we launched for a 16, 17 year old adolescents. And we had over 2000 adolescents apply to that. Meeting them where they were at in their social media spaces. So I think that's one thing that's really fantastic. Another thing that some of us are faced with is like administration of structured clinical interviews for diagnostic purposes.

And often those have to be clinical Raider administered, or they have to be physician administered. And when you're a patient to your point, right, Kimberly, you might have to come in for that. That's a lot of. A lot of hassle. If you have children, if you're a single parent, if you have to pay for childcare.

And so the idea of offering to meet someone where they're at, where they can get that assessment done in the, in the privacy of their own home at their convenience is another huge benefit of thinking through decentralized chemicals. I think it just cost, you know, cost effective. I do think when you're looking at the cost of overall running a traditional study versus a de-centralized study, it's a cost effectiveness.

And this is why I think you can see a lot of digital therapeutics, you know, liking that idea. Right. Because they can save a lot of money from an operational standpoint. I don't think you're saving that much money. Perhaps not in recruitment because it may be taking still a little bit longer to recruit patients remotely.

Right? Do you have to verify diagnosis perhaps and not having your data available from EMR or clinical data available? But I do think you're saving costs from operational standpoint. Also there's a less burden on the PTO. You can have a lot of remote, you know, data analysis or data verification. You can do it.

There's probably less cost and staff you, as you mentioned, right? Like you don't have to a bunch of stuff that's going around. And per site also cost associated with setting up sites and bonding them and training individuals training, you know, physicians to kind of get these sites up and running. So.

Yeah. And I think building off that there are also some important time efficiencies and optimal levels. So for the patient, not having to commute to a clinic to be assessed, or, you know, even in terms of efficiencies of automatic data, capturing and automation ensuring. Safeguards in place around safety.

I think in terms of data capturing Adam, you had mentioned during your talk like patient retention and we all are invested in patient retention. One of the patient have a good experience. We want enough data at the end to analyze. So I think that's really helpful too, in streamlining those efficiencies that you all have, right.

So when we comparing de-centralized clinical trials to traditional studies like any other, I know we've spoken about a few comparison points where deep decentralized trials really are winning the game. If we were to compare them, any other points that we haven't touched on that you'd like to discuss?

Well, the only thing I was going to say with these centralized, I think we, we keep forgetting and we shouldn't forget that, you know, there, the human interaction and finding that right balance between the, you know, human interaction versus this decentralized is the key here that hybrid somewhat of an, a hybrid model.

Right. So I think that's the key because there are some patients perhaps, you know, especially. We're working in mental health environment, right. And conducting studies related to that, there was a key there that a patient may not understand. The study may not fully you know, e-consent process or data, how their data is being utilized or in digital health.

It's like also data privacy, right? People are afraid. And then you bring on AI, right. And you're talking to them or chatting with them. And I think it's important to have that human interaction to kind of walk them through. What is the treatment they're getting what's what is the process? How has their data being collected and how are we going to use it?

Right. And just sort of security around it as well would be. I think those are some of the things that are a little bit different when it comes to it or traditional versus, you know, a digital. It's a decentralized to go. Yeah, no, and I, I would just build on that to say that the majority of trials are hybrid.

So I think that I mentioned that only five to 10% are fully virtual. So we're big believer our ourselves in that hybrid approach. Specifically related to e-consent having that ability to do that video visit and walk the patient through what does the e-consent actually say? Because unfortunately they still are really long.

They can send documents. And so without someone to help them understand all the legal ease, most people are not going to understand what's in that consent document. What are they actually consenting to? So it's nice again, to have that human interaction, even if it's on video, you know, they're getting that human interaction.

And then the other thing I would say is that we have seen an acceleration in patient recruitment with decentralized clinical trial methods. So we did it was called the COVID red study with the world health organization last year. And we were able to recruit 17,000 patients in 15 weeks, which would normally take.

So we deployed, yeah, we deployed a method of just saying, you know, getting the word out, having them go through the pre screener, being able to consent them rapidly. And obviously with the pandemic, that was very important. So, and, and the good news about the pandemic if there, if there is some good news, is that we can do it.

So we we've proven as an industry that we can accelerate time to. And, and, you know, kudos to Pfizer and Madonna J and J who were able to successfully do that and get a vaccine to the masses. So Yeah. And not just numbers, but also as you said, representation and the people who were hearing their voice from is more representative as well.

Absolutely. And I think, I think that's a good point. I think there's still areas for us to grow and learn together in, right. Like there's a caster and base and the people, the sponsors, as well as we're continuing to administer these. It's not a panacea yet. We want to really understand how do we reach those minoritized populations?

How do we make clinical trial awareness? Easy for patients in need, who don't have other access, maybe there's registries, et cetera. I think there's also, there's a new association might hopefully I get the name, right, but it's like decentralized trials, research association or something like that who is having their inaugural meeting, I think in November of this year.

But there's all, I think there's some sponsorship from people in the room of that organization, but it's really helpful to think about how much commitment is going into understanding decentralized trials towards the patients now. Building off that point. I did wonder if you know, you cells, if your experience with decentralized a hybrid trials, trying to get a representative patient sample in terms of engagement and recruitment, other tips, strategies, ideas of how you can do that with hybrid or decentralized trials.

Complete. I'd love to hear. I mean, this is, this is always like someone earlier today mentioned humility when it comes to like DTX and growth, and this is where recruitment, we will always be evergreen in humility. Right. We're trying to get our patients. We're trying to get representative samples. So if anybody else has ideas that we met all last, let us know.

But I think. To be very, very thoughtful and recruitment. At some point you made to monitor in real time, someone spoke about dashboards earlier that socio-demographic variables that you are recruiting. So at some point maybe you are already reaching the amount that you want to hit with. You know, non-Hispanic white females.

And so you decide to limit the amount of those moving forward and open the spaces up for other minoritized population. There's also some panels that you can set to try to hit target census numbers and Qualtrax offers a panel of that nature as well. And of course the traditional things that many of us may be accustomed to in terms of being very, very thoughtful of your ads, your wording, your images, all of those things to really be inviting to people in addition to liaising with other providers and things that we've done, sort of traditional.

I think one size fits all. And I think somebody else said that too, with ads, right? It doesn't work right. An ad that might work in Boston in this downtown area, right around the language, right. About where you're trying to recruit patients versus someone in Vegas, right. Or a California or someone, you know, in areas that maybe the message is not well understood.

Maybe the wording on the message is not what they are. The, or, you know, want. And so I think having multiple ads, understanding your demographics, understanding your patient target, and then speaking sort of in their language to recruit them. But also, you know, there's a compliance aspect of what you can or cannot say, but I think it matters.

I've seen ads, you know, you know, basically working for a mental health company. I get targeted on Instagram with a bunch of ads, right. Whoever is monitoring that. So it's just, I think it just depends on the areas and demographics. I think those, we need to kind of come out of that one, one ad will work across the board in us.

Right. I don't think that's approach that's feasible or should be reasonable across the board for recruitment. Yeah, and I, I think it's important to, to look at new and novel ways of, of reaching underserved populations. So I'm doing a diversity panel in a couple of weeks with J labs. So Johnson and Johnson is obviously taking a close look at this.

I think they have a hundred million dollars invested in really improving overall diversity. But as we went out and talked with people, one of the things we heard consistently was. We need support as a community. And so all of you, all of us in this room have a lot of resources at our disposal and things like, you know, food deserts and impact that we can have.

And th the, the idea is that we create a long standing relationship and partnership with underserved communities, and it goes something like if you're serving. Those communities. And then, and then you have an ask at some point where you're saying, Hey, we have, we have a clinical trial. We'd love for you to participate in.

You've already built the relationship you've already established trust. And that community is going to be much more likely to want to engage. So I think there's a lot that we can be doing as an industry to really be kind of the tide that raises all the boats. It's a very important point. So actually building sustainable relationships and connecting with communities and meeting.

So as we look to the future, what do we think the future of clinical trials, the digital therapeutics holds? I mean, the, I think it's more of a, I think it's more of what we've been talking about a little bit, you know, continuing to lean into growth mindset and working together as an industry and a field to share.

What's what we've learned and how we continue to improve. All aspects, particularly again, the patient centricity and understanding what is most helpful for the patient. We've also talked to know Kimberly and I were chatting a little bit earlier today about interoperability, interoperability, right.

And really thinking about making sure everything is functioning on the backend. And because without that, it's not useful for any of the parties, parties involved. So I think that that is really helpful. I also. I think that, you know, making sure that we understand what different stakeholders want, you know, in terms of the sponsor regulatory the patient payers or other people that we are keen to you know, want to look at our data so that they reimburse our product.

So there's a lot of learning that needs to be done. And a lot of investment, I think, from the community as well, to continue to try decentralized trial, but it's here. It's already here. I mean, COVID made it here. We've said it time and time again, it really accelerated things. So it's, it's time to kind of.

Yeah. And I would just add to similarly seamless patient journey is really and, and I think what excites me is digital therapeutics really give us the opportunity to try that out. And I think it's something that hopefully will eventually become. True for everyone, even for large-scale pharma's phase three trials, but I think we're a long way from that yet.

So there still is kind of this idea that I've just switched different apps and I'm kind of going from here to there and, and we really have a vision that, that. Seamless. It becomes very easy for people. And I think for all of us, we know in our personal lives, you know, it's not easy, we're not doing it.

You know, we need that ease of use. I love my Instacart. I don't go to the grocery store anymore, you know, so it's just, you know, we have to create that experience for people if we want them to participate. So I think that's a, that's a big piece. I also think flexibility and choice. So you've heard a lot today about people talking about this idea of, you know, having a visit at home or going into the office.

Sometimes people want to decide, right? So I may say this week, I'm busy. I have a thousand things going on. So I want to have a remote visit. However, you know, two weeks from now, I might be feeling really bad. And I might say, you know, I really want to go see my physician and that shouldn't be rate limiting for people like our technology should support that people should be able to do that.

I love the idea of, you know, if you're in an urban region, you can pick your site. You can pick your time if you want to go in or, you know, So I think we need to, to be able to support that. And the last piece, I would just say an echoing, some of what Adam said earlier, too, about data collection.

We've got to have that standardized data. We've got to be thinking about it when we collect it. It's not just, you know, your old, old school EDC data. I like to say, but we've got all kinds of data coming in now from wearables and devices and your digital. All of that needs to be standardized so that you can get it out in a meaningful way.

Because if we're collecting a bunch of data and we can't really use it in a, in a meaningful way, then we've kind of missed the point. So I think the future of clinical trials we'll, we'll see more of.

I just want to say, I think we need to work on demystifying clinical trials. All right. I think I think of clinical trials and you think of like, something is happening. So, you know, we're testing something out. Right. I think it's important for the regulators. It's important for the payers. It's important for the patients, right.

Just demystifying what it is and educating. The newer generation of ACPS that are coming in understanding how clinical trials are run at what we're collecting and be part of that innovation as well. Right. And it should be somehow then become part of the clinical pathway of how they're looking at their patients and having this sort of availability of clinical trials available to them across the board.

Right. So it's, shouldn't be, you know, on some clinical trials.gov, you have to go in and see what's happening, but in this integration platform, Where everybody's sort of involved, right. As a community, we heard that throughout the day today, we need to work as a community, right? So we need to advance this innovation in clinical trials as a community, every person playing their role to the betterment of our society.

Absolutely. So I'd like to now open it up to the audience, to see what questions you might have for our panel about clinical trials and, and the next generation of what they're going to look like.

Yeah, I have a question. In regards to that question about what did the clinical trials at the future look like for not only digital therapeutics, but kind of all devices, you know, thinking about how, you know, with drugs as sort of a model, one of the major, I wouldn't say revolutions, but evolutions was going from your typical phase 1, 2, 3 to more adaptive design.

And that's something that I haven't really seen with devices and certainly not in digital therapeutics. So I'm just wondering if any of the panels, especially those of you who are actually have product products in the clinic, if you guys have thought about maybe different ways to assess, you know, and, you know, rather than going feasibility, pilot and and pivotal to something that's more modular.

And as you're getting data back from the patients, you know, in some of these models, To actually get more, more data and potentially a shorten the timeline between, you know, the start in market. That's a great question. So I can take that ask me developing medical devices. I think it's some, again, I think it's newer, right?

So we're just evolving and we're learning what to do. I think for us, I can speak from our experience or some of the digital therapeutics company we work for. If your core mechanism isn't changing, you can take the learnings of your previous trials that you've done. Right. And see what you can do to improve it.

Perhaps you don't have to run a pilot day. On new newer population, if there was some sort of maybe AI model, right. That we can leverage and run some sort of a feasibility, right? Like data you plug in the data, the AI models should be able to push it back and say, what's the likelihood of success, right?

For this pilot. And instead of recreating rerunning similar pilots across the board. Yeah, I think that, I think that's a great point and, you know, and there are ways to be creative here. And, and this group really brings together a bunch of creative, scientific minds. One thing, you know, to consider as like synthetic arms, adding that to your clinical trial.

So you have a method of getting real world data, you know, in a historical way and integrating that. So there are definitely ways to continue to think about creative science today. And I think also recognizing one trial, isn't going to answer all the questions like, you know, it's not necessarily going to answer the questions around return investment and or cost savings versus clinical effectiveness.

So, you know, it's, it's figuring out what's the purpose of this trial and what's the next step we're trying to take
out the questions.

Hi, I'm Cindy Williams from Quinton's a digital therapeutic for self self-management of diabetes, and really enjoyed the discussion. So thank you very much. I'm curious that you mentioned on the side of recruitment and kind of multi-site trials. And how do you just have this diverse, a population as possible?

The things that you talked about are all things that as a direct to consumer therapeutic provider, you would do. Really fine tune the messaging and the creatives and you target the audience as well just to acquire customers and then to apply that also to the, the recruitment for the trials. And I'm just curious about other things.

Are there areas where that doesn't work is where there's, for some reason, sort of regulatory, can you just take whatever you did in Happify and then we'll. Get consumers on your app and how you've targeted folks and apply that to join this trial or are there. Yeah, no, I think it's a good question. If I, if I understand correctly, Mon I'd love to hear your thoughts as well.

You know, so we have a suite of products, emotional support products that will Woebot how so our direct to consumer application is, is not the same as our prescription digital therapeutic pipeline. And so we would really want to be careful about the level of advertising and setting patient expectations about what participation would involve.

So it is slightly different from, from that route. In terms of our general kind of Woebot app that's readily available in market, we haven't fiscally invested in marketing to date that the people that come have just come that's very different than recruitment, where we are investing money for clinical trials to get a particular patient population.

So those dashboards dashboards are real time data that I was mentioning about monitoring. The amount or socio-demographic variables that people and their self identity as they come through has been really helpful in one of our recent national Institute of drug abuse, funded studies. We increased our male participant population by 10% to be over a third when it was just a quarter before.

So that was one way that we helped to do it. So I see that it's a little bit different than our general DTC. Thank you for the question. I agree. I think that the claims are different, right? So it's hard to then take the data that you have for your consumers and then apply it for a chronic condition population.

Right. What you can do is learn maybe usability of your app, right? Maybe software validation, data that could be used across the board. But I think when it comes to claims, there's limited amount where you can kind of transfer from your consumer apps to your a little bit more. Patient Pacific, you know, chronic conditions, population,

some questions from the online audience here. So what are some considerations regarding outcome measures slash metrics, the DTX trolls as compared to traditional pharma trials?

Well, in. I don't have a farming background, so I might have to, but I mean, thank goodness we're together in TTX and I don't know, like your experience has been MoMA Luna. We haven't been sending people in for labs, you know? And so a lot of pharmas pharmacotherapy trials, there might be scheduled labs where you have to get blood draws, et cetera, your analysis.

What have you, in order to understand patients in order to make sure your. Safety assessments regularly throughout the way. So there, there are some things that would be different in regards to that, but that being said to your earlier point, it sounds like the decentralized trial ecosystem can accommodate patients like patient preference and site selection.

Yes. So I think the end points are different, right? So the end points doesn't really, to me, they're kind of separate, right? So you got traditional trials, how you run trials, but your digital endpoints or your end points are based on what you were collecting, the claims you want to make or an indication for use.

Right. So I'm not sure the outcomes we're looking at on would be to D you know, depending on, again, your indication for you is it's not so much about the type of trial you're running. So that's what I would say. I think it just depends on what you're looking at. And it may be. Yeah. And I mean, the only thing I would add is, you know, obviously in big pharma, you know, you, you've got a lot of safety and advocacy, you know issues.

And I would say with digital therapeutics, we're looking at efficacy, you know, primarily. So I mean, obviously safety too, but if you're not adjusting, obviously a drug you know, a little bit different. But I do agree. I think, you know, you still, if you think about clinical research, you have a scientific hypothesis.

This is what your hypothesis is. This is what you believe the efficacy of that therapy is going to be, whether it's a drug, a medical device or digital therapeutic, and then you would expect your outcomes, data to support, you know, that. And regardless we want to stick to the protocol, right? We want to identify the methodology for assessing safety because regulatories can be quite interested in safety.

So I know a lot of pharma pharmacotherapy trials, as well as digital therapeutic trials utilize, you know, things like the Hamdi, the madrassas example, for examples, for outcomes, for depression or the Columbia suicide risk assessment scale. So things like that. It all depends on how you've set up your protocol as well.

One final question from the. What are your thoughts about potential needs for durability data? Meaning looking at what happens to the treatment effects once the VTX has stopped to say three to six months out is the Bohai of versus drugs. I'll give you that one. Chris, can you repeat the question? I'm not sure I heard all of it.

What are your thoughts about potential needs for durability data? Meaning looking at what happens to the treatment effects once the DTX has stopped I'd say three to six months. The high of vessels drugs. There's the bar higher. Yeah. So I, I think this is such a great question and I wish we had I'm trying to remember their names that Liz from Anthem today's earlier panel discussion, and I believe it was Ryan from another pair.

Thank you. Thank you, Chris. I knew you'd know you're with us to answer that. Very good question. Because to something we were mentioning earlier, I think different stakeholders might have different interest in duration, duration, durability, right? Some people might be like, Hey, show me what. Some folks might want to see six months or 12 months or even longer.

And it's also interesting as we look at digital therapeutics can set up frameworks for brief briefer interventions yet are the periods of durability, the same someone who might be on SSRI for six to 12 months, whatever. And what are the expectations for durability of remission there versus, you know, an eight week or 12 week DTX program.

So it's really something that I think we have to put our theoretical brains together for. And as well as the conversations with the stakeholders to understand what type of durability is most meaningful. Yeah. And I think related to that, like, Some of the ups in the mental health space or about skills training.

So it's interesting to think about, are people still using these skills, even when they're maybe not using the device or the app versus medication, which may be more, you've got to keep taking it right. In order to achieve a desirable clinical outcomes. That's a fantastic point. And how do we measure that?

What regularity? Exactly.

Any other questions? I'd like to think of very big,

closing out for the day. We have covered a lot of grounds within one day. I mean, it's hard to believe we have two days to come, but I did want to just close out with an analogy that came to mind as I was listening to the presentations today and that's around where our industry is at. And I think, you know, we've moved past our infancy.

We've moved past establishing clinical evidence to show that there's great value for digital therapeutics in each event. So I very much see us as in our adolescents and with adolescents comes lots of turbulence and potential for more growth and, and really thinking and being very mindful around whose attention we're trying to attract.

For what reasons, and what's the best approach to doing that as we talk about greater alignment between key stakeholders to achieve a unified vision of how we're going to move forward for the greater good of all society. Thanks so much, everyone have a wonderful night and we'll see you tomorrow morning.