Really interesting analysis of the incentives of clinical trials! Having that many stakeholders definitely makes solving for optimality very complex, but lots of room for improvement from how things are done now. I don't know a lot about how clinical trials are run but while researching gene and cell therapies for an essay I learned about the RMAT designation and accelerated approval pathway, which has a mechanism for pooling data (for similar manufacturing protocols), which allows small sponsors to basically run trials together. I wonder if similar structural mechanisms can be extended to trials outside gene and cell therapies?
Also, the mention of game theory and repeated games reminded me of this incredible "game" (explorable explanation) by Nicky Case. Very useful for understanding the complex mechanisms behind the theory: https://ncase.me/trust/
Played the simulation with my morning coffee and I wish they taught this at all schools :). One unique thing about clinical trials is the layers *on top* of the “simple” drug approval game. When you start layering in different indications (patient finding, trial administration complexities), time to market (first in market, second, follow-on), geographic time of approval (US first, EU first, etc), market access strategies (how will the approved strategy get to patients? Will insurance companies put the drug on their formulary?) the game gets really complex, really fast. Inspired by the race to get GLP-1s, I tried to do some modeling with time to market, but as soon as differential equations came into play, I had to put part 2 down for another day :).
Fascinating article. Given the recent rise of China as a source of innovation (documented on Biotech hangout for example), I wonder if this collaborative approach may gain traction (necessity being the mother of invention). One of China’s advantages is the ability to significantly accelerate both pre-clinical and early clinical trials vs US. Now the downside of such an approach which prioritises speed above all else, may be trial quality.
Interesting proposal, Saul, and I think some of this has been put to test. TL;DR: Possible to do, but ideally not a single-country trial. IMHO, a few things need to be in place for this to work for US regulators:
1/ Trial setup and data must meet FDA standards. This includes GCP-compliant conduct, clear protocol and PI quals, and the ability for FDA to verify the data in whatever mode of inspection it chooses (including unannounced and on site). If the latter is not possible (e.g., for geopolitical tension issues) then this would be a non-starter.
2/ Representativeness and genetic relevance. Honestly, I think US trials have an UNDER-representation of East Asian populations today, so I think a China-based trial would augment data. However, a trial based only, say, on a Han Chinese population, might require additional data or confirmatory trials elsewhere to show generalizability.
We can look at precedent here from a couple of years ago. Eli Lilly and its partner Innovent Biologics ran a lung cancer trial for an anti-PD1 inhibitor in China and submitted those data to FDA for a US approval. FDA ultimately rejected the approval with a vote of 14-1, citing two issues. One was the comparator arm chosen. This study went for chemotherapy as standard of care, even though NSCLC has different established SoC modalities in the US today. Secondly, the lack of racial/ethnic diversity could not assure that the drug would be safe and efficacious in a more diverse US population.
Given the recent carnage at the FDA, it’s difficult to see much novelty happening any time soon. Have to wait until the dust settles (whenever that is).
This was an absolutely amazing take! Also whenever I get the intersection of game theory and clinical trials I am all ears, lol. I have developed a solution to optimize participant referrals. The inefficiency in the space is frustrating, however I feel like your voice will push people to challenge their way of thinking and working. Those changes could truly lead to better therapies and outcomes for patients.
Thanks Melanie! Good luck with your quest. It is so frustrating how many people want to get into clinical trials, and yet how many trial sites close without a single enrollment… and how it seems like those supply / demand lines rarely cross!
I've been thinking about this a lot in the context of our start-up, ImYoo, and I wonder if one potential way of nudging the system in the collaborative direction could be to have a platform for trial participants to directly contribute their information about their participation in a clinical trial, run by a third party. That could feed the "AI-Powered Trial Engine". Given the growth of consumer data rights laws, I wonder if now is a good time to build such a foundation. The participants could get some tokens or credits (or maybe good old-fashioned dollars) for their contributions, and the third party could aggregate and sell that data to buyers. Surely failed clinical trial data is of value to many players in the space - other pharma companies, the sponsor themselves (if for example the data includes things not collected as part of the trial), or groups that want data to do better at the "Prediction Market for Clinical Trials".
Have you seen efforts in this direction, or have data or anecdotes for or against it?
Very interesting idea (ps: big fan of the ImYoo crew!). I have not seen this in practice as far as I can remember. Most of what I’ve seen is data collection (e.g., from patient advocacy groups) to indicate whether someone is interested in participating in a trial, should one become available.
The one thing to consider is how much (or little) participants know about the clinical trial they’re in. I have actively participated in two RCTs for example. I could reasonably share what the studies were about, but still don’t know neither my randomization status nor the results of the study (granted, both are ongoing). I know what was on the informed consent form, but I’m not sure I have the rights to share that info with a third party. The rest of what I know can also be found in clinicaltrials.gov. Is the mere knowledge that a participant was in a trial (perhaps alongside one’s medical record and self reported data) useful as part of a database?
Maybe second guessing my thought experiment here, but if the prediction market in itself becomes too valuable/lucrative, will there be incentives to go on the hunt for participants to see if they had some info that would give an investor an early edge? 😅 That could actually backfire!
Going on the hunt for participants to find clinical trial info already happens! Especially in rare diseases with small trials, investors in public companies are constantly monitoring message boards, and I suspect finding patients to see their results
Hmm, that's really interesting! I didn't know that. Not sure how I feel about investors snooping around on message boards for details. If that's already happening, might as well make this information more publicly available so there's an even playing field.
Interesting, I am curious to hear if you do eventually get that information! I would hope so, even if for your own medical history. I'd go so far as to say I think it should be illegal to not release that, if it's not illegal already. Otherwise, yeah - I don't think participation in the trial itself is enough info - to get a lot more value, you need to know that they did or did not get a placebo. Alternatively, you could get a sample from them and biobank it. You could then assess who got placebo vs not by seeing who has the drug in their system, or if there were any measurable biological changes.
Really interesting analysis of the incentives of clinical trials! Having that many stakeholders definitely makes solving for optimality very complex, but lots of room for improvement from how things are done now. I don't know a lot about how clinical trials are run but while researching gene and cell therapies for an essay I learned about the RMAT designation and accelerated approval pathway, which has a mechanism for pooling data (for similar manufacturing protocols), which allows small sponsors to basically run trials together. I wonder if similar structural mechanisms can be extended to trials outside gene and cell therapies?
Also, the mention of game theory and repeated games reminded me of this incredible "game" (explorable explanation) by Nicky Case. Very useful for understanding the complex mechanisms behind the theory: https://ncase.me/trust/
That was great and in some way exactly what I was trying to get at! Thank you for sharing :).
Played the simulation with my morning coffee and I wish they taught this at all schools :). One unique thing about clinical trials is the layers *on top* of the “simple” drug approval game. When you start layering in different indications (patient finding, trial administration complexities), time to market (first in market, second, follow-on), geographic time of approval (US first, EU first, etc), market access strategies (how will the approved strategy get to patients? Will insurance companies put the drug on their formulary?) the game gets really complex, really fast. Inspired by the race to get GLP-1s, I tried to do some modeling with time to market, but as soon as differential equations came into play, I had to put part 2 down for another day :).
Next week I go back to posting about proteins. 🥲
Fascinating article. Given the recent rise of China as a source of innovation (documented on Biotech hangout for example), I wonder if this collaborative approach may gain traction (necessity being the mother of invention). One of China’s advantages is the ability to significantly accelerate both pre-clinical and early clinical trials vs US. Now the downside of such an approach which prioritises speed above all else, may be trial quality.
Interesting proposal, Saul, and I think some of this has been put to test. TL;DR: Possible to do, but ideally not a single-country trial. IMHO, a few things need to be in place for this to work for US regulators:
1/ Trial setup and data must meet FDA standards. This includes GCP-compliant conduct, clear protocol and PI quals, and the ability for FDA to verify the data in whatever mode of inspection it chooses (including unannounced and on site). If the latter is not possible (e.g., for geopolitical tension issues) then this would be a non-starter.
2/ Representativeness and genetic relevance. Honestly, I think US trials have an UNDER-representation of East Asian populations today, so I think a China-based trial would augment data. However, a trial based only, say, on a Han Chinese population, might require additional data or confirmatory trials elsewhere to show generalizability.
We can look at precedent here from a couple of years ago. Eli Lilly and its partner Innovent Biologics ran a lung cancer trial for an anti-PD1 inhibitor in China and submitted those data to FDA for a US approval. FDA ultimately rejected the approval with a vote of 14-1, citing two issues. One was the comparator arm chosen. This study went for chemotherapy as standard of care, even though NSCLC has different established SoC modalities in the US today. Secondly, the lack of racial/ethnic diversity could not assure that the drug would be safe and efficacious in a more diverse US population.
https://www.fiercepharma.com/pharma/lilly-innovent-hit-fda-no-go-discounted-pd-1-immunotherapys-lung-cancer-bid-after-bleak
Dr. Pazdur of the Oncology CoE at FDA even penned a NEJM article on this topic: https://www.nejm.org/doi/10.1056/NEJMp2116863
Given the recent carnage at the FDA, it’s difficult to see much novelty happening any time soon. Have to wait until the dust settles (whenever that is).
Yes, there's that, too 🥲
This was an absolutely amazing take! Also whenever I get the intersection of game theory and clinical trials I am all ears, lol. I have developed a solution to optimize participant referrals. The inefficiency in the space is frustrating, however I feel like your voice will push people to challenge their way of thinking and working. Those changes could truly lead to better therapies and outcomes for patients.
Thanks Melanie! Good luck with your quest. It is so frustrating how many people want to get into clinical trials, and yet how many trial sites close without a single enrollment… and how it seems like those supply / demand lines rarely cross!
I've been thinking about this a lot in the context of our start-up, ImYoo, and I wonder if one potential way of nudging the system in the collaborative direction could be to have a platform for trial participants to directly contribute their information about their participation in a clinical trial, run by a third party. That could feed the "AI-Powered Trial Engine". Given the growth of consumer data rights laws, I wonder if now is a good time to build such a foundation. The participants could get some tokens or credits (or maybe good old-fashioned dollars) for their contributions, and the third party could aggregate and sell that data to buyers. Surely failed clinical trial data is of value to many players in the space - other pharma companies, the sponsor themselves (if for example the data includes things not collected as part of the trial), or groups that want data to do better at the "Prediction Market for Clinical Trials".
Have you seen efforts in this direction, or have data or anecdotes for or against it?
Very interesting idea (ps: big fan of the ImYoo crew!). I have not seen this in practice as far as I can remember. Most of what I’ve seen is data collection (e.g., from patient advocacy groups) to indicate whether someone is interested in participating in a trial, should one become available.
The one thing to consider is how much (or little) participants know about the clinical trial they’re in. I have actively participated in two RCTs for example. I could reasonably share what the studies were about, but still don’t know neither my randomization status nor the results of the study (granted, both are ongoing). I know what was on the informed consent form, but I’m not sure I have the rights to share that info with a third party. The rest of what I know can also be found in clinicaltrials.gov. Is the mere knowledge that a participant was in a trial (perhaps alongside one’s medical record and self reported data) useful as part of a database?
Maybe second guessing my thought experiment here, but if the prediction market in itself becomes too valuable/lucrative, will there be incentives to go on the hunt for participants to see if they had some info that would give an investor an early edge? 😅 That could actually backfire!
Going on the hunt for participants to find clinical trial info already happens! Especially in rare diseases with small trials, investors in public companies are constantly monitoring message boards, and I suspect finding patients to see their results
The pseudo market for clinical trials is biotech but I agree a purer version would be interesting. https://x.com/plainyogurt21/status/1878566217335607466?s=46&t=-LxTDgsdTu2N6UPGgjAwUg
Love the X thread! Thanks for sharing, Adu!
Hmm, that's really interesting! I didn't know that. Not sure how I feel about investors snooping around on message boards for details. If that's already happening, might as well make this information more publicly available so there's an even playing field.
Interesting, I am curious to hear if you do eventually get that information! I would hope so, even if for your own medical history. I'd go so far as to say I think it should be illegal to not release that, if it's not illegal already. Otherwise, yeah - I don't think participation in the trial itself is enough info - to get a lot more value, you need to know that they did or did not get a placebo. Alternatively, you could get a sample from them and biobank it. You could then assess who got placebo vs not by seeing who has the drug in their system, or if there were any measurable biological changes.