The intersection of AI and pharmaceutical regulation is rapidly evolving, with potential implications for IP strategy. The recently issued draft guidance on the Considerations for the Use of AI To Support Regulatory Decision-Making for Drug and Biological Products (FDA-2024-D-4689) is the FDA’s first attempt to standardize how AI-generated data and models will be evaluated in the context of drug safety and effectiveness. New AI applications in the drug regulatory process are emerging every day, from personalised medicine to clinical trial design and manufacturing quality control. As innovators make more and more use of these technologies, they will face the challenge of conforming to the regulatory transparency requirements whilst protecting their IP.
Use of AI to support regulatory decision making
The focus of the FDA guidance is the application of AI to processes directly supporting regulatory decision-making about the safety, effectiveness, or quality of drugs. As such, the guidance explicitly excludes pre-clinical drug discovery activities that occur prior to any interaction with regulatory authorities. However, whilst much of the public interest for the use of AI in pharma has focused on drug discovery tools such as AlphaFold, AI companies are already moving beyond drug discovery to develop solutions for the whole drug development process.
The FDA identifies numerous potential applications for AI throughout drug development. These include computer models for pharmacokinetic, pharmacodynamic, and toxicological studies, predictive modelling for dose determination and the integration of data from different sources such as clinical trials, genetic databases, and social media to improve understanding of a disease.
Current AI applications in drug development already demonstrate the regulatory complexities that the FDA guidance seeks to address, particularly in the realm of personalised medicine. For Moderna’s cancer vaccine mRNA-415/V940, for example, AI algorithms analyse patient tissue samples to identify tumour-specific neoantigens for creating patient-unique mRNA vaccines. This approach raises questions for the regulatory authorities, such as how the safety and efficacy of the vaccine can be assessed when each patient receives a potentially unique vaccine sequence.
Challenges and limitations of AI systems
The FDA identifies several key challenges associated with AI use in drug development. First, the FDA guidance points out that the variability in quality, size, and representativeness of training datasets may introduce bias and raise questions about the reliability of AI-driven results. Additionally, the complexity of the underlying AI models can make it difficult to understand how these systems reach their conclusions. This lack of interpretability necessitates what the FDA terms “methodological transparency”, namely detailed disclosure of the methods and processes used to develop AI models in regulatory submissions.
Here we see a parallel with patent law, specifically with the requirement in patent law for the invention to be sufficiently disclosed (Europe) or described and enabled (US). The patent law on the sufficiency of AI inventions is still developing. The question thus remains whether the regulatory requirements relating to the use of AI in pharmaceutical development will be more or less stringent than the patentability requirements for the AI itself.
To satisfy the requirement for methodological transparency, the FDA guidance places particular emphasis on disclosure of the quality and characteristics of the training data used to develop an AI model. To ensure that the data is as representative of the target patient population or manufacturing process involved, the guidance recommends describing data collection, processing, annotation, storage, and control procedures, along with rationales for dataset selection and methods for establishing labels or annotations. The guidance also suggests including descriptions of model inputs and outputs, architecture, features, selection processes, and training hyperparameters. For models using pre-trained components, the guidance indicates that the datasets used for pre-training should be specified, and there should be a description of how the models were obtained or developed. The level of disclosure required by the guidance is therefore currently far greater than is necessitated for obtaining a patent relating to an AI model.
The credibility assessment approach
At the heart of the FDA guidance is our old friend “credibility”. Credibility (and its synonym plausibility) is of course a loaded term in European patent law, applied to the evidence standard for therapeutic inventions and the patentability of software inventions, see for example, Case Law of the Boards of Appeal I-D.9.2.8. The FDA Guidance also chooses “credibility” to define the evidence standard for AI in the context of the drug regulatory process.
FDA guidance provides a 7-step risk-based credibility assessment process for AI in the drug regulatory process:
Step 1: What is the question of interest to be addressed by the AI model?
Step 2: What is the “context of use” for the AI model? The 2nd step involves defining the “context of use” (COU) for the AI model. The COU is supposed to define the AI system’s role and scope, describing in detail what will be modelled and how model outputs will be used.
Step 3: What is the risk of the AI model? This step requires an assessment of the risk of using the AI model, which combines two factors: model influence and decision consequence. Model influence represents the contribution of AI-derived evidence relative to other contributing evidence used to inform the question of interest. In other words, what is the benefit of using AI over more traditional approaches? Decision consequence is supposed to describe the significance of an adverse outcome resulting from an incorrect decision by the AI. For these two steps, the guidance provides a risk matrix showing how combinations of model influence and decision consequence determine overall model risk. For instance, if an AI system is the sole determinant for a high-stakes decision (high model influence) and errors could result in serious patient harm (high decision consequence), the model carries high risk requiring stringent credibility assessment activities.
Step 4: Develop a plan to establish the credibility of AI model output within the COU. Step 4 involves developing a “credibility assessment plan” tailored to the model risk and COU. For high-risk models, this plan should include comprehensive documentation of model architecture, training data characteristics, development processes, and evaluation methods. The guidance provides detailed recommendations for describing models, training data, model development processes, and evaluation methodologies. It is clear from the details and complexity of this process that innovators developing an AI model should engage with the FDA as early as possible to ensure they are documenting the correct information and mitigating relevant risks appropriately.
Steps 5-7 involve executing the credibility assessment plan, documenting the results of the credibility assessment plan and establishing whether the AI is adequate for the COU.
When model credibility proves insufficient for the intended context of use, the guidance suggests some actions that may be taken to remedy the situation. These include reducing the dependency on the AI model by submitting alternative evidence, conducting more stringent credibility assessment activities or improving the AI model performance with additional training data.
Impact on IP strategy for AI
One of the most critical decisions for AI companies and companies using AI is whether to disclose and protect their models via patent filings, or to keep the details of the model a trade secret. What is clear from even this brief outline of the FDA 78-step process, is that clear documentation and description of what, when and where AI was used in regulatory submissions will be critical. It is unclear from the guidance how much of this information may be made public. If full or even partial public disclosure of the intricate details of an AI model used in the regulatory process is required by the FDA, then this may significantly impact the IP strategy of the AI provider, and necessitate a patent-focused, as opposed to a trade secret-focused, strategy
Furthermore, even if the full information provided to the FDA about an AI model is not made publicly available, there are still potential IP implications. In many cases, an innovator company may not itself have all the details about all the AI models to the level of detail stipulated by the FDA guidance. In these circumstances, the AI provider may have to provide more information about their product and processes to the pharma company than they are comfortable with from a trade secrets perspective. Additionally, there are some aspects of the disclosure required by the guidance that simply cannot be protected by patents, including the detailed information about the training data.
Response from the industry: What about foundation models?
Over a hundred comments on the guidance have been submitted by interested parties, including big pharma and AI biotech companies. There are a number of recurring themes in the comments including the need for more comprehensive examples of how the 7-step credibility assessment process will work. There are also calls for greater harmonisation with international standards, particularly the OECD’s AI definition and the International Medical Device Regulators Forum’s Good Machine Learning Practice principles, to ensure global consistency in regulatory approaches.
A number of technical challenges were also highlighted by commentators including the unique considerations required for foundation models and agentic AI platforms. Foundation models are large-scale models, such as ChatGPT, are trained on vast datasets. Foundational models can then be fine-tuned for numerous downstream applications, including those in the drug regulatory process such as patient stratification, biomarker discovery, and clinical trial design. LLMs, for example, may well be used to write clinical trial protocols and regulatory submissions. Foundation models thus operate as general-purpose platforms that may not fit into context-of-use categorisation. However, the draft guidance focuses on context-specific credibility assessments. Commenters such as the AI biotech Owkin wondered whether there should therefore be a dual assessment structure, evaluating both the foundation model itself for data governance and privacy compliance and each specific application derived from it.
Final thoughts
It is clear from the draft guidance that, as the use and complexity of AI models in the pharmaceutical industry inexorably grows, the FDA is likely to face an ever increasing burden of reviewing the safety and efficacy of AI models. Innovators will also face an increasing burden of documenting and reviewing the AI models they use throughout the regulatory process, including those provided by third parties. Layered on top of this is the complexity of the IP strategy that will be required to ensure alignment between the regulatory process and maintaining a competitive IP position. A key lesson that I have learnt throughout her career is reinforced by the emergence and challenges of AI in the drug regulatory process. To be effective, IP cannot exist in a silo, but must be integrated and aligned with the drug development and R&D teams and the entire regulatory process. This latest FDA guidance highlights that alignment between the IP and regulatory strategy for AI models and AI generated data continues to be absolutely crucial.
Author: Rose Hughes (Patent Attorney)
Further reading
- USPTO call for comments: Impact of AI on patentability
- OpenAI’s large language model (LLM) patents
- New USPTO Guidance on the use of AI in precision medicine
- When is the inventor of an AI model also an inventor of the model’s output? A closer look at the USPTO Guidance for AI-Assisted Inventions
- IP strategy for AI-assisted drug discovery
- AlphaFold: From Nobel Prize to drug-discovery gold mine?
- Insilico Medicine: Lessons in IP strategy from a front-runner in AI-drug discovery
- Too broad, too early? AI platform for cell analysis found to lack technical character and sufficiency (T 0660/22, Cell analysis/NIKON)
This post is based on a previous article on IPKat.
