Manufacturing - Greenleaf Health

This article was originally published as a guest column in Outsourced Pharma.

Considering the feverish pace of innovation in the field of AI/ML and the inevitable impact this family of technologies has on drug development, an overview of the approaches to AI/ML regulation by the leading medical product regulatory authorities, the FDA and European Medicines Agency (EMA), is timely. Below, we outline the documents and guidances the two regulators have released thus far, comparing and contrasting their areas of focus and concern.

A Comparison of the Definitions of AI and ML

Despite the lack of a universally accepted definition of AI among experts,¹ both regulatory agencies have settled on a working definition of AI.

In its definition, FDA acknowledges the breadth and multidisciplinary nature of the field, defining AI as “[a] branch of computer science, statistics, and engineering that uses algorithms or models to perform tasks and exhibit behaviors such as learning, making decisions, and making predictions.”² Meanwhile, FDA identifies ML as a subset of AI that allows “[m]odels to be developed by ML training algorithms through analysis of data, without being explicitly programmed.”³

EMA, however, takes a more mechanistic approach, defining AI as “systems displaying intelligent behavior by analyzing data and taking actions with some degree of autonomy to achieve specific goals.”⁴ Meanwhile, EMA’s definition of ML — “models [that] are trained from data without explicit programming” — mirrors FDA’s ML definition.

The FDA’s Approach

In May 2023, FDA began to consider the implications of AI/ML technologies for drug development with the publication of two discussion papers: Artificial Intelligence in Drug Manufacturing⁵ and Using Artificial Intelligence and Machine Learning in the Development of Drug and Biological Products.⁶ These two discussion papers highlight the agency’s areas of concern related to the incorporation of AI/ML in drug development and manufacturing.

Chief among these concerns are the governance, accountability, and transparency of AI/ML systems. For ML models, transparency and accountability are particularly challenging considering they are sub symbolic, or a “stack of equations — a thicket of often hard-to-interpret operations on numbers.”⁷ Thus, the nature of these systems makes its outputs difficult to interpret, presenting obvious regulatory challenges. To address these challenges, the FDA emphasizes the importance of “tracking and recording … key steps and decisions, including the rationale for any deviations and procedures that enable vigilant oversight and auditing.”⁸ The problem of transparency and accountability is further compounded by competitive concerns, as many of these models are proprietary.

Data quality is another concern the FDA addresses in its discussion papers, noting that the application of AI/ML systems in drug manufacturing can significantly increase the frequency and volume of data exchanges in the manufacturing process, thereby exponentially increasing the quantity of data. This increase in data output may require new considerations relating to data storage, retention, and security. In terms of data input, sponsors must be cognizant of any preexisting biases in the training data, as ML systems can easily duplicate or even amplify these biases.

The FDA also highlights reliability as another area of focus and concern. As recent experiences with large language models may attest, some AI systems are prone to hallucination, “a phenomenon where AI generates a convincing but completely made-up answer.”⁹ Indeed, in a recent study on AI hallucination, a group of researchers prompted a chatbot to generate a list of research proposals with reliable references. Of the 178 references provided by the chatbot, 69 did not have a digital object identifier (DOI), while 28 did not turn up on internet searches.¹⁰ Thus, FDA’s concern about reliability seems well founded, especially in the context of a drug development program.

The EMA’s Approach

Following the FDA’s recent publications, the EMA released a reflection paper¹¹advocating for a risk-based approach that considers patient safety and the reliability of development data. In April 2021, the European Union (EU) introduced a coordinated plan and a regulation proposal for AI, aimed at promoting innovation and ensuring AI benefits society. The reflection paper is an extension of this plan, outlining considerations for AI usage in drug development and emphasizing regulatory oversight based on risk assessment. It highlights three key concerns, specifically, the need for:¹²

risk-based oversight,
the establishment of strong governance for AI deployments, and
guidelines covering data reliability, transparency, and patient monitoring.

The paper categorizes the risk of AI application in drug development stages. AI use in early drug discovery is deemed low risk, while its use in clinical trials spans various risk levels depending on factors like human oversight and potential impact on regulatory decisions. To manage risks, the paper recommends transparent AI models (the idea to fully trace information flow within a ML model), cautious handling of issues like overfitting (the result of non-optimal modeling practices wherein you learn details from training data that cannot be generalized to new data), and appropriate performance assessment metrics. Ethical and privacy issues, such as human agency and oversight, technical robustness and safety, privacy and data governance, transparency, accountability, societal and environmental well-being, diversity, non-discrimination, and fairness, are addressed and outlined.

Specific considerations for AI usage include ensuring accurate AI-generated text through quality review procedures and high-risk AI decisions in precision medicine settings, AI use in manufacturing adhering to quality risk management principles, and the importance of regulatory interactions during development. The reflection paper acknowledges that it is not an exhaustive source of regulatory insight on AI but serves as a starting point for further discussions. Stakeholders can provide feedback until Dec. 31, 2023.

Conclusion

While both the FDA and the EMA strive to provide a framework that balances innovation and patient safety, nuances emerge in their respective approaches. Stakeholder input and evolving industry practices are critical to shaping future regulatory guidelines. Collaboration among regulators, manufacturers, and researchers will be pivotal in fostering a transparent, accountable, and efficient AI ecosystem that enhances the development and deployment of medical products for the betterment of global health.

Stanford University, “Artificial Intelligence and Life in 2030,” 2016, 12; https://ai10020201023.sites.stanford.edu/sites/g/files/sbiybj18871/files/media/file/ai100report
10032016fnl_singles.pdf.
FDA, “Using Artificial Intelligence and Machine Learning in the Development of Drug and Biological Products,” (May 2023), https://www.fda.gov/media/167973/download.
Ibid.
EMA, “5 Reflection paper on the use of Artificial Intelligence (AI) in 6 the medicinal product lifecycle.” 13 July 2023, https://www.ema.europa.eu/en/documents/scientific-guideline/draft-reflection-paper-use-artificial-intelligence-ai-medicinal-product-lifecycle_en.pdf.
FDA, “Artificial Intelligence in Drug Manufacturing,” May 2023, https://www.fda.gov/media/165743/download.
FDA, “Using Artificial Intelligence and Machine Learning in the Development of Drug and Biological Products,” May 2023, https://www.fda.gov/media/167973/download.
Mitchell, Melanie. Artificial Intelligence: A Guide for Thinking Humans, p. 12.
FDA, “Using Artificial Intelligence and Machine Learning in the Development of Drug and Biological Products,” p. 20.
Athaluri SA, Manthena SV, Kesapragada VSRKM, Yarlagadda V, Dave T, Duddumpudi RTS. Exploring the Boundaries of Reality: Investigating the Phenomenon of Artificial Intelligence Hallucination in Scientific Writing Through ChatGPT References. Cureus. 2023 Apr 11;15(4):e37432. doi: 10.7759/cureus.37432. PMID: 37182055; PMCID: PMC10173677.
Ibid.
European Medicines Agency. (2023, July 13). Reflection Paper on the Use of Artificial Intelligence (AI) in the Medicinal Product Lifecycle. European Medicines Agency. https://www.ema.europa.eu/en/documents/scientific-guideline/draft-reflection-paper-use-artificial-intelligence-ai-medicinal-product-lifecycle_en.pdf.
European Medicines Agency. (2021, August 16). Artificial Intelligence in Medicine Regulation. European Medicines Agency. https://www.ema.europa.eu/en/news/artificial-intelligence-medicine-regulation.

Greenleaf Regulatory Landscape Series

For nearly two decades, the Food and Drug Administration (FDA or the Agency) has supported the development of innovative manufacturing technologies that modernize quality management systems and provide greater quality assurance across medical product supply chains. Evidence of the FDA’s commitment to the development and implementation of this technology appeared most recently in January 2021 with the creation of a new collaboration between the FDA and the National Institute of Standards and Technology (NIST), under a Memorandum of Understanding (MOU), aiming to increase U.S. supply chain resilience and advanced domestic manufacturing by adopting innovative technologies, such as continuous manufacturing processes, as well as artificial intelligence and machine learning. The purpose of this effort is to mitigate risks of supply chain disruptions leading to product shortages, a concern that has grown more problematic under an increasingly globalized approach to medical product production.

Since the onset of the coronavirus pandemic (COVID-19) in Spring 2020, similar efforts to support uptake of advanced manufacturing technologies have been met with a greater sense of urgency across other parts of the federal government as well. Consistent with these efforts, actions to on-shore and increase domestic manufacturing capacity and modernize medical product manufacturing have been bolstered by pandemic-related legislation and executive orders alike.

Federal Initiatives Related to Medical Product Supply Chains During COVID-19

Even though acute awareness of the potential benefits of advanced manufacturing resurfaced during the pandemic, the origins of the FDA’s pursuit of modern quality systems through advanced manufacturing began in August 2002 with the launch of its “Pharmaceutical Current Good Manufacturing Practices (cGMPs) for the 21st Century” initiative. This initiative evaluated the Agency’s pharmaceutical regulatory programs and released a final report in September 2004, introducing a new risk-based quality assessment system that would replace the chemistry, manufacturing, and controls (CMC) review process, encouraging implementation of process analytical technologies, and framing innovative technologies as essential components of a modern quality system. Later that same year, the FDA issued final guidance for industry, “Process Analytical Technology (PAT) and a Framework for Innovative Pharmaceutical Development, Manufacturing, and Quality Assurance,” establishing a regulatory framework intended to support more innovation and quality modernization in pharmaceutical production.

Part of the Agency’s intention in its initial push for adopting advanced manufacturing technologies stemmed from concerns about quality management deficits associated with conventional batch manufacturing processes. The decades-old batch manufacturing model consists of frequent testing, storage, and shipping across regions, making it time-sensitive and more prone to product contamination. In addition, the batch manufacturing model was linked to reactive and wasteful discarding of final drug products due to quality issues. Instead, the FDA envisioned transitioning to a risk-based, quality management framework involving a more controlled and efficient pharmaceutical production regime.

Benefits of a Continuous Manufacturing System Versus

a Batch Manufacturing System

Pursuant to this vision, the FDA supported implementation of advanced manufacturing technologies, such as continuous manufacturing, by establishing platforms for engagement between the Agency and companies interested in producing products using innovative technologies. These platforms involve CDER’s Emerging Technology Program (ETP), CBER’s Advanced Technology Teams (CATT), and CDRH’s Case for Quality initiative. Additional final guidance on “Advancement of Emerging Technology Applications for Pharmaceutical Innovation and Modernization” continued to support companies seeking to adopt advanced manufacturing technologies. Under this regime, nine drug products manufactured with advanced technologies, one of which uses biomanufacturing processes, have been approved by the FDA – three of these products were approved in 2020 according to CDER’s Office of Pharmaceutical Quality (OPQ) Annual Report.

The National Academies of Sciences, Engineering, and Medicine (NASEM) held a workshop in June 2020 on barriers that hinder pharmaceutical manufacturing innovation, finding that external, regulatory challenges “loom large.” At this workshop, CDER-OPQ Director Mike Kopcha, Ph.D., distilled what he saw as regulatory barriers to the adoption of advanced manufacturing, including the need to fit new technologies into existing regulatory frameworks and the need for global regulatory harmonization. That is, because the current regulatory framework is “based in offline testing of batch processes, regulatory requirements do not currently translate well into new manufacturing technologies that allow for varied batch sizes, inline analytics, and higher-fidelity methods for detecting batch-to-batch variation.”

While the FDA has done much in the way of encouraging industry implementation of innovative and emerging technologies, barriers to achieving more substantial implementation persist. Earlier this year, NASEM released a report highlighting gaps where future guidance and clarity would be helpful in mounting identified regulatory barriers. These include:

Consideration of more fluid and targeted guidance that is shorter and published promptly to allow for industry comment.
Greater focus on underlying technology, as opposed to individual product approvals.
Creation of new mechanisms and pilot programs for incorporation of industry input and collaboration.
Expanded scope of the ETP to create greater opportunities for interested companies to engage.

With that said, the FDA is working to finalize its February 2019 draft guidance on “Quality Considerations for Continuous Manufacturing,” in which it has the opportunity to respond to gaps identified by NASEM, as well as others. With Acting Commissioner Janet Woodcock, M.D., at the helm, the FDA’s commitment towards advanced manufacturing technologies is likely to remain a top priority – Dr. Woodcock has long championed the increased adoption of advanced manufacturing technologies throughout her FDA career. Even if a permanent nominee other than Dr. Woodcock were to take her place as FDA Commissioner, advanced manufacturing is still seen as a key element to modern, risk-based quality systems and part of secure and efficient supply chains. Therefore, the FDA’s support of the adoption of advanced manufacturing will continue. Additionally, as previously noted, efforts to strengthen medical product supply chains by focusing on advanced technologies and re-focusing on U.S.-based manufacturing has enjoyed recent bipartisan support, in large part due to lessons learned from the pandemic. Thus, although industry adoption has been slow, advanced manufacturing technologies are expanding in scope and will likely become more of a norm in a post-COVID-19 world.

Resources

FDA & NIST MOU, “Accelerating the Adoption of Advanced Manufacturing Technologies to Strengthen Our Public Health Infrastructure” (January 2021)
Trump Executive Order, “Buy American for ‘Essential Drugs’ and Medical Supplies” (August 2020)
Biden Presidential Campaign, “Rebuild U.S. Supply Chains and Ensure the U.S. Does Not Face Future Shortages of Critical Equipment” (June 2020)
Biden Executive Order, “Sustainable Public Health Supply Chain” (January 2021)
Biden Executive Order, “America’s Supply Chains,” (February 2021)
FDA Initiative, “3D Printing in FDA’s Rapid Response to COVID-19” (November 2020)
FDA PREPP Initiative, “FDA’s COVID-19 Pandemic Recovery and Preparedness Plan (PREPP) Initiative: Summary Report” (January 2021)
FDA In Brief, “FDA Provides Update on COVID-19 Pandemic Recovery and Preparedness Plan Initiative” (April 2021)
FDA Initiative – Final Report, “Pharmaceutical CGMPs for the 21^st Century: A Risk-Based Approach” (September 2004)
FDA Final Guidance, “PAT – A Framework for Innovative Pharmaceutical Development, Manufacturing, and Quality Assurance” (October 2004)
FDA Final Guidance, “Advancement of Emerging Technology Applications for Pharmaceutical Innovation and Modernization” (September 2017)
FDA Draft Guidance, “Quality Considerations for Continuous Manufacturing” (February 2019)
NASEM Workshop – Proceedings in Brief, “Barriers to Innovations in Pharmaceutical Manufacturing” (September 2020)
NASEM Report, “Innovations in Pharmaceutical Manufacturing on the Horizon: Technical Challenges, Regulatory Issues, and Recommendations” (2021)