A Cross-Disciplinary Training Program for the Advancement of Medical Countermeasures

In September 2012, the United States Department of Health and Human Services Food and Drug Administration’s (FDA) Ofﬁce of Counterterrorism and Emerging Threats and the University of Texas Medical Branch at Galveston entered into a collaborative educational partnership for the academic development of a robust training program for good laboratory practices in high-biocontainment environments. The implementation of problem-based learning techniques encouraged researchers and regulators to cross-educate each other on the challenges related to the conduct of regulated studies in biological safety level 4 (BSL-4) laboratories and identiﬁed solutions that were acceptable from scientiﬁc and regulatory perspectives. The result was the development of a face-to-face course entitled Achieving Data Quality and Integrity in Maximum Containment Laboratories and an additional online companion course covering the FDA regulation, Good Laboratory Practice for Nonclinical Laboratory Studies (21 CFR Part 58). The course offers a unique opportunity for members of the regulatory and scientiﬁc communities to solve complex issues in an interactive educational environment, especially for the advancement of medical countermeasures (MCMs) via the FDA Animal Rule (21 CFR Parts 314 and 601 (2002)). The program occurs annually and is expanding in 2019 to include a course addressing data quality and integrity in clinical trials involving the evaluation of MCMs for high-consequence pathogens. To date, 311 individuals have attended the course. Based on attendance numbers, diversity of participation (by afﬁliation and area of expertise), and self-reported evaluation results, course attendees indicate that the training program addresses a knowledge gap and that they will implement knowledge gained.

In the introduction to the supplementary information of the Animal Rule is this statement: ''All studies subject to this rule must be conducted in accordance with preexisting requirements under the good laboratory practices (21 CFR part 58) regulations 3 and the Animal Welfare Act (7 U.S.C. 2131 et. seq.).'' 4 Because of the increased safety risk and highly infectious nature of the agents that fall under the purview of the Animal Rule, these studies are often performed in high or maximum (BSL-3 or -4) biocontainment laboratories. The logistics of assuring accurate and reliable data as collected and transferred from a BSL-4 (ie, maximum biocontainment) laboratory, conducting the study under regulatory oversight, maintaining animal records equating to clinical case files, characterizing the diseasecausing agent, and designing the appropriate study to satisfy the regulatory reviewers that study data are equivalent to the outcome in humans are challenging and complex. According to the FDA Compliance Program Inspection of Nonclinical Laboratories Conducting Animal Rule-Specific Studies, ''The Good Laboratory Practice (GLP) regulations (21 CFR part 58) were developed as a quality system for nonclinical safety studies, and thus, do not address the Animal Rule-specific studies. FDA, however, considers the GLP regulations to be a well-established and relevant framework (e.g., definitions, procedures, roles and responsibilities, and controls) for ensuring data quality and integrity in Animal Rule-specific studies.'' 5 Only 3 products-pyridostigmine bromide, Cyanokit, and levofloxacin (for pneumonic plague)-were approved in the 10 years (May 2002 through May 2012) following publication of the Animal Rule, and none were novel drugs or therapeutics. 6 While the FDA was developing guidance (a concept paper 7 in 2008 followed by a Draft Guidance for Industry 8 in 2009), there were no existing GLP training mechanisms addressing the complexities of performing regulated studies in maximum biocontainment and no existing education mechanisms for FDA inspectors or regulators.
To gain a better understanding of the complexities involved in executing regulated studies in maximum biocontainment laboratories, the FDA Office of Counterterrorism and Emerging Threats (OCET) and the University of Texas Medical Branch at Galveston, Texas (UTMB), collaborated in 2012 to design and implement an educational training program. The purpose of the program was to cross-educate members of the medical countermeasure community to enable the conduct of regulated studies in maximum biocontainment laboratories.
A Funding Opportunity Announcement, 9 published in the Federal Register on June 19, 2012, reported FDA's intention to develop an educational training program with an award commencement date of September 1, 2012. The training program would include 2 parts: a face-to-face course with a partnering on-line course introducing the GLP regulations. The activity met the FDA's OCET mission to provide leadership and participate in the public health community and with the military defense community in helping to advance the development, evaluation, and approval of medical countermeasures for use against threats involving chemical, biological, radiological, or nuclear (CBRN) agents. The role of UTMB included the Galveston National Laboratory (GNL), known for educational excellence in the sciences, medicine, and research, as well as for its Laboratory Biosafety Training Program and the Institutional Office of Regulated Nonclinical Studies, which offered an extensive, high-quality GLP training program to support faculty and staff conducting nonclinical studies in BSL-3 and -4 laboratories.

Background
According to the 2012 education Funding Opportunity Announcement in the Federal Register, there were no BSL-4 facilities capable of performing pivotal studies under GLP. 9 Development of medical countermeasures against category A BSL-4 agents (eg, Ebola virus, Marburg virus, Nipah virus, and Lassa fever virus) requires specialized laboratory space for safe handling and containment. 10 Although these environments are specialized in design, safety, and personnel training, acceptance of study results used to draw a regulatory conclusion is still contingent on the quality and integrity of the data as outlined in the GLP regulations. The Funding Opportunity Announcement identified challenges in meeting GLP requirements in BSL-4 laboratories to include appropriate data recording, record keeping, inspections, and equipment validation. Further challenges, including challenges in BSL-3 laboratories, were identified during education development.
As potential drug and therapeutic candidates advance, multiple studies must be conducted to refine the animal model(s) that adequately mimic disease in humans such that product efficacy and dosing may be translational. The Funding Opportunity Announcement states, ''Training on the development of strategies to meet GLP requirements in high and maximum biocontainment laboratories can be realized when everyone has a common understanding of the challenges and requirements. In such a case, the scientific validity and regulatory acceptance of the study can be ensured early on, reducing the need for repeat studies, thereby reducing the numbers of animals needed to address the scientific and regulatory objectives.'' 9 The training program was designed to decrease redundancy and link GLP regulatory requirements with the BSL-4 laboratory environment to increase the efficiency of FDA data review and facilitate approval of medical countermeasures.

Course Development
A face-to-face course was systematically designed following the ADDIE model for instructional design; the acronym ADDIE represents the 5 major phases of instructional systems development: analysis, design, development, implementation, and evaluation. 11 To analyze knowledge gaps and identify current issues, the agenda and content from a prior FDA workshop conducted at the Centers for Disease Control and Prevention (CDC) in Atlanta, GA, on August 23 to 25, 2011, were reviewed, and a subsequent symposium was organized and conducted with members of the medical countermeasure community. The symposium, entitled ''Ensuring the Quality and Integrity of Animal Model and Efficacy Studies at BSL-4,'' took place on October 25 to 26, 2012, in the GNL at UTMB. Objectives of this symposium were to (1) identify and discuss the challenges of applying the principles of the GLP regulations to Animal Rule studies; (2) seek consensus on the most critical elements that could potentially affect data quality and integrity in maximum biocontainment and explore potential options for implementing practices to address them; and (3) identify topics for the training course.
The original symposium provided novel engagement with experts from the FDA, academia, business, and industry addressing topics critical to the success of animal model qualification and subsequent product testing and approval via the Animal Rule. Identified BSL-4 regulated study challenges included: physical restrictions inside containment, personnel sustainability (training considerations), shared equipment with research staff, lack of regulatory awareness (applicability of the FDA Animal Rule), scientific versus regulatory terminology, limited space for specimen and record retention, conduct of FDA inspections, use of telemetry, facility differences, document transfer out of containment, applicability of GLP during animal model development, sponsor expectations, the role of a study director, surrogate endpoints, and others.
Representatives from the FDA, the National Institutes of Health (NIH), and UTMB met immediately after the close of the symposium to develop a preliminary list of topics, identify the target audience and methods for faculty selection, and determine the appropriate course size. Comments were also solicited from business and industry through the Alliance for Biosecurity.
During the design phase, topics were grouped together by subject matter, and a draft course outline was subsequently developed. Learning objectives for each topic were prospectively developed, and faculty were identified based on subject matter expertise. Education materials were developed via a mixture of active and passive learning activities, including lectures, case studies, panel discussions, videos, and demonstrations. A mock BSL-4 laboratory exercise was included to provide a hands-on demonstration of the differences between BSL-2 and BSL-4 environments. A broad target audience was identified to facilitate crosseducation-specifically, medical countermeasure product sponsors, contracting officers, scientists (including graduate students), veterinarians, pathologists, physicians, nurses, quality assurance professionals, regulators, reviewers, and policymakers. A steering committee consisting of FDA and UTMB subject matter experts reviewed and provided continual feedback during the design and development process.
Course development included a pilot course, held April 1 to 5, 2013, at the GNL and used UTMB's mock BSL-4 training laboratory. While course attendance was open, interested individuals were placed on a waiting list and admitted to the pilot course in a manner that assured representation from the target audience; course faculty were also encouraged to attend the entire course.
Initial funding allowed for the course to be offered annually from 2013 to 2017 at UTMB; however, the course was relocated in 2014 to the NIH campus in Bethesda, MD, in order to accommodate key US government participants. The NIH agreed to collaborate by providing access to the NIH mock BSL-4 training laboratory facility and financially offsetting the costs associated with use of meeting rooms and equipment. The resulting move also allowed for support of the course from all members of the National Interagency Confederation of Biological Research (NICBR) at the Fort Detrick campus in Maryland. The original grant funding period occurred from September 1, 2012, through August 31, 2017. The FDA and UTMB entered into a second 5-year funding cycle from 2017 to 2022 in order to continue and expand the training program. There are no fees associated with attending the course.
In addition, an online prerequisite course introducing the GLP regulations was developed as part of the training program for attendees not familiar with the regulation requirements. The 11-module online course was developed by a UTMB subject matter expert and reviewed by FDA subject matter experts prior to launch. A course assessment was developed consisting of a pool of 50 multiple-choice and true or false questions. The questions were reviewed for relevance, accuracy, and clarity by subject matter experts who possessed content and/or educational development experience. Language was clarified and questions omitted based on feedback. Comments were also solicited during pilot testing in 2013 and were reviewed by the FDA Continuing Education Office in advance of final question selection for the 2014 course. A pool of 49 questions has remained unchanged since 2014. From the pool, 25 questions are randomly selected, with a passing score of 80% required to receive a certificate of successful completion. Summative evaluation gauged the effectiveness of the online course design, delivery platform, and content. credits and 4.75 CNE credits were awarded in 2018 for the face-to-face course, and 11.5 CME credits were awarded for the corresponding online GLP course.

Evaluation Results
Formative course evaluation of the 2013 pilot course occurred through anonymous pre-and post-course knowledge assessments consisting of questions provided by course faculty. Results were shared with course developers and used as a general indicator of knowledge gain. The topics of facility management responsibilities, animal model qualification, and data integrity issues when using polyolefin dunkable paper were topics that showed normalized knowledge gain based on response averages.
Summative evaluation occurred for the 2013 pilot course and every course offering thereafter through the distribution of a post-course anonymous evaluation designed to measure the effectiveness of course content against established objectives and to provide an opportunity for feedback. Evaluation results were compiled and provided as independent reports by the entity granting continuing education units. The results were reviewed, and the content subsequently reassessed and modified, as necessary, prior to the next course.
Based on the evaluation results from the pilot course, education gaps were identified, and the following topics were subsequently added to the 2014 course: veterinary pathology, disease agent characterization, good documentation practices, supportive care and euthanasia, the role and impact of quality assurance (QA), testing facility management, and clinical application of Animal Rule studies and how these studies can save lives when bridged to the proper emergency use authorization data. Topics consistently addressed in all courses (conducted annually since 2013) include: the Animal Rule regulations, FDA inspec-tions of Animal Rule studies, the GLP regulations, IACUC protocols, surrogate endpoints, BSL-4 equipment, the role of the study director, and physiologic monitoring via telemetry. The overall course structure is outlined in Table 1.
In 2018, the topic of international considerations was expanded to include faculty members from Public Health England and the Paul Erlich Institut. Dr. Anthony Fauci, director of the National Institute of Allergy and Infectious Diseases, served as the 2018 distinguished closing keynote speaker; his presentation addressed the challenge of pandemic preparedness. Information presented provided a global perspective on the emergence of infectious diseases and the need for medical countermeasures.
To date, self-reported evaluation results have been consistently positive, indicating the education materials addressed knowledge and skill gaps and the learning activity will improve patient and public health outcomes ( Table 2). Course evaluations also allowed attendees to elaborate on changes they intend to integrate into practice. Table 3 includes a sample of responses from 2013 to 2018 course evaluations demonstrating planned changed behavior as a result of completing the educational activity. In 2018, 87% of evaluation respondents indicated they would attend the course in the future (n = 22), and 96% responded they would recommend the course to a colleague (n = 23).
Attendance is tracked and trended by category based on registration information: international (ie, non-US), industry, academia, or government, with FDA participation tracked as a subset of the government category. International participation (faculty and/or attendee) has been documented in all years and has included participants from Public Health England; the Liberian Institute for Biomedical Research and the National Public Health Laboratory, Ministry of Health and Social Welfare, Republic of Liberia; the Australian Animal Health Laboratory; the Paul Ehrlich Institut, Germany;  Table 4 illustrates the distribution of attendance by category for course attendees based on course registration demographics. Total attendance peaked in 2015; however, the number of individuals admitted to the course was subsequently restricted in order for attendees to effectively participate in the mock BSL-4 laboratory activity. According to 2018 post-course evaluation results, participants self-identified their profession as follows: 4.4% nurses, 4.4% practicing physician (MD or DO), 21.7% quality assurance, 43.5% DVM/PhD, and 21.2% other government professional (23 respondents). Each year, the course reaches capacity, and a subsequent waiting list for the next year's course is initiated. Combined attendance for all 6 years totals 311 attendees.

Discussion
Prior to the course, the only document available to provide guidance on the conduct of Animal Rule studies was a draft concept paper titled ''Animal Models-Essential Elements to Address Efficacy Under the Animal Rule'' (September 2008). This subsequently evolved into the ''Draft Guidance The guidance offers clarity on the requirement for studies to be conducted under the Good Laboratory Practice regulations by adding the qualifier ''to the extent practicable'' for model defining natural history studies and applicable animal efficacy studies. Using the GLP regulations as a quality system provides a mechanism to ensure the quality and integrity of data from these studies.
From 2013 to present, 10 additional products have been approved through the Animal Rule pathway (see Figure 1). The timeline illustrates products approved since the codification of the Animal Rule (2002-2018) coinciding with the maturation of FDA's Draft Concept Paper into Final Guidance and the award of the FDA education grant. Although course impact cannot be directly correlated to the increased number of product approvals, self-reported course evaluation results indicate attendees will apply knowledge learned, course content addressed a need or gap in their knowledge or skill, and they intend to integrate newfound knowledge and skills into their practice or work environment. Course interest remains high, as each year an attendee wait list is generated several months prior to course registration. The course continues to educate the scientific and regulatory community that is involved with the development and approval of medical countermeasures for high-consequence pathogens.

Training Program Expansion
The original grant funding period occurred from September 1, 2012, through August 31, 2017. As a result of pos-itive evaluation results, demand, and attendance records, the FDA and UTMB entered into a second 5-year funding cycle from 2017 to 2022 to continue and expand the program. The expansion includes the addition of a partnering course for achieving data quality and integrity in clinical trials involving the evaluation of medical countermeasures for high-consequence pathogens. A workshop was held on April 25, 2018, with invited stakeholders from the regulatory, scientific, and medical communities to gather information on the education gaps. The workshop was part of an overall gap analysis and needs assessment for the development of the new course. Leadership from the National Ebola Training and Education Center (NETEC), composed of subject matter experts from Emory University, New York Health and Hospitals Corporation, and the University of Nebraska Medical Center, are involved in continual course development. An advisory committee that included volunteers from the April 2018 workshop along with other identified subject matter experts reviewed the preliminary grid and provided feedback on content, learning objectives, and potential course faculty. Attendees of the pilot course included representatives of the targeted audience; formative and summative evaluations will evaluate the effectiveness of course design, course content, and course delivery. Information will be used to develop the first full course in 2020. There are no planned course fees associated with either the nonclinical or clinical course.
The first full clinical course is scheduled to be held in Course organizers will continue to explore opportunities for the curriculum to satisfy continuing education requirements, allowing medical and regulatory professionals to obtain credits for participating in the courses.

Conclusion
The training program began in 2013 with the course Achieving Data Quality and Integrity in Maximum Containment Laboratories, which is offered annually and provides a unique opportunity for members of the regulatory and scientific communities to discuss and solve complex issues in an interactive educational environment. Faculty and attendees of varied educational backgrounds and professional fields collaborate to increase their knowledge of the maximum biocontainment laboratory environment and to discuss mechanisms for the conduct of regulated studies under a quality system (ie, GLP to the extent practicable) that assures the quality and integrity of study data. The course includes applicability of the FDA Animal Rule pathway for medical countermeasure development and approval. A partnering self-paced modular online course provides an overview of the FDA GLP regulatory requirements. Based on the attendance numbers, diversity of participation (by affiliation and area of expertise), and selfreported evaluation results, the course has been successfully designed and executed to achieve the objective of cross-educating the medical countermeasure community to promote data quality and integrity in maximum biocontainment laboratories. Course evaluation data show intended application of gained knowledge. A post-course survey of past attendees is planned to analyze whether prior attendees implemented changes as a result of course attendance.
The expansion of the training program to include the addition of a companion course titled Achieving Data Quality and Integrity in Clinical Trials Involving High-Consequence Pathogens will allow the program to provide education for nonclinical and clinical studies conducted in support of the development and approval of medical countermeasures. The goal of the companion clinical course is to provide a learning environment that cultivates collaboration of ideas; yields tools for clinical study conduct; enhances mutual understanding of clinical, scientific, and regulatory complexities; and promotes the data quality and integrity derived from these regulated studies.
The educational outcome of the training program is a better-prepared, cohesive community of clinical, scientific, and regulatory experts mutually engaged in, and committed to, the development, research, and approval of medical countermeasures for high-consequence pathogens.