FDA Draft Guidance on PCCP: Data Management | RegDesk

Table of Contents

The Food and Drug Administration (FDA or the Agency), the US regulating authority in the sphere of healthcare products, has published a draft guidance document dedicated to the regulatory requirements for a predetermined change control plan (PCCP) to be included in marketing submissions related to medical devices utilizing artificial intelligence/machine learning (AI/ML) technologies. Once finalized, the guidance will provide an overview of the existing regulatory framework and additional clarifications and recommendations to be considered by medical device manufacturers and other parties involved to ensure compliance thereto. At the same time, provisions of the guidance are non-binding in their legal nature, nor are they intended to introduce new rules or impose new obligations. Moreover, the authority explicitly states that an alternative approach could be applied, provided such an approach is in line with the underlying legislation and has been agreed with the authority in advance. 

The approach described in the guidance is intended to reduce the regulatory burden for medical device manufacturers with respect to modifications to existing products, the way they should be introduced, and the regulatory requirements associated thereto. Under the general rule, modifications included in a predetermined change control plan will not require a separate marketing submission, provided the plan was approved (authorized) by the FDA. 

The scope of the guidance covers, inter alia, the elements to be included in a Modification Protocol, which is a part of PCCP. In particular, the authority provides examples of elements of Modification Protocol components for machine learning device software functions (ML-DSFs).

Content of a Modification Protocol

As it was mentioned before, a Modification Protocol should include the four main components addressing the following aspects:

1. Data management practices;
2. Re-training practices;
3. Performance evaluation protocols; and
4. Update procedures. 

The abovementioned aspects should be covered for each modification included in the Description of Modifications, which is another important section of a PCCP. At the same time, as it is stated by the authority, medical device manufacturers are also allowed to include additional components, should they reasonably deem it necessary. 

With respect to the above, it is important to mention that the regulatory approach could be subject to changes based on the new information becoming available to the authority. The authority explicitly states that the items below [in the guidance] are also not an exhaustive list of topics that a manufacturer is expected to cover, and all questions may not apply to all marketing submissions. Furthermore, the authority is entitled to request additional components to be included in a Modification Protocol for specific medical device types. The level of detail the authority will expect will depend on the complexity of specific modifications and risks associated thereto. 

The guidance further describes each of the main aspects outlined hereabove and highlights the key points to be taken into consideration by medical device manufacturers in this respect.

Data Management

In this section, the authority expects medical device manufacturers to provide detailed information on the way the data will be collected and used for training and testing. 

For instance, with respect to data collection, the aspects of being addressed include, inter alia, the following:

• What are the inclusion/exclusion criteria for data collection, and how are they linked to the intended use population?
• Will the data be collected prospectively or retrospectively? Will the data set include consecutive cases within a given date range?
• Are there any plans for enrichment or stratified sampling to include specific patient subgroups (e.g., sex, age, race)?
• What is the number and geographical distribution of data collection sites?
• What are the measures to mitigate potential unwanted bias in learning or performance estimation, for example, due to issues related to new training or testing data, respectively? 

Apart from the aspects outlined hereabove, the document also pays attention to assurance of data quality, as the latter significantly impact the actual performance of an ML-DSF. In this respect, the main questions to be addressed are:

• What techniques will be employed to fortify data consistency and completeness?
• What are the strategies used to promote data authenticity, transparency, and integrity?
• How will potentially missing data elements within a case/record be handled? 
• Are there criteria for including/excluding cases/records based on data quality, and what are the criteria and rationale?
• If data might be excluded as a result of the quality assurance process, what methods are planned to minimize the impact on the generalisability of training and accuracy of testing? 

The authority also mentions the importance of ensuring the traceability of data issues to specific records and also establishing proper controls to prevent unauthorized third parties from accessing the data and making changes thereto. 

Another aspect addressed in the guidance with respect to data management includes the ones related to the determination of reference standards to be used in the context of device training and testing. The authority mentions that the particular method used for this purpose should be justified, with the said justification to be provided to the authority. 

The document also pays attention to the matters related to the sequestration of test data sets, which ensures medical device manufacturers will have no access to the test data set for ML-DSF development. The main questions to be addressed in this respect are:

• What strategies will be employed at the outset of data collection to shield the test data set from the ML-DSF development?
• What specific procedures should be followed so that the test data set remains sequestered during re-training?
• If test data are planned to be used multiple times for performance evaluation, what measures are in place to prevent unwanted bias from being introduced through ML model manufacturers learning substantial information about the test data set and results? 

In summary, the draft guidance issued by the FDA covers the most important aspects to address in a predetermined change control plan. The document describes in detail the approach to be applied in the context of data management in order to ensure the proper performance of ML-based products.

Sources:

https://www.fda.gov/regulatory-information/search-fda-guidance-documents/marketing-submission-recommendations-predetermined-change-control-plan-artificial

How Can RegDesk Help?

RegDesk is a holistic Regulatory Information Management System that provides medical device and pharma companies with regulatory intelligence for over 120 markets worldwide. It can help you prepare and publish global applications, manage standards, run change assessments, and obtain real-time alerts on regulatory changes through a centralized platform. Our clients also have access to our network of over 4000 compliance experts worldwide to obtain verification on critical questions. Global expansion has never been this simple.