About the FDA (U.S. Food and Drug Administration) Guidance Document “Computer Software Assurance for Production and Quality System Software”

FDA just released the new guidance document about “Computer Software Assurance for Production and Quality System Software” [1] on 13.09.2022. It is about the safety and quality of the software. It will be a guide and helpful document who work in pharmaceutical companies.

The FDA wants medical devices to increase product quality and safety. FDA issues this draft document to be guidance for IT and manufacturing personnel on software quality and security. As they stated on page 5 of the draft document “…FDA envisions a future state where the medical device ecosystem is inherently focused on device features and manufacturing practices that promote product quality and patient safety…” [1] Software must serve product quality and patient security. They can be helpful in manufacturing order, sequence, and testing; however, the main focus must be product quality and patient safety.

As said on page 6 of the draft document “…Traditionally, software validation has often been accomplished via software testing and other verification activities conducted at each stage of the software development lifecycle…” [1], traditionally software validation is performed under the software development lifecycle. However, as software, and hardware gets more complicated and more functional, they must be tested with a special risk-based approach. The software development lifecycle is not provided with appropriate safety for product quality right now. They do not establish confidence that the software is fit for its intended use. The software only measures and monitors the defects during production, software does not help to manufacture on the quality level. Because of these reasons, pharmaceutical companies must implement a risk-based approach, continuous monitoring, data monitoring, and unscripted testing of the software as said in the document “…This is important because manufacturers increasingly rely on computers and automated processing systems to monitor and operate production, alert responsible personnel, and transfer and analyze production data, among other uses. By allowing manufacturers to leverage principles such as risk-based testing, unscripted testing, continuous performance monitoring, and data monitoring, as well as validation activities performed by other entities (e.g., developers, suppliers), the computer software assurance approach provides flexibility and agility in helping to assure that the software maintains a validated state consistent with 21 CFR 820.70(i)…” [1].

As stated before, commercial off-the-shelf software does not require any testing other than vendor assessment and software installation and configuration. These can be enough for the widely used software. However, when the functionalities get complicated and various, a risk-based approach must take place within the company. As stated on page 9; “…However, if a manufacturer utilizes built-in functions of the COTS spreadsheet to create custom formulas that are directly used in production or the quality system, then additional risks may be present. For example, if a custom formula automatically calculates time and temperature statistics to monitor the performance and suitability of the curing process, then additional validation by the manufacturer might be necessary…” [1].

As the software gets more complicated and various, new guidelines must be prepared. As a leading institution, the FDA prepared this document to help pharma companies. It guides the companies to prepare a risk-based approach to software.

It begins with “Identifying the Intended Use”. As software gets complicated each day, its intended use becomes vague. The first step must be the identification of the intended use of the software. As stated in the “21 CFR 820.70(i))”, all production-related software must be tested for its intended use. FDA split the software into two categories, which are “software that is part of the production”, and “software that supports the manufacture”. Both of them must be tested under “21 CFR 820.70(i))” however, as “support software” carries much lower risk than direct production software, the effort for the validation can be low. As stated on page 8 “Both kinds of software are used as “part of” production or the quality system and must be validated under 21 CFR 820.70(i). However, as further discussed below, supporting software often carries a lower risk, such that under a risk-based computer software assurance approach, the effort of validation may be reduced accordingly without compromising safety.“ [1].

FDA wants companies to determine the intended use of software to develop the proper, neat risk-based approach. As custom functions of the software increase, testing must be performed individually and in more detail. Old software, which has only one functionality still can be tested with the risk-based approach by taking its intended use overall.

As the second step in software validation activities, companies must “Determine the Risk-Based Approach”. The risk-based approach depends on the possibilities. On the possibility that software can fail in “that” functionality, what will happen? The guidance document indicates just that. The risk-based approach must determine whether such a failure creates a major, critical, or minor defect. BCP, (business continuity plan), was created to take action after a disruptive event happened in the plants, etc. Software risk-based approach must be written in the same mentality. As stated in the document on page 9 “Instead, the risk-based analysis for production or quality system software considers those factors that may impact or prevent the software from performing as intended, such as proper system configuration and management, security of the system, data storage, data transfer, or operator error. Thus, a risk-based analysis for production or quality system software should consider which failures are reasonably foreseeable (as opposed to likely) and the risks resulting from each such failure. “.

Software is used on medical device risks and process risks. The guidance document discussed both of them. FDA considers software failure as a high process risk when “…its failure to perform as intended may result in a quality problem that foreseeably compromises safety, meaning an increased medical device risk. “. High process risk can occur when the critical parameters are affected by the software failure, testing performed by the software cannot be performed by the human operators with the same confidence, if adjustments are made by the data monitoring of software, labeling cannot be performed other than software, etc. These are all important process risks that can be resulted in bad consequences. If the company’s software’s intended use is one of these, a more thorough risk-based approach must be performed.

The risk-based approach must be detailed. It cannot be just two options “high process risk” and “not high process risk”. It must be split into different categories such as “low risk”, “moderate risk”, “intermediate risk” etc. As performed in the equipment risk assessments, actions to lower risks must be sequenced and performed. Determining the risk level of an error FDA gave an example on page 13 “A feature, function, or operation that could lead to severe harm to a patient or user would generally be high device risk. In contrast, a feature, function, or operation that would not foreseeably lead to severe harm would likely not be high device risk. In either case, the risk of the software’s failure to perform as intended is commensurate with the resulting medical device risk.”

Software development led to more automated processes controlled by the software. Testing must be executed to know what will happen when an error happens in the automated process.

FDA states that if there change and it affects the process even by low risk, it must be submitted in a 30-day notice. Manufacture process of medicines is highly complicated and interconnected. This can lead to risky changes that can affect manufacture. As stated in the FDA guidance report on page 12 [1], “In general, if a change may result in a quality problem that foreseeably compromises safety, then it should be submitted in a 30-day notice. If a change would not result in a quality problem that foreseeably compromises safety, an annual report may be appropriate. “. However, if the software is used for only data tracking, thresholds based on validated parameters, etc. failure of such software did not affect the process from a quality-based perspective. These software changes can be the subject of the annual report.

As the third step in software validation activities, companies must “Determine the Appropriate Assurance Activities”. After completing the risk assessment, companies must lower the risks associated with them by taking quality assurance precautions.

FDA suggested some risk-lowering activities including;

· Unscripted Testing such as;

o Ad-hoc testing

o Error-guessing

o Exploratory testing

· Scripted testing

o Robust scripted testing

o Limited scripted testing

As we see from the activities, FDA suggests more and more unscripted testing for software. Because unscripted tests are more versatile and more comprehensive, they can lead to findings hard-to-find bugs and out-of-scope errors. Equipment qualification is performed with prescriptions determined strictly. However, the equipment has known features for centuries they have prescribed actions and specialties. The first theory of software showed up in 1935 by Alan Turing, however, machines are at the center of our lives for centuries. We need to be more flexible; we need to communicate and understand them to control and lead them. This is the exact FDA’s stance according to the guidance document.

As additional assurance FDA suggests;

· “Activities, people, and established processes that provide control in production. “

· Manufacturer testing (performing conventional tests on the factory)

· Additional security and control programs, people.

· Additional supervisor software for determining software failure, trend, etc.

· “Using of Computer System Validation tools…”

· Iterative approach to testing (decreasing test cycles for low-risk software)

As the fourth step in software validation activities, companies must “Establish the Appropriate Record”. Data integrity and data collection have a special place in quality. Data must be collected about the software’s performance and integrity. The record, according to the FDA guidance document [1] comprises;

· Intended use

· Determination of the risk of the software feature

· Documentation of;

o Description of the testing

o Issues found

o Conclusion statement

o Established review and approval

There is a useful table about the “Examples of Assurance Activities and Records” on page 16 of the guidance document which explains the scripted and unscripted testing more holistically [1]. Also, there is an example of the record of assurance in a scenario. There are examples of risk assessment on pages 20 to 25.

FDA released this guidance because of the confusion and concern regarding the application of Part 11. As software gets more complicated, documents and guides must be changed and improved too. Introducing unscripted testing to software, is a big step for more flexible, more dynamic testing, also needed. The industry was waiting for guidance and FDA did not fail them.

Bibliography

[1] FDA, “Computer Software Assurance for Production and Quality System Software,” FDA, 13/09/2022.