Contamination Recovery Rate Using Statistical Process Control Sean Toler October 5, 2017
Statisitics and Microbiology Challenges On Data Sets Cleanroom microbiology testing results have a majority of samples with results of 0 CFU. On Test Assays Different tests have different recovery abilities. On Personnel Statistical knowledge among microbiologists varies person to person. Resource Constraints Tools need to be flexible to allow use for multiple sample types. External Presentation 2
What is Contamination Recovery Rate? Enumeration Individual results (Count / Plate) Compare magnitude to existing Limits React when limits exceeded Contamination Recovery Rate By Sample Site / Type (%) Regardless of count magnitude (1=70CFU) Samples over time (e.g. monthly, weekly) 70 CFU/Plate 11% Recovery Rate External Presentation 3
Why Contamination Recovery Rate? Microorganisms may be in a clump when recovery assay occurs. Can lead to colony growth that makes it difficult to count/compare results to Action/Alert Limits External Presentation 4
Why Contamination Recovery Rate? Here is a normality test of the presterilization bioburden example used later on in this presentation a. Normally distributed data allows use of more statistical tools b. Trend Analysis much easier to technically justify with normal distribution External Presentation 5
Why Contamination Recovery Rate? Different recovery methods have different recovery percentages: a. Example: Surface Sampling Difference in recovery ability Inherent differences in personnel technique Membrane Filtration Direct Swab Contact Plate Flexibility: Can be used on any non-sterile assay: a. Product BB, EM, WFI b. Especially useful in Grade C/D environments (more likely to recover) c. We strive for no growth, so a grow/no grow monitoring system is desirable/useful. External Presentation 6
What is Statistical Process Control? Definition Statistical process control involves getting a process to consistently operate to its full potential and continually improving in performance through the use of control chart tools. About Data Sets Variation always exists. Even identical processes using the same equipment, personnel, product, etc. will experience variation. Variation can be routine or exceptional. Process Behavior Charts Data points plotted in a time series. Central Line (Mean) plotted. 3σ limits plotted for upper control and lower control. Contamination Recovery Rate Individual Values charts are utilized when subgroup n=1. External Presentation 7
Recovery Rate Statistical Process Control Analyzing Data Set via Individual Values Control Charts 3σ (UCL) 2σ (UCL) 1σ (UCL) Center Line (Mean) 1σ (LCL) 2σ (LCL) 3σ (LCL) Time External Presentation 8
Month 1 Month 2 Month 3 Month 4 Month 5 Month 6 Month 7 Month 8 Month 9 Month 10 Month 11 Month 12 Month 13 Recovery Rate (%) How to Identify Exceptional Variation Rule 1 Rule 1: One data point that is greater than the 3σ UCL. 16 14 12 3σ (UCL) 2σ (UCL) 10 1σ (UCL) 8 6 4 Mean 1σ (LCL) 2σ (LCL) 3σ (LCL) 2 0 Time Based on Nelson s Rules. We have highlighted 4 rules most useful for our recovery rate analysis. External Presentation 9
Month 1 Month 2 Month 3 Month 4 Month 5 Month 6 Month 7 Month 8 Month 9 Month 10 Month 11 Month 12 Month 13 Recovery Rate (%) How to Identify Exceptional Variation Rule 2 Rule 2: Two out of three consecutive data points that are greater than the 2σ UCL. 16 14 12 10 8 6 4 3σ (UCL) 2σ (UCL) 1σ (UCL) Mean 1σ (LCL) 2σ (LCL) 3σ (LCL) 2 0 Time Based on Nelson s Rules. We have highlighted 4 rules most useful for our recovery rate analysis. External Presentation 10
Month 1 Month 2 Month 3 Month 4 Month 5 Month 6 Month 7 Month 8 Month 9 Month 10 Month 11 Month 12 Month 13 Recovery Rate (%) How to Identify Exceptional Variation Rule 3 Rule 3: Four out of five consecutive data points that are greater than the 1σ UCL. 14 12 10 3σ (UCL) 2σ (UCL) 8 1σ (UCL) 6 Mean 4 1σ (LCL) 2 2σ (LCL) 0 3σ (LCL) Time Based on Nelson s Rules. We have highlighted 4 rules most useful for our recovery rate analysis. External Presentation 11
Month 1 Month 2 Month 3 Month 4 Month 5 Month 6 Month 7 Month 8 Month 9 Month 10 Month 11 Month 12 Month 13 Recovery Rate (%) How to Identify Exceptional Variation Rule 4 Rule 4: Eight consecutive data points that are greater than the mean. 35 30 25 3σ (UCL) 2σ (UCL) 20 1σ (UCL) Mean 15 10 1σ (LCL) 2σ (LCL) 3σ (LCL) 5 0 Time Based on Nelson s Rules. We have highlighted 4 rules most useful for our recovery rate analysis. External Presentation 12
Responses to Process Instability (Exceptional Variation) Justifiable to Look for Assignable Cause: When Cause Is Identified, Act: If Assignable Cause Is Not Immediately Identifiable, Keep Track: Special Project / Initiative may be required External Presentation 13
Month 1 Month 2 Month 3 Month 4 Month 5 Month 6 Month 7 Month 8 Month 9 Month 10 Month 11 Month 12 Month 13 Recovery Rate (%) Voice of the Process vs. Voice of the Customer 12.00% 10.00% 8.00% ISO 8 3σ (UCL) 2σ (UCL) 1σ (UCL) Voice of the Customer Requirements that the product / process must meet Voice of the Process 6.00% Mean 1σ (LCL) How the process is actually performing 2σ (LCL) 4.00% 3σ (LCL) 2.00% The Voice of the Process will be within the Voice of the Customer range for a process within control. 0.00% Time External Presentation 14
Voice of the Process vs. Voice of the Customer Important! Never confuse the control chart limits with specification limits. Control Chart limits are calculated each time from the process data and represents Voice of the Process Guide for predicting behavior and/or preemptively act when variation in data exists. Specification limits represent Voice of the Customer requirements product must meet. External Presentation 15
Case Study Data Disclaimer All data shared throughout the presentation has been developed to illustrate the concepts of the SPC approach. External Presentation 16
Intro to Case Study #1 Background The following slides contain information on viable surface samples (EM) collected on a monthly basis at facility with product that is Aseptically Processed. Scope All surface samples in scope were collected in an ISO Class 8 environment. For each month, an average of ~250 surfaces samples were collected. Specific Case Details At the time of the data collection, statistical process control was not in use to analyze the data (this is a retroactive review). The facility has a specification contamination recovery rate limit of 10% (USP <1116>) External Presentation 17
Month 1 2 3 4 5 6 7 8 Month 2 3 4 5 6 7 8 9 Month 10 3 4 5 6 7 8 9 Month 10 11 4 5 6 7 8 9 Month 10 11 12 5 6 7 8 9 Month 10 11 12 13 6 7 8 9 Month 10 11 12 13 14 7 8 9 Month 10 11 12 13 14 15 8 9 Month 10 11 12 13 14 15 16 9 Month 10 11 12 13 14 15 16 17 Month 11 12 13 14 15 16 17 18 Month 12 13 14 15 16 17 18 19 Month 13 14 15 16 17 18 19 20 Recovery Rate (%) Environmental Monitoring Surfaces - Case Study #1 12.00% 10.00% 8.00% 6.00% 4.00% Out of Specification Limit (Rule 1; Rule 2 Voice of Customer) Exceptional Variation (Rule 1 Voice of the Process) Corrective Action 3σ (UCL) ISO 8 3σ (UCL) 3σ 3σ 2σ (UCL) 3σ 2σ (UCL) 3σ 2σ (UCL) 2σ 1σ 1σ (UCL) (UCL) 2σ 1σ (UCL) 1σ 1σ Mean (UCL) Mean (UCL) 1σ Mean Mean 1σ (LCL) (LCL) 1σ 1σ (LCL) 2σ (LCL) 1σ 2σ (LCL) 3σ (LCL) 2σ (LCL) 3σ (LCL) 2σ (LCL) 3σ (LCL) 3σ (LCL) 2.00% 0.00% Time External Presentation 18
Summary of Case Study #1 What Happened? Starting in Month 14, the facility would have seen an uptick in recovery rate that would have led up to an Exceptional Variation rule violation (Rule 1-one point above the 3σ limit) in Month 18; however, at this time no Preventative Action took place because the facility was not using SPC. The Month 19 data showed that data was above the USP <1116> guidance limit for contamination recovery rate when it exceeded 10%. The reaction led to a Corrective Action which drove the recovery rate back down to the mean in Month 20. Significance If using SPC and contamination recovery rate, the out of specification condition could have been potentially avoided because the previous month signaled Exceptional Variation, which would lead to a Preventative Action. SPC offers proactive response driven by data. External Presentation 19
Intro to Case Study #2 Background The following slides contain mix tank product solution bioburden samples collected on a monthly basis at a facility with product that is Terminally Sterilized via moist heat. Scope All bioburden samples in scope were collected in a Grade D environment from one mix tank. For each month, an average of ~125 samples were collected from the specific mix tank. Specific Case Details At the time of the data collection, statistical process control was not in use to analyze the data (this is a retroactive review). This facility uses a P-chart to determine the monthly limit for ratio of samples that are above alert limits for trend analysis. External Presentation 20
Month 1 2 3 4 5 6 7 8 9 Month 10 2 3 4 5 6 7 8 9 Month 10 11 3 4 5 6 7 8 9 Month 10 11 12 4 5 6 7 8 9 Month 10 11 12 13 5 6 7 8 9 Month 10 11 12 13 14 6 7 8 9 Month 10 11 12 13 14 15 7 8 9 Month 10 11 12 13 14 15 16 8 9 Month 10 11 12 13 14 15 16 17 9 Month 10 11 12 13 14 15 16 17 18 Month 11 12 13 14 15 16 17 18 19 Month 12 13 14 15 16 17 18 19 20 Month 13 14 15 16 17 18 19 20 21 Recovery Rate (%) Mix Tank Bioburden SPC Trend Analysis - Case Study #2 6.00% 5.00% 4.00% 3.00% 2.00% 1.00% Exceptional Variation (Rule 1) 3σ (UCL) 3σ 3σ 2σ (UCL) (UCL) 2σ 2σ (UCL) 1σ (UCL) 1σ (UCL) 1σ 1σ (UCL) (UCL) Mean Mean Mean 0.00% Time External Presentation 21
Month 10 2 3 4 5 6 7 8 9 Month 1 Month 10 11 3 4 5 6 7 8 9 Month 2 Month 10 11 12 4 5 6 7 8 9 Month 3 Month 10 11 12 13 5 6 7 8 9 Month 4 Month 10 11 12 13 14 6 7 8 9 Month 5 Month 10 11 12 13 14 15 7 8 9 Month 6 Month 10 11 12 13 14 15 16 8 9 Month 7 Month 10 11 12 13 14 15 16 17 9 Month 8 Month Month 10 11 12 13 14 15 16 17 18 9 Month 10 11 12 13 14 15 16 17 18 19 Month 12 13 14 15 16 17 18 19 20 11 Month 12 13 14 15 16 17 18 19 20 21 Month 13 14 15 16 17 18 19 20 21 22 OOL Rate(%) Mix Tank Bioburden SPC Trend Analysis - Case Study #2 10.00% 10% 9.00% 9% 8.00% 8% 7.00% 7% One month Data following Set for Month Exceptional 21 had Exceptional Variation (Month Variation 22) (Previous Slide) (Rule 1) 6.00% 6% 5.00% 5% 4.00% 4% Out of Limit Trend Identified Formal Investigation Within Trend Limits (No Action) 3.00% 3% 2.00% 2% 1.00% 1% 0.00% 0% Time External Presentation 22
Summary of Case Study #2 What Happened? In Month 21, the facility would have identified an Exceptional Variation rule violation (Rule 1- one point above the 3σ limit); however, at this time no Preventative Action took place because the facility was not using SPC. The traditional trending tool (comparing # samples over Alert Limit to trend limit) did not signal an investigation for Month 21. The following month s data (Month 22) did signal an investigation due to an elevated number of samples over the Alert Limit. Significance If using SPC and contamination recovery rate, the out of trend condition could have been potentially avoided because the previous month signaled Exceptional Variation, which would lead to a Preventative Action. SPC offers a more sensitive monitor of system quality. If SPC is detecting drifts, but Alert/Action Levels do not, perhaps these levels should be assessed. External Presentation 23
Intro to Case Study #3 Background The following slides contain WFI bioburden samples collected on a monthly basis at two (2) facility locations (A and B). Scope All WFI samples in scope collected at the respective facilities. Each facility collects >40 samples per month. Both systems utilized different combinations of equipment/processes to produce WFI. Specific Case Details At the time of the data collection, statistical process control was not in use to analyze the data in either facility (this is a retroactive review). The following slide shows a side-by-side comparison of two (2) facilities. External Presentation 24
Month 1 2 3 4 5 6 7 Month 2 3 4 5 6 7 8 Month 3 4 5 6 7 8 9 Month 10 4 5 6 7 8 9 Month 10 11 5 6 7 8 9 Month 10 11 12 6 7 8 9 Month 10 11 12 13 7 8 9 Month 10 11 12 13 14 8 9 Month 10 11 12 13 14 15 9 Month 10 11 12 13 14 15 16 Month 11 12 13 14 15 16 17 Month 12 13 14 15 16 17 18 Month 13 14 15 16 17 18 19 Month 7 1 2 3 4 5 6 Month 8 2 3 4 5 6 7 Month 9 3 4 5 6 7 8 Month 10 4 5 6 7 8 9 Month 11 10 5 6 7 8 9 Month 12 10 11 6 7 8 9 Month 13 10 11 12 7 8 9 Month 14 10 11 12 13 8 9 Month 15 10 11 12 13 14 9 Month 16 10 11 12 13 14 15 Month 17 11 12 13 14 15 16 Month 18 12 13 14 15 16 17 Month 19 13 14 15 16 17 18 Multiple Facility WFI Comparison - Case Study #3 4.00% 3.50% 3.00% 2.50% 2.00% 1.50% 1.00% 0.50% 0.00% Facility A 3σ 3σ (UCL) (UCL) 2σ (UCL) 1σ (UCL) Mean 12.00% 10.00% 8.00% 6.00% 4.00% 2.00% Facility B 3σ (UCL) 3σ 3σ (UCL) 2σ (UCL) 3σ 3σ 2σ (UCL) 2σ 2σ 2σ 1σ 1σ (UCL) (UCL) 1σ 1σ Mean Mean (UCL) (UCL) Mean Mean 0.00% External Presentation 25
Summary of Case Study #3 What Happened? In Month 17, Facility B would have identified an Exceptional Variation rule violation, Facility B also experienced Exceptional Variation data sets for the following two (2) months. Over the same time period, Facility A experienced only Routine Variation. From Month 8 through Month 11, Facility B experienced a sustained downtrend in recovery. What was going on during this time period to account for the increased control? Can that be duplicated? Facility A had a mean of ~1% and Facility B had a mean of ~5%. What can account for these differences? Significance Production of WFI is not standardized for each facility (different combinations of equipment can produce WFI as necessary for the product). The Exceptional Variations in the data set would intiate Preventative Actions in Facility B that could potentially improve quality to match Facility A. Comparing the SPC allows the corporate functions to monitor the differences between facility systems and standardize those which have better data sets, which can lead to continuous improvement through harmonization and collaboration. External Presentation 26
Why Statistical Process Control w/ Recovery Rate? Advantages Data sets follow a normal distribution. Statistical Process Control (SPC) is an established mechanism for Quality Control in industry (easy to learn, use and maintain). Flexibility The tool can be Plug and Play. Combination of validated tool and guidance can lead to easy to follow stats training/knowledge for local microbiologists. Can be used retrospectively to compare past with present. More Sensitive monitor that can preemptively identify signals compared to Action/Alert excursions. Investigations signaled before system is off the rails. Ratio of Preventative to Corrective Actions (health of CAPA). Greater confidence in data analysis. Data driven Preventative Actions. Harmonization of facility systems for companies with multiple locations producing similar products/materials. External Presentation 27
References 1. United States Pharmacopeia. USP 40 NF 35, Microbiological Control and Monitoring of Aseptic Processing Environments <1116> Rockville, MD: USP; 2017:1430-1443. Acknowledgements / Thank You-s PDA Midwest Roy McLean and Mike Sadowski. External Presentation 28
Thank You sean_toler@baxter.com