An auditor is entitled to determine the scope of its own audit and surely quality personnel have the best intentions when developing paper-based questionnaires, but auditing activities are not a one-size-fits-all exercise. With risk assessment being at the forefront of FDAs focus, more supplier evaluations should be incorporating risk at the level of the audit process. Risk is being used as a tool to evaluate the format of the audit (onsite vs. paper survey) as well as the frequency of the audit, but not always to determine the appropriate audit initiatives. Natural product companies fully understand the risk an ingredient supplier has on their operations, but more emphasis should be placed on the risks within the suppliers organization. Those are the areas where the most time and effort should be placed when evaluating supplier quality systems. If the audit does not focus on the most critical aspects of the service or product that supplier is providing, it doesnt matter how frequently a supplier is being evaluated or in what format.

Establishing that suppliers are in compliance with GMPs (good manufacturing practices) requires a comprehensive understanding of the work they are performing for your organization. If they are not a manufacturing operation, it may be true that only a subset of regulatory requirements are applicable. Time and effort must be placed on developing a technical understanding of the suppliers systems and processes. This allows for time to be well-focused while at the suppliers facility. If submitting a questionnaire, taking the time to develop applicable content will result in better responses and more willingness to provide responses. Obtaining an outline or summary of the technology used before the audit can help a company prepare, especially for audits of contract service laboratories that provide microbial identification services. Prior to the audit or at the opening meeting of the audit, attempt to establish critical parameters for the processes and critical pieces of equipment used to provide the service. As part of this determination, the advantages, as well as limitations, of the systems used should be established. This should ensure the most important processes and systems will be the focus and the appropriate use of the systems can be verified. 

Microbial identification laboratories provide an invaluable service to FDA-regulated industries. Tracking microbiological flora in manufacturing environments requires accurate and reproducible identification of those organisms. Using a service laboratory that demonstrates a robust quality system and state of control can greatly enhance environmental monitoring (EM) programs. The most compliant providers will show evidence that they have built quality into their product at all levels: people, equipment, sample processing, data review and interpretation, and reference databases. These are all key factors that have interdependencies and can impact the quality and accuracy of the microbial identification.

Providers of microbial identification services utilize various commercially available and propriety systems for obtaining their identifications. The identifications may be based on phenotypic, proteotypic or genotypic characteristics. It is well-known that some systems are limited in their ability to accurately and reproducibly identify environmental microorganisms that are commonly recovered from manufacturing environments. A proper evaluation of the system should consist of two components: 1) Determine if the system can accurately and reproducibly identify the organisms present in your facility, and 2) assess the compliance of each input to the system that ultimately produces the identification. The most important inputs to an automated identification system that can impact the resulting identification are the instrument itself, the microbial reference databases used by the instrument to report the identification, and the people who manually prepare the sample, manage the reagents, run the instrument and interpret the resulting data.

The lab should provide evidence that the instrument has been appropriately qualified. This is typically accomplished via an Installation, Operation and Performance Qualification (IQ, OQ and PQ). Many manufacturers of automated identification systems will make an IQ/OQ available as part the initial purchase, but most do not develop a PQ, nor will they provide any support for developing the appropriate content of one. The PQ, however, is the most critical qualification test as it establishes that the instrument is performing as needed for a given application. The PQ should result in sufficient data that demonstrates the systems ability to consistently produce accurate identifications.

Within the range of automated systems that produce identifications, they have varying degrees of operator intervention, which can impact the quality and accuracy of the data that is produced. Some highly automated systems require minimal preparation of the sample being analyzed; others require several interdependent tasks to be manually performed prior to placing a sample onto the instrument. For some systems, improper preparation of the sample or making an improper assessment of the phenotypic characteristics of the sample (e.g., gram reaction) can cause the system to report the wrong identification. Irrespectively, operator training should be formally documented. The person preparing the samples and/or running the instrument should be required to complete a proficiency evaluation prior to working independently. These evaluations should be repeated at pre-defined intervals.  Operators should also be trained on evaluating data quality and subsequent interpretation of the data. It is important to determine what criteria have been established for assessing data quality and  interpreting the data as well as ensure it is consistent with manufacturers or industry guidelines.

The instrument's microbial reference database is the final, very important but often overlooked, aspect of an automated system. The database is referenced to determine the actual identity of the unknown organism. This is the most critical component since errors or gaps in the library database will inhibit the system from accurately identify microorganisms. It is important to ascertain if the system has the capability of reporting inconclusive results or an incorrect identification if a library entry is missing. Most microbial identification systems have been developed for clinical applications and, therefore, have many gaps in the reference database if trying to identify environmental microorganisms found in regulated manufacturing environments. When performing the PQ, most fail to include adequate testing of the reference database that is included with the instrument. Organizations that understand the limits of commercial, off-the-shelf reference databases have procedures for adding organisms to the library and maintaining the database in a validated state. Whether the library entry came with the instrument or was added after purchase, there should be evidence that the entries have been adequately tested to confirm their validity. Library entry errors do occur within manufacturers databases and there needs to be assurance that these errors will not result in inaccurate test results. Performing an additional library validation provides evidence that the library entries are accurate and consistent with current taxonomy.

Using these guidelines as part of a risk-based approach at the level of the audit process will ensure that the most critical aspects of the laboratories quality system are prioritized. Having the right focus will benefit your supplier qualification program.

Tricia Ehmann, M.S., has been with Charles River Accugenix Services for nearly a decade. She helps maintain and improve the company's quality systems. As an ASQ-certified quality auditor and quality manager, she manages all quality control activities, programs and personnel associated with the microbial identification services.