IN THE MIND OF AN EX-FDA INVESTIGATOR - INTERNAL AUDITS

Both during and after my career at FDA, individuals have asked me, “As an FDA investigator what are you looking for when…?”. I thought perhaps I would try a new concept, “In the Mind of an Ex-FDA Investigator…” and share some insights from my enforcement days on a variety of topics.

While conducting inspections, I always enjoyed asking questions about the internal audit program. I could immediately see the defensiveness waiting to be vocalized. “You aren’t allowed to see our internal audits!”. Yes, thank you, I know.

So how do FDA Investigators know if your internal audit program is working? I always started with the SOP. The SOP was key to knowing how well the audit program was likely functioning. For example, the key words are mostly always there. “Not responsible for the area being audited”. “Minimally audited annually”. “Audit schedule will be maintained”. I wasn’t really interested in those things anyway.

I wanted to know how the auditors were qualified to perform the audits. I wanted to understand the audit techniques. And I really wanted to see what was done with the audit findings.

Then and now, I ask who performs the internal audits. Most often I am told Quality or perhaps the Corporate Audit Team. The next obvious question regards qualifications. I was frequently told that only certified auditors are used or that having 20+ years of experience in quality was more than enough to be qualified to perform an audit.

In trying to make my point, I would ask specifically who audited certain quality system elements. For example, the QC laboratory. A company making small volume sterile injectable drug products operates a complex chemistry laboratory most often with GC/MS, HPLC and a LIMS, at a minimum. Asking for the credentials of the auditor was of huge importance. Finding that the QC auditor had no degree in chemistry and no work history in chemistry or a QC laboratory was a curiosity at best.

Next, I would move on to understanding how the audits were performed. Aside from reading the SOP, I wanted to know if they could tell me what they look for and how they know if the unit being audited was complying with the Quality Management System. My favorite answer was always that the firm follows their SOP. When asked for further elaboration, I most often heard and continue to hear, that the auditor reviews the unit’s SOPs and then asks questions and reviews records to determine if the SOPs are being followed. I don’t believe I have ever had anyone tell me on the first try that they start by asking if the SOP meets the intent of the regulation.

Directing attention to audit findings, I wanted to determine how those findings were managed as important quality indicating data. I received many different descriptions of how internal audit findings were managed. Some stated that all internal audit findings were maintained and tracked within the audit report only. Others report that all audit findings became CAPAs. And still others stated that there was a tiered process, recommendations within the report remained there, major findings were deviations/non-conformances and critical findings were CAPA.

Of course, I wanted to see how they followed up, so I would ask for evidence that corrective actions had been implemented. The worst thing I encountered was someone telling me that I was not seeing audit stimulated deviations/non-conformances or CAPA because they kept those in a separate quality system because FDA was not allowed to see those. Yikes.

Finally, I would look for repetitive failures, deviations, non-conformances and CAPA that related to previous internal audit corrective actions. Finding repetitive failures was a sure sign that the actions coming from the internal audit findings were not effective.

And now for an aside…Under the FDA’s Quality System Inspection Technique (QSIT), Management Controls, which includes internal audits, was frequently found as a low value system to inspect. In fact, when QSIT first came out, FDA investigators were allowed to select CAPA + 1 (Management, Design (DC), Production and Process (PPC)). With time, FDA discovered that Investigators took the easy way out, Management Controls were fast to review and when given 2.5 days for an inspection, it helped keep the trains running on time, so to speak. FDA moved away from that and required CAPA + either PPC or DC, which meant Management was only covered under the full inspection requirement. All of this is likely to change under the new QMSR.

What is the point, you ask? For internal audit programs to be successful you need at least three things:

1. Auditors who are qualified for

the area they are auditing. It is one thing to follow the SOP, it is another to ensure that the records and the data that are supporting the activities meet quality standards. Meeting quality standards exceeds the SOP. It means good decision making, logic, reasoning, and risk management; things not well written into procedures. If the auditor does not understand the process, equipment, concepts, how are they qualified to determine if the decisions, logic, reasoning, and risk management have been appropriately applied?

2. A Quality System predicated on the regulations and requirements of the market you are operating in. Although your procedures may be reviewed as part of the document control process every 2-3 years, changes often occur between those reviews. And mistakes happen. Blindly reading the SOP and asking for evidence it was followed should never be the first step. The first step is to ensure the procedure meets the intent of the regulation. If the SOPs do not drive toward a compliant quality system, then following those procedures only means you are now creating records of further non-compliance. Aside from changes that may lead to mistakes rendering your SOPs non-compliant, interpretation of the regulation and improper translation into procedures happens. And, finally, some inexperienced firms may simply not have enough knowledge to write compliant procedures from inception. Whatever the case may be, the audit should start by ensuring the procedures are appropriate before assessing if the implementation is correct.

3. Audit findings are by nature a notation of a Quality System discrepancy. Discrepancies are captured in your quality system as a deviation, non-conformance or CAPA. These are the regulatory requirements and expectations. Just because you identified the discrepancy in an internal audit, which is a good thing, does not mean that it should not be captured and resolved within one of these systems. FDA has all rights to review the corrective actions stemming from audit findings. Maintaining a separate file or culling out these actions when requested by FDA is a serious finding and can result in an observation for partial refusal.

As noted in the Quality System Regulation pre-amble, "FDA will review the corrective and preventive action procedures and activities performed in conformance with those procedures without reviewing the internal audit reports. FDA wants to make it clear that corrective and preventive actions, to include the documentation of these activities, which result from internal audits and management reviews are not covered under the exemption at 820.180(c)."

Internal audit data are of great value to a company when qualified auditors assess systems based on compliant procedures and processes. The internal audit findings when compared to discrepancies prior to the audit, and discrepancies compared to audit findings after the audit, are both ways of telling you if your day-to-day deviation/non-conformance/CAPA management systems are working, and if your internal audit process is working. However, as they say, “junk in, junk out”.

When I see recurrent quality problems, one additional thing I consider is over familiarity. This often occurs in smaller firms; however, I have seen it in multinationals. This is when the same auditor or audit team is used to assess the same unit each year. As with anything else, when we feel like we know something, we sometimes do not look as hard. Knowing that the last two audits only turned up recommendations may lead to discrepancy blindness. Frequently this applies to firms, where there is a small audit group responsible for the entire quality system and a fresh set of eyes is nowhere to be found.

Perhaps you can see how a process that should be so straightforward can be damning when not properly designed and implemented. The internal audit program can be your firm’s friend, helping to identify and resolve issues before they become unmanageable and better yet, before a regulatory body finds them for you. The program can also be a gateway to multiple FDA-483 observations or other regulatory findings.

Look at where your internal audit program lands and let me know if you need some help.

---