On a recent long-haul flight, I got to thinking about the processing of TMF documents. Many organisations and eTMF systems seem to approach TMF documents with the idea that every one must be checked by someone other than the document owner. Sometimes, the document owner doesn’t even upload their own documents but provides them, along with metadata, to someone else to upload and index. And then their work is checked. There are an awful lot of documents in the TMF and going through multiple steps of QC (or inspection as W Edwards Deming would call it) seems rather inefficient – see my previous posts. But we are a risk-averse industry – even having been given the guidance to used risk-based approaches in ICH E6 (R2) and so many organizations seem to use this approach.
So what is the implication of 100% QC? I decided I would model it via an Excel spreadsheet. My assumptions are that there are 1000 documents submitted per week. Each document requires one round of QC. The staff in Central Files can process up to 1100 documents per week. I’ve included a random +/-5% to these numbers for each week (real variation is much greater than this I realise). I assume 10% of documents are rejected at QC. And that when rejected, the updated documents are processed the next week. I’ve assumed First In, First Out for processing. My model looks at the inventory at the end of each week and the average cycle time for processing. It looks like this:
It’s looking reasonably well in control. The cycle time hovers around 3 days after 20 weeks which seems pretty good. If you had a process for TMF like this, you’d probably be feeling pretty pleased.
So what happens if the rejection rate is 15% rather than 10%?
Not so good! It’s interesting just how sensitive the system is to the rejection rate. This is clearly not a process in control any more and both inventory and cycle time are heading upwards. After 20 weeks, the average cycle time sits around 10 days.
Having every document go through a QC like this forms a real constraint on the system – a potential bottleneck in terms of the Theory of Constraints. And it’s really easy to turn this potential bottleneck into a real bottleneck. And a bottleneck in a process leads to regular urgent requests, frustration and burn-out. Sound familiar?
In my next post, I’ll take a look at what happens when an audit is announced and the volume of documents to be processed jumps for a couple of weeks.
Text: © 2019 DMPI Ltd. All rights reserved.
Picture: CC BY 2.0 Remko Van Dokkum
Keith, you are on target with your assumptions and conclusions. Been there through many crisis cycles as a result of this 100%QC process. Not sustainable!
Hi Keith,
There is another potential pitfall here – now the inspectors look not only at the presence of the dox in the eTMF (like in “good old” paper TMF), but also at the timeliness of their filing – now the eTMF provides the filing time easily visible in the audit trails. So, in your example “imagine the audit is announced”, or even worse “imagine the inspection is announced”, not only there will be an increased stress, frustration, burn-out etc., but also a potential finding that the dox were delayed to file…
Thanks for your comment Oleg. Great point. This is why we need to think of TMF as a process. Documents should be continually flowing through – not in large batches when there is an audit or inspection.
Keith- I also noticed that independent QC’s does not help unless each document owner review document for accuracy and adequacy prior to submitting it. Submission of document appears to be item to be checked in the check box. Many times, it results in adding additional QC steps for TMF in the process instead of tackling the issue of submission of inadequate and/or inaccurate documents.
Interesting point, Sachin. This is actually one of the problems with too much QC. It can take away responsibility. If I know someone is going to check my work 100% of the time, perhaps I don’t take as much care as I should. I’m in a rush anyway and someone else will pick up any issues and let me know. But what if the person carrying out the QC is also in a rush? Or maybe they don’t really understand the document and what to look for (your point). 100% QC in this scenario can actually lead to worse quality! Counter-intuitive at first.
Fully agree Sachin and Keith 🙂