keith@dorricottmpi.com

Don’t Waste a Good Mistake…Learn From It

Everyone is so busy. There’s not enough time to even think! This seems to be a challenge in many areas of business – we expect more and more from fewer people. Tom DeMarco describes this situation in his book “Slack” which I have recently re-read. And I think he’s on to something when he quotes “Lister’s Law – People under time pressure don’t think faster.” And of course, that’s right. Put people under time pressure and they will try to cut out wasted time. And they can re-prioritize so they spend more time on that task. They can work longer hours. But eventually, there is a limit and so people start to take cognitive short-cuts…”this problem is the same as one I’ve encountered before and so the solution must be the same”. Of course, that might be the right conclusion but if you don’t have the available time to interrogate it a little further then you run the risk of implementing the wrong solution and even making the problem worse.

One of the reasons I often hear as to why people don’t do root cause analysis is that they don’t have the time. People don’t want to be seen analysing a problem – much better to be taking action. But what if the action is the wrong action and is not based on the root cause? If the action is “re-training” you can be sure no-one has taken the time to really understand why the problem occurred. Having a good method you can rely on is part of the battle (I suggest DIGR^® of course). But even knowing how is no good if you simply don’t have the time. Not having the time is ultimately a management issue. If managers asked “why” questions more and encouraged their staff to take time to think, question and get to root cause rather than rushing to a short-term fix, we would have true learning.

If we are not learning from things that go wrong to try to stop it recurring then we have missed an opportunity. If the culture of an organization is for learning and improvement then management must support staff with the right encouragement to understand, and good tools. But above all they must provide the time to really understand an issue, get to root cause and implement actions to try to stop recurrence. And if your manager isn’t providing time and encouraging you in this, challenge them on it – and get them to read this blog!

As Robert Kiyosaki said “Don’t waste a good mistake…learn from it.”

DIGR^® is a registered trademark of Dorricott Metrics & Process Improvement Ltd.

Is more QC ever the right answer? Part II

In part I of this post, I described how some processes have been developed that they can end up being the worst of all worlds by adding a QC step – they take longer, cost more and give quality the same (or worse) than a one step process. So why would anyone implement a process like this? Because “two sets of eyes are better than one!”

What might a learning approach with better quality and improved efficiency look like? I would suggest this:

In this process, we have a QC role and the person performing that role takes a risk-based approach to sampling the work and works together with the Specialist to improve the process by revising definitions, training etc. The sampling might be 100% for a Specialist who has not carried out the task previously. But would then reduce down to low levels as the Specialist demonstrates competence. The Specialist is now accountable for their work – all outputs come from them. If a high level of errors is found then an escalation process is needed to contain the issue and get to root cause (see previous posts). You would also want to gather data about the typical errors seen during the QC role and plot them (Pareto charts are ideal for this) to help focus on where to develop the process further.

This may remind you of the move away from 100% Source Document Verification (SDV) at sites. The challenge with a change like this is that the process is not as simple – it requires more “thinking”. What do you do if you find a certain level of errors? This is where the reviewer (or the CRA in the case of SDV) need a different approach. It can be a challenge to implement properly. But it should actually make the job more interesting.

So, back to the original question: Is more QC ever the answer? Sometimes – But make sure you think through the consequences and look for other options first.

In my next post, I’ll talk about a problem I come across again and again. People don’t seem to have enough time to think! How can you carry out effective root cause analysis or improve processes without the time to think?

Windrush: Use and Abuse of Metrics

Here in the UK, a huge scandal has blown up in the government’s face recently. The so-called Windrush generation are members of the former British Empire in the Caribbean who came to the UK after the war at Britain’s request. They were wanted to help rebuild Britain. And they made the UK their home. They worked, had families, paid taxes and made a difference. They are British and recognised as such since an act in 1971. Approximately 500,000 came to the UK (including from other countries such as India and Pakistan). Now as they reach retirement, they have fallen foul of successful measures taken by the government to make a hostile environment for illegal immigrants. They are not illegal but are caught out by the rules – they have to prove they are not illegal and not all have been able to do so. There are tragic stories in the news of people being detained, deported, denied work, housing, healthcare even though they are British. The country has been horrified – how on earth is this possible?

As more and more leaks out, it is becoming clear that metrics are an important part of the story. The government has been desperate since 2010 to reduce immigration. They have been proud of creating a hostile environment for illegal immigrants and deporting them if they cannot prove their right to be here. And some of the Windrush generation have been caught up in this. It has emerged that the Home Office had a target to forcibly return around 12,000 illegal immigrants per year. These outrageous examples of deporting British people with the right to be here are included in that total and help the department meet the target. With huge pressure to meet a target, people will try their hardest to do so – whatever the means. If you had a target to deport illegal immigrants and your job (or bonus) depended on it then what would you do? How sympathetic would you be to someone who could not prove they were legally here?

Metrics can drive the wrong behaviour as well as the right behaviour. The country is appalled at the way these people have been treated including newspapers who are normally strongly supportive of the government. This was clearly the wrong behaviour – being driven to meet a metric target. And the evidence of these cases has been there for several years. But the target was always more important.

This is the wrong use of metrics. They should not be used at the exclusion of thinking and compassion. Whatever metrics you use, they can drive the wrong behaviour – always look behind them to understand what is happening. Ask the right questions. If the metric is improving, ask why. Ask how. Is it driving the right behaviour or are there negative consequences? Think.

Thanks to a free press, these stories have come into the open and the government has apologised and taken action to reverse the injustices. But these injustices should not have happened in the first place and some people’s lives have been turned upside-down. The lesson – use metrics carefully and thoughtfully and watch for them driving the wrong behaviour.

A great book that makes clear the use and abuse of metrics in the public sector is “Systems Thinking in the Public Sector” by John Seddon.

Picture: kmusser

Is More QC Ever the Right Answer? Part I

In a previous post, I discussed whether retraining is ever a good answer to an issue. Short answer – NO! So what about that other common one of adding more QC?

An easy corrective action to put in place is to add more QC. Get someone else to check. In reality, this is often a band-aid because you haven’t got to the root cause and are not able to tackle it directly. So you’re relying on catching errors rather than stopping them from happening in the first place. You’re not trying for “right first time” or “quality by design”.

“Two sets of eyes are better than one!” is the common defence of multiple layers of QC. After all, if someone misses an error, someone else might find it. Sounds plausible. And it does make sense for processes that occur infrequently and have unique outputs (like a Clinical Study Report). But for processes that repeat rapidly this approach becomes highly inefficient and ineffective. Consider a process like that below:

Specialist I carries out work in the process – perhaps entering metadata in relation to a scanned document (investigator, country, document type etc). They check their work and modify it if they see errors. Then they pass it on to Specialist II who checks it and modifies it if they see any errors. Then the reviewer passes it on to the next step. Two sets of eyes. What are the problems with this approach?

It takes a long time. The two steps have to be carried out in series i.e. Specialist II can’t QC the same item at the same time as Specialist I. Everything goes through two steps and a backlog forms between the Specialists. This means it takes much longer to get to the output.
It is expensive. A whole process develops around managing the workflow with some items fast-tracked due to impending audit. It takes the time of two people (plus management) to carry out the task. More resources means more money.
The quality is not improved. This may seem odd but if we think it through. There is no feedback loop in the process for Specialist I to learn from any errors that escape to Specialist II so Specialist I continues to let those errors pass. And the reviewer will also make errors – in fact the rework they do might actually add more errors. They may not agree on what is an error. This is not a learning process. And what if the process is under stress due to lack of resources and tight timelines? With people rushing, do they check properly? Specialist I knows That Specialist II will pick up any errors so doesn’t check thoroughly. And Specialist II knows that Specialist I always checks their work so doesn’t check thoroughly. And so more errors come out than Specialist II had not been there at all. Having everything go through a second QC as part of the process takes away accountability from the primary worker (Specialist I).

So let’s recap. A process like this takes longer, costs more and gives quality the same (or worse) than a one step process. So why would anyone implement a process like this? Because “two sets of eyes are better than one!”

What might a learning approach with better quality and improved efficiency look like? I will propose an approach in my next post. As a hint, it’s risk-based!

I Must Do Better Next Time

I was interviewed recently by LMK Clinical Research Consulting (podcast here). I was intrigued when in the interviewer’s introduction, he said that from reading my blog he knew that I “have a fundamentally positive outlook with how humans interact with systems”. I suppose that’s true but I’d not thought of it that way before. I do often quote W. Edwards Deming “Nobody comes to work to do bad job” and “A bad system will beat a good person every time”. The approach is really one of process thinking – it’s not that people don’t matter in processes, they are crucial. But processes should be designed to take account of the variation in how people work. They should be designed around the people using them. No point blaming the individual when things go wrong – time to learn and try to stop it going wrong next time. I wrote previously about the dangers of a culture of blame from the perspective of getting to root cause. Blame is corrosive. Most people don’t want to open up in an environment where people are looking for a scape-goat – so your chance of getting to root cause is much less.

Approaching blame in this way has an interesting effect on me. When things go wrong in everyday life, my starting point isn’t to blame myself (or someone else) but rather to think “why did that go wrong?” A simple everyday example…I was purchasing petrol (“gas” in American English) and there were two card readers at the till. The retailer asked me to put my card in – which I did. He immediately said “No – not that one!” So, I took it out and put it in the other one. “That’s pretty confusing having two of them,” I said. To which he replied, “no it’s not!” I can see how it’s not confusing to him because he is using the system every day but to me it was definitely confusing. I don’t think he was particularly interested in my logic on this, so I paid and said “Good-bye”. Of course, I don’t know why he had two card readers out – what was the root cause? But even without knowing the root cause, he certainly could have put a simple correction in place by telling me which card reader to put my card in to.

There’s no question, we can all learn from our mistakes and we should take responsibility for them. But perhaps by extending the idea of no blame to ourselves, we can focus on what we can do to improve rather than simply thinking “I must do better next time.”

Some of My Recent Posts

A number of people on my subscriber list have told me they thought I had stopped posting. I haven’t – but it seems my posts have not been going out to many of you since about June last year. I hope this post reaches you. I’ve changed some of the settings. In case you missed any of my posts and wanted to take a look here are some of the popular ones since June:

Don’t blame me! The corrosive effect of blame

To Err is Human But Human Error is Not a Root Cause

Don’t waste people’s time on root cause analysis

Not everything that counts can be counted!

Stop Issues Recurring by Retraining?

Get rid of plastic packaging – are you mad?

And do let me know if you received this. Let’s hope I got to root cause…

Get rid of plastic packaging – are you mad?

There has been much in the UK media recently about the need to eliminate plastic packaging. The shocking pictures from the BBC series “Blue Planet” showing just how much plastic ends up in our oceans has been a wake-up call. We have to do something to fix this. And it seems that the solution is obvious – we even have 200 members of parliament writing to major supermarkets calling for plastic-free aisles. Let’s rid the world of plastic packaging. But I worry that we are in danger of “throwing the baby out with the bath water.”

It is important to understand what the problem is first – what is the problem we are trying to solve? Then let’s investigate the problem and see if we can understand the root causes. After that, we can focus our solutions on the root causes. This is the most efficient and effective way to solve problems. Jumping straight to solutions without even understanding the problem risks unfocused, inefficient actions and unintended consequences.

So what is the problem? Too much plastic in the oceans. What sort of plastic? Mostly packaging. Where does it come from? Mainly 10 rivers – 8 in Asia and 2 in Africa. Why from those rivers and not other ones? They pass through very populated areas where there is limited collection and even less recycling of plastic waste. Of course, the reasons for this limited collection and recycling are many and varied but by focusing on those, we have a good chance of having a real impact on the problem and reducing the new plastic going in to the oceans. We could also work on ways to try to reduce the plastic that is already there and reduce our use of unnecessary plastic packaging such as bottled water and plastic-coated single-use coffee cups.

But – the focus in the media and by the MPs seems to be on getting rid of plastic packaging in the UK all together. Given what the problem is and the source for much of the plastics in the oceans, getting rid of plastic packaging in the UK seems an odd solution. It doesn’t appear to be focused on the root cause(s). And, of course, it does not consider the unintended consequences. In many circumstances, plastics are the most effective and efficient type of packaging. Using films and modified atmosphere packaging to wrap fresh meats can more than double shelf life. Cucumbers can last weeks rather than days when shrink-wrapped. These huge increases in shelf life mean much more efficient supply chains with larger, more efficient production runs, fewer deliveries, less stock rotation and, most importantly, much less waste from farm to plate. 1/3 of food in the UK is thrown away – food that uses resources to be grown, processed and transported. Plastics, when used appropriately and handled well at end-of-life are a real boon to the environment by substantially reducing food waste.

We must try to reduce packaging to a minimum – reduce, reuse, recycle. But let’s define the problem first before we go rushing off into seemingly simple, populist solutions that may have unintended consequences. The first step in solving a problem is to define the problem. Then try to understand the root cause(s) and develop solutions focused on theose root cause(s).

Packaging has an important job to do. And plastic packaging plays a very important role in keeping food waste down.

Let’s not get rid of plastic packaging!

Stop Issues Recurring by Retraining?

“That issue has happened again! We really need to improve the awareness of our staff – anyone who has not used the right format needs to be retrained. We can’t tolerate sloppy work. People just need to concentrate and do the job right!”

You may recall a previous post about human factors where I looked at why people make errors and the different types of errors. If the error was a slip (a type of action error where someone planned to do the right thing but did the wrong thing) then retraining won’t help. The person already knows what the right thing to do is. Similarly if the error was a lapse (where someone forgot to do it). Of course, with both of these error types, making people aware will help temporarily. But over time, they will likely go back to doing what they were doing before unless some other change is made.

If the error was a rule-based thinking error where the knowledge is there but was misapplied, it is unlikely that retraining will work long term. We would need to understand the situation and why it is that the knowledge was misapplied. If the date is written in American format but read as European (3/8/18 being 8-Mar-2018 rather than 3-Aug-2018) then can we change the date format to be unambiguous in the form dd-mmm-yyyy (03-Aug-2018)?

What if the error is a non-compliance? If someone didn’t carry out the full procedure because they were rushed and they get retrained, do we really think that in the future when they are rushed they are going to do something different? They might do short term but longer term it is unlikely.

For all these errors, retraining or awareness might help short term but they are unlikely to make a difference longer term. To fix the issue longer term, we need to understand better why the error occurred and focus on trying to stop its recurrence by changes to process/systems.

A thinking error that is knowledge-based is different though. If someone made an error because they don’t know what they should be doing then clearly providing training and improving their knowledge should help. But even here, “retraining” is the wrong action. It implies they have already been trained and if so, the question is, why didn’t that training work? Giving them the same training again is likely to fail unless we understand what went wrong the first time. We need to learn from the failure in the training process and fix that.

Of course, this does not mean that training is not important. It is vital. Processes are least likely to have errors when they are designed to be as simple as possible and are run by well-trained people. When there are errors, making sure people know that they can happen is useful and will help short term but it is not a long term fix (corrective action). Longer term fixes need a better understanding of why the error(s) occurred and this is where the individuals running the process can be of vital help. As long as there is a no-blame culture (see previous post) you can work with those doing the work to make improvements and help stop the same errors recurring. Retraining is not the answer and it can actually have a negative impact. We want those doing the work to come forward with errors so we can understand them better, improve the process/system and reduce the likelihood of them happening again. If you came forward acknowledging an error you had made and were then made to retake an hour of on-line training on a topic you already know, how likely would you be to come forward a second time? Retraining can be seen as a punishment.

So, to go back to the post title “Stop errors recurring by retraining?” No, that won’t work. Retraining is never a good corrective action.

What about that other corrective action that comes up again and again – more QC? That’s the subject of a future post.

Not everything that counts can be counted!

It’s coming to that time of year again – performance appraisals! Do you know anyone who likes them? When I first became a line manager, I was lucky, I had actually received training on the importance of performance appraisals and how to run them. The time came and I was ready. Except I wasn’t ready for the wave of negativity. Employees came in to my office and slumped on the chair ready for what they clearly considered was a pointless annual appraisal. I tried my best, I used all the techniques I had been taught. I got feedback from peers, I considered strengths, weaknesses, opportunities. I spent hours putting the text together for the appraisal and further hours working out the SMART goals for the next year. But to no avail. Perhaps I wasn’t doing it right. I tried other ways but somehow over many cycles of these and different employers and employees I have never managed to work out the right formula. It was the same with my own appraisals – I never felt they really added much to what I already knew. And my sense of disappointment and unfairness at getting a “Meets Expectations” one year where I thought I had achieved so much lasted a long time.

Could it be that actually the annual appraisal process itself is not fit for purpose in the modern world of work? In fact, perhaps it never was. So much of what we do is team work, it is actually quite tricky to separate out the individual contribution. And things change so quickly in organizations. What seems important when goals are set may be irrelevant even two months later. As for SMART goals, there are many critiques of these and I have really been left questioning their value. They encourage you to set targets you know you’re going to achieve rather than challenging ones you might fail at. The challenging ones will bring your performance rating down. I’ve been in organizations that spent months going round and round trying to agree SMART objectives for the year and only getting there by May (when the appraisal year started in January, five months earlier!) And there were way too many goals that employees only looked at when the manager reminded them.

As to the actual rating process, I have never come across one that worked well. Many seemed based on the 5 point scale (1=Unacceptable Performance, 2=Needs Improvement, 3=Meets Expectations, 4=Exceeds Expectations, 5=Exceptional Performance). As the resident data nerd in one organization, I was given all the data on ratings to crunch to see what the distribution looked like. You can probably guess. 0.2% got a 1 rating, 0.8% got a 2 rating, and 0.2% got a 5 rating. In other words, 98.8% of people got a 3 or 4 rating. Managers tend to choose the middle ratings because it is easier – explaining the extreme values to employees and superiors is hard work.

Of course, good managers do much better. They manage performance on an ongoing basis. The annual performance appraisal becomes more of an administrative burden. Isn’t it about time we got rid of this failed approach and got managers managing performance with their employees on an ongoing basis? They could talk about strengths, weaknesses, opportunities, ways to grow without worrying about comparing individuals using numbers.

With employees and managers hating the process of annual performance appraisals, isn’t it about time we ditched them in favour of a continuous assessment approach and an ongoing focus on goals – for both the employee and organization? If you’re wondering how that might work, take a look here.

A phrase often wrongly attributed to Einstein but actually thought to be from the sociologist, William Bruce Cameron should give us pause for thought when using ratings for annual performance appraisals: “not everything that can be counted counts, and not everything that counts can be counted”.

Want to learn more about using KPIs correctly? Drop me a line! Or take a look at the training opportunities.

Picture: Rizkyharis CC BY-SA 4.0

Don’t waste people’s time on root cause analysis

In an earlier post, I described a hypothetical situation where you are the Clinical Trial Lead on a vaccine study. Information is emerging that a number of the injections of trial vaccine have actually been administered after the expiry date of the vials. This has happened at several sites. I then described actions you might take without the need for root cause analysis (RCA) such as – review medical condition of the subjects affected, review stability data to try to estimate the risk, ask CRAs to check expiry dates on all vaccine at sites on their next visit, remind all sites of the need to check the expiry date prior to administering the vaccine. So if you were to now go through the time and effort of a DIGR^®RCA and you still end up with these and similar actions, why did you bother with the RCA? RCA should lead to actions that tackle the root cause and try to stop the issue recurring – to help you sleep at night. If you or your organization is not going to implement actions based on the RCA then don’t carry out the RCA. A couple of (real) examples from an office environment might help to illustrate the point.

In a coffee area there are two fridges for people to store milk, their lunch etc. One of them has a sign on it. The sign is large and very clear “Do not use”. And yet, if you open the fridge, you will see milk and people’s lunch in it. No-one takes any notice of the notice. But why not? In human factors analysis, the error occurring as people ignore the sign is a routine non-compliance. Most people don’t pay much attention to signs around the office and this is just another sign that no-one takes notice of. Facilities Management occasionally sends out a moaning email that people aren’t to use the fridge but again no-one really takes any notice.

What is interesting is that the sign also contains some root cause analysis. Underneath “Do not use” in small writing it states “Seal is broken and so fridge does not stay cold”. Someone had noticed at some point that the temperature was not as cold as it should be and root cause analysis (RCA) had led to the realisation that a broken seal was the cause. So far, so good. But the action following this was pathetic – putting up a sign telling people not to use it. Indeed, when you think about it, no RCA was needed at all to get to the action of putting up the sign. The RCA was a waste of time if this is all it led to. What should they have done? Replaced the seal perhaps. Or replaced the fridge. Or removed the fridge. But putting a sign up was not good enough.

The second example – a case of regular slips on the hall floors outside the elevators – including one minor concussion. A RCA was carried out and the conclusion was that the slips were due to wet surfaces when the time people left the office coincided with the floors being cleaned. So the solution was to make sure there were more of the yellow signs warning of slips at the time of cleaning. But slips still occurred – because people tended to ignore the signs. A better solution might have been to change the time of the cleaning or to put an anti-slip coating on the floor. There’s no point in spending time on determining the root cause unless you think beyond the root cause to consider options that might really make a difference.

Root cause analysis is not always easy and it can be time consuming. The last thing you want to do is waste the output by not using it properly. Always ask yourself – could I have taken this action before I knew what the root cause was? If so, then you are clearly not using the results of the RCA and it is likely your action on its own will not be enough. Using this approach might help you to determine whether “retraining” is a good corrective action. I will talk more about this in a future post.

Here’s a site I found with a whole series of signs that helps us understand one of the reasons signs tend to be ignored. Some of them made me cry with laughter.

Photo: Hypotheseyes CC BY-SA 4.0

DIGR^® is a registered trademark of Dorricott Metrics & Process Improvement Ltd.