Checklists Will Not Solve All Your Problems (But They May Help). Checklist Series-Part 2

In the first blog post in this series of posts about checklist development and usage I described checklists in general and how they may be used to improve operational performance. If you missed the first post in this series of 3 posts, you can read it here. In that post I shared some stories about checklist usage and some of the benefits and limitations associated with checklists. In this post (the second in this series) I will describe some of the challenges with checklist development and usage and how you may address these challenges in your efforts to develop excellent checklists.

Challenge 1. Checklists should not be the immediate “go-to” strategy for risk reduction before other controls are considered. One of the points I stressed in the first post was how checklists fit within the hierarchy of controls for risk reduction. Checklists typically fall third from the bottom in order of effectiveness, in the administrative controls section of the hierarchy. Following the hierarchy of controls from ANSI/AIHA Z10-2012 Occupational Health and Safety Management System Standard1 typically, before simply going to checklists as a risk control solution we would want to first determine if we could eliminate the specific hazard under consideration. If that isn’t feasible we would seek to find substitute work methods that were less hazardous. If that wasn’t feasible we could see if engineering controls, such as barriers or interlocks, would be feasible. If the previous 3 levels of the hierarchy wouldn’t be feasible in terms of risk reduction, we would look at administrative controls (checklists would fall in that category), warnings, and Personal Protective Equipment (PPE). In some cases a combination of controls could be used, such as a engineering controls with a checklist or checklists with PPE. One of the main points worth emphasizing is that lower order controls should not simply be the “go-to” strategy for risk reduction.

In many cases lower order controls are used right away because they may be seen as the low-hanging fruit that is easily accessible. This isn’t always the case and a solid risk assessment should help teams determine if a checklist by itself will lower risk to acceptable levels, or if it will further reduce risk where other higher-order controls leave off. Checklists may be certainly be effective, but there must be an understanding of how they work to reduce risks. Checklists should not be used to make up for a deficient system safety process. Unfortunately, the reality is that in many cases, crews are dealing with legacy systems full of hazards and there are only so many ways teams can reduce risk using higher order controls. In those cases, checklists may also help to further reduce risk and may help teams and crews to gain clarity on task requirements and status. We must also remember that checklists are often used as a tool to improve operations performance, as well as safety performance.

Challenge 2. Checklists may help control risks in one area while increasing risk in other areas. Just like any decision, there will be consequences (intended and unintended) with checklist development and usage. While checklists may help to reduce risk in some areas there may be unintended consequences associated with checklist usage, and those consequences could actually increase risk in other areas. For example, if checklists are used for a period of time and nothing bad happens (no accidents, incidents, or near-misses) crews and teams may become complacent and may stop looking beyond the checklist for risks. A culture of high-reliability must be developed where crews and teams seek out information rather than simply getting comfortable with a checklist. This is just one example, and with most risk controls there will be unintended consequences, so a solid risk management process should use a system approach with multiple affected players and groups, who can weigh in on the costs and benefits of any type of hazard control or operations performance tool.

Challenge 3. A simple “check-the-box” mentality for checklist usage may actually increase the likelihood of performance failures. If checklists are not emphasized with a strong checklist management culture, they could come across as a simple perfunctory action where crews and teams just “check the box” whether or not the specific task was actually completed according to standards or not. In this fashion  crews may become too comfortable with the checklist. They may also “check the box” before a task is completed in anticipation of the satisfactory results. However, what happens if an employee is called away from a procedure, yet has already checked the box or called out an item as complete? Upon task resumption he or she may skip ahead and finish the job incorrectly. So checklists have to be managed. In the world of Marine Corps aviation we called this process of checklist management “running the checklist,” where the checklist is worked from start to finish and if any interruptions occur, the task or item is called out, with a statement such as “holding the checklist there.” This indicates to the crew that the checklist hasn’t yet been completed. 

Challenge 4. Checklists may be committed to memory after crews gain proficiency. This may not even be intentional. I can still remember most of the pre-start checklist from the T-34C aircraft. Ask any other aviator who flew that aircraft within the past few years and many of them will likely be able to rattle off the first steps, such as “seat and rudder pedals adjusted, parking brake set…” We are cautioned to not memorize checklists, but it can be difficult. This is why deliberate checklist habits must be embedded into operational crews and teams so they understand the purpose of the checklist, which is often to check that certain switches, knobs, functions, modes, or actions have been successfully completed. A good checklist culture will emphasize the importance of checklist discipline, just like the professional discipline of other trades and crafts. This culture often starts with top-level support in the organization, but may be most effectively managed by front line supervisors. Crews should be trained to not only follow the checklist steps, but to think about what they are doing during each step as well. This is a deliberate though process that goes along with a habit. In Weick and Sutcliffe’s High-Reliability Organizing (HRO) terms this may be thought of as part of the Sensitivity to Operations HRO principle.2

Challenge 5. Developing a checklist that fails to take into consideration user input, task design, and workflows. Can you imagine being a worker and someone handing you a checklist to follow without asking your input? Now imagine if the checklist takes the efficient steps you used to perform and makes them more cumbersome and less efficient? Checklist development should include line operator input, particularly from those doing the work under consideration, so they can provide feedback about the checklist effectiveness and task, procedure, or process flows. If a checklist is unorganized and causes unnecessary steps or makes employees double back over certain areas unnecessarily it can become very frustrating. Therefore the checklist should be optimized to maximize human output while minimizing their effort where feasible. In military aviation we would often refer to task execution as “flows,” where the checklist and steps flowed from one direction to another in a fairly efficient manner, while not neglecting key steps. Flows might move from left to right and/or top to bottom, and helped aircrew to develop habit patterns.

This list of challenges isn’t exhaustive, but is intended on providing you with a few cautionary notes as well as a few ideas about how to address these challenges during checklist development. In part 3 of this checklist development series I will attempt to synchronize many of these issues from Part 1 and Part 2 into some actionable next steps.

I would also like to invite you to put your email address on the waiting list for the upcoming instructor-led live online seminar where I will give a presentation on checklist development. Attendees will also receive a FREE copy of V-Speed’s forthcoming checklist development eBook. Why am I writing this eBook and presenting this online seminar? Because I am tired of hearing stories about incidents where so much emphasis was placed on human behavior. I hear stories about incidents where employees were blamed. When I dig deeper into the conversations I ask about hazard controls and the potential hierarchy of controls, and in many cases people don’t even think about controls and they can be dismissive of checklists and their ability to help maintain consistent performance. Let me state again that checklists will not solve all performance and safety problems. They are not a panacea for incidents in the workplace, but they have demonstrated the ability to help with consistent performance in some industries.

I want you to have access to the knowledge of how to develop checklists. Right now I am only accepting names on a waiting list, and do not have the course registration form yet. If you are interested in this class and eBook, please sign up. If there is sufficient interest, I will proceed with full-scale development of the material. If you want to learn how to take crew performance to the next level and wish to be contacted when the course and eBook are ready, please enter your email address by clicking the image or link below the references. Thanks for reading and have a great, productive, and safe day!

References:

1. American Industrial Hygiene Association American National Standard for Occupational Health and Safety Management Systems ANSI Z10-2012. Fairfax:  American Industrial Hygiene Association, 2012. Print.

2. Weick, K. E., and K. M. Sutcliffe. Managing the Unexpected, Resilient Performance in an Age of Uncertainty. Second. San Francisco, CA: Jossey-Bass Inc., 2007. Print.