Incident Evaluation

I’ve written at length about the importance of quality evaluation of exercises.  Essentially, if we don’t evaluate exercises, and do it well, the benefits of the exercises are quite limited.  Generally, we don’t see a benefit to incidents.  By their very nature, incidents threaten and impact life, property, and environment – things we don’t view as being beneficial.  However, benefits are often a product of opportunity; and we absolutely should take the opportunity to evaluate our responses.

Many incidents do get evaluated, but through research after the fact.  We retrace our steps, review incident documents (such as incident action plans), interview personnel, and examine dispatch logs.  These efforts usually paint a decent picture of intent and result (things that are often different), but often miss the delta – the difference between the two – as well as other nuances.  When we evaluate an exercise, we do so in real time.  Th evaluation effort is best done with preparation.  Our evaluation plans, methodologies, and personnel are identified in the design phase of the exercise.  Just as we develop emergency operations plans and train personnel to respond, we can develop incident evaluation plans and train personnel to evaluate incident responses.

Understandably, a hurdle we might have is the availability of personnel to dedicate solely to evaluation, especially on larger incidents – but don’t be afraid of asking for mutual aid just to support incident evaluation (just be sure to include them in your preparedness efforts).  Just as regional exercise teams should be developed to provide cooperative efforts in exercise design, conduct, and evaluation; incident evaluation teams should be developed regionally.  To me, it makes sense for many of these personnel to be the same, as they are already familiar with how to evaluate and write up evaluations.

In exercises, we often use Exercise Evaluation Guides (EEGs) to help focus our evaluation efforts.  These are developed based upon identified Core Capabilities and objectives, which are determined early in the exercise design process.  While we don’t know the specific objectives we might use in an incident, we can identify these in general, based upon past experiences and our preparedness efforts for future incidents.  Similarly, our emergency planning efforts should be based around certain Core Capabilities, which can help inform our incident evaluation preparedness efforts.  Job aids similar to EEGs, let’s call them incident evaluation guides (IAGs), can be drafted to prepare for incident evaluation, with adjustments made as necessary when an incident occurs.

Evaluating an incident, in practice, is rather similar to how we would evaluate an exercise, which is why the training for these activities is relatively portable.  Evaluation efforts should avoid evaluating individuals, instead focusing on the evaluation of functions and processes.  Don’t reinvent the wheel – evaluate based upon documented (hopefully!) plans and procedures and use the Homeland Security Exercise and Evaluation Program (HSEEP) standards to guide your process. Incident evaluation must be managed to ensure that evaluation gaps are minimized and that evaluation progresses as it should.  Observations should be recorded and, just as we would for an exercise, prepared for and eventually recorded in an after action report (AAR).

I favor honest after action reports.  I’ve seen plenty of after action reports pull punches, not wanting the document to reflect poorly on people.  Candidly, this is bullshit.  I’ve also heard many legal councils advise against the publication of an after action report at all. Similarly, this is bullshit.  If our actions and the need to sustain or improve certain actions or preparations is not properly recorded, necessary changes are much less likely to happen.  If an AAR isn’t developed, a corrective action plan certainly won’t be – which gives us no trackable means of managing our improvements and disavows our intent to do so.

As a profession, public safety must always strive to improve.  We have plenty of opportunity to assess our performance, not just through exercises, which are valuable, but also through the rigors of incident responses.  Prepare for incident evaluation and identify triggers in your emergency plans for when evaluation will be employed, how, and who is involved.  Begin evaluation as early as possible in an incident – there are plenty of lessons learned in the early, and often most critical moments of our incident response.  Finally, be sure to document lessons learned in an AAR, which will contribute to your overall continuous improvement strategy.

How does your agency accomplish incident evaluation?  If you don’t, why?

Need help with the evaluation of incidents?  We are happy to help!

© 2017 – Timothy Riecker, CEDP

Emergency Preparedness Solutions, LLC

2017 HSEEP Course Information

Emergency Preparedness Solutions is regularly looking for Homeland Security Exercise and Evaluation Program (HSEEP) trained personnel to help us design, conduct, and evaluate exercises.  The following training bulletin from FEMA’s Emergency Management Institute provides updated information on the course offerings and application process.  Below the training bulletin is a listing of the webinar-based offerings of HSEEP (K0146).

Training Bulletin

Course:  K/L0146 – Homeland Security Exercise Evaluation Program

 (HSEEP): Basic Course

Emmitsburg, MD — You are subscribed to EMI News for FEMA. The following information has recently been updated, and is now available on http://training.fema.gov/EMI/

1263 – REVISED – Training Bulletin – K-L0146 – updated – Jan 12, 2017

The K/L0146 HSEEP is an intermediate-level course that provides a comprehensive overview of exercise design along with practical skill development in accordance with the HSEEP Doctrine.  The course uses activities that will give participants an opportunity to interact with many of the templates and other materials that are provided by the National Exercise Division to ensure exercises are conducted in a consistent manner.  Upon completion of this course, participants will gain a better understanding of what constitutes a HSEEP consistent exercise.

Read more in Training Bulletin 1263.

 

Course Start End
K0146: Homeland Security Exercise and Evaluation Program (HSEEP) Training Course
(M,T,W,R 1:00-5:00 p.m. EST)
4/10/2017 4/13/2017
K0146: Homeland Security Exercise and Evaluation Program (HSEEP) Training Course
(M,T,W,R 1:00-5:00 p.m. EST)
5/8/2017 5/11/2017
K0146: Homeland Security Exercise and Evaluation Program (HSEEP) Training Course
(M,T,W,R 1:00-5:00 p.m. EST)
6/5/2017 6/8/2017
K0146: Homeland Security Exercise and Evaluation Program (HSEEP) Training Course
(M,T,W,R 1:00-5:00 p.m. EST)
7/10/2017 7/13/2017
K0146: Homeland Security Exercise and Evaluation Program (HSEEP) Training Course
(M,T,W,R 1:00-5:00 p.m. EST)
8/7/2017 8/10/2017
K0146: Homeland Security Exercise and Evaluation Program (HSEEP) Training Course
(M,T,W,R 1:00-5:00 p.m. EST)
9/18/2017 9/21/2017
K0146: Homeland Security Exercise and Evaluation Program (HSEEP) Training Course
(M,T,W,R 1:00-5:00 p.m. EST)
3/13/2017 3/16/2017
K0146: Homeland Security Exercise and Evaluation Program (HSEEP) Training Course
(M,T,W,R 1:00-5:00 p.m. EST)
2/6/2017 2/9/2017

 

Incident Action Planning

Hello readers!  It’s been a couple of months since I’ve last posted – along with the holidays, I’ve been fortunate to be busy with some great projects and a bit of travel.  While I’m still busy, I’ve been itching to get back to posting.

Over the past few months, I’ve had an opportunity to look at some different aspects of incident action planning.  Having worked as a Planning Section Chief for numerous state and federally declared disasters, I’ve created Incident Action Plans (IAPs), part and parcel, for a variety of incidents.  Teaching hundreds of course offerings of ICS and EOC management, I’ve also had a lot of opportunity to help others understand and appreciate the value of an IAP.

What is an IAP?

The Incident Action Plan is really the culmination of the planning process.  It documents our expected actions for the next operational period, which, through that planning process, should be ready to be supported through an identified organization and necessary resources.  It begins, foundationally, with an understanding of the situation and objectives built to solve the problems that situation imposes upon us.  IAPs identify some situational information, the incident objectives, and tactics – which specific resource assignments – to accomplish those objectives.  To support the tactics, additional elements are added to the IAP, such as the organizational structure, a medical plan (for responders), a communications plan, and specific safety messages.  Based upon the unique characteristics of each incident, additional material can be added.

The planning process, and thus IAPs, are a standard of practice within NIMS/ICS.

Issues with Teaching the Planning Process

For those coming up through the ranks, as it were, of ICS training, the forms integrated into ICS are often one of the most frustrating aspects.  Responders want to do the hands-on stuff, not fill out forms.  Most areas and systems, however, are able to find the responders who are interested in becoming involved in incident management aspects, and engage them in formal or ad-hoc incident management teams.

Often the only exposure that responders have to IAPs is from the ICS 300 course, where they are able to see some samples and are walked through the planning process and associated forms of the IAP.   Having taught hundreds of these, I know it’s a challenge*.  We inundate course participants with a pile of forms and expect them to leave the class to go out and do great things.  While it might be a reasonable initial exposure, this needs to be followed up on, practices, and reinforced if we expect anyone to be successful, much less use the system.

*note: if you aren’t familiar with my position on the current state of ICS training, here are a few other blog posts to orient you.  In short… ICS Training Sucks!

The planning process can be confusing – who does what?  When?  Based upon what information?  The Planning P is the best visual out there and incident management handbooks (aka Field Operations Guides or FOGs) are references that every IMT member should have in the utility pocket of their 5.11s.  While all based on the same system, I find the United States Coast Guard to have the best handbook out there.  These are great job aids for something we don’t go to work and do every day, no matter how proficient you might think you are.

planning P for Planning

Uses of IAPs

IAPs can be applied to anything we can/should apply the planning process to.  These include incidents, pre-planned events, and exercises.  Planned events and exercises provide a great opportunity to practice the planning process and development of IAPs.  It is certainly something that requires practice to be proficient.  Every member of the Command and General staff, as well as a number of support positions, have an important role to play and responsibilities to contribute to the planning process.  The forms themselves even require some practice to ensure that the right information is obtained.  Large incidents can also require a great deal of tactical planning, which means greater time for that and for documenting the tactics and necessary support.

One aspect that is often forgotten in the heat of battle is that our response should, ideally, be based on our emergency operations plans (EOPs).  These should be a regular reference to the Command and General staff, as they can, at the very least, provide some general guidance.  Good EOPs, and their associated annexes, should provide some detailed guidance on certain aspects of response, which can prevent the IMT from having to re-invent a plan in the midst of chaos.  The direction of EOPs, to the greatest extent possible, should be referenced in the planning process and reflected in the IAPs.  EOPs should serve as the foundation for the planning process – with that in mind, EOPs can be implementation-ready.  More thoughts on emergency plan development here.

As mentioned, exercises are a great opportunity for participants to practice the planning process and IAP development, along with other facets of ICS – especially those that we don’t get to apply so often.  Also consider, however, the use of an IAP for exercise management.  Our company, Emergency Preparedness Solutions, worked with the State of Vermont this past summer providing exercise control and evaluation services for their Vigilant Guard exercise.  This week-long exercise spanned nearly 60 distinct venues across the state and involved thousands of participants.  We coordinated over 100 control and evaluation staff throughout the exercise.  While staffing decisions were made weeks in advance, we knew that with so many variables, needs and assignments were bound to change.  For each operational period of the exercise, venues had staff assigned, a point of contact they should coordinate with, a need to communicate with the simcell and exercise management staff, a need to be aware of weather and safety matters, and processes to follow in regard to exercise management and reporting.  We recognized the similarities to a tactical deployment and decided to develop incident action plans for exercise management.  We called these eXercise Action Plans (XAPs).  Within just a couple of operational periods, we were receiving great feedback on the documents from exercise staff.  Use of XAPs were identified as a best practice in exercise management for that project.

Final Thoughts

Incident Action Plans are great tools that can help us put our emergency plans in action.  They allow us to apply incident or event-specific ground-truths and the realities of incident needs and resources.  While it’s understanding that some are frustrated with the forms used, the forms are job aids, tested through decades, to help us navigate complex incident management.  When I walk into a command post or an EOC, the IAP is the first thing I look for.  Because the format and forms are relatively standardized, I can flip through it, and in a couple of minutes have a good sense of the activity and who is responsible for what.  Someone told me long ago that ICS is forms facilitated, not forms driven – and that’s very true.  The forms (and thus the IAP) are products of the planning process, which is another decades-old practice in incident management.

What best practices have you seen in the application of the planning process and incident action plans?

Need assistance with planning, training, or exercises?  Emergency Preparedness Solutions can help!

Until next time.

© 2017 – Timothy Riecker, CEDP

 

ICS and the Human Factor

A number of my articles have mentioned the unpredictable human factor in executing emergency plans and managing incidents, particularly for complex incidents.  We can build great plans and have a great management system to facilitate the incident management process, but the human factor – that largely intangible level of unpredictability of human behavior – can steer even the best emergency plan astray or derail an incident management process.

An article published in the Domestic Preparedness Journal yesterday, written by Eric McNulty, reflects on this.  Mr. McNulty cites several human factors which have relevance within incident management and encourages leaders to understand these factors within themselves and others to bring about more effective leadership.  The introductory paragraph of his article suggests the need for integrating behavior training into ICS training to ‘improve performance and outcomes’.  Given the impact of behavior factors on how we respond, this is a concept I can certainly endorse for a much-needed rewrite of the ICS curriculum.

I’ve heavily referenced Chief Cynthia Renaud’s paper, The Missing Piece of NIMS: Teaching Incident Commanders How to Function in The Edge of Chaos, in the past and continue to hold her piece relevant, especially in this discussion.  Chief Renaud’s suggestions draw lines parallel to behavioral factors, which suggest to me that we certainly need to integrate leadership training into ICS training.  The current ICS 200 course attempts to do so, but the content simply panders to the topic and doesn’t address it seriously enough.  We need to go beyond the leadership basics and explore leadership training done around the world to see what is the most effective.

Incident management is life and death – not a pick-up game of stick ball.  Let’s start taking it more seriously and prepare people better for this responsibility.

© 2016 – Timothy Riecker, CEDP

Emergency Preparedness Solutions, LLCYour Partner in Preparedness

Do We Need Different Systems for Catastrophic Incidents?

We’ve long heard, albeit in small pockets, people proclaiming that emergency management and public safety need different systems for larger incidents vs smaller incidents.  For years, the Incident Command System (ICS) fought that stigma, with many saying that ICS is only used for hazardous materials incidents (specifically because of OSHA requirements) or for large incidents that required such a high degree of organization.  Following the release of HSPD-5 and the resultant requirements for everyone to use the National Incident Management System (NIMS), we finally seemed to transcend that mentality – although we are still seeing people apply ICS poorly, and often with the thought that it will all work out fine when a large incident occurs.

Since the mid-2000s, coupled with the push for ‘catastrophic planning’, I’ve been hearing people proclaiming that catastrophic incidents require different systems – be it for planning or management.  Recently, I’m hearing this mentioned again.  Yet, interestingly enough, none of the arguments identify specifically what it is about our current systems of preparedness or incident management that fail at the sight of a catastrophic incident.

While I’m a critic of various aspects of our current systems, I’m a believer in them overall.  Do we need a new system of planning?  No, we just need to do it better.  When we plan for a catastrophic level event, we must consider that NOTHING will work in the aftermath of such an incident.  I’m shocked that some people are still counting on the existence and functionality of critical infrastructure following a catastrophic event.  No roads, no communications, no life lines.  These surprised disclosures are revealed in the After Action Reports (AARs) of incidents and exercises that test catastrophic incidents, such as the recent Cascadia Rising exercise.

Fundamentally, are these losses all that different than what we experience in smaller disasters?  Not so much.  Smaller disasters still take out our roads and disable our communications systems – but such disasters are small enough that we can work around these issues.  So how is it a surprise that a large hurricane or earthquake will do even more damage?  It really shouldn’t be.  It’s essentially a matter of scale.

That said, I certainly acknowledge the difficulties that come with the combined impacts of a catastrophic disaster, coupled with the sheer magnitude of it all.  There are challenges offered that we don’t normally see, but a new system of planning is not the answer.  The current frameworks and standards, such as CPG-101 and NFPA 1600 are absolutely substantial.  The processes are not flawed.  The issue is a human one.  We can’t blame the standards.  We can’t blame the plans.  The responsibility lies with the people at the table crafting the plan.  The responsibility lies with them to fully understand the hazards and the potential impacts of those hazards.  Conducting a hazard analysis is the first step for a planning team to accomplish, and I think this is often taken for granted.  While the traditional hazard analysis has value, the current standard is the Threat and Hazard Identification and Risk Assessment (THIRA).  It is an exhausting and detailed process, but it is highly effective, with engaged teams, to reveal the nature and impacts of disasters that can impact a community.  Without a solid and realistic understanding of hazards, including those that can attain catastrophic levels, WE WILL FAIL.  It’s that simple.

As we progress through the planning process and identify strategies to accomplish objectives, alternate strategies must be developed to address full failures of infrastructure and lack of resources.  Assumptions are often made in plans that we will be able to apply the resources we have to fix problems; and if those resources are exhausted, we will ask for more, which will magically appear, thus solving our problems.  Yes, this works most of the time, but in a catastrophic incident, this is pure bullshit.  This assumption needs to be taken off the table when catastrophic incidents are concerned.  The scarcity of resources is an immediate factor that we need to address along with acknowledging that a severely damaged infrastructure forces us out of many of the technological and logistical comforts we have become accustomed to.  It doesn’t require a new system of planning – just a realistic mentality.

This all logically ties to our incident management system – ICS.  ICS is fully able to accommodate a catastrophic-level incident.  The difficulties we face are with how we apply it (another human factor) and integration of multiple ICS organizations and other incident management entities, such as EOCs.  The tenant in ICS is that one incident gets one incident command system structure.  This is obviously not a geo-political or practical reality for a catastrophic incident that can have a large footprint.  This, however, doesn’t mean that we throw ICS out the window.  This is a reality that we deal with even on smaller disasters, where different jurisdictions, agencies, organizations, and levels of government all have their own management system established during a disaster.  Through implementations such as unified command, multi-agency coordination, agency representatives and liaison officers, and good lines of communication we are able to make effective coordination happen.  (Side note: this is absolutely something I think we need to plan for and tighten up conceptually.  It’s often pulled together a bit too ad-hoc for my comfort).

While some time and effort needs to be applied to develop some solid solutions to the issues that exist, I’m confident that we DO NOT need to create alternate preparedness or response systems for addressing catastrophic incidents – we simply need to apply what we have better and with a more realistic perspective.  The answers won’t come easy and the solutions might be less than ideal, but that’s the nature of a catastrophic event.  We can’t expect it to be easy or convenient.

© 2016 – Timothy Riecker, CEDP

Emergency Preparedness Solutions, LLCYour Partner in Preparedness

EMS is in Trouble

If you’ve worked in or with Emergency Medical Services (EMS) over the past couple of decades, you probably know it’s in trouble in many areas around the nation.  As with many organizations, finances are the culprit.  Many EMS organizations simply can’t make ends meet.  Costs for equipment, insurance, fuel, training, and facilities often can’t be balanced in the black against recovered income.  It’s not to say it can’t be done – some are doing it, and successfully.  But many are having a difficult time.  Staffing is another problem.  Volunteer services must constantly recruit and work to retain staff.  Volunteer fire services are experiencing similar recruitment and retention problems.  Even with diligent efforts, day time coverage in some areas is a challenge while many of these volunteers are working at their primary jobs.  Paid services struggle with staffing as well.  It’s no mystery that EMTs and Paramedics don’t get into this business to make good money.  According to a study posted by Monster, the highest paid states provide pay in the $20-$35/hour range, but it slides quickly, with not only the lowest paying states paying in the teens, but the average also sitting in the teens.  Yep, you could be out saving lives and someone working at McDonald’s makes more money than you do.  It’s a rather depressing valuation.

When you couple these two big factors – volunteer staffing and finances – it gets even more difficult.  Paid and volunteer services alike are kicking calls over to mutual aid providers because of being short staffed, which means they miss revenue, which continues to make matters worse.  Many volunteer EMS services, as well as volunteer fire services, are hiring day-shift staff so they can continue to meet needs in their communities during these more difficult timeframes.  For those of you keeping score at home, that’s more cash out.  It might pay off for a busier provider, but certainly not for a provider whose call volume doesn’t balance the checkbook.  Yes, it continues to provide a service to the community, but it’s not sustainable in the long run.

How are the private paid services doing?  Many aren’t doing so well, either.  We see service areas shrinking all around the nation, with paid services seeing diminishing revenues from less dense population areas.  Quite a few paid services make ends meet from interfacility transfers, which are low cost but require volume to make reasonable revenue.

Municipal services are another category which generally have a poor income statement.  I think it’s great that some municipalities provide EMS transport services.  Financially, these services are underwritten by tax payers, with some cost recovery possible through billing insurance companies.  The costs of most municipal services, however, are generally higher, as EMTs and paramedics are government employees, often unionized, and with benefits.  It’s great for them, but not good for the municipal comptroller.  That said, it’s one of the most sustainable models since the underlying financing is spread across all the jurisdiction’s tax payers.  Still a challenge, though, when you consider the tough financial constraints many jurisdictions are facing.

So what’s to be done?  We will eventually need to see a shift in how EMS is provided across the nation.  It is an absolutely necessary service, just as important as roads, fire protection, or law enforcement.  While we won’t see a sudden change, I believe the way forward will be municipal services, or municipally-funded services (those being private or volunteer, but under contract with one or more municipalities).  EMS, similar to other disciplines in public safety, is a public service, and foundationally will need to be publicly funded in order to sustain.  This is nothing new, as there are a number of EMS providers already following this model – that being the maintenance of a contract with one or more municipalities to provide EMS services, for a fee, while also gaining revenue from third party billing, as well as fees for stand-by services for sporting events and other mass gatherings.

What trends do you see in EMS organizational models where you are?  Are the current models sustainable?  Do you view EMS as a public safety endeavor similar to law enforcement and the fire service?

© 2016 – Timothy M. Riecker, CEDP

Emergency Preparedness Solutions, LLCYour Partner in Preparedness

 

 

Embracing the Playbook in Emergency Management

This morning I suggested to a client the possible use of a playbook as a training supplement for their duty officer program.  A playbook is a job aid, but more than just a check list or a chart.  Ideally, a playbook is a structured collection of job aids, broken down by situation for easy reference by whomever needs the information.

Playbooks are best known to be most extensively used in American football.  They contain a variety of strategies for accomplishing certain objectives (short yardage gain, long yardage gain, end zone scoring, kick return, etc.), with each strategy often accompanied by one or more diagrams of Xs (the opposition) and Os (your team), with lines showing movement and action by certain positions.  Playbooks are tested, practiced, and memorized by players before the season begins, but are constantly referenced every game by coaches and players alike.

playbook-diagram

Fundamentally, the play book exists because there are so many situations the team needs to respond to and several different ways in which they can respond.  While some of their plays are fairly routine, others are rarely, if ever, used.  Just like emergency management, however, it’s good to have the plan available if we need it.

Assembling a playbook for emergency management purposes doesn’t have any set standard, but as with any other tool or plan we assemble, we must first identify the purpose and the audience.  Play books can be assembled for elected officials and executives, duty officers and middle management, or dispatchers and first responders – it all depends on what is needed.  One thing that should be stressed, though… playbooks are NOT plans, nor should they be treated as a replacement for training.  Playbooks should be based upon established policy, plans, and procedures; and training still should be conducted on those policies, plans, and procedures.  A playbook is the collection of job aids which will help us to navigate the things people may not routinely do – which makes them excellent for emergency management applications.

Consider that the content of the playbook won’t (and probably shouldn’t) try to map out an incident from inception to completion.  The playbook is intended to address critical actions within an early timeframe of the incident.  Following this, a team should be assembled and striving to manage the incident using the planning process and referencing the foundational plans and policies.  There may be a few nuanced circumstances and activities within the playbook that are still relevant in the extended response, but these should be few.

The organization and content of the playbook should depend on the objectives and audience of the tool.  It should not be heavy in narrative and doesn’t require a lot of background as a plan might.  For emergency management purposes, most playbooks are organized by hazard and/or impact, with topics such as Mass Shooting, Severe Storm, or Power Outage.  For each topic, the essential elements of response can be identified, perhaps in check-list format.  Additional job aids such as decision tables, flow charts, and references can also be incorporated.  While I’m a big fan of keeping things digital, playbooks and similar references are better in hard copy, as this helps to ensure the information is always accessible.  Bind it (a three ring binder is best, so content can be updated as needed) and insert tabs for easy reference.  Be sure to give all users their own copy.

When creating the playbook, consider what information needs to be conveyed.  While some repetition may be necessary from topic to topic, reduce the amount of in-document references (i.e. See page 19 for additional information), as the purpose of the document is to help keep people organized.  Always keep the user in mind, ensuring that they understand each step and that they have the ability, resources, and authority to perform the identified actions.  Ensure that the playbook is a proper reflection of established policy and procedure and be sure to test it for effectiveness.  Train people in its content and use, and be sure to provide regular training and content updates and to incorporate its use into exercises.

I’m interested in hearing your thoughts on playbooks and if you have created and/or used them.  Of course, if your jurisdiction or organization is looking for assistance in developing a playbook for your critical activities, let us know!

© 2016 – Timothy M. Riecker

Emergency Preparedness Solutions, LLCYour Partner in Preparedness