Failed Attempts to Measure NIMS Compliance – How can we get it right?

Yesterday the US Government Accountability Office (GAO) released a report titled Federal Emergency Management Agency: Strengthening Regional Coordination Could Enhance Preparedness Efforts.  I’ve been waiting for a while for the release of this report as I am proud to have been interviewed for it as a subject matter expert.  It’s the second GAO report on emergency management I’ve been involved in through my career.

The end game of this report shows an emphasis for a stronger role of the FEMA regional offices.  The GAO came to this conclusion through two primary discussions, one on grants management, the other on assessing NIMS implementation efforts.  The discussion on how NIMS implementation has thus far been historically measured shows the failures of that system.

When the National Incident Management System (NIMS) was first created as a nation-wide standard in the US via President Bush’s Homeland Security Presidential Directive (HSPD) 5 in 2003, the NIMS Integration Center (NIC) was established to make this happen.  This was a daunting, but not impossible task, involving development of a standard (lucky much of this already existed through similar systems), the creation of a training plan and curricula (again, much of this already existed), and encouraging something called ‘NIMS implementation’ by every level of government and other stakeholders across the nation.  This last part was the really difficult one.

As identified in the GAO report: “HSPD-5 calls for FEMA to (1) establish a mechanism for ensuring ongoing management and maintenance of the NIMS, including regular consultation with other federal departments and agencies and with state and local governments, and (2) develop standards and guidelines for determining whether a state or local entity has adopted NIMS.”

While there was generally no funding directly allocated to NIMS compliance activities for state and local governments, FEMA/DHS associated NIMS compliance as a required activity to be eligible for many of its grant programs.  (So let’s get this straight… If my jurisdiction is struggling to be compliant with NIMS, you will take away the funds which would help me to do so????)  (the actual act of denying funds is something I heard few rumors about, but none actually confirmed).

NIMS compliance was (and continues to be) a self-certification, with little to no effort at the federal level to actually assess compliance.  Annually, each jurisdiction would complete an online assessment tool called NIMSCAST (the NIMS Compliance Assistant Support Tool).  NIMSCAST ran until 2013.

NIMSCAST was a mix of survey type questions… some yes/no, some with qualified answers, and most simply looking for numbers – usually numbers of people trained in each of the ICS courses.  From FEMA’s NIMS website: “The purpose of the NIMS is to provide a common approach for managing incidents.”  How effective do you think the NIMSCAST survey was at gauging progress toward this?  The answer: not very well.  People are good at being busy but not actually accomplishing anything.  It’s not to say that many jurisdictions didn’t make good faith efforts in complying with the NIMS requirements (and thus were dedicated to accomplishing better incident management), but many were pressured and intimidated, ‘pencil whipping’ certain answers, fearing a loss of federal funding.   Even for those will good faith efforts, churning a bunch of people through training courses does not necessarily mean they will implement the system they are trained in.  Implementation of such a system required INTEGRATION through all realms of preparedness and response.  While NIMSCAST certainly provided some measurable results, particularly in terms of the number of people completing ICS courses, that really doesn’t tell us anything about IMPLEMENTATION.  Are jurisdictions actually using NIMS and, if so, how well?  NIMSCAST was a much a show of being busy while not accomplishing anything as some of the activities it measured.  It’s unfortunate that numbers game lasted almost ten years.

In 2014, the NIC (which now stands for the National Integration Center) incorporated NIMS compliance questions into the Unified Reporting Tool (URT), including about a dozen questions into every state’s THIRA and State Preparedness Report submission.  Jurisdictions below states (unless they are Urban Area Security Initiative grant recipients) no longer need to provide any type of certification about their NIMS compliance (unless required by the state).  The questions asked in the URT, which simply check for a NIMS pulse, are even less effective at measuring any type of compliance than NIMSCAST was.

While I am certainly being critical of these efforts, I have and continue to acknowledge how difficult this particular task is.  But there must be a more effective way.  Falling back to my roots in curriculum development, we must identify how we will evaluate learning early in the design process.  The same principal applies here.  If the goal of NIMS is to “provide a common approach to managing incidents”, then how do we measure that?  The only acceptable methodology toward measuring NIMS compliance is one that actually identifies if NIMS has been integrated and implemented.  How do we do that?

The GAO report recommends the evaluation of after action reports (AARs) from incidents, events, and exercises as the ideal methodology for assessing NIMS compliance.  It’s a good idea.  Really, it is.  Did I mention that they interviewed me?

AARs (at least those well written) provide the kinds of information we are looking for.  Does it easily correlate into numbers and metrics?  No.  That’s one of the biggest challenges with using AARs, which are full of narrative.  Another barrier to consider is how AARs are written.  The HSEEP standard for AARs is to focus on core capabilities.  The issue: there is no NIMS core capability.  Reason being that NIMS/ICS encompasses a number of key activities that we accomplish during an incident.  The GAO identified the core capabilities of operational coordination, operational communication, and public information and warning to be the three that have the most association to NIMS activities.

The GAO recommends the assessment of NIMS compliance is best situated with FEMA’s regional offices.  This same recommendation comes from John Fass Morton who authored Next-Generation Homeland Security (follow the link for my review of this book).  Given the depth of analysis these assessments would take to review AAR narratives, the people who are doing these assessments absolutely must have some public safety and/or emergency management experience.  To better enable this measurement (which will help states and local jurisdictions, by the way), there may need to be some modification to the core capabilities and how we write AARs to help us better draw out some of the specific NIMS-related activities.  This, of course, would require several areas within FEMA/DHS to work together… which is something they are becoming better at, so I have faith.

There is plenty of additional discussion to be had regarding the details of all this, but its best we not get ahead of ourselves.  Let’s actually see what will be done to improve how NIMS implementation is assessed.  And don’t forget the crusade to improve ICS training!

What are your thoughts on how best to measure NIMS implementation?  Do you think the evaluation of AARs can assist in this?  At what level do you think this should be done – State, FEMA Regional, or FEMA HQ?

As always, thanks for reading!

© 2016 – Timothy Riecker

Emergency Management: Coordinating a System of Systems

Emergency management, by nature, is at the nexus of a number of other practices and professions, focusing them on solving the problems of emergencies and disasters.  It’s like a Venn diagram, with many entities, including emergency management, having some overlapping interests and responsibilities, but each of them having an overlap in the center of the diagram, the place where coordination of emergency management resides. That’s what makes the profession of emergency management fairly complex – we are not only addressing needs inherent in our own profession, we are often times doing it through the application of the capabilities of others.  It’s like being the conductor of an orchestra or a show runner for a television show. It doesn’t necessarily put emergency management ‘in charge’, but they do become the coordination point for the capabilities needed.



This high degree of coordination depends on the functioning and often integration of a variety of systems.  What is a ‘system’?  Merriam-Webster offers that a system is a “regularly interacting or interdependent group of items forming a unified whole.”  Each agency and organization that participates in emergency management has its own systems.  I’d suggest that these broadly include policies, plans, procedures, and the people and technologies that facilitate them – and not just in response, but across all phases or mission areas.  Like the Venn diagram, many of these systems interact to (hopefully) facilitate emergency management.

There are systems we have in many nations that are used to facilitate components of emergency management, such as the National Incident Management System (NIMS), the Incident Command System (ICS) (or other incident management systems), and Multi-Agency Coordination Systems (MACS).  These systems have broad reach, working to provide some standardization and common ground through which we can manage incidents by coordinating multiple organizations and each of their systems.  As you can find indicated in the NIMS doctrine, though, NIMS (and the other systems mentioned) is not a plan.  While NIMS provides us with an operational model and some guidance, we need plans.

Emergency Operations Plans (EOPs) help us accomplish a coordination of systems for response, particularly when written to encompass all agencies and organizations, all hazards, and all capabilities.  Likewise, Hazard Mitigation Plans do the same for mitigation activities and priorities.  Many jurisdictions have smartly written disaster recovery plans to address matters post-response.  We also have training and exercise plans which help address some preparedness measures (although generally not well enough).  While each of these plans helps to coordinate a number of systems, themselves becoming systems of systems, we are still left with several plans which also need to be coordinated as we know from experience that the lines between these activities are, at best, grey and fuzzy (and not in the cuddly kitten kind of way).

The best approach to coordinating each of these plans is to create a higher level plan.  This would be a comprehensive emergency management plan (CEMP).  Those of you from New York State (and other areas) are familiar with this concept as it is required by law.  However, I’ve come to realize that how the law is often implemented simply doesn’t work. Most CEMPs I’ve seen try to create an operational plan (i.e. an EOP) within the CEMP, and do very little to actually address or coordinate other planning areas, such as the hazard mitigation plan, recovery plans, or preparedness plans.

To be successful, we MUST have each of those component plans in place to address the needs they set out to do so.  Otherwise, we simply don’t have plans that are implementation-ready at an operational level.  Still, there is a synchronicity that must be accomplished between these plans (for those of you who have experienced the awkward transition between response and recovery, you know why).  The CEMP should serve as an umbrella plan, identifying and coordinating the goals, capabilities, and resources of each of the component plans.  While a CEMP is generally not operational, it does help identify, mostly from a policy perspective, what planning components must come into play and when and how they interrelate to each other.  A CEMP should be the plan that all others are built from.


I’m curious about how many follow this model and the success (or difficulty) you have found with it.

As always, if you are looking for an experienced consulting firm to assist in preparing plans or any other preparedness activities, Emergency Preparedness Solutions is here to help!

© 2016 – Timothy Riecker

Emergency Preparedness Solutions, LLC

H.R. 4397 – The Rail Safety Act

A few days ago I came across a notice of the introduction of HR 4397, aka the Rail Safety Act.  Text of the bill can be found here:  This bill was introduced by Rep. Ron Kind (D-WI) and had been assigned to House Committee on Transportation and Infrastructure on January 28th.  The essence of the bill… “To direct the Administrator of the Federal Emergency Management Agency to provide for caches of emergency response equipment to be used in the event of an accident involving rail tank cars transporting hazardous material, crude oil, or flammable liquids.”

If you follow the link provided above, you will get the full text of the bill, which honestly doesn’t tell much more.  I’m rather ambivalent about things like this.  We have a history of pre-positioning equipment and supplies for a variety of disasters.  Organizations such as the American Red Cross function this way, as do various agencies of the US federal government.  In 1999 the National Pharmaceutical Stockpile (NPS) was expanded to preposition medical supplies around the nation as a preparedness effort for a biological or chemical attack.  This program expanded in 2003 and became the Strategic National Stockpile (SNS).  In 2006, FEMA/DHS developed a program to pre-position disaster supplies (mostly mass care types of supplies) in certain disaster prone areas around the nation.  While we also have a variety of specialized teams, that’s a slightly different matter.

One key struggle of prepositioning supplies and equipment largely boils down to who will be responsible for them.  Supplies and equipment need to be secured and maintained.  This requires some regularity of check in to ensure they are ready to be deployed at a moment’s notice.  Each location needs a deployment plan, identifying how these assets will be deployed.  As part of this planning, there must be a trigger mechanism for requesting these supplies.  The supplies must deploy and reach their destination in such a time frame to be effective.  Of course upon arrival of the cache, responders must be familiar with what is there, take time to unpack it and inspect it, and be readily able to use it (therefore they must be pre-trained in the use of the equipment).  So who will be responsible for these caches?  State governments?  Local governments?  Rail carriers?  First responders?

Hazardous materials response is one of the most highly regulated aspects of public safety.  It is governed in the US by the Occupational Health and Safety Administration (OSHA) regulation 1910.120.  There is a strong emphasis on preparedness and protocol.  Only individuals trained to certain levels can conduct certain actions in a hazardous materials response.  Most responders, due to the fairly low ranking hazard of a major hazardous material release in their jurisdiction, do not have the degree of training needed to utilize some of what I expect would be in a cache of supplies as ordered by the Rail Safety Act.  That said, every jurisdiction in the US has access to a hazardous materials team – either from a nearby jurisdiction or from the state.  These teams have the specialized training and equipment needed to address a hazmat incident.  Now that we’ve gotten to that, what, exactly, is the need that the Rail Safety Act is addressing?

Sure, these caches of supplies may provide more of whatever is needed, but there are a few issues here.  First of all, it will take people to examine what is being delivered, to unpack it, and to ready it for deployment.  Often, the biggest issue on a response such as this is a lack of trained personnel.  Second, will the materials being provided by the cache be interoperable with what the hazmat team is using?  While we have gotten better at standardizing equipment, there are still many issues out there.  Tab A requires Slot A.  Slot B simply won’t work.

I suppose what I’m really interested in here is a definition of need.  Has there been any type of needs assessment or feasibility study conducted for this?  I’m doubtful.  Most bills are generally introduced at a whim by well-intentioned but ill-informed politicians.  The last thing we need is another requirement to work within that does us little good – even if it is to be funded by the rail carriers.

I’m curious if anyone out there happens to know about this bill or any need supporting it.

© 2016 – Timothy Riecker

Book Review – The Storm of the Century by Al Roker

Yep, THE Al Roker.  The weather guy.  Fellow SUNY Oswego alum.  Smart, funny, the kind of guy you want to have over for poker night.

Roker has actually written a few books, covering cooking, murder mysteries, family, and weight loss.  The Storm of the Century was released late last year and offers a compelling historical review of the Gulf Hurricane of 1900 that destroyed Galveston, Texas.


Admittedly, the book was not what I expected.  I anticipated a book that had more structure and was a bit more proper and history-bookish.  While I was pleasantly surprised by the book’s more narrative approach, switching gears mentally took me a while, which is I think why I had a hard time with the first few chapters.  That’s on me, though, and not a reflection on the book itself.

The book is set up almost like a work of fiction, setting the stage of the time and place of our environment and introducing and developing the main characters.  Don’t be fooled, though – this is no work of fiction.  The events described in the book are real, as were the stories of the people.  Roker emphasizes this at the end of book, as he details both formally through a bibliography and informally through narrative, the sources of his information, which include newspapers, scholarly works, historical accounts, and documented eye witness reports.

The book follows the lives of several individuals and families, with the primary focus on Isaac and Joseph Cline, who worked for what became the National Weather Service.  Roker mixes in a number of other personalities from Galveston and other areas.  Notables, such as Clara Barton, Joseph Pulitzer, and William Randolph Hearst also figure into the events of this devastating hurricane.  Roker provides insight on the state of politics and society in the post-Civil War United States, external political relations, and certain beliefs of those in meteorological science at the time, including the Jesuit priests of Cuba.

Roker details the interesting history of the National Weather Service, with its roots in the US Army Signal Corps, as well as some of the science and instrumentation of meteorology.  It’s interesting to see how much we have advanced in the science, yet how much still reflects back on what was done almost 120 years ago.

In the end, the events surrounding The Storm of the Century create a story of human error, tragedy, and perseverance.  In the practices of emergency management we must always keep in mind the human element.  Ultimately, that’s why emergency management exists.  While our focus might be on critical infrastructure, NIMS, or the current organization of FEMA, the reason why must ALWAYS reflect on people and our need to protect them from the impacts of disasters.  The Storm of the Century does just this, putting society front and center.

The Storm of the Century is overall a good read and accessible to many audiences including disaster and meteorology buffs, social scientists, and even those interested in US history.  The book is a great read for colleges and high schools alike, offering insights on society, politics, and science.  And hey, mine is even autographed.  Thanks Al!


© 2016 – Timothy Riecker

Continuity of Government – Preservation of Records and Data

A common but often low priority issue in emergency management is the loss of physical records and electronic data from a disaster.  To be honest, I ignored the issue for much of my career.  It wasn’t until working on a contract in the northeast and meeting with a lot of local governments did my eyes really open to the importance of the issue.  While this article focuses on preservation of records for governments, it can certainly apply to businesses, not for profits, and even individuals.

Many of the local governments we interfaced with on a completely unrelated contract, were talking about their experiences with Tropical Storm Irene.  Town officials told of their efforts hauling boxes of town records either to a higher floor of town offices or removing them offsite, with water to their knees or even waist high.  Needless to say, many records were lost.

While some of these offices were in known floodplains, others simply suffered from an extraordinary event and the fault of a place where we commonly store things – the basement.  Towns (and other municipal offices) often store physical copies of tax maps and records, property deeds, permits, flood insurance information (ironic, isn’t it?), human resources data for town employees, town financial records, court records, birth certificates, death certificates, marriage certificates, divorce certificates, and other information.  The loss of this information can have an impact, not only historically, but also on current government operations.

Continuity of government and continuity of operations plans should identify those records which are most important.  These are called vital records.  Vital records should have the highest degree of protection.  The National Archives offers some guidance on the protection of vital records.  While the guidance applies to federal agencies, there is still plenty of valuable information which can be applied to other organizations.

Every municipality should examine records storage as part of their continuity of operations and continuity of government planning.  It’s not to say that records can’t be stored in the basement of a building, but mitigation efforts must be made to flood proof the building as much as possible, including water alarms and sump pumps connected to emergency power systems.  Paper and water don’t mix – so get your records off the floor and consider waterproof storage solutions.  Ventilation is also important to prevent molding.

If mitigation is too costly, then you need to consider relocating the records.  Regardless of where your records are, you should have a component of your continuity of operations plan that addresses emergency relocation of records – when, how, to where, and who.  Digital storage is obviously a great solution.  Some towns I spoke with had decided after the storm to scan their records.  Catching up to a hundred plus years of records can be pretty time consuming and practically unsurmountable for most municipal offices.  This is a service that can be hired out.  Be sure to follow sound data protection standards for both storage and access to ensure the continuity of these records.

In the event that records do get wet, all is not necessarily lost.  The Preservation Directorate of the US Library of Congress has a lot of information on preservation of records, including a variety of resources and training opportunities.  There are also companies that specialize in document preservation and recovery after a disaster.  While it’s probably a good idea to identify who you might reach out to in the event of such a loss, know that this is expensive and it’s generally far more cost effective to mitigate against the risk.

Need assistance with government continuity or continuity of operations planning?  EPS can help!

© 2016 – Timothy Riecker

Emergency Preparedness Solutions, LLC

Seek First to Understand

‘Seek first to understand.’  It’s one of Stephen Covey’s 7 Habits of Highly Effective People.

This past weekend I came across a blog in a prominent industry magazine’s online edition which was highly critical of a recent response and the state of preparedness of a major metropolitan area.  I was quite set back by how outwardly critical this post was, particularly since the author is rather experienced in emergency management.

No matter what field we are in, we have a tendency to examine, critique, analyze, and criticize.  This is generally healthy and important, especially when there is something that can be learned and applied from the experience.  Things can easily go ugly, though.

The nitty gritty of this is that if you weren’t involved and aren’t providing a critique through something more or less official and reasonably objective, such as an after action report, you generally shouldn’t be commenting (at least publicly).  Why?  Primarily, you very likely don’t have all the information.  Second, what is the criticism gaining you aside from looking like an ass?

Seek first to understand.  That’s the main reason why we, particularly in emergency management, should be looking at other people’s incidents.  Yes, we can examine media reports and other sources of information, but be holistic and comprehensive.  If the people involved in managing the incident made mistakes, then learn from their mistakes.  Don’t criticize them for it – they very likely are already receiving that criticism internally.  They certainly don’t need you to Monday morning quarterback.  It does no one any good.

Pointing fingers at other people only makes them point fingers back and creates a culture of negativity.  In emergency management, we are fortunate enough to have a culture of collaboration, where we are generally willing to share our success and failures with others so that they may learn from them as well.  When we become critical, people become bitter, defensive, and isolationist.

It’s not to say that it’s inappropriate to use an incident as an example.  In December I wrote a post about how People Should Not Die in Exercises, in response to an article about an active shooter exercise in Kenya gone wrong. Was I harsh?  You bet your ass I was – and rightfully so.  The occurrence I wrote about was a great example of what not to do in exercises and an important lesson learned that a lot of people should know about to prevent further loss of life.

While I have as much a history of putting my foot in my mouth as the next person, all I’m saying is be careful how you spend your criticism credits.  When you start to criticize you are no longer seeking to understand.  If you aren’t seeking to understand, then no one learns.


Another Great Emergency Management and Homeland Security Podcast

A couple months ago I came across another great podcast.  This one is done by a company called PreparedEx – and on Twitter @preparedex.  They link to their podcast from their website but you can also find it in the iTunes podcast listing.  They are only seven episodes in, and most episodes are about a half hour long, so you can catch up pretty quickly.  They generally post two episodes a month, which is excellent frequency.

PreparedEx is a consulting firm specializing in preparedness exercises.  Yes, they are technically a competitor of my company, but from what I’ve seen and heard, they are quite capable and do some really cool stuff.

The host of their podcast is Robert Burton, who is the company’s managing director.  Robert has some great counterterrorism creds and facilitates the podcast well.  What I love most about this podcast is the interview format.  Nearly every episode focuses on an interview, and they have gotten some great subjects – from state emergency management directors to corporate security specialists.  The interviews offer excellent insight and are very conversational and easy to listen to. They cover topics in emergency management, homeland security, and business continuity.

Go check them out and enjoy!

– TR