My name is Sue Ann Corell Sarpy.  I am Principal of Sarpy and Associates, LLC and Program Chair of the Disaster and Emergency Management Evaluation (DEME) TIG.  This week, I am curating six AEA365 contributions that represent research and practice in DEME encompassing a national and international scope.  I am introducing this week with an evaluation study I conducted concerning resiliency training for workers and volunteers responding to large-scale disasters.  

The National Institute of Environmental Health Sciences (NIEHS) Worker Training Program with the Substance Abuse and Mental Health Services Administration (SAMSHA) identified a need to create trainings that promoted mental health and resiliency for workers and volunteers in disaster impacted communities.  We used a developmental evaluation approach that spanned two distinct communities (e.g., Gulf South region; New York/New Jersey region) disasters (e.g., Gulf of Mexico Oil Spill; Hurricane Sandy) and worker populations (e.g., disaster workers/volunteers; disaster supervisors) to identify effective principles and address challenges associated with these dynamic and complex training needs.  We used an iterative evaluation process to enhance development and delivery of the training such that project stakeholders provided active support and participation in the evaluation and discussion of findings and incorporated the effective principles (best practices/lessons learned) into the next iteration of training. 

The evaluation results supported the usefulness of this type of developmental evaluation approach for designing and delivering disaster worker trainings.  The effectiveness of the training was demonstrated in different geographic regions responding to different disaster events with different target audiences.  The graphic below depicts percentage of ratings of agreement with how well the training met the needs of the various target audiences.  We found that ratings increased as we continued to integrate the best principles in the training, starting with the disaster worker training in the Gulf South region to the final phase of the project with the disaster supervisory training in the New York/New Jersey region.

Lessons Learned:

Several key factors were critical for success in evaluating resiliency training for workers and volunteers responding to large-scale disasters:

• Major stakeholders actively involved in development, implementation, and evaluation of trainings.  We included information from workers, supervisors, community-based organizations, and subject matter experts in the evaluation data and discussion of findings.

• Evaluation conducted early on in the training design and feedback of effective principles used in each iteration.  We, the evaluators, were brought in as a key stakeholder early in the process and we were integral in revising and refining the training products as the project progressed.

• Build relationships and trust with various stakeholders to gather information and refine curriculum.  The inclusion of stakeholder feedback, so that everyone gets a voice in the system, built trust and buy-in in the evaluation process and was key to its success.

• Creating balance between standardized training and flexibility to tailor training to meet needs (adaptability).  The best principles that emerged across worker populations, communities, and disasters/emergencies provided a framework that allowed for a structure for the trainings but afforded flexibility/adaptability needed to meet specific training needs.

Rad Resources:

Please visit the NIEHS Responder and Community Resiliency web site for training resources related to this project.