STREET: Swedish Tool for Risk/Resource Estimation at EvenTs. Part one, risk assessment – face validity and inter–rater reliability

Abstract

Objective: To develop a validated and generalized high reliability organizations collaborative tool in order to conduct common assessments and information sharing of potential risks during mass-gatherings. Methods: The Swedish resource and risk estimation guide was used as foundation for the development of the generalized collaborative tool, by three different expert groups, and then analyzed. Analysis of inter-rater reliability was conducted through simulated cases that showed weighted and unweight κ-statistics. Results: The results revealed a mean of unweight κ-value from the three cases of 0.37 and a mean accuracy of 62% of the tool. Conclusions: The collaboration tool, “STREET”, showed acceptable reliability and validity to be used as a foundation for high reliability organization collaboration in a simulated environment. However, the lack of reliability in one of the cases highlights the challenges of creating measurable values from simulated cases. A study on real events can provide higher reliability but need, on the other hand, an already developed tool.

Keywords

• Events
• Mass-gatherings
• Assessments
• Face validity
• Inter-rater reliability
• High reliability organizations
• Healthcare
• Rescue teams
• Police