National review of community risk methodology across UK Fire and Rescue Service

Testing assumptions

Cheshire FRS uses historical data from the previous 5 years incident data to predict future risk. However, where Cheshire’s good practice is unique is that they first ‘tune’ the software by testing the previous year’s data to situations in the present. The back testing of new approaches through older data not only refines newer methodologies, but also tests specificity of the method and the compliance of the structure of the datasets to the new methodology.

Conceptualisation of risk

Across the submissions we identified differing philosophical positions of representation of risk as a concept. For example West Midlands, Cumbria, Cheshire and Buckinghamshire conceptualise and understand risk through the approach of absolute risk (i.e. the likelihood of an individual experiencing an incident), relative risk (i.e. the likelihood of an incident for different demographics), and comparative risk (i.e. the likelihood of an incident happening in the population). These FRSs also demonstrated a good understanding of the fallibility of risk scores. Alternatively, London represents risk as all risk identified by their community members, regardless of whether it sits within the statutory duties of the FRS or not. These differing approaches do not really have a ranking order theoretically as both can be supported in the academic literature that considers the conceptualisation of risk. Although this sounds theoretical in nature, this does have implications as to the initial starting point for risk identification.

Identification of risk

It is common across the UK FRS to use national risk registers to articulate and inform a local perspective of risk. This aligns with statutory duties. Aligned to this was the identification of a smaller set of FRSs who are understanding and using national patterns and trends to benchmark and inform their understanding of local risks. This activity will not only highlight areas of discrepancy, but it would also provide the opportunity to horizon scan (West Midlands, London, Leicestershire, Humberside and Tyne and Wear all articulate that they complete horizon scanning for future risks/hazards explicitly). Buckinghamshire and other FRSs are developing practice in this area, but the initial submissions did not articulate the full process with which they complete this consideration. This is good practice to also inform FRSs articulation, understanding and plan of response to regional and national level risks.

Statistical analysis of risk

There is good practice in FRSs applying caution when using and interpreting weighted percentages. Devon and Somerset provide a good understanding of the fallibility of weighted percentages in their submission. West Midlands and East Sussex also provide a good understanding of what evidence informs the weightings of those weighted percentages. Buckinghamshire, West Midlands and London all provide a good understanding of how to use outliers, quartiles and standard deviations to localise risk scores. West Midlands also use Pearson’s correlation analyses. The use of descriptive and inferential statistics in modelling risk allows a more sophisticated analysis of the data and more assured inferences to be made from findings. Transferring this knowledge across the sector would increase the ability of the sector to analyse the risks identified.

Understanding the nature of risk

There was a discrepancy between FRSs in how they would prioritise risks. Many FRSs used fire fatality and casualty as a category of risk. London (amongst others) used incident of fire with or without fatality or casualty as a category to acknowledge the economic scale of impact of those fires. This is to acknowledge the impact on the individual, similar to the positions taken relating to personal, community or local disasters in the sociology of disaster literature. A small-scale incident in a localised area maybe experienced as a disaster (through economic or social impact) by those in that area. This can be scaled and extrapolated to fire risk as while a fire may not have a fatality or casualty, it may be a significantly impactful fire to those inhabiting or using the building, with regards to the effects on the individual of withdrawing access to the building due to damage or the effects of loss of valuable (sentimental or financial) belongings in fire.

Risk analysis

Across the UK FRS there are many different ways in which risks and hazards are identified. These component parts of the identification of different risks are not clearly identified by the submissions, however each FRS clearly combines these in different ways to produce a risk rating or matrix. This is typically a likelihood by severity model which aligns to good practice and theory in risk management. Some of this is done at LSOA level, although other sizes of geography are selected (borough, area risk etc). How these data sources and resulting risks are integrated was not detailed in most submissions. For example, whether the risks are layered or integrated to provide a single output of priorities was not clear. Most FRSs used their own products / models to perform this task, but named processes included; Fire Risk Assessment Methodology, Likelihood x Severity model, Prioritisation Tool, Community Risk Model, Risk Matrix etc. Having a clear method of integrating the output from analysis of individual datasets is an area for FRSs to improve their transparency of approach.

Methodological limitations

There was evidence of data expertise from some FRSs, who reported limitations with the methodology with regards to the quality (and therefore usefulness) of the data, and of concerns with the use of non-evidence-based weighting of contributing factors of risk. We encourage all practitioners involved in risk management to have a good awareness of the limitations of their risk management methodologies, as this will influence the relative validity of findings, and push FRSs to continue to improve their methodologies.