National Highways has made significant progress on smart motorway safety according to a new report from the roads monitor however concerns remain about the strength of the underlying testing data.
Most notably, the national roads operator hit all its key targets for stopped vehicle detection (SVD) - meaning the complex system can officially become 'business as usual' now.
While software fixes were cited as the key reason for the progress, the improvements also followed a change in testing methodology, which saw about 85% of the stopped vehicle alerts thrown out of the 'ground truthing' process as being 'unverifiable'.
The 'ground truthing' process uses recorded CCTV footage to check alerts and determine the proportion of stopped vehicles where an alert was correctly raised by the SVD system, and the time taken for an alert to be generated.
Changes to the methodology meant that any incident where the CCTV footage was incomplete - due to the way a camera might have panned, for instance - was counted as 'unverifiable'.
As a result, the ORR has asked National Highways to review how it 'collects and analyses information about unverified alerts'.
Software upgrade
The latest ORR strategic road network safety report states that National Highways 'identified a series of potential software fixes that included improvements to the performance of radar equipment and filtering out multiple alerts created by vehicles stopped in queuing traffic'.
The national roads operator implemented these software fixes by June 2023 and subsequently made a huge leap forward from its results in the ORR's 2022 safety report, which found National Highways to be failing in all the key SVD areas.
The latest data from ground truthing tests this year shows that the performance targets for the three metrics were being met:
- Detection rates (i.e., the proportion of stopped vehicles correctly identified by SVD): 89% (target >80%)
- False detection rates (i.e., the proportion of SVD alerts that are reported incorrectly because the alert does not relate to a true stopped vehicle): 6% (target <15%)
- Average time to detect stopped vehicles (i.e., the elapsed time between a vehicle stopping and an SVD alert being generated): 12.2 seconds (target <20 seconds)
These three targets were also met in all six National Highways operational regions, except in one case. In the Yorkshire and North East region, the performance for detection rates was 76% though it had significantly fewer SVD alerts compared to the other regions.
National Highways also reported to the ORR that 92% of stopped vehicles were being detected within 20 seconds.
Methodology change
However, ORR notes that a key impact of methodological changes to the testing process was that '85% of all stopped vehicle alerts were classed as unverified and therefore excluded from the performance calculation'.
The methodological changes were to:
(a) Exclude any alerts beyond defined SVD coverage areas. This means any alerts outside defined coverage areas, for example, short segments of the hard shoulder, were excluded from the testing.
(b) Exclude ‘unverified’ alerts: Any alerts categorised as ‘unverified’ were excluded from the analysis. An alert is classified as unverified when, during the ground truthing testing process, it has not been possible to use the recorded CCTV footage to verify all the performance metrics which are measured in a ‘stopped vehicle event’. For example, if recorded CCTV footage does not capture the start of the event because it was pointed in a different direction, this would be excluded from the calculation as it is not possible to measure the time taken to detect the stopped vehicle.
The ORR report adds: 'There are different levels of certainty associated with the various scenarios but, under the new methodology, National Highways only uses data where it has 100% certainty about how well the system has performed in detecting a stopped vehicle.'
This means it only has 100% certainty around 15% of the stopped vehicle alerts that could have initially come under the ground truthing exercise.
Across the 21 all lane running (ALR) smart motorway schemes - those without a hard shoulder - there were 3,850 alerts. Of these, 3,286 were classified as unverified, which left a total sample size of 564 used for testing SVD detection rate and time to detect.
Of the tested events, 80% occurred in an emergency area and the remaining 20% were live lane stops.
The software improvements were first tested on three 'spearhead' smart motorway schemes before being rolled out to the full 21 ALR schemes.
The ORR said that it had challenged National Highways to demonstrate the benefits of the software improvements compared to the impact of the change in methodology and the company 'did this by sharing details from testing it had undertaken on the three spearhead schemes to assess the effectiveness of the software fixes'.
It applied a consistent methodology to ground truthing data from before the software upgrade (in 2022) and after the upgrade (in 2023) and repeated this for both the old and new methodologies, the ORR states.
'In both cases, the data showed improved performance across all three metrics of detection rate, time to detect and false detection rates in 2023 (after the software fixes).’
The data from this exercise was not included in the ORR report, however. Highways has asked for the data to be provided.
Attendance and availability
Four of National Highways’ six operational regions achieved annual average attendance times for stopped vehicles of under 10 minutes in the year from September 2022 to August 2023.
The two regions that did not achieve annual average response times of 10 minutes were: the East region (10 minutes 22 seconds); and the South East (10 minutes 55 seconds).
In 2022, National Highways had also been failing a wider technology availability target of 95%.
After committing to a £105m investment plan to improve operational technology (OpTech) on ALR motorways, National Highways aimed for 97% average availability for the four asset types by the end of this road period (2025).
The four key elements of operational technology (OpTech) are:
- the Motorway Incident Detection and Automatic Signalling (MIDAS); which had a score of 97% availability for the year from September 2022 to August 2023,
- variable message signs, which had a score of 91%;
- signals (speed and lane control signals) (96%)
- and CCTV Cameras (96%).
The availability score in September 2023 across these four assets was 95.6%.
Emergency rest areas
National Highways appears to be far behind on its commitment to roll out over 150 additional emergency areas on ALR smart motorways by the end of RP2 (2025), under a £390m allocated budget.
By September 2023, National Highways had only completed work on 13 emergency areas, on the M1 and M6. The ORR said the next phase of work is expected to deliver a further 40 on the M1 and M25.
Final results
Shortly before the ORR report was released, National Highways released its own third annual progress report following the Government's smart motorway stocktake.
It found that 'the latest safety data continues to show that overall, all three types of smart motorway [all lane running, dynamic hard shoulder, controlled] are better than conventional motorways for those safety metrics which consider the most significant impacts, such as deaths or serious injuries'.
However, this is based on using a five-year rolling average to 'smooth some of the peaks and troughs caused when safety across all roads in individual years is very different to its usual levels'.
The effect also masks the impact of dramatically increasing traffic levels on smart motorways and the knock-on impact on safety figures.
Due to new schemes opening and changes in driving patterns, all lane running schemes increased from 14.3hmvm (hundred million vehicle miles) in 2015 to 67.7hmvm in 2021 - the last year of smart motorway data provided in the National Highways report.
In the same period, conventional motorways decreased from 482.3hmvm to 369.3hmvm.
National Highways reports that between September 2022 and August 2023, live lane incident numbers on ALR smart motorways increased by 38%, from 1,308 to 1,808 and adds 'this is likely to be related to increased traffic, as four new ALR schemes opened in this period, bringing the total to 21'.
For 2021, the Department for Transport statistics show ALR schemes had higher personal injury collision rates per hmvm than conventional motorways (6.19 to 5.10), higher killed and seriously injured rates per hmvm (1.66 to 1.38) and higher fatal and weighted injuries per hmvm (0.39 to 0.36).
