National Highways has claimed improvements to the performance of its stopped vehicle detection (SVD) technology - crucial to the safety of smart motorways - although the new figures coincide with methodology changes known to improve the performance score.
Serious concerns about SVD have been raised since Highways revealed doubts about the presentation of data in an early evaluation of the system.
Despite initial denials from National Highways - some even made to MPs on the Transport Select Committee - its monitor, the Office of Rail and Road (ORR) corroborated Highways' findings in a report last December, which revealed that the system was falling significantly short of National Highways' own performance requirements.
The national roads operator has since been working on improving the SVD system and rolling out upgrades to all its smart motorway routes - with a deadline of the end of June this year to meet the performance specifications. Significantly, the ORR has not confirmed that this has been achieved.
National Highways chief executive Nick Harris said: ‘We have successfully completed our upgrades and operational data from three pilot schemes shows the technology has improved and is now meeting the performance expectations.'
However, in a letter to the ORR, he revealed that the company had also made a number of what he called ‘improvements’ to its methodology for calculating SVD performance, which it had discussed with the ORR.
He said these ‘were to more accurately reflect the performance of the system’.
Although the ORR has not been able to set out the extent of the methodological changes, it told Highways that the data collected no longer includes ‘unverified’ alerts, such as when vehicles are not in a live lane, or where alerts cannot be confirmed by CCTV.
In last December's report, the ORR acknowledged that SVD performance figures depended 'on how unverified events are treated (whether the detection rate is based on the total number of alerts, or only those that can be verified)'.
Using the data from unverified events means worse SVD performance results, but as the ORR report explained, the previous methodology employed unverified events because 'if this worst case is higher than the minimum level, it would demonstrate that the system is meeting the requirement’.
This means that under the new methodology, National Highways has made at least one change that the ORR has publicly described as providing higher, but potentially unreliable scores.
The latest results provided to the ORR from the initial round of testing at three smart motorway all lane running (ALR) sites showed:
- detection rates ranging from 91% to 97% against a requirement of 80%,
- false detection rates ranging from 1% to 10%, against a target of below 15%.
Mr Harris did not say how much of the claimed improvements resulted from changes to the methodology. In fact, he said: ‘Due to differences in the methodology, any comparisons with previous data should be made with significant caution.’
National Highways also reported that the time to detect a stopped vehicle ranged from 5.8 to 9.7 seconds across the three schemes against a target of 20 seconds.
The National Highways' performance requirement for speed of detection
This appears to be an average, although the requirement is in fact that all stopped vehicle detections should be under 20 seconds (see above). The ORR has repeatedly misstated this target, for example in its December report.
In his reply to Mr Harris, ORR chief executive John Larkinson implicitly admitted that his organisation could not quantify the impact of the changes yet. He added: 'We will continue to monitor the data until we are satisfied that these levels of performance are being achieved across all sections of ALR smart motorways.'
The ORR is due to give a full analysis of the impact of the methodology changes later this year. It has admitted that currently, it is dependent on data provided by National Highways and that the results should be treated with caution.
Despite this, Mr Larkinson said in a statement: ‘We welcome the latest data that shows National Highways has responded well to our challenge to improve the performance of Stopped Vehicle Detection technology on All Lane Running (ALR) sections of smart motorways.’
Highways understands that no hardware improvements have been made to the system, only unspecified software tweaks alongside the methodology changes.
The ORR said National Highways had implemented a software fix which it then tested on the three ALR smart motorway schemes; however, the company has not said what this fix involved.
In his letter, Mr Harris did not provide any detail on the improvements either, stating only that the company had ‘set up a team of specialists to analyse every part of the system and determine the required improvements’ and ‘worked with our operators to help ensure they could provide the best service to road users’.
He added that National Highways had piloted the improvements on three ALR motorways and, based on the results of those schemes, rolled them out to other ALR motorways over a period of a month, concluding on 14 June 2023.
The three trial sites were: the M20 between J3 and J5, the M1 between J16 and J19, and the M6 between J2 and J4.
The ORR said its current understanding of the timeline is that fixes were rolled out to trial sites in April/May, with testing in May, before the fixes were rolled out across the network last month.
AA president Edmund King said: 'Last year we expressed severe concerns at the missed targets for operating SVD so it is encouraging to hear that improvements have been made.
'However, the true test will be how the system works on the ground, 24 hours per day, 365 days per year, on all ALR motorways, as ultimately drivers’ lives are at risk if they breakdown in a live lane and the system misses them.'
