Highways England has admitted a string of errors were made in a report on the effectiveness of the stopped vehicle detection (SVD) system used to increase safety on smart motorways, casting doubt on whether the system managed to pass the company’s own evaluation.
The SVD system being retrofitted across the smart motorway network at a cost of at least £18m only sent alerts for 65% of incidents in trials.
This was well below a required detection rate of 80% - a relatively low benchmark that reflected the fact Highways England was only testing the system to see if it ‘would provide sufficient additional safety benefit’ to be worth fitting.
The system passed the evaluation in 2016 because some of the events for which alarms were not sent were put towards the successful detections total, which saw the success rate boosted to 86.4% - a pass mark even after the 3.9% margin of error was applied.
However, Highways England has admitted to Highways that a significant number of events were incorrectly classified as detected.
Subsequently, a highway expert told Highways that these events should have been classified as missed detections. If they had the system would have failed the evaluation.
Evidence suppressed?
Highways England has also told Highways that a second report, in 2018, tested the same system. In high traffic volumes, this report found an even lower headline detection rate of 82.4%. However, Highways England has refused to release this report on grounds of ‘commercial confidentiality’.
A Department for Transport spokesperson said: ‘Since taking office, the current Secretary of State has expressed his concerns over smart motorways. He has committed £500m to safety improvements and has recently pressed Highways England to further accelerate work in certain areas.’
The spokesperson declined to comment on Highways England’s refusal to publish the 2018 report.
Edmund King, AA president, who has long campaigned for safer ‘smart’ motorways told Highways: 'Any questions about the efficiency of stopped vehicle detection should be investigated as a matter of urgency. Smart motorways are meant to rely on technology but if that technology is now being questioned it might be time to go back to the drawing board.'
Detections, alerts and alarms
The SVD system uses a combination of radar, a server and an algorithm. When it detects a stopped vehicle, it sends an alert to the Highways England regional control centre, which creates an alarm for operators.
In the 2016 report, which focused on an early trial of the SVD system across 13km of the M25, the system detected and sent alerts for 192 stationary vehicle events in six days. A review of CCTV footage identified 294 events during the same period.
'Unknown reason' missed detections were in fact 11.2% of the total
Of the 102 events for which alerts were not sent: 40 were classified as missed detection for ‘unknown reason’ (33) or ‘blind spots’ (7), while 62 were classified as ‘allowable missed detection’ - although elsewhere in the report it labels the 'allowable missed detections' as 'SVD system detected'.
Highways England has claimed that 45 of these were in fact incorrectly labelled ‘allowable missed detections' and should have been termed 'SVD system detected', as they were detected but no alert was sent because the system was suppressed. The report does not provide evidence of this or describe a methodology by which it was established.
The government-owned company admitted that a further 17 of these cases were not even detected by the system.
These were one ‘short stop’ event, where a vehicle stopped for less than 12 seconds, and 16 ‘out of radar range’ – ‘valid stationary vehicle which falls outside of the coverage area of the radar. This includes maintenance access areas, and locations at the extremity of the SVD trial area’.
So the report's claim that '254 of the 294 ground truthing SVD events were successfully detected by the SVD radar system, equivalent to 86.4%' is untrue. Highways England admits that only 237 were detected.
The 40 indisputably 'missed detections' in the six days of the ground truth trial mean that around 2,400 stopped vehicle events could be missed every year in just this 13km stretch of the M25. By 2025, there could be 788 miles of smart motorway network in England.
Out of range?
A Highways England spokesperson said the CCTV field of vision extended beyond the range of the radar, which created a situation where vehicles out of the range of the radar could be seen on camera.
Highways England has argued that the out of radar range events should be excluded altogether from the totals. However, the spokesperson conceded these events were still inside the trial area.
The report does not make clear exactly where each out of range event took place and Highways England did not confirm.
A survey of the stretch of motorway suggests there are very few areas that a vehicle could have stopped where its occupants would not be at risk. Occupants stopped within the trial area on the motorway would be no less at risk in these out of range events.
It would only take half of the out of range events to be classified as missed to mean the system fails.
Breakdown of the calculations:
- Original calculation: 192 + 62 divided by 294 = 86.4% detection rate
- Highways England revised calculation: 192 + 45 divided by 277 = 85% detection rate
- Calculation with none of the missed out of range events excluded: 192 + 45 divided by 293 = 80.8% detection rate. Fail under the margin of error
- Calculation with half of the out of range events excluded: 192 + 45 divided by 285 = 83.1%. Fail under the margin of error.
Or missed detections?
The trial used a thorough site survey to establish ‘optimum locations’ and the radar system achieved 96.7% coverage of the carriageway, against Highways England’s benchmark of ‘90% of the carriageway where no hard shoulder exists’.
The remaining 3.3% was not out of range but in blind spots close to the radar devices, which were assumed to have an operational range of 250m in each direction – well within the capabilities of the radar devices used.
In fact, because of ‘line of sight issues’ in the trial area, an extra radar unit was used, so that 27 units covered a 13km stretch of motorway that could otherwise be covered by 26 units.
This suggests that ‘out of radar range’ missed detections resulted from and reflect the limitations of the SVD system being trialled.
Highways spoke to former roads and bridges engineer, Alan Hames, who is a chartered member of both the Institution of Civil Engineers and the Chartered Institution of Highways and Transportation. He has over 50 years of experience in roads and bridges design and construction on the strategic road network and two motorway reconstructions in 1962 and 1982.
He said: ‘The out of range events should definitely be counted as missed detections in my opinion. If these were in the trial area and the radar system did not pick them up they must be seen as “missed detections”.
'On the basis of this report, the SVD system is so inaccurate as not to be able to attain even the minimal target of 80%, which is in itself derisory. This leaves two in 10 breakdowns undetected and at continued risk.'
A Highways England spokesperson said: 'Analysis from the trial clearly indicates that over 85% of incidents were identified.
‘Not only that, but the system for detecting stopped vehicles has been developed even further since the 2016 trial. Stopped vehicle detection, along with other systems along our motorways, helps to keep motorists safe.’
Signs suppressed
Mr Hames also told Highways that the need to suppress alarms could itself see even more incidents being missed.
He said: ‘The different “allowable missed detections” have to be taken separately. Some of them are very important to a stopped vehicle detection system. For instance, manually suppressed alerts for maintenance vehicles are important detections because maintenance vehicles can cause incidents and not every stopped works vehicle will be known to Highways England control.
‘Others like congestion suppression are of less concern as risks, because these may not be stopped vehicle events, instead, they could just be slow-moving traffic.
‘Either way, the idea of suppressing alarms is a difficult one because it can lead to either actual dangerous incidents being missed with the suppression on, or the system being overloaded with false alarms when the suppression is off.’
Mr Hames pointed out that when the suppression events are taken away from the total, it could leave around 25% of breakdowns not accounted for.
'[On the basis of this report], the stopped vehicle detection system is therefore unacceptable and unfit for purpose as a safety solution in my opinion,' he said.
'This returns Highways England to the position where, as a minimum, refuge bays at 500 metres must be retrofitted, or more appropriately reinstatement of hard shoulders should be carried out without delay.'
Even if alerts are made, there is no guarantee that Highways England operators will be able to react in time. The 2016 study found 8.5% of events for which alerts were sent were 'auto cleared' while the vehicle was still there and before the operators had time to react.
Slow progress or lack of confidence?
In 2016 – the same year as the published report – Highways England told MPs that the SVD system ‘works’ and would be fitted to all new and existing all lane running (ALR) schemes – where the hard shoulder is removed. However it then made almost no progress for four years.
Last year, the Department for Transport’s Stocktake and Action Plan pledged to complete the retrofitting process within three years, after which Highways England sought bidders for the programme. In January, transport secretary Grant Shapps told MPs that retrofitting would be completed by the end of next year.
He said one of the reasons for Highways England’s failure to implement SVD earlier ‘was that the supplier is a single company’, but has also argued smart motorways should never have been rolled out without the SVD system.
The new contract, worth £18m, was awarded to UK firm Navtech, suppliers of the system tested in 2016 and 2018.
It is the first of two separate lots tendered by Highways England for SVD, worth a total £32m. A second lot, worth £14m, will cover SVD on new schemes.
A Navtech spokesperson told Highways: 'The system has a very sophisticated algorithm for detecting stopped vehicles. The system operates in both high and low traffic and works independently. Each ClearWay sensor scans 360°, four times a second and provides a high-resolution radar image of the road that it is covering.
‘The software then interprets the data and tracks all the vehicles on the road all the time and we use this information to make decisions about what is a stopped vehicle. Specifically, with stopped vehicle detection we are monitoring speed, how long they have been stopped for and comparing this against overall traffic flow on the road.'
