Exclusive: Errors in report cast doubt on smart motorway safety system

16/03/2021
Chris Ames and Dominic Browne

Highways England has admitted a string of errors were made in a report on the effectiveness of the stopped vehicle detection (SVD) system used to increase safety on smart motorways, casting doubt on whether the system managed to pass the company’s own evaluation.

The SVD system being retrofitted across the smart motorway network at a cost of at least £18m only sent alerts for 65% of incidents in trials.

This was well below a required detection rate of 80% - a relatively low benchmark that reflected the fact Highways England was only testing the system to see if it ‘would provide sufficient additional safety benefit’ to be worth fitting.

The system passed the evaluation in 2016 because some of the events for which alarms were not sent were put towards the successful detections total, which saw the success rate boosted to 86.4% - a pass mark even after the 3.9% margin of error was applied.

However, Highways England has admitted to Highways that a significant number of events were incorrectly classified as detected.

Subsequently, a highway expert told Highways that these events should have been classified as missed detections. If they had the system would have failed the evaluation.

Evidence suppressed?

Highways England has also told Highways that a second report, in 2018, tested the same system. In high traffic volumes, this report found an even lower headline detection rate of 82.4%. However, Highways England has refused to release this report on grounds of ‘commercial confidentiality’.

A Department for Transport spokesperson said: ‘Since taking office, the current Secretary of State has expressed his concerns over smart motorways. He has committed £500m to safety improvements and has recently pressed Highways England to further accelerate work in certain areas.’

The spokesperson declined to comment on Highways England’s refusal to publish the 2018 report.

Edmund King, AA president, who has long campaigned for safer ‘smart’ motorways told Highways: 'Any questions about the efficiency of stopped vehicle detection should be investigated as a matter of urgency. Smart motorways are meant to rely on technology but if that technology is now being questioned it might be time to go back to the drawing board.'

Detections, alerts and alarms

The SVD system uses a combination of radar, a server and an algorithm. When it detects a stopped vehicle, it sends an alert to the Highways England regional control centre, which creates an alarm for operators.

In the 2016 report, which focused on an early trial of the SVD system across 13km of the M25, the system detected and sent alerts for 192 stationary vehicle events in six days. A review of CCTV footage identified 294 events during the same period.

'Unknown reason' missed detections were in fact 11.2% of the total

Of the 102 events for which alerts were not sent: 40 were classified as missed detection for ‘unknown reason’ (33) or ‘blind spots’ (7), while 62 were classified as ‘allowable missed detection’ - although elsewhere in the report it labels the 'allowable missed detections' as 'SVD system detected'.

Highways England has claimed that 45 of these were in fact incorrectly labelled ‘allowable missed detections' and should have been termed 'SVD system detected', as they were detected but no alert was sent because the system was suppressed. The report does not provide evidence of this or describe a methodology by which it was established.

The government-owned company admitted that a further 17 of these cases were not even detected by the system.

These were one ‘short stop’ event, where a vehicle stopped for less than 12 seconds, and 16 ‘out of radar range’ – ‘valid stationary vehicle which falls outside of the coverage area of the radar. This includes maintenance access areas, and locations at the extremity of the SVD trial area’.

So the report's claim that '254 of the 294 ground truthing SVD events were successfully detected by the SVD radar system, equivalent to 86.4%' is untrue. Highways England admits that only 237 were detected.  

The 40 indisputably 'missed detections' in the six days of the ground truth trial mean that around 2,400 stopped vehicle events could be missed every year in just this 13km stretch of the M25. By 2025, there could be 788 miles of smart motorway network in England.

Out of range?

A Highways England spokesperson said the CCTV field of vision extended beyond the range of the radar, which created a situation where vehicles out of the range of the radar could be seen on camera.

Highways England has argued that the out of radar range events should be excluded altogether from the totals. However, the spokesperson conceded these events were still inside the trial area.

The report does not make clear exactly where each out of range event took place and Highways England did not confirm.

A survey of the stretch of motorway suggests there are very few areas that a vehicle could have stopped where its occupants would not be at risk. Occupants stopped within the trial area on the motorway would be no less at risk in these out of range events.   

It would only take half of the out of range events to be classified as missed to mean the system fails.

Breakdown of the calculations:

  • Original calculation: 192 + 62 divided by 294 = 86.4% detection rate
  • Highways England revised calculation: 192 + 45 divided by 277 = 85% detection rate
  • Calculation with none of the missed out of range events excluded: 192 + 45 divided by 293 = 80.8% detection rate. Fail under the margin of error
  • Calculation with half of the out of range events excluded: 192 + 45 divided by 285 = 83.1%. Fail under the margin of error.

Or missed detections?

The trial used a thorough site survey to establish ‘optimum locations’ and the radar system achieved 96.7% coverage of the carriageway, against Highways England’s benchmark of ‘90% of the carriageway where no hard shoulder exists’.

The remaining 3.3% was not out of range but in blind spots close to the radar devices, which were assumed to have an operational range of 250m in each direction – well within the capabilities of the radar devices used.

In fact, because of ‘line of sight issues’ in the trial area, an extra radar unit was used, so that 27 units covered a 13km stretch of motorway that could otherwise be covered by 26 units.

This suggests that ‘out of radar range’ missed detections resulted from and reflect the limitations of the SVD system being trialled.

Highways spoke to former roads and bridges engineer, Alan Hames, who is a chartered member of both the Institution of Civil Engineers and the Chartered Institution of Highways and Transportation. He has over 50 years of experience in roads and bridges design and construction on the strategic road network and two motorway reconstructions in 1962 and 1982.

He said: ‘The out of range events should definitely be counted as missed detections in my opinion. If these were in the trial area and the radar system did not pick them up they must be seen as “missed detections”.

'On the basis of this report, the SVD system is so inaccurate as not to be able to attain even the minimal target of 80%, which is in itself derisory. This leaves two in 10 breakdowns undetected and at continued risk.'

A Highways England spokesperson said: 'Analysis from the trial clearly indicates that over 85% of incidents were identified.

‘Not only that, but the system for detecting stopped vehicles has been developed even further since the 2016 trial. Stopped vehicle detection, along with other systems along our motorways, helps to keep motorists safe.’

Signs suppressed

Mr Hames also told Highways that the need to suppress alarms could itself see even more incidents being missed.

He said: ‘The different “allowable missed detections” have to be taken separately. Some of them are very important to a stopped vehicle detection system. For instance, manually suppressed alerts for maintenance vehicles are important detections because maintenance vehicles can cause incidents and not every stopped works vehicle will be known to Highways England control.

‘Others like congestion suppression are of less concern as risks, because these may not be stopped vehicle events, instead, they could just be slow-moving traffic.

‘Either way, the idea of suppressing alarms is a difficult one because it can lead to either actual dangerous incidents being missed with the suppression on, or the system being overloaded with false alarms when the suppression is off.’

Mr Hames pointed out that when the suppression events are taken away from the total, it could leave around 25% of breakdowns not accounted for.

'[On the basis of this report], the stopped vehicle detection system is therefore unacceptable and unfit for purpose as a safety solution in my opinion,' he said.

'This returns Highways England to the position where, as a minimum, refuge bays at 500 metres must be retrofitted, or more appropriately reinstatement of hard shoulders should be carried out without delay.'

Even if alerts are made, there is no guarantee that Highways England operators will be able to react in time. The 2016 study found 8.5% of events for which alerts were sent were 'auto cleared' while the vehicle was still there and before the operators had time to react.

Slow progress or lack of confidence?

In 2016 – the same year as the published report – Highways England told MPs that the SVD system ‘works’ and would be fitted to all new and existing all lane running (ALR) schemes – where the hard shoulder is removed. However it then made almost no progress for four years.

Last year, the Department for Transport’s Stocktake and Action Plan pledged to complete the retrofitting process within three years, after which Highways England sought bidders for the programme. In January, transport secretary Grant Shapps told MPs that retrofitting would be completed by the end of next year.

He said one of the reasons for Highways England’s failure to implement SVD earlier ‘was that the supplier is a single company’, but has also argued smart motorways should never have been rolled out without the SVD system.

The new contract, worth £18m, was awarded to UK firm Navtech, suppliers of the system tested in 2016 and 2018.

It is the first of two separate lots tendered by Highways England for SVD, worth a total £32m. A second lot, worth £14m, will cover SVD on new schemes.

A Navtech spokesperson told Highways: 'The system has a very sophisticated algorithm for detecting stopped vehicles. The system operates in both high and low traffic and works independently. Each ClearWay sensor scans 360°, four times a second and provides a high-resolution radar image of the road that it is covering.

‘The software then interprets the data and tracks all the vehicles on the road all the time and we use this information to make decisions about what is a stopped vehicle. Specifically, with stopped vehicle detection we are monitoring speed, how long they have been stopped for and comparing this against overall traffic flow on the road.'

Latest Issue

latest magazine issue

INSIDE:

  • COP26: The sector reacts
  • Road pricing: Putting the theory into practice
View the latest issue


Highways jobs

Highways Projects Manager

Up to £38,326.08 p.a.
We are looking for an experienced, skilled and competent Highways Projects Manager to help deliver a wide range of... Broxbourne, Hertfordshire
Recruiter: Broxbourne Borough Council

Assistant Director of Environment and Regulatory Services

circa £84,000
As Assistant Director of Environment and Regulatory Services, you will be responsible for core ‘front-line’ place-based services, including... Bolton, Greater Manchester
Recruiter: Bolton Metropolitan Borough Council

Heads of Service

£57,636 - £58,509 pa
It’s an exciting time to join us here in West Lancs as we are looking for 3 inspirational leaders who have the energy and drive to... Lancashire
Recruiter: West Lancashire Borough Council

Senior Engineer- Highways

£23,080- £30,451
Our Highways Service is growing to reflect the increase in investment. Huddersfield, West Yorkshire
Recruiter: Kirklees Metropolitan Council

Environmental Data Officer

£26,592
In this role you will collect, maintain and manage high quality energy and environmental data, and providing... Camden, London (Greater)
Recruiter: Camden London Borough Council

Senior Engineer - Highways Asset Maintenance

£39,854 to £43,741 (Pay Award Pending)
There’s never been a more exciting time to join Doncaster Council. Doncaster, South Yorkshire
Recruiter: Doncaster Council

Director of Environmental & Community Services

Up to £90,096 per annum
We seek an energetic and committed Director who has the right motivations and values; someone who is... Long Eaton, Nottingham
Recruiter: Erewash Borough Council

Transport Infrastructure Commissioning Officer

£30,451 to £34,728 per annum
We have a great opportunity in Somerset to join our Highways and Transport Service. Taunton, Somerset
Recruiter: Somerset County Council

Team Manager Band 3 – Environment Strategy & Change

£46,845 - £48,849 per annum
This is the perfect time to join St Helens and become part of our exhilarating journey and help build our legacy for the future. St Helens, Merseyside
Recruiter: St. Helens Metropolitan Borough Council

Noise and Nuisance Officer

£32,301 - £35,949 per annum
To be successful in the Noise and Nuisance Team you will need a genuine commitment to... Kensington and Chelsea, London (Greater)
Recruiter: The Royal Borough of Kensington & Chelsea Council

Team Manager Band 3 - Fleet, Depot & Security

£46,845 - £48,849 per annum
This is the perfect time to join St Helens and become part of our exhilarating journey and help build our legacy for the future. St Helens, Merseyside
Recruiter: St. Helens Metropolitan Borough Council

Minibus Driver

Hourly rate
North Yorkshire County Council
Recruiter: North Yorkshire County Council

Minibus Driver

Hourly rate
North Yorkshire County Council
Recruiter: North Yorkshire County Council

Minibus Driver

Hourly rate
North Yorkshire County Council
Recruiter: North Yorkshire County Council

Minibus Driver

Hourly rate
North Yorkshire County Council
Recruiter: North Yorkshire County Council

Minibus Driver

Hourly rate
North Yorkshire County Council
Recruiter: North Yorkshire County Council

Asset Manager

£42,821 - £46,845 per annum
The Highway Asset Manager is responsible for the strategic management of all the council’s highway assets... Blackburn, Lancashire
Recruiter: Blackburn with Darwen Borough Council

Digital Evidence Officer

£25.137 - £26.520
The Digital Evidence (CCTV) Team are responsible for improving community safety outcomes throughout the Royal Borough of Greenwich by... Greenwich, London (Greater)
Recruiter: Royal Borough of Greenwich

Guaranteed Relief Cleaners North and South Kirklees Area

£9.25 per hour
Some of the key features and benefits of being part of the Kirklees Cleaning Service include... Kirklees, West Yorkshire
Recruiter: Kirklees Metropolitan Council

Group Engineer - Safety Design

£40,876 - £45,859
We are looking for an experienced engineer to join our Highways Safety Design team. Kirklees, West Yorkshire
Recruiter: Kirklees Metropolitan Council

Highways Presents

 

Highways on Fridays

Register now!

Latest Video