Exclusive: Errors in report cast doubt on smart motorway safety system

16/03/2021
Chris Ames and Dominic Browne

Highways England has admitted a string of errors were made in a report on the effectiveness of the stopped vehicle detection (SVD) system used to increase safety on smart motorways, casting doubt on whether the system managed to pass the company’s own evaluation.

The SVD system being retrofitted across the smart motorway network at a cost of at least £18m only sent alerts for 65% of incidents in trials.

This was well below a required detection rate of 80% - a relatively low benchmark that reflected the fact Highways England was only testing the system to see if it ‘would provide sufficient additional safety benefit’ to be worth fitting.

The system passed the evaluation in 2016 because some of the events for which alarms were not sent were put towards the successful detections total, which saw the success rate boosted to 86.4% - a pass mark even after the 3.9% margin of error was applied.

However, Highways England has admitted to Highways that a significant number of events were incorrectly classified as detected.

Subsequently, a highway expert told Highways that these events should have been classified as missed detections. If they had the system would have failed the evaluation.

Evidence suppressed?

Highways England has also told Highways that a second report, in 2018, tested the same system. In high traffic volumes, this report found an even lower headline detection rate of 82.4%. However, Highways England has refused to release this report on grounds of ‘commercial confidentiality’.

A Department for Transport spokesperson said: ‘Since taking office, the current Secretary of State has expressed his concerns over smart motorways. He has committed £500m to safety improvements and has recently pressed Highways England to further accelerate work in certain areas.’

The spokesperson declined to comment on Highways England’s refusal to publish the 2018 report.

Edmund King, AA president, who has long campaigned for safer ‘smart’ motorways told Highways: 'Any questions about the efficiency of stopped vehicle detection should be investigated as a matter of urgency. Smart motorways are meant to rely on technology but if that technology is now being questioned it might be time to go back to the drawing board.'

Detections, alerts and alarms

The SVD system uses a combination of radar, a server and an algorithm. When it detects a stopped vehicle, it sends an alert to the Highways England regional control centre, which creates an alarm for operators.

In the 2016 report, which focused on an early trial of the SVD system across 13km of the M25, the system detected and sent alerts for 192 stationary vehicle events in six days. A review of CCTV footage identified 294 events during the same period.

'Unknown reason' missed detections were in fact 11.2% of the total

Of the 102 events for which alerts were not sent: 40 were classified as missed detection for ‘unknown reason’ (33) or ‘blind spots’ (7), while 62 were classified as ‘allowable missed detection’ - although elsewhere in the report it labels the 'allowable missed detections' as 'SVD system detected'.

Highways England has claimed that 45 of these were in fact incorrectly labelled ‘allowable missed detections' and should have been termed 'SVD system detected', as they were detected but no alert was sent because the system was suppressed. The report does not provide evidence of this or describe a methodology by which it was established.

The government-owned company admitted that a further 17 of these cases were not even detected by the system.

These were one ‘short stop’ event, where a vehicle stopped for less than 12 seconds, and 16 ‘out of radar range’ – ‘valid stationary vehicle which falls outside of the coverage area of the radar. This includes maintenance access areas, and locations at the extremity of the SVD trial area’.

So the report's claim that '254 of the 294 ground truthing SVD events were successfully detected by the SVD radar system, equivalent to 86.4%' is untrue. Highways England admits that only 237 were detected.  

The 40 indisputably 'missed detections' in the six days of the ground truth trial mean that around 2,400 stopped vehicle events could be missed every year in just this 13km stretch of the M25. By 2025, there could be 788 miles of smart motorway network in England.

Out of range?

A Highways England spokesperson said the CCTV field of vision extended beyond the range of the radar, which created a situation where vehicles out of the range of the radar could be seen on camera.

Highways England has argued that the out of radar range events should be excluded altogether from the totals. However, the spokesperson conceded these events were still inside the trial area.

The report does not make clear exactly where each out of range event took place and Highways England did not confirm.

A survey of the stretch of motorway suggests there are very few areas that a vehicle could have stopped where its occupants would not be at risk. Occupants stopped within the trial area on the motorway would be no less at risk in these out of range events.   

It would only take half of the out of range events to be classified as missed to mean the system fails.

Breakdown of the calculations:

  • Original calculation: 192 + 62 divided by 294 = 86.4% detection rate
  • Highways England revised calculation: 192 + 45 divided by 277 = 85% detection rate
  • Calculation with none of the missed out of range events excluded: 192 + 45 divided by 293 = 80.8% detection rate. Fail under the margin of error
  • Calculation with half of the out of range events excluded: 192 + 45 divided by 285 = 83.1%. Fail under the margin of error.

Or missed detections?

The trial used a thorough site survey to establish ‘optimum locations’ and the radar system achieved 96.7% coverage of the carriageway, against Highways England’s benchmark of ‘90% of the carriageway where no hard shoulder exists’.

The remaining 3.3% was not out of range but in blind spots close to the radar devices, which were assumed to have an operational range of 250m in each direction – well within the capabilities of the radar devices used.

In fact, because of ‘line of sight issues’ in the trial area, an extra radar unit was used, so that 27 units covered a 13km stretch of motorway that could otherwise be covered by 26 units.

This suggests that ‘out of radar range’ missed detections resulted from and reflect the limitations of the SVD system being trialled.

Highways spoke to former roads and bridges engineer, Alan Hames, who is a chartered member of both the Institution of Civil Engineers and the Chartered Institution of Highways and Transportation. He has over 50 years of experience in roads and bridges design and construction on the strategic road network and two motorway reconstructions in 1962 and 1982.

He said: ‘The out of range events should definitely be counted as missed detections in my opinion. If these were in the trial area and the radar system did not pick them up they must be seen as “missed detections”.

'On the basis of this report, the SVD system is so inaccurate as not to be able to attain even the minimal target of 80%, which is in itself derisory. This leaves two in 10 breakdowns undetected and at continued risk.'

A Highways England spokesperson said: 'Analysis from the trial clearly indicates that over 85% of incidents were identified.

‘Not only that, but the system for detecting stopped vehicles has been developed even further since the 2016 trial. Stopped vehicle detection, along with other systems along our motorways, helps to keep motorists safe.’

Signs suppressed

Mr Hames also told Highways that the need to suppress alarms could itself see even more incidents being missed.

He said: ‘The different “allowable missed detections” have to be taken separately. Some of them are very important to a stopped vehicle detection system. For instance, manually suppressed alerts for maintenance vehicles are important detections because maintenance vehicles can cause incidents and not every stopped works vehicle will be known to Highways England control.

‘Others like congestion suppression are of less concern as risks, because these may not be stopped vehicle events, instead, they could just be slow-moving traffic.

‘Either way, the idea of suppressing alarms is a difficult one because it can lead to either actual dangerous incidents being missed with the suppression on, or the system being overloaded with false alarms when the suppression is off.’

Mr Hames pointed out that when the suppression events are taken away from the total, it could leave around 25% of breakdowns not accounted for.

'[On the basis of this report], the stopped vehicle detection system is therefore unacceptable and unfit for purpose as a safety solution in my opinion,' he said.

'This returns Highways England to the position where, as a minimum, refuge bays at 500 metres must be retrofitted, or more appropriately reinstatement of hard shoulders should be carried out without delay.'

Even if alerts are made, there is no guarantee that Highways England operators will be able to react in time. The 2016 study found 8.5% of events for which alerts were sent were 'auto cleared' while the vehicle was still there and before the operators had time to react.

Slow progress or lack of confidence?

In 2016 – the same year as the published report – Highways England told MPs that the SVD system ‘works’ and would be fitted to all new and existing all lane running (ALR) schemes – where the hard shoulder is removed. However it then made almost no progress for four years.

Last year, the Department for Transport’s Stocktake and Action Plan pledged to complete the retrofitting process within three years, after which Highways England sought bidders for the programme. In January, transport secretary Grant Shapps told MPs that retrofitting would be completed by the end of next year.

He said one of the reasons for Highways England’s failure to implement SVD earlier ‘was that the supplier is a single company’, but has also argued smart motorways should never have been rolled out without the SVD system.

The new contract, worth £18m, was awarded to UK firm Navtech, suppliers of the system tested in 2016 and 2018.

It is the first of two separate lots tendered by Highways England for SVD, worth a total £32m. A second lot, worth £14m, will cover SVD on new schemes.

A Navtech spokesperson told Highways: 'The system has a very sophisticated algorithm for detecting stopped vehicles. The system operates in both high and low traffic and works independently. Each ClearWay sensor scans 360°, four times a second and provides a high-resolution radar image of the road that it is covering.

‘The software then interprets the data and tracks all the vehicles on the road all the time and we use this information to make decisions about what is a stopped vehicle. Specifically, with stopped vehicle detection we are monitoring speed, how long they have been stopped for and comparing this against overall traffic flow on the road.'

Latest Issue

latest magazine issue

Inside

  • Smart motorways' BCR problem
  • New lighting decision support tool

 

 

View the latest issue


Highways jobs

Senior Project Manager

Up to £50,823 per annum
The role of this post is to oversee the project management, development and implementation of the County Council’s Major Transport Projects.. Leicestershire
Recruiter: Leicestershire County Council

Programme and Change Delivery Manager (Spatial Planning and Urban Design)

£43,833 - £53,115 Depending on skills and experience
Are you a change delivery manager with expertise in England’s local government spatial planning and urban design sector? Wandsworth, London (Greater)
Recruiter: London Borough of Richmond upon Thames and London Borough of Wandsworth

Bridges Manager – Highways and Transportation

£49,320 to £57,376
You will be responsible for... Northallerton, North Yorkshire
Recruiter: North Yorkshire County Council

Head of Major Projects and Infrastructure – Highways and Transportation

£59,526 to £67,815 plus 10% recruitment payment
This role will be pivotal to the effective delivery of our Highways and Transportation service, therefore requires an... Northallerton, North Yorkshire
Recruiter: North Yorkshire County Council

Principal Travel Plan Officer

£34,728 to £38,890 per annum
We have a fantastic opening within our team for a Principal Travel Plan Officer. Taunton, Somerset
Recruiter: Somerset County Council

Director of Economy & Environment

£96,414 to £106,650 per annum
You will play a leading role in driving forward the county’s ambitious growth programme. Gloucester, Gloucestershire
Recruiter: Gloucestershire County Council

Director of Transport & Highways

£96,414 to £106,650 per annum
You will play a leading role in driving forward Gloucestershire’s ambitious, multi-million pound transport and highways programme. Gloucester, Gloucestershire
Recruiter: Gloucestershire County Council

Board and Partnership Board Chair

Up to £60,000 per annum
This is unique opportunity to lead the Northern agenda and play a pivotal role in creating new opportunities for 15 million people in the North. Manchester/Leeds (Flexible)
Recruiter: Transport for the North

Waste Strategy Manager

£47,672 per annum
We are looking to recruit a waste strategy specialist to add the necessary technical expertise to our team. North Tyneside, Tyne and Wear
Recruiter: North Tyneside Metropolitan Borough Council

Highways Officers

£26,999 to £29,636
Are you someone who is looking to progress their career with one of the most highly regarded highway services in the country? Selby, North Yorkshire
Recruiter: North Yorkshire County Council

Highways Operative

£19,312 to £21,748
We are looking for Highways Operatives to work as part of our operations teams here at Kirklees Council. Kirklees, West Yorkshire
Recruiter: Kirklees Metropolitan Council

Senior Environmental Health Officer

£44,370 - £50,964 per annum
As a Senior Officer in the Food Safety & Infectious Disease team, you will be one of the operational leads on... Kensington and Chelsea, London (Greater)
Recruiter: The Royal Borough of Kensington & Chelsea Council

Permit Officer

£24,982 - £27,041
In October 2016 Cambridgeshire County Council launched it Permit Scheme for Road Works and Street Works. Huntingdon, Cambridgeshire
Recruiter: Cambridgeshire County Council

Traffic Systems Engineer

Competitive Salary
This is a full time role, with a minimum of 37 hours a week for a 6 month interim period, with the potential of the role going permanent. York, North Yorkshire
Recruiter: City of York Council

Transport Compliance Officer

£27,741 - £30,451 per annum
An exciting opportunity has arisen for two Transport Compliance Officers to join our Transport Services team at... Kirklees, West Yorkshire
Recruiter: Kirklees Metropolitan Council

Client Team Leader - Highway Maintenance

£40,876 - £42,821 per annum
Are you looking for a challenge that widens your experience, adds something unique to your CV and helps to make Warwickshire the best it can be?  Warwickshire
Recruiter: Warwickshire County Council

Head of Waste & Parking

£62,235–£68,615 pa
Are you passionate about waste recycling and the bigger picture of sustainability and tackling the climate emergency? Horsham, West Sussex
Recruiter: Horsham District Council

Project Engineer x2

Grade H £35,745 - £43,857
We’re looking for experienced civil engineers to join our well-regarded design and implementation team. Havant, Hampshire
Recruiter: Havant Borough Council

Environment Locality Officer

£27,741 - £29,577 per annum
The officer will play a key role in collaborative working with other internal & external stakeholders specifically... Telford, Shropshire
Recruiter: Telford & Wrekin Council

Senior Principal Structures Engineer

£39,880 - £42,821 per annum
We are looking for a dynamic structurally competent and relevantly experienced officer, to deliver the structural requirements and disciplines on... Bristol
Recruiter: Bristol City Council

Highways Presents

Highways on Fridays

Register now!

Latest Video