Stopped vehicle detection hits target but 85% of testing data unusable

18/12/2023 | DOMINIC BROWNE

National Highways has made significant progress on smart motorway safety according to a new report from the roads monitor however concerns remain about the strength of the underlying testing data.

Most notably, the national roads operator hit all its key targets for stopped vehicle detection (SVD) - meaning the complex system can officially become 'business as usual' now.

While software fixes were cited as the key reason for the progress, the improvements also followed a change in testing methodology, which saw about 85% of the stopped vehicle alerts thrown out of the 'ground truthing' process as being 'unverifiable'.

The 'ground truthing' process uses recorded CCTV footage to check alerts and determine the proportion of stopped vehicles where an alert was correctly raised by the SVD system, and the time taken for an alert to be generated.

Changes to the methodology meant that any incident where the CCTV footage was incomplete - due to the way a camera might have panned, for instance - was counted as 'unverifiable'.

As a result, the ORR has asked National Highways to review how it 'collects and analyses information about unverified alerts'.

Software upgrade

The latest ORR strategic road network safety report states that National Highways 'identified a series of potential software fixes that included improvements to the performance of radar equipment and filtering out multiple alerts created by vehicles stopped in queuing traffic'.

The national roads operator implemented these software fixes by June 2023 and subsequently made a huge leap forward from its results in the ORR's 2022 safety report, which found National Highways to be failing in all the key SVD areas.

The latest data from ground truthing tests this year shows that the performance targets for the three metrics were being met:

  • Detection rates (i.e., the proportion of stopped vehicles correctly identified by SVD): 89% (target >80%)
  • False detection rates (i.e., the proportion of SVD alerts that are reported incorrectly because the alert does not relate to a true stopped vehicle): 6% (target <15%)
  • Average time to detect stopped vehicles (i.e., the elapsed time between a vehicle stopping and an SVD alert being generated): 12.2 seconds (target <20 seconds)

These three targets were also met in all six National Highways operational regions, except in one case. In the Yorkshire and North East region, the performance for detection rates was 76% though it had significantly fewer SVD alerts compared to the other regions.

National Highways also reported to the ORR that 92% of stopped vehicles were being detected within 20 seconds.

Methodology change

However, ORR notes that a key impact of methodological changes to the testing process was that '85% of all stopped vehicle alerts were classed as unverified and therefore excluded from the performance calculation'.

The methodological changes were to:

(a) Exclude any alerts beyond defined SVD coverage areas. This means any alerts outside defined coverage areas, for example, short segments of the hard shoulder, were excluded from the testing.

(b) Exclude ‘unverified’ alerts: Any alerts categorised as ‘unverified’ were excluded from the analysis. An alert is classified as unverified when, during the ground truthing testing process, it has not been possible to use the recorded CCTV footage to verify all the performance metrics which are measured in a ‘stopped vehicle event’. For example, if recorded CCTV footage does not capture the start of the event because it was pointed in a different direction, this would be excluded from the calculation as it is not possible to measure the time taken to detect the stopped vehicle.

The ORR report adds: 'There are different levels of certainty associated with the various scenarios but, under the new methodology, National Highways only uses data where it has 100% certainty about how well the system has performed in detecting a stopped vehicle.'

This means it only has 100% certainty around 15% of the stopped vehicle alerts that could have initially come under the ground truthing exercise.

Across the 21 all lane running (ALR) smart motorway schemes - those without a hard shoulder - there were 3,850 alerts. Of these, 3,286 were classified as unverified, which left a total sample size of 564 used for testing SVD detection rate and time to detect.

Of the tested events, 80% occurred in an emergency area and the remaining 20% were live lane stops.

The software improvements were first tested on three 'spearhead' smart motorway schemes before being rolled out to the full 21 ALR schemes.

The ORR said that it had challenged National Highways to demonstrate the benefits of the software improvements compared to the impact of the change in methodology and the company 'did this by sharing details from testing it had undertaken on the three spearhead schemes to assess the effectiveness of the software fixes'.

It applied a consistent methodology to ground truthing data from before the software upgrade (in 2022) and after the upgrade (in 2023) and repeated this for both the old and new methodologies, the ORR states.

'In both cases, the data showed improved performance across all three metrics of detection rate, time to detect and false detection rates in 2023 (after the software fixes).’

The data from this exercise was not included in the ORR report, however. Highways has asked for the data to be provided. 

Attendance and availability 

Four of National Highways’ six operational regions achieved annual average attendance times for stopped vehicles of under 10 minutes in the year from September 2022 to August 2023.

The two regions that did not achieve annual average response times of 10 minutes were: the East region (10 minutes 22 seconds); and the South East (10 minutes 55 seconds).

In 2022, National Highways had also been failing a wider technology availability target of 95%.

After committing to a £105m investment plan to improve operational technology (OpTech) on ALR motorways, National Highways aimed for 97% average availability for the four asset types by the end of this road period (2025).

The four key elements of operational technology (OpTech) are:

  • the Motorway Incident Detection and Automatic Signalling (MIDAS); which had a score of 97% availability for the year from September 2022 to August 2023,
  • variable message signs, which had a score of 91%; 
  • signals (speed and lane control signals) (96%)
  • and CCTV Cameras (96%).

The availability score in September 2023 across these four assets was 95.6%.

Emergency rest areas

National Highways appears to be far behind on its commitment to roll out over 150 additional emergency areas on ALR smart motorways by the end of RP2 (2025), under a £390m allocated budget.

By September 2023, National Highways had only completed work on 13 emergency areas, on the M1 and M6. The ORR said the next phase of work is expected to deliver a further 40 on the M1 and M25.

Final results 

Shortly before the ORR report was released, National Highways released its own third annual progress report following the Government's smart motorway stocktake.

It found that 'the latest safety data continues to show that overall, all three types of smart motorway [all lane running, dynamic hard shoulder, controlled] are better than conventional motorways for those safety metrics which consider the most significant impacts, such as deaths or serious injuries'.

However, this is based on using a five-year rolling average to 'smooth some of the peaks and troughs caused when safety across all roads in individual years is very different to its usual levels'.

The effect also masks the impact of dramatically increasing traffic levels on smart motorways and the knock-on impact on safety figures.

Due to new schemes opening and changes in driving patterns, all lane running schemes increased from 14.3hmvm (hundred million vehicle miles) in 2015 to 67.7hmvm in 2021 - the last year of smart motorway data provided in the National Highways report.

In the same period, conventional motorways decreased from 482.3hmvm to 369.3hmvm.

National Highways reports that between September 2022 and August 2023, live lane incident numbers on ALR smart motorways increased by 38%, from 1,308 to 1,808 and adds 'this is likely to be related to increased traffic, as four new ALR schemes opened in this period, bringing the total to 21'.

For 2021, the Department for Transport statistics show ALR schemes had higher personal injury collision rates per hmvm than conventional motorways (6.19 to 5.10), higher killed and seriously injured rates per hmvm (1.66 to 1.38) and higher fatal and weighted injuries per hmvm (0.39 to 0.36).

Highways InProfile

latest magazine issue
Highways jobs

Director of Transport

£112,924
East Midlands Combined County Authority (EMCCA) is on a bold journey. East Midlands Region
Recruiter: East Midlands Combined County Authority

Ferry Host - WMF3049i

£25,584 - £25,992
Ferry Host - WMF3049i Windermere, Cumbria
Recruiter: Westmorland and Furness Council

Director of Streetscene

£94,972 - £108,767
We continue to have big ambitions for our borough Fareham, Hampshire
Recruiter: Fareham Borough Council

Principal Engineer - Urban Traffic Control (UTC)

£44,711 - £49,764
Liverpool is home to vibrant, energetic and engaged communities. Liverpool, Merseyside
Recruiter: Liverpool City Council

Roadworker/Driver

Grade 6 £25,183 - £25,638 per annum (Pay Award Pending)
We’re looking for Roadworker/trainee LGV drivers to be part of our team Derbyshire
Recruiter: Derbyshire County Council

Project Chargehand (2 Posts)

Grade 9 £31,344 - £33,253 per annum (Pay Award Pending)
We’re looking for experienced Project Chargehands to be part of the Highways Construction Services team Derbyshire
Recruiter: Derbyshire County Council

Lawyer 2- Planning and Highways

£49,764 - £54,181
We are looking for a fresh thinking and enthusiastic Planning and Highways Lawyer Hereford, Herefordshire
Recruiter: Herefordshire Council

Development Engineer

£35,967 - £41,268 per annum
Are you ready to play a key role in shaping and safeguarding the future of our highways and public spaces? Bexley (City/Town), London (Greater)
Recruiter: London Borough of Bexley

Professional Services Partnering Manager

Grade 13 £47,653 - £51,296 per annum (pay award pending)
We are seeking a dynamic and experienced Professional Services Partnering Manager to lead the strategic management Darley Dale, Matlock
Recruiter: Derbyshire County Council

Technical Supervision Manager

Grade 13 £47,653 - £51,296 per annum (pay award pending)
We are looking for an experienced and driven Technical Supervision Manager to lead our technical supervision teams within the Highways Service. Derbyshire
Recruiter: Derbyshire County Council

Street Lighting Electrician (CDC)

Grade 8, £32,662 - £36,239 (pay award pending)
Highways Operations is a growing service area within the Place directorate. Doncaster, South Yorkshire
Recruiter: City Of Doncaster Council

Highways Assistant Engineer WMF2133e

£39,513 to £40,476
We have an exciting opportunity for the right person to work within the challenging and ever-changing landscape. Penrith, Cumbria
Recruiter: Westmorland and Furness Council

Highways Inspector/NRSWA

£36,345 - £40,344
We are looking for an organised and self-motivated individual to join our Streetworks Team based in Hackney Central. Hackney, London (Greater)
Recruiter: London Borough of Hackney Council

Structures Commissioner

Grade 14 £53,166 - £59,080 per annum (Pay Award Pending)
We are seeking an experienced and strategic leader to join our team as the Structures Commissioner. Derbyshire
Recruiter: Derbyshire County Council

Team Leader Network Co-ordination

£51,429 - £58,140
As Team Leader for Network Coordination, you’ll manage a team that oversees all works and events on our public highways. Kensington and Chelsea, London (Greater)
Recruiter: The Royal Borough of Kensington & Chelsea Council

PRINCIPAL TRANSPORT PLANNING OFFICER

£43693 - £46731
We have an exciting opportunity for an experienced professional to join our team in Hull as a Principal Transport Planning Officer. Kingston upon Hull, East Riding of Yorkshire
Recruiter: Hull City Council

HIGHWAYS AND TRAFFIC ENGINEER - 4 POSTS

£43693 - £46731
We have several exciting opportunities for suitably qualified Highways Civil Engineers and Traffic Engineers Wilson Centre, 1st floor, Hull HU1 2AG
Recruiter: Hull City Council

Principal Transport Planner

£45,718 to £49,764
Do you have considerable experience in the transport strategy and policy, and are you looking for your next career development opportunity? Sheffield, South Yorkshire
Recruiter: Sheffield City Council

Project Lead - Bridges and Structures WMF3037e

£45,718 to £46,731
We are currently seeking to recruit a Project Lead to join our Bridges and Structures team Kendal, Cumbria
Recruiter: Westmorland and Furness Council
SUBSCRIBE NOW

Latest Video

Subscribe to Highways today to ensure you keep your finger on the pulse of everything happening in the UK road network throughout the year.

SUBSCRIBE NOW