Former director at the Department for Transport and current director of the RAC Foundation, Steve Gooding, picks up on a recent Highways investigation into smart motorways to ask important questions about models, predictions and above all what we want.
Like many others I’ve been following the furore – is that too strong a word? – over two recent POPE reports from National Highways: one for the M1 junctions 10 to 13 hard shoulder running & junction improvements scheme; the other for the M6 Junction 8-10a Smart (Managed) Motorway scheme.
For neither scheme was the scale of monetised net benefit that had been sought forthcoming by year five.
More worryingly, nor is it looking likely to be forthcoming over the longer period covered by the original appraisal based on the five-year trajectory.
Let me start by being up front about the fact that I was in the Department for Transport at the time the original investment cases were presented to ministers seeking the go-ahead for these schemes. I wanted, I hoped, I expected the POPE analyses to prove these schemes to be positive.
But there are two phrases that leap off the page to me from the two reports.
From the M1: ‘In this case, the monetisation of journey time benefit is not a good measure of value for money and the qualitative evidence presented in the evaluation is considered a more robust measure.’
And from the M6: ‘Journey time savings of 10% have been achieved in the southbound direction during the AM Peak; this represents a reduction of approximately 147 seconds.’
The DfT, in my view, deserves great credit for being one of, if not the, trailblazer for bringing economic rigour to decision-making. The thing is, though, that there are so many variables involved that modelling the outcome of any multi-million-pound transport scheme is, in reality, something of a guessing game. The only thing any of us can say with confidence about the future is that it hasn’t happened yet.
So, there is uncertainty about costs – the M6 scheme had costs around 20% higher than predicted, the M1 scheme had costs 20% lower than those forecast.
The weighting given to various types of benefit is critical. And here I can confess my long-standing anxiety about journey time savings: 147 seconds is, to most of us, just over two minutes. Huge weight rests on this proxy measure of value. I’m not saying that it is wrong; I am concerned that our ability to monetise it had the potential to give it undue prominence in the original scheme appraisal when the layman’s perfectly reasonable question might be: ‘Why on earth do you want to do this?’
In practice that question is the backbone of what’s called the ‘strategic case’. None of us thinks that congestion is a good thing. Traffic jams are bad from every angle.
But is the case for accommodating more road trips, more reliably positive or not?
All those people making additional trips on the M1 were doing so for a reason. Many were doing so on a safer route than they otherwise would have chosen. But more trips, in an internal-combustion engined world, is potentially a bad thing. And that’s long before we get to the ‘yay’ or ‘nay’ about dynamic hard shoulders or all-lane running.
I’m left thinking that the most important thing – and the really important thing that Highways has brought to the fore – is that we should look at what we forecast to happen, and we should look at what actually happened, and we should ask ourselves what we need to learn as a result. POPE studies need to be open data documents, and they should form the basis for an open and honest debate about where we go from here.
One of the most thought-provoking reports I have seen in recent years was the CPRE’s 2017 study: The Impact of Road Projects in England. The authors point to numerous instances where evidence for the delivery of anticipated scheme benefits was somewhere between lacking and non-existent.
I was, and I still am, a sceptic of the argument that all road building does is generate more traffic, with the presumption that this is a bad thing. Those extra trips surely reveal a desire by people to make a trip and who am I to judge whether their desire should not have been accommodated?
Benefit:cost analysis is probably the best way we have of deciding between investment options to achieve an outcome. It’s far from perfect in supporting the case for doing something, which is not to say that we shouldn’t be trying, just that we are trying to do something hard and riddled with uncertainty.
We can’t blame economics for delivering mathematical conclusions that are out of whack with what we – as a society – want, if we don’t then calibrate our mathematical inputs to the calculation correctly. Which forces us to ask some very big questions about how well we value carbon and air quality emissions, or even the real impact of unpredictable delay (which I think is woefully undervalued at present).
As a highly qualified chum of mine once observed: ‘Modelling is easy; it’s getting the input assumptions right that’s hard.’
