I first came across RCF (Reference Class Forecasting) at Oxford in 2010. It would be another eight years before I actually saw it applied in the real world. It proved to be extremely effective, but as with any technique, how it was applied made all the difference, and we now use it extensively in all client scenarios, whether it’s an outset, onset or reset situation.
Quantitative vs. qualitative Reference Class Forecasting
RCF, as originally explained to me, relies on having a normalised dataset of previous programmes that allows for analysis of overspends so that any projections for the current programme can be compared to previous relevant examples. In other words, if a particular type of programme overran/overspent by X% previously, the assumption is that it will likely do so again, so adjust budgets and timeframes accordingly. The logic presupposes that the data is properly normalised and the examples are truly relevant, so that the comparisons are genuinely like-for-like. However, even if that is the case, the quantitative use of RCF doesn’t make for a better programme: It simply means that the budget and timeframe are adjusted to be more realistic. But there is also a different way to look at it, which is that RCF misapplied bakes in old mistakes and resulting overruns/overspends.
The Commonwealth Games 2022: a masterclass in programme delivery
What I witnessed in 2018 was a much smarter application of RCF qualitatively. The context was the Commonwealth Games 2022 (CG2022), which in essence was a programme coordinated between a UK Government Department and a disparate collection of other organisations, ranging from the Commonwealth Games Federation to the local authority in the host city, so the collaboration challenges far exceed those found in a typical corporate programme. Degree of difficulty? Probably a 12 out of 10.
CG2022 should have been an absolute disaster. Normally, a Commonwealth Games is seven years in the making. Originally, Durban was the successful bidder, but when the bid collapsed and Birmingham agreed to take it on, there were only four and a half years left to deliver something that normally takes seven years. Then Birmingham experienced its wettest winter on record – never helpful in a construction programme. Then the programme started to experience all of the supply-chain disruptions of Brexit, affecting everything from acquisition of overseas materials to labour markets. And then, just when seemingly the challenges couldn’t get any worse, along came a once-a-century pandemic; Covid.
Any one of the above should have been the final nail in the coffin, but the cumulative effect of all four? Despite these challenges, CG2022 came together on time and in fact everything was sufficiently ready that there was even time enough to run two test events in advance of the real thing. So, how did that happen and what role did RCF play in beating the rather daunting odds?
Anyone who thinks that all public sector programmes are HS2 ‘mini-me’s, and that the civil service can’t run a programme well, clearly hasn’t seen the HM Government Department, DCMS (formerly Digital, Culture, Media and Sport) in action. During my tenure I witnessed them think their way through a lot of the same problems that prove fatal to private sector programmes, and above all, their willingness to contemplate new ideas and approaches is exceptional – and this was a major factor in the successful outcome of CG2022.
Applying RCF to the aquatic centre build
Whether it’s a Commonwealth Games or an Olympics, one of the toughest aspects is the aquatic centre. It’s technically complex, a big investment and unlike some aspects of staging a Games, there is no real workaround. It simply has to be ready on time. So RCF was used to assess many of the previous builds of aquatic centres, specifically looking for what went wrong: What did cause the budget to be exceeded and prompted a lot of costly late-night and last-minute fixes across everything from valves and heat exchangers to seating and sealants.
So far, so good: Sounds a bit like a typical lessons learned exercise? Not really, because of two factors:
- It was based on multiple examples, so if it was lessons learned, it was lessons learned over many programmes all with their own permutations.
- It went beyond what the programme got wrong and devised countermeasures to each and every challenge.
Beyond 'lessons learned': a clean-sheet approach to planning
The second factor is hugely important. When a lessons learned session for a programme is run, it usually begins with three questions on a whiteboard: What did we get wrong? What did we get right? And what would we do differently?
A better question to ask is this: “If we were starting this programme today, knowing what we now know, how would we do it?” It may sound like a subtle distinction, but it doesn’t use the negatives as a starting point and then heap in a little bit of positivity like an employee performance appraisal. Instead, it is a genuine clean-sheet approach, informed by prior experience.
RCF, used qualitatively, is simply that question on steroids. And it’s the one time when steroid use is actually desirable in an international sporting event.
