Megaproject Management

post by ryan_b · 2021-01-20T18:55:50.132Z · EA · GW · 5 comments

Contents

  History
  Four Sublimes
  The Iron Law of Megaprojects
  Break-Fix Model
  Hirschman's Hiding Hand
  Survival of the Unfittest
  Light at the End of the Tunnel
  End
None
5 comments

This post originally appeared on LessWrong [LW · GW]. It has been very lightly edited.

Megaproject management is a new-ish subfield of project management. Originally considered to be the special case of project management where the budgets were enormous (billions of dollars), it is developing into a separate specialization because of the high complexity and tradition of failure among such projects. The driving force behind treating it as a separate field appears to be Bent Flyvbjerg, previously known around here for Reference Class Forecasting as the first person to develop an applied procedure. That procedure was motivated by megaprojects. For context, these projects are things like powerplants, chip fabs, oil rigs, et cetera; in other words, the building blocks of modernity.

I will make a summary of the paper "What you should know about megaprojects, and why: an overview" from 2014. For casual reading, there is an article about it from the New Yorker here. There is also an EconTalk episode with Flyvbjerg, pointed out to me by EdoArad.

History

Megaprojects got their name from the association of mega with big, so think mega-city rather than mega-joule. It did match the unit prefix in the beginning however, as such projects were mostly dams, bridges, or very large buildings in the early 20th century.

The next shift upward took place with the Manhattan Project and then the Apollo program, which are also frequently drawn on as positive examples. The term 'megaproject' picked up steam in the 1970s, at the same time project costs crossed over into the billions.

Currently project costs of 50-100 billion are common, with even larger projects less common but not rare. If you were to view certain things which need dedicated management as a project, like the stimulus packages from 2008 or US defense procurement, then we have crossed over into the trillions and are entering a 'tera era' of megaprojects.

Ignoring these special cases, but counting infrastructure and industries where billion dollar projects are common, megaprojects account for ~8% of global GDP.

Four Sublimes

These are four reasons which drive the popularity of megaprojects. They are kind of a group bias for each type of stakeholder. They are:

Predictably with biases, there are side effects:

The following characteristics of megaprojects are typically overlooked or glossed over when the four sublimes are at play and the megaproject format is chosen for delivery of large-scale ventures: 

1. Megaprojects are inherently risky due to long planning horizons and complex interfaces (Flyvbjerg, 2006). 

2. Often projects are led by planners and managers without deep domain experience who keep changing throughout the long project cycles that apply to megaprojects, leaving leadership weak. 

3. Decision-making, planning, and management are typically multi-actor processes involving multiple stakeholders, public and private, with conflicting interests (Aaltonen and Kujala, 2010). 

4. Technology and designs are often non-standard, leading to "uniqueness bias" amongst planners and managers, who tend to see their projects as singular, which impedes learning from other projects. 3 

5. Frequently there is overcommitment to a certain project concept at an early stage, resulting in “lock-in” or “capture,” leaving alternatives analysis weak or absent, and leading to escalated commitment in later stages. "Fail fast" does not apply; "fail slow" does (Cantarelli et al., 2010; Ross and Staw, 1993; Drummond, 1998). 

6. Due to the large sums of money involved, principal-agent problems and rent-seeking behavior are common, as is optimism bias (Eisenhardt, 1989; Stiglitz, 1989; Flyvbjerg el al., 2009). 

7. The project scope or ambition level will typically change significantly over time. 

8. Delivery is a high-risk, stochastic activity, with overexposure to so-called "black swans," i.e., extreme events with massively negative outcomes (Taleb, 2010). Managers tend to ignore this, treating projects as if they exist largely in a deterministic Newtonian world of cause, effect, and control. 

9. Statistical evidence shows that such complexity and unplanned events are often unaccounted for, leaving budget and time contingencies inadequate. 

10. As a consequence, misinformation about costs, schedules, benefits, and risks is the norm throughout project development and decision-making. The result is cost overruns, delays, and benefit shortfalls that undermine project viability during project implementation and operations.

The Iron Law of Megaprojects

These aren't little, either: cost overruns of 1.5x are common, in bad cases they can run more than 10x, and 90% of projects have them; it is common for projects to have 0.5x or less utilization once complete. This holds for the public and private sectors, and also across countries, so things like excessive regulation or corruption aren't good explanations.

They start off badly, but they do still manage to get completed, which is due to...

Break-Fix Model

Since management of megaprojects doesn't know what they are doing or don't have the incentives to care, inevitably something breaks. Then additional time and money are spent to fix what broke, or the conditions of the project are renegotiated, and it limps along to the next break. This process continues until the project is finished.

If it is so terrible and we know it is terrible, why do we do it this way?

Hirschman's Hiding Hand

Because a lot of important stakeholders don't know how terrible it is. From Willie Brown, former mayor of San Francisco:

"News that the Transbay Terminal is something like $300 million over budget should not come as a shock to anyone. We always knew the initial estimate was way under the real cost. Just like we never had a real cost for the [San Francisco] Central Subway or the [San Francisco-Oakland] Bay Bridge or any other massive construction project. So get off it. In the world of civic projects, the first budget is really just a down payment. If people knew the real cost from the start, nothing would ever be approved. The idea is to get going. Start digging a hole and make it so big, there's no alternative to coming up with the money to fill it in."

Nor are they without justification, for arguments have been made that support it. The first argument is exactly as Willie made it: if we knew how difficult large projects were, we would never build them.

For the second, note the title of the section is hiding, not hidden. This argument was made by Albert O. Hirschman on the basis of earlier work by J.E. Sawyer, and it says that there is an error in both the estimations of costs, and in the estimation of benefits, and this error should roughly cancel out. The problem is that Sawyer's work just pointed out that this was possible based on a picked sample of 5 or so. Hirschman then generalized it into a "Law of the Hiding Hand" and thereby legitimated lying to ourselves.

Alas it is bunk. Aside from being falsified by the actual data, Flyvbjerg points out the non-monetary opportunity costs through the example of the Sydney Opera House. It's architect, Dane Jorn Utzon, won the Pritzker Prize (the Nobel of architecture) in 2003 for the Sydney Opera House. It is his only major work - the catastrophic delays and cost overruns destroyed his career. Contrast with Frank Gehry, another inspired architect, and it looks like management's bungling of the Opera House probably cost us half a dozen gorgeous landmarks.

Survival of the Unfittest

The prevailing attitude that it is perfectly acceptable to lie about and then badly manage megaprojects leads to a weird scenario where worse projects are more likely to be chosen. Consider two competing projects, one with honest and competent management, and one with dishonest and incompetent management. The costs look lower for the latter project, and the benefits look higher, and the choosers between them probably expect them to both be over budget and behind schedule by about the same amount. Therefore we systematically make worse decisions about what projects to build.

Light at the End of the Tunnel

Fortunately there are bright spots. During the Obama administration these failings were identified as an important policy area for the US government. It is now much more common for a megaproject failure to result in consequences for leadership, like the CEOs of BP and Deepwater Horizon, or Airbus and the A380 superjumbo. There are megaprojects that go well and serve as examples of how to do it right, like the Guggenheim Museum in Bilbao. Lastly, there is scattered adoption of varying levels of good practices, like reference class forecasting and independent forecasting.

End

For those interested in learning more, Oxford has Major Programme Management at the masters level. In books there is the Oxford Handbook of Megaproject Management, and Megaproject Planning and Management: Essential Readings, both from Flyvbjerg.

5 comments

Comments sorted by top scores.

comment by Aaron Gertler (aarongertler) · 2021-01-21T10:11:46.599Z · EA(p) · GW(p)

Thanks for sharing this! I have a few large projects on the horizon, and while none of them are within five orders of magnitude of the examples mentioned here, I feel like I can still identify some snags to watch out for (in particular, doing more research to ensure the end results won't be "underutilized").

comment by ryan_b · 2021-01-21T16:32:56.005Z · EA(p) · GW(p)

Investigating the field more deeply this year is going to be one of my hobby projects, but my early impression is that a big part of the claim is that when you do regular project management things wrong, the penalty at least scales. It also looks like the cost of doing the analysis right, such as reference class forecasting, doesn't come remotely close to scaling with the needs of the project.

I'm glad about this, because I was worried at first the whole inquiry might be useless except to people in a position of responsibility. Instead, it looks like there will be a lot of methods that are always a good idea but also scale really well. Bonus!

comment by EdoArad (edoarad) · 2021-01-21T06:01:09.303Z · EA(p) · GW(p)

Thank you for writing this! I'll also throw in the EconTalk episode with Flyvbjerg on the topic 🙂

comment by ryan_b · 2021-01-21T18:45:42.563Z · EA(p) · GW(p)

This talk is great, and hits the exact same points as the paper. Would it be alright with you if I put the link to the talk in post with the other resources?

comment by EdoArad (edoarad) · 2021-01-21T20:22:32.518Z · EA(p) · GW(p)

Of course!  :)