Managing the uncertainties of a large-scale IT project

In 2008, the ministers responsible for health and infrastructure in Québec announced with great fanfare the start of construction on the new Centre Hospitalier de L’Universite de Montréal (CHUM), which would cost nearly $1.3 billion and welcome its first patients in 2015.

In 2021, when the CHUM officially opened its doors—six years late—the bill was closer to $3.6 billion. In other words, the hospital cost three times as much for work that took twice as long.

The CHUM may go down in history as an extreme case, but large-scale projects that exceed time, budgetary and contractual estimates are numerous. In fact, according to experts, this is the case in almost all cases.

In practice, estimating the time and cost of a project is the most complex and arduous task that engineers and managers can undertake. When you think of recent projects that have gone off the rails, IT and digital transformation projects probably come to mind—regardless of the level of seniority and skills from the IT team.

The real risks

Chances are that the actual cost of a project will range between 25 to 400 per cent of its initial estimate, according to a study by Steve McConnell, an expert on software development in the United States. In other words, a project estimated at $100,000 will cost somewhere between the modest sum of $25,000 and the staggering sum of $400,000.

McConnell knows what he’s talking about: He worked on major transformation projects for more than 30 years, including Microsoft, Boeing and the U.S. military. By his own admission, he was always wrong.

There are a range of cognitive biases that can explain this inability to respect an initial estimate. Some come from pseudoscience, while others—such as the “planning fallacy” or “optimism bias”—come from more robust studies. But one fact prevails: the more humans involved in coordinating an estimate for a given project, the greater the likelihood that the initial figures will be erroneous.

On the other hand, without humans, there is no project, or even a business. So humans will continue to play an essential role in these projects, regardless of their cognitive biases.

The cone of uncertainty

McConnell understood the impacts that a bad estimate could have on his organization. Indeed, a cost overrun could lead to the abandonment of the project altogether, while a stretched schedule could lead to the loss of a competitive advantage or business opportunity.

To manage the risk of his IT projects, McConnell decided to approach the estimation issue differently. Rather than tackling the human ability to estimate, he tackled the risks linked to each stage, or “milestones,” of a project. The basis of this idea is that no matter the project, the milestones will be the same. This is where the concept of a “cone of uncertainty” for IT projects comes from.

On the Y axis, we find the variability factor of the estimate, which ranges from 0.25 to 4. On the X axis, we find the generic stages of an IT project. 

The philosophy behind the model is simple: The further you are from the delivery date, the more erroneous your estimate will be. As a result, McConnell suggests that even your estimation be iterative. 

A concrete example  

 Say you want to roll out new software to improve the customer experience and limit service costs, so you create a call for tenders and send it to four specialized firms. When you receive their offers, you find that the price of the highest bidder is four times higher than the price of the lowest bidder. Did they read the same document? You check your emails again. This is indeed the case. So how is this possible? 

During the initial concept phase, you’re at the furthest point from project delivery, which leaves room for a vague interpretation of your requirements. After all, “connecting the new software to accounting software” can mean many things. 

 Under these circumstances, how do you choose a partner? How will you manage your risks? According to McConnell’s approach, rather than estimating your entire project upfront, you should focus only on the path to the next step and complete those business requirements.  

Instead of setting aside a large budget that will likely turn out to be inaccurate—and could damage your reputation—plan on a smaller budget to get to the next iterative step. With this strategy, you will already see your margin of error cut in half. At the same time, developers can more easily understand the technical requirements of getting from A to B, rather than the complexities of getting from A to Z. Your risk drops by another half. 

The idea is to move forward iteratively until you are comfortable with the level of risk. If there is a 50 per cent difference between the lowest and highest bidder, you are probably in the right place. 

This article is part of a series of articles on the major questions facing CIOs. To receive our next articles on the subject or to read the articles already on file, go to logient.com. Subscribe to our newsletter and learn how to face the complex business challenges of today and tomorrow—and stay tuned via our LinkedIn and Facebook accounts.