Sign Up
ENعربي
All articles

Project Management

From Bias to Balance: Merging Cognitive Insights and External Data in Project Forecasting

  • Introduction

The pervasive issue of optimism bias in project cost forecasting significantly undermines project success, leading to frequent cost overruns and schedule delays. Traditional risk management approaches often fall short in addressing this bias because they neglect the influence of cognitive factors on decision-making. This literature review aims to synthesize and evaluate existing research on mitigating optimism bias in project cost forecasting by examining the theoretical foundations, exploring different perspectives, and assessing the methodologies used in this field. The review will highlight the main themes, points of agreement and debate, research gaps, and the evolution of the field over time. Ultimately, this review argues for a holistic approach that integrates both internal and external views to enhance the accuracy of project cost estimations.

  • The Theoretical Underpinnings of Optimism Bias
  • Cognitive Biases and Bounded Rationality: Traditional project risk management, rooted in decision sciences, has been criticized for its failure to adequately address uncertainty and cognitive biases. These biases, such as optimism bias, distort project appraisals, contributing to cost underestimation and unrealistic schedules. The concept of bounded rationality, introduced by Simon, acknowledges that decision-makers do not have perfect information or rationality, and therefore do not make optimal decisions aimed at utility maximization. Instead, they make satisfactory ones, acting reasonably within existing limits. Ecological rationality further explains how decision-makers adapt their behavior to their environment, considering both the physical and social contexts. Individuals use heuristics, which are quick, instinctive decision-making techniques, in situations of uncertainty. While these heuristics can be efficient, they can also lead to cognitive biases. According to Kahneman and…
  • Prospect Theory and the Planning Fallacy: Prospect theory, developed by Kahneman and Tversky, explains non-maximizing behaviors that traditional expected utility theory cannot account for. It posits that decision-making occurs in two phases: an editing phase, where individuals simplify the decision, and an evaluation phase, where they choose the prospect with the highest payoff. The theory highlights how the perception of change influences decisions, depending on the decision-maker's reference point, which can lead to loss aversion, where losses are overestimated and gains are underestimated. This theory also relates to the planning fallacy, where the reference point is a distorted image of the future. The planning fallacy leads to overly optimistic time and cost estimates, where individuals underestimate the time needed to complete a task compared to past averages. In a project management context, the planning fallacy is linked to optimism bias, which leads planners to overvalue posi…
  • Risk vs. Uncertainty: Winch extends the critique of traditional risk management by emphasizing that project uncertainty includes unforeseen complexities, not just known risks. Drawing on the cognitive perspectives of Knight and Keynes, Winch distinguishes between risk, where probabilities can be assigned to outcomes, and uncertainty, where probabilities cannot be meaningfully determined. Kay and King further develop this idea with the concept of radical uncertainty, where outcomes are fundamentally unknowable. Traditional project risk management relies too heavily on predictive models and fixed outcomes, making them ill-suited for complex projects. Therefore, a move towards more adaptive and flexible planning tools is needed to respond to both known risks and unforeseen complexities.

III. The Inside View vs. The Outside View

  • The Inside View: The inside view refers to the natural tendency of decision-makers to focus on the unique aspects of a given project or task, gathering information and developing scenarios based on a few specific analogies. This focus on the uniqueness of the endeavor leads to overconfidence and excessive optimism, contributing to the planning fallacy. By focusing on the specifics of the project, decision-makers often overlook historical patterns that could provide a more realistic assessment.
  • The Outside View: In contrast, the outside view emphasizes using historical data from similar past projects to create more accurate estimates. This view highlights the commonalities between the current project and previous ones, thus avoiding an overemphasis on unique aspects. A key methodology for implementing the outside view is Reference Class Forecasting (RCF). This method involves three main phases: 1) identifying the most relevant reference class, 2) establishing a probability distribution, and 3) positioning the current project within the reference class. The adoption of the outside view has gained traction, particularly in the construction industry, with governments and policy makers beginning to incorporate it into guidelines. RCF, however, faces limitations related to data availability, incorrect grouping of asset classes and geographical variability.
  • Integrating the Inside and Outside Views
  • Support Theory: To improve upon the limitations of each of these perspectives, the integration of the inside and outside views has been proposed. One framework for doing this is through the application of support theory, which focuses on subjective probabilities and how they are influenced by how events are described. According to support theory, when an event is unpacked into subcategories, its perceived likelihood increases. This occurs because the unpacking of a task enhances its "support," which is the mental weight assigned to the event's probability. The concept of unpacking is different from decomposing a task; unpacking involves figuratively breaking down a task to enhance accessibility, whereas decomposition involves dividing a task into subcomponents with separate forecasts for each.
  • Combining Perspectives: By integrating unpacking with the outside view, the shortcomings of RCF can be mitigated. For example, by unpacking a project and considering its workstream milestones, the appraiser may identify characteristics previously overlooked before conducting the RCF analysis. Also, support theory's reliance on subjective probabilities offers a more tailored approach by allowing forecasters to adjust probabilities based on project-specific characteristics compared to standardized measures that may be used for optimism bias. Therefore, a more holistic approach to forecasting is needed, combining the strengths of both the inside and outside views.
  • Prospect Theory and Support Theory: Both prospect theory and support theory offer different interpretations on overcoming the planning fallacy. Prospect theory emphasizes the importance of comparing the task with similar past cases and using a probabilistic mindset, while support theory focuses on increasing awareness of the components that constitute the task. These theories can be interpreted as complementary, providing a more holistic view of the planning fallacy and opening new routes for exploration.
  • Methodologies The research reviewed utilizes a variety of methodologies, including empirical studies, theoretical frameworks, and case-based reasoning. Empirical studies, which include experiments in real-world settings, are used to assess how individuals make decisions under uncertainty. Theoretical frameworks, such as prospect theory and support theory, are developed to explain and predict behavior, and explore the underlying causes of cognitive biases. Case-based reasoning methods, such as RCF, use historical data to inform current decisions. Each methodology has strengths and limitations: while empirical studies provide real-world data, they can be challenging to generalize, theoretical frameworks can be abstract, and the effectiveness of case-based reasoning relies heavily on the quality and availability of historical data.
  • Development in the Field

The field has evolved from traditional risk management approaches to incorporating cognitive and behavioral economics perspectives. Initially, the focus was on internal project factors, such as technical and economic aspects, to explain project underperformance. Over time, the field has shifted to incorporating external historical data, introducing the outside view, and addressing psychological factors, like the planning fallacy. There has also been a shift in the debate between inside and outside views, toward an approach that recognizes the importance of both.

VII. Critical Insights

The literature highlights a tension between the inside and outside views. While the outside view is useful for mitigating optimism bias, it may lead to overlooking the uniqueness of individual projects. Conversely, while the inside view can provide valuable project-specific insights, it is prone to biases such as overconfidence. There is general agreement on the significance of optimism bias and the planning fallacy as major contributors to cost overruns, but debates arise when it comes to the extent to which these biases are responsible for project underperformance. Some scholars emphasize the role of errors and unforeseen events, not just human behavior. Gaps in the research include the practical implementation of integrated approaches, and further studies are needed to see how these theoretical frameworks can be operationalized. Additionally, the field has yet to develop a consensus on how to balance standardized objective probability measures with the flexibility of subjective probabilities when making decisions.

VIII. Conclusion

This literature review underscores the need for a holistic approach to mitigating optimism bias in project cost forecasting. Integrating the inside and outside views, by using techniques such as unpacking alongside reference class forecasting, can enhance the accuracy of project estimates. While the framework does not eliminate uncertainty, it provides tools for managing and reducing the impact of biases. The use of subjective probabilities in decision-making, can further refine forecasts and help create more reliable outcomes. Future research should focus on how to operationalize this integration to make project cost forecasting tools more effective.

The most important points in the response have been bolded to make it easier to understand. This response includes all the sections requested and provides an analysis and synthesis of the provided sources. The sources are cited appropriately with the relevant number in brackets.

FAQ: Understanding and Mitigating Cost Underestimation in Projects

  • What is optimism bias and how does it contribute to cost underestimation in projects? Optimism bias is a cognitive bias where individuals tend to overestimate positive outcomes and underestimate negative ones, leading to unrealistic expectations. In project management, this manifests as an overvaluation of project success, an oversimplification of project activities, and a neglect of potential risks. Consequently, project planners often underestimate costs and time requirements, leading to cost overruns and schedule delays. This bias is a significant contributor to the "planning fallacy," where project estimates are consistently more optimistic than actual outcomes.
  • What is the "planning fallacy" and how does it relate to the "inside view" and "outside view" of project forecasting? The planning fallacy is the tendency to underestimate the time and resources needed to complete a task or project. This is often linked to the "inside view" – where project managers focus on the unique aspects of a project and develop detailed, optimistic scenarios based on internal information. By emphasizing the uniqueness of a project, planners can become overly confident, ignoring the broader trends and patterns revealed in similar past projects. The "outside view," in contrast, advocates for examining the distribution of outcomes from similar projects to create more accurate forecasts. It emphasizes the common traits between the current project and previous ones rather than focusing solely on the project's unique attributes.
  • What is "Reference Class Forecasting" (RCF) and how does it work in practice? Reference Class Forecasting (RCF) is a method used to implement the "outside view" in project forecasting. It involves three main phases: First, identifying a relevant reference class (a group of similar past projects); second, establishing a probability distribution of costs/schedules based on the data from the reference class; and third, positioning the current project within that distribution. The assumption is that the project should be positioned close to the median point in the distribution of past project data for accurate forecasting. RCF aims to reduce optimism bias by grounding project estimates in the historical performance of comparable endeavors. It provides a more objective, statistically-backed approach to forecasting as opposed to focusing on the unique aspects of a given project and what we think will happen.
  • What are the limitations of relying solely on the "outside view" and how can they be addressed? While the "outside view" and techniques like RCF are useful for mitigating optimism bias, they can be limited by the availability of precise and comparable historical data and the risk of incorrectly grouping projects from different asset classes. Over-reliance on the "outside view" may lead to overlooking the specific complexities and unique attributes of the current project. In this scenario, the estimation process may become a mere statistical exercise, losing sight of project-specific nuances. To address these limitations, the outside view can be complemented by adopting attributes from an inside view, specifically the concept of "unpacking" from support theory.
  • What is "support theory" and how does it introduce the concept of "unpacking" into the forecasting process? Support theory suggests that when making a decision, individuals rely on subjective probabilities, based on their beliefs and perceptions of events. It also introduces the concept of "unpacking," where a generalized event is broken down into more specific subcategories. For example, rather than estimating an entire project budget, one might break it down into subcategories of different tasks or milestones. This enhances the perceived likelihood of the overall event, and increases the awareness of various factors that might not be initially considered. Unpacking also increases the "support" for an event, or the mental weight assigned to the probability of its occurence, because specific details may be more clearly focused on. Unpacking thus brings the inside view's understanding of project specifics into a methodology that can be integrated with external forecasting.
  • How can the concept of "unpacking" from support theory enhance the "outside view" on project forecasting? Integrating "unpacking" from support theory with the "outside view" can improve the quality of forecasts by allowing project appraisers to consider various elements of the project with more focus. By breaking down a project into subcategories before assessing its fit within a reference class, this method highlights the unique characteristics of the current project and may help unveil hidden risks or opportunities. This combination ensures that forecasting is not just a statistical exercise based solely on past trends but also considers the particulars of the project itself. For example, by thinking of all potential causes of a delay before using a reference class, risk assessment can be more thorough, leading to a more accurate positioning of the project within the distribution of historical data. This process aims to reduce the likelihood of inappropriately placing a project due…
  • How does subjective probability relate to risk assessment in project planning? While objective probabilities and risk factors are important, support theory emphasizes the role of subjective probabilities and the need to refine forecasts by allowing for flexible adjustments based on the specifics of a project. By unpacking different elements, project forecasters can then refine the probabilities associated with various contributory risk factors and project milestones in a unique way, rather than strictly using standardized measures. This allows a more tailored and realistic risk assessment, mitigating the likelihood of underestimation due to a singular focus on a predetermined set of risks.
  • What does a "holistic approach" to cost forecasting look like and what is it designed to accomplish? A "holistic approach" to cost forecasting integrates the strengths of both the inside and outside views, acknowledging the interdependence of project-specific details and distributional patterns from past data. This approach involves using techniques such as unpacking in combination with methods such as RCF to create more accurate and flexible forecasts. The goal is to move away from the idea that the inside and outside views are mutually exclusive and to leverage the advantages of each approach to create better risk assessments that mitigate cost underestimation. Ultimately, this method aims to build new knowledge and more effective techniques that will allow for more realistic project forecasting

(Sassano, 2025)

Reference:

Sassano, G. (2025). The holistic view in forecasting: A conceptual framework to analyze and mitigate cost underestimation arising from optimism bias. Project Leadership and Society, 6. https://doi.org/10.1016/j.plas.2025.100177