Practical Application and Empirical Evaluation of Reference Class Forecasting
Fleming, Q., & Koppelman, J. (2010).
Earned value project management
(4th ed.). Newtown Square, PA: Project
Flyvbjerg, B. (2006). From Nobel Prize to
project management: Getting risks right.
Project Management Journal, 37( 3), 5–15.
Flyvbjerg, B. (2007). Eliminating bias
in early project development through
reference class forecasting and good
governance. In K. J. Sunnevåg (Ed.),
Decisions based on weak information:
Approaches and challenges in the
early phase of projects (pp. 90–110).
Trondheim, Norway: Concept Program,
The Norwegian University of Science and
Flyvbjerg, B., & Cowi. (2004). Procedures
for dealing with optimism bias in
transport planning: Guidance document.
London, England: UK Department for
Flyvbjerg, B., Holm, M., & Buhl, S.
(2002). Underestimating costs in public
works projects: Error or lie? Journal of the
American Planning Association, 68( 3),
Flyvbjerg, B., Holm, M., & Buhl, S.
(2005). How (in)accurate are demand
forecasts in public works projects? The
case of transportation. Journal of the
American Planning Association, 71( 2),
Jacob, D., & Kane, M. (2004). Forecasting
schedule completion using earned value
metrics? Revisited. The Measurable News
Kahneman, D. (1994). New challenges
to the rationality assuption. Journal of
Institutional and Theoretical Economics,
150( 1), 18–36.
Kahneman, D., & Tversky, A. (1979a).
Intuitive prediction: Biases and
corrective procedures. In S. Makridakis
& S. Wheelwright (Eds.), Studies in
the management sciences: Forecasting
(p. 12). Amsterdam, Netherlands:
Kahneman, D., & Tversky, A. (1979b).
Prospect theory: An analysis of decisions
under risk. Econometrica, 47 ( 2),
practical applicability and utility of RCF,
but also of many other project management techniques.
We acknowledge the support provided by
the “Nationale Bank van België” (NBB)
and by the “Bijzonder Onderzoeksfonds”
(BOF) for the project with contract
number BOF12GOA021. Furthermore,
we would also like to thank Gilles
Bonne, Eveline Hoogstoel, and Gilles
Vandewiele for their efforts in developing
Anbari, F. (2003). Earned value project
management method and extensions.
Project Management Journal, 34 ( 4),
Batselier, J., & Vanhoucke, M.
(2015a). Construction and evaluation
framework for a real-life project
database. International Journal of
Project Management, 33( 3), 697–710.
Batselier, J., & Vanhoucke, M. (2015b).
Empirical evaluation of earned value
management forecasting accuracy for
time and cost. Journal of Construction
Engineering and Management, 141( 11),
Carbone, R., & Armstrong, J. (1982).
Evaluation of extrapolative forecasting
methods: Results of a survey of
academicians and practitioners.
Journal of Forecasting, 1( 2), 215–217.
Caron, F., Ruggeri, F., & Merli, A.
(2013). A Bayesian approach to improve
estimate at completion in earned value
management. Project Management
Journal, 44( 1), 3–16.
Colin, J., & Vanhoucke, M. (2016).
Empirical perspective on activity
durations for project management
simulation studies. Journal of
Construction Engineering and
Management, 142( 1), 04015047.
Covach, J., Haydon, J., & Reither, R.
(1981). A study to determine indicators
and methods to compute estimate at
completion (EAC). Fairfax, VA: Man Tech
technique can be evaluated on a quantitative basis through comparison with
other existing forecasting methods.
Although this study provides interesting
insights into the workings and performance of RCF and other forecasting
methods, its results may not be readily
generalized because of the restricted
number of real-life projects from which
the reference classes were selected.
To increasingly substantiate the validity of the RCF technique, it should be
applied and tested on an ever-growing
empirical project database.
Furthermore, following the concept of combining outside view with
inside view for project forecasting (Kim
& Reinschmidt, 2011), we identify the
future research topic of integrating
RCF in EVM. By replacing the baseline
estimates with the forecasts from RCF,
more accurate EVM performance metrics could be obtained. In turn, this
would lead to more reliable warning signals and thus more adequate corrective
actions. Therefore, it would ensure more
effective project control in general. This
assertion should, of course, be validated
by extensive empirical research.
Although the RCF technique in itself
is fairly straightforward, it is relatively
difficult to correctly implement in practice because of its strong dependence
on the selected reference class. As this
research has shown, a reference class
of (highly) similar projects is needed
to provide (highly) accurate forecasts.
Such a collection of (highly) similar
projects is not always readily available
in practice. Even less often is the collection of adequate size; indeed, forecasting bias increases as the reference class
gets smaller, which can undermine the
performance and applicability of RCF.
That is why it is of great importance to
have available many (and correct) real-life project data. Organizations should
make a point of collecting their projects’ progress and performance data
in a structured way, as described, for
example, in Batselier and Vanhoucke
(2015a) or at www.or-as.be/research/
database. This would not only boost the