Impact case of the month: ‘IT investment decision-making in the public sector’

The implementation of IT systems in the public sector does not always go smoothly. Cokky Hilhorst, Professor of Business Information Systems at Nyenrode Business University, conducted research into decision-making in IT investments, specifically focused on public organizations. This has led to valuable insights that help organizations gain more control over their IT expenditure.

IT investments in the public sector often make the news, and usually not in a positive way. An example is the system of the Dutch Tax Authorities, which, according to the media, is 'on its last legs'. The research project that Hilhorst carried out together with researchers from national and international universities consisted of various studies. “First, we looked into the impact of IT investment costs on efficiency in public organizations”, Hilhorst explains. “To improve this efficiency, decision-makers must make the right IT investments. That is why we, subsequently, researched the influence of language on decision-making. We looked for red flags regarding planning and budget overruns in IT projects. Finally, we came to some interesting insights when we studied the application of Reference Class Forecasting (RCF) as a method to improve the estimation of IT investment projects.”

IT investment pays off

The research conducted by Hilhorst and her fellow researchers shows that making IT investments in the public sector certainly pays off. An increase of €1.00 in the share of IT costs relative to total organizational costs is related to an average net cost efficiency gain of €1.08 two years later, the analysis showed. Net cost efficiency gain refers to the positive change in an organization's efficiency, reducing costs after deducting all relevant costs. Organizations can thus measure how effective their efforts are. Hilhorst emphasizes that there are differences between organizations and that not every organization benefits from IT investments.

Influence of language

In an initial study, Hilhorst and her colleagues discovered that the degree of use of abstract language in a business case can serve as an early warning signal. Abstract language seems to indicate more psychological distance that someone can have from a project. This leads to less overoptimism about the project, meaning that the schedule and budget are also less likely to be exceeded. Conversely, concrete language has a stronger relationship with project overruns. A second study showed that new IT systems are often presented more positively than existing IT systems. “For example, a new system can be described as 'fast and modern', and an existing system as 'inefficient and error-prone'”, Hilhorst explains. “Decision-makers could therefore be unconsciously influenced to invest in the wrong projects.” 

Reference Class Forecasting

Reference Class Forecasting (RCF) is used in project management and risk assessment. It is a method that improves the accuracy of forecasts of the current project by comparing it with similar projects from the past. The research of Hilhorst and her colleagues shows how RCF can improve project plans and how this can be tailored to the risk a decision-maker is willing to take. The study also provides evidence that there would have been significantly fewer schedule and budget overruns, if RCF had been applied based on the historical project data available at the time.

Impact research

As a result of the research, Hilhorst was asked to serve on various committees in the field of IT in the Dutch public sector. In addition, the research has resulted in several publications in top scientific journals and has received attention in the media and from the Dutch government. Dutch media, including the Volkskrant, NOS and FD, also contact Hilhorst for her expert opinion on a regular basis.