Project risk

Estimated reading time: 6 minutes

When an organization wants to take on a larger project, it often faces the challenge of obtaining a reliable estimate of the financial risk involved. Let’s review two steps that can help us. We start with a qualitative assessment and proceed with a quantitative risk assessment.

Step I: Qualitative risk assessment

A project or risk manager normally starts with a qualitative risk assessment. He asks insiders to provide an estimate of the chance and impact of potential risks (e.g. on a scale from 1 to 5, or from very low to very high).

Although such an assessment is useful as a first indication of the biggest risks, it is often not clear how reliable such an estimate would be.

It is therefore not surprising that we often read in the news about projects with substantial budget overruns where expectations have not been met. Reasons for those overruns could be due to changes in the economic or business environment, or in the requirements for the project. Whatever the case, it means that the real risk drivers and their context were not (fully) considered or estimated. Eventually this results in goals that are only partially met or in cost cuts in other parts of the organisation to fill in the gaps. In case of public projects it might lead to increases in the taxes.

The ultimate responsible stakeholders want to avoid big misses. Therefore they often have interest in methods that lead to more realistic estimates of the financial risks. This implies a balanced assessment of risks, avoiding both substantial under- or overestimation of the risks.¹ 

How does quantitative risk assessment help us to achieve this goal?

Step II: Quantitative risk assessment

To calculate the project risks, we:

1. Identify the main underlying risk factors
2. Collect appropriate historical and benchmark data
3. Calculate the value in a median (expected) and negative scenario (VaR 90%)

Identification of risk factors

Similar to other projects (e.g. see the land development project), we map the various revenue and cost categories to applicable risk factors.

Regarding the operating budget we identified the number of visitors, applicable spending per visitor and the weather as main risk drivers on the revenue side. We identified staff salary, energy, maintenance and capital costs as the main expense risk drivers.

Further, the investment costs have a disproportionate effect on the total spending with a significant impact of both building and financing costs (see diagram 1).

---

Data collection

To obtain a proper risk estimate for each of the underlying risk drivers, we use a combination of historical data and annual reports where applicable.

For the estimation of the entry fee and the inflation of building costs, for instance, we gather several CBS datasets that contain information on leisure spending and building materials inflation respectively.

Check on the appropriateness of the data

Taking statistics out of readily available public datasets might pose some challenges, nevertheless. One needs to question how appropriate the data is for the specific use case at hand.

The reported average number of visitors for a specific swimming pool size across the Netherlands (CBS dataset) might not be the accurate number to use if regional peculiarities or differences in the density of people in the area are not considered. To obtain a proper benchmark or reference class we therefore first search for the most important underlying risk factors by testing its sensitivities statistically.

For example, for construction projects the design phase (conceptual, preliminary or definitive) can have substantial impact on the risks in additional costs. This implies that temporary additional surcharges might be relevant if the project is still in a preliminary concept phase and construction requirements can still change.

For the interest rate stress scenarios calculation, we could use the standard Solvency II model for the insurance sector (see also the articles on interest rate risk framework and on the validation of Solvency II interest rate model). However, we can also use models that automatically take into account the current context.

Calculation of the negative scenario

After obtaining appropriate statistical data on the various risk factors on revenues and costs we can calculate the total operating result (revenue – costs) both in the projected, median and the negative scenario. For the negative scenario we use a 90% certainty level, which means there is a 10% chance the results are worse than the estimate in the negative scenario.

Calculate Total Risk

The correlation between the risk factors enables us to derive a correlation benefit as risks do not always happen at the same time. This leads us to the reduction in the total project risk after considering correlation effects. We then calculate the total combined risk either through a mathematical formula or Monte Carlo simulation².

Summary: from qualitative to quantitative risk assessment

When we list the preceding steps we obtain the following approach:

In practise a risk manager would start with a qualitative assessment. For smaller projects this would often suffice. However if the projects and risk become larger, an additional quantitative assessment becomes more and more important. In this process it is important to always focus on the most important risk factors. We want to substantiate those not only ‘by gut feel’ but also statistically on representative reference data. Because: ‘garbage in is garbage out’.

Advantages of a quantitative risk assessment

Having an objective statistical estimation of the risks helps a lot in the decision making process.

In this case our financial risk analysis of the project was provided as a supplementary material to the decision makers in the municipality. It gave more clarity about the uncertainties around estimates from the initial business case. As one can imagine it makes a big difference if the project could be potentially 30%, 100% or 200% more expensive than projections in a negative scenario.

The project risk estimations were within the levels that the municipality was able to handle. We could take doubts away about the financial feasibility and it became easier to give green light for the next phase of the project.

Would you also like to get more certainty about project risks, feel free to contact us. We would be happy to show how we estimate project risk in a context dependent and data driven way.

---

1. In the Netherlands often the SSK method is used for large infrastructure projects. This implies manual input of an expected and lower and upper bound of the risk. When the statistical substantiation is missing, it becomes unclear what the value of the risk estimation is and we would classify it as only a qualitative estimation. ↩︎
2. A Monte Carlo simulation is fully dependent on the correct input about the risk factors. A Monte Carlo simulation on itself therefore does not mean that the resulting output is ‘probably ok’. When the input is incorrect, then the output also will be. Or in other words: ‘garbage in is garbage out’.  ↩︎