Convincing AI analysis. But would you rely on it?
AI can generate convincing analyses and conclusions. But what are they actually based on? Are the key assumptions valid, or do they rely on widely held but incorrect or outdated beliefs, speculation, or hallucinations? And what does the relevant empirical data tell us about their validity, key drivers, and uncertainties?
That is why we developed an automated validation engine.
It tests key assumptions underlying capital decisions against relevant empirical data and helps assess how reliable they are.
This approach aligns with the growing emphasis on transparency, reliability, and accountability in the use of AI, as reflected in the EU AI Act.
Empirical validation levels
The following levels of empirical support can be distinguished:
| Validation level | Evidence type | Interpretation | Example approaches |
|---|---|---|---|
| 1. Initial | In-sample | Captured relationships; might be in-sample data fit. | LLMs or foundation models without out-of-sample validatie. |
| 2. Predictive | Out-of-sample with statistical support | Generalisable predictability within the evaluated domain. | Forecasting models with independent holdout validation. |
| 3. Predictive per reference class | Out-of-sample with statistical support per reference class | Generalisable predictability within reference classes. | Predictive Reference Class Retrieval, with validation within the relevant reference classes. |
Why is out-of-sample testing important? Our article on empirically validated risk forecasts explains why generating probabilities is not sufficient evidence of their reliability.
Broadly applicable
From land development risk and early-stage infrastructure estimates to macroeconomics, interest rate risk, and investments, our validation engine can be applied across a broad range of capital decisions.
Assumptions are easy to make.
But informed decisions demand more.
