Model validation: Providing an external perspective to optimize economic modelling
Providing external validation of three models, designed to support the launch of a new treatment in multiple sclerosis
The challenge
Our client had developed a budget impact model (BIM), cost-consequence model and cost-effectiveness model (CEM) to support the launch of a new treatment. Since the models were developed internally by the client, they requested support to validate them. The objective was to ensure internal validity of all code, formulae and architecture in the three models, as well as provide recommendations on user experience and ease of use for country teams.
The solution
There were a number of critical elements to ensuring the models were appropriately validated. Firstly, the team performed an in-depth review of each model to fully understand the objectives, target patient population, treatment, comparators, perspective, time horizon, schedule of treatment, efficacy and safety data, costs and outputs. Additionally, the model architecture was examined to assess if the layouts of the models were clear and intuitive to facilitate the user-experience and flexibility. All cell calculations and macros were checked and any errors identified were recorded. The outputs of the models, including the base case and sensitivity and scenario analyses, were explored to ensure they made logical sense. Secondly, given that the intended use included customer-facing discussions, we provided recommendations for model updates and enhancements to facilitate both adaptations by country teams and also the overall ease of interpretation by end users. We provided mock-ups of recommended front-end enhancements, including a patient flow diagram and model order. An example of the model order is presented below:
Finally, each model underwent thorough quality control.
Key results
We delivered three annotated models in Microsoft Excel with validation comments on each sheet (with recommended solutions for any errors or issues identified), alongside accompanying summary reports in Microsoft Word with recommendations for model updates, improvements and enhancements.
Value to the client
The validation of the three models allowed the client to be confident in their results and analysis before submitting the models to HTA agencies and utilising them in discussions with payers and other decision-makers.
The pragmatic recommendations provided by Adelphi Values PROVE were well received and will be utilised by the client in future iterations of the models and in other projects.