Skip to main content
SpringerLink
Log in

Assessing Quality in Decision Analytic Cost-Effectiveness Models

A Suggested Framework and Example of Application

  • Consensus Conference Papers
  • Published:
PharmacoEconomics Aims and scope Submit manuscript

Abstract

Despite the growing use of decision analytic modelling in cost-effectiveness analysis, there is a relatively small literature on what constitutes good practice in decision analysis. The aim of this paper is to consider the concept of ‘validity’ and ‘quality’ in this area of evaluation, and to suggest a framework by which quality can be demonstrated on the part of the analyst and assessed by the reviewer and user.

The paper begins by considering the purpose of cost-effectiveness models and argues that the their role is to identify optimum treatment decisions in the context of uncertainty about future states of the world. The issue of whether such models can be defined as ‘scientific’ is considered. The notion that decision analysis undertaken at time t can only be considered scientific if its outputs closely predict the results of a trial undertaken at time t+1 is rejected as this ignores the need to make decisions on the basis of currently available evidence. Rather, the scientific characteristic of decision models is based on the fact that, in principle at least, such analyses can be falsified by comparison of two states of the world, one where resource allocation decisions are based on formal decision analysis and the other where such decisions are not. This section of the paper also rejects the idea of exact codification of scientific method in general, and of decision analysis in particular, as this risks rejecting potentially valuable models, may discourage the development of novel methods and can distort research priorities. However, the paper argues that it is both possible and necessary to develop a framework for assessing quality in decision models.

Building on earlier work, various dimensions of quality in decision modelling are considered: model structure (disease states, options, time horizon and cycle length); data (identification, incorporation, handling uncertainty); and consistency (internal and external). Within this taxonomy a (nonexhaustive) list of questions about quality is suggested which are illustrated by their application to a specific published model. The paper argues that such a framework can never be prescriptive about every aspect of decision modelling. Rather, it should encourage the analyst to provide an explicit and comprehensive justification of their methods, and allow the user of the model to make an informed judgment about the relevance, coherence and usefulness of the analysis.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Table I
Table II

Similar content being viewed by others

References

  1. Sonnenberg FA, Beck JR. Markov models in medical decision making. Med Decis Making 1993; 13: 322–38

    Article  PubMed  CAS  Google Scholar 

  2. Keeler E. Decision trees and Markov models in cost-effectiveness research. In: Sloan F, editor. Valuing health care: costs, benefits and effectiveness of pharmaceuticals and other medical technologies. Cambridge: Cambridge University Press, 1995: 185–205

    Chapter  Google Scholar 

  3. Gold MR, Siegel JE, Russell LB, et al. Cost-effectiveness analysis in health and medicine. New York (NY): Oxford University Press, 1996

    Google Scholar 

  4. Briggs A, Sculpher MJ. An introduction to Markov modelling for economic evaluation. Pharmacoeconomics 1998; 13: 397–409

    Article  PubMed  CAS  Google Scholar 

  5. Sheldon TA. Problems of using modelling in the economic evaluation of health care. Health Econ 1996; 5: 1–11

    Article  PubMed  CAS  Google Scholar 

  6. Buxton MJ, Drummond MF, Van Hout BA, et al. Modelling in economic evaluation: an unavoidable fact of life. Health Econ 1997; 6: 217–27

    Article  PubMed  CAS  Google Scholar 

  7. Popper KR. The logic of scientific discovery. London: Hitchinson, 1972

    Google Scholar 

  8. Lakatos I. Falsification and the methodology of scientific research programmes. In: Lakatos I, Musgrave, A, editors. Criticism and the growth of knowledge. Cambridge: Cambridge University Press, 1970: 91–195

    Google Scholar 

  9. Manning R, Claxton K. Experimental and econometric solutions to selection bias. The International Health Economics Association Meeting, 1999 Jun 6–9; Rotterdam

  10. Schwartz D, Lellouch J. Explanatory and pragmatic attitudes in therapeutic trials. J Chronic Dis 1967; 20: 637–48

    Article  PubMed  CAS  Google Scholar 

  11. Sonnenberg FA, Roberts MS, Tsevat J. Toward a peer review process for medical decision analysis models. Med Care 1994; 32 Suppl.: JS52–64

    Google Scholar 

  12. Russell B. My philosophical development. London: Routledge, 1959

    Google Scholar 

  13. Sokal A, Bricmont J. Intellectual impostures. London: Profile Books, 1998

    Google Scholar 

  14. Feyerabend P. Against method. London: New Left Books, 1975

    Google Scholar 

  15. Begg C, Cho M, Eastwood S, et al. Improving the quality of reporting of randomized controlled trials. JAMA 1996; 276: 637–9

    Article  PubMed  CAS  Google Scholar 

  16. Guyatt GH, Sackett DL, Cook DJ. Users’ guides to the medical literature: II. How to use an article about therapy or prevention. B. What were the results and will they help me in caring for my patients? JAMA 1994; 271: 59–63

    Article  PubMed  CAS  Google Scholar 

  17. Pocock SJ. When to stop a clinical trial. BMJ 1992; 305: 235–40

    Article  PubMed  CAS  Google Scholar 

  18. Eddy DM. Technology assessment: the role of mathematical modeling. In: Mosteller F, editor. Assessing medical technologies. Washington, DC: National Academy Press, 1985: 144–53

    Google Scholar 

  19. Halpern MT, Luce BR, Brown RE, et al. Health and economic outcomes modeling practices: a suggested framework. Value Health 1998; 1: 131–47

    Article  PubMed  CAS  Google Scholar 

  20. Drummond MF, O’Brien BJ, Stoddart GL, et al. Methods for the economic evaluation of health care programmes. New York (NY): Oxford University Press, 1997

    Google Scholar 

  21. Drummond M, Jefferson T. Guidelines for authors and peer reviewers of economic submissions to the BMJ. BMJ 1996; 313: 275–83

    Article  PubMed  CAS  Google Scholar 

  22. Nuijten MJC, Starzewski J. Applications of modelling studies. Pharmacoeconomics 1998; 13: 289–91

    Article  PubMed  CAS  Google Scholar 

  23. Davies LM, Drummond MF. Assessment of costs and benefits of drug therapy for treatment-resistant schizophrenia in the United Kingdom. Br J Psychiatry 1993; 162: 38–42

    Article  PubMed  CAS  Google Scholar 

  24. Palmer CS, Revicki DA, Genduso LA, et al. A cost-effectiveness clinical decision analysis model for schizophrenia. Am J Manag Care 1998; 4: 345–55

    PubMed  CAS  Google Scholar 

  25. Mohr P, Neumann P, Claxton K. Letter to the editor. Am J Manag Care 1998; 4: 1200–1

    Google Scholar 

  26. Peralta V, Cuesta MJ, de Leon J. An empirical analysis of latent structures underlying schizophrenic symptoms: a four-syndrome model. Biol Psychiatry 1994; 36: 726–36

    Article  PubMed  CAS  Google Scholar 

  27. Campeau L. Grading of angina pectoris. Circulation 1976; 54: 522–3

    CAS  Google Scholar 

  28. Patrick DL, Erickson P. Health status and health policy: allocating resources to health care. New York (NY): Oxford University Press, 1993

    Google Scholar 

  29. Sugar CA, Sturm R, Lee TT, et al. Empirically defined health states for depression from the SF-12. Health Serv Res 1998; 33: 911–28

    PubMed  CAS  Google Scholar 

  30. Deyo RA, Centor RM. Assessing the responsiveness of functional scales to clinical change: an analogy to diagnostic test performance. J Chronic Dis 1986; 39: 897–906

    Article  PubMed  CAS  Google Scholar 

  31. Van Doorslaer EKA. Economic evaluation of antifungal agents in the treatment of toenail onychomycosis. Dermatology 1996; 193: 239–44

    Article  PubMed  Google Scholar 

  32. Sculpher M, Michaels J, McKenna M, et al. Acost-utility analysis of laser-assisted angioplasty for peripheral arterial occlusions. Int J Technol Assess Health Care 1996; 12: 104–25

    Article  PubMed  CAS  Google Scholar 

  33. Daly E, Vessey MP, Barlow D, et al. Hormone replacement therapy in a risk-benefit perspective. Maturitas 1996; 23: 247–57

    Article  PubMed  CAS  Google Scholar 

  34. Coulter A, Bradlow J, Agass M, et al. Outcomes of referrals to gynaecology outpatient clinics for menstrual problems: an audit of general practice records. Br J Obstet Gynaecol 1991; 98: 789–96

    Article  PubMed  CAS  Google Scholar 

  35. Luoto R, Kaprio J, Uutela A. Age at natural menopause and sociodemographic status in Finland. Am J Epidemiol 1994; 139: 64–76

    PubMed  CAS  Google Scholar 

  36. Miller DK, Homan SM. Determining transition probabilities: confusion and suggestions. Med Decis Making 1994; 14: 52–8

    Article  PubMed  CAS  Google Scholar 

  37. Dowie J. Evidence-based, cost-effective and preference-driven medicine: decision analysis based medical decision making is the pre-requisite. J Health Serv Res Policy 1996; 1: 104–13

    PubMed  CAS  Google Scholar 

  38. Fenwick E, Claxton K, Sculpher M, et al. Improving the efficiency and relevance of health technology assessment: the role of iterative decision analytic modelling. Centre for Health Economics (CHE) discussion paper. York: CHE, University of York. In press

  39. Briggs A, Sculpher M, Buxton M. Uncertainty in the economic evaluation of health care technologies: the role of sensitivity analysis. Health Econ 1994; 3: 95–104

    Article  PubMed  CAS  Google Scholar 

  40. Thompson KM, Graham JD. Going beyond the single number: using probabilistic risk assessment to improve risk management. Hum Ecological Risk Assess 1996; 24: 1008–34

    Article  Google Scholar 

  41. Thompson KM, Burmaster DE, Crouch EAC. Monte Carlo techniques for quantitative uncertainty analysis in public health risk assessments. Risk Anal 1992; 12: 53–63

    Article  PubMed  CAS  Google Scholar 

  42. Stinnett A, Paltiel D. Estimating CE ratios under second-order uncertainty: the mean ratio versus the ratio of the means. Med Decis Making 1997; 17: 483–9

    Article  PubMed  CAS  Google Scholar 

  43. Tosteson ANA, Rosenthal DI, Melton J, et al. Cost-effectiveness of screening perimenopausal white women for osteoporosis: bone densitometry and hormone replacement therapy. Ann Intern Med 1990; 113: 594–603

    PubMed  CAS  Google Scholar 

  44. Briggs A, Sculpher M, Britton A, et al. The costs and benefits of primary total hip replacement: how likely are new prostheses to be cost-effective? Int J Technol Assess Health Care 1998; 14 (4): 743–61

    Article  PubMed  CAS  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Centre for Health Economics, University of York, Heslington, York YO10 5DD, England

    Mark Sculpher & Elisabeth Fenwick

  2. Department of Economics and Related Studies, University of York, York, England

    Karl Claxton

Authors
  1. Mark Sculpher
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Elisabeth Fenwick
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Karl Claxton
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Sculpher, M., Fenwick, E. & Claxton, K. Assessing Quality in Decision Analytic Cost-Effectiveness Models. Pharmacoeconomics 17, 461–477 (2000). https://doi.org/10.2165/00019053-200017050-00005

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.2165/00019053-200017050-00005

Keywords

Search

Navigation