Predictive Modeling in Practice: Improving the Participant Identification Process for Care Management Programs Using Condition-Specific Cut Points
Publication: Population Health Management
Volume 14, Issue Number 4
Abstract
The objective of this study was to optimize predictive modeling in the participant selection process for care management (CM) programs by determining the ideal cut point selection method. Comparisons included: (a) an evidence-based “optimal” cut point versus an “arbitrary” threshold, and (b) condition-specific cut points versus a uniform screening method. Participants comprised adult Medicaid health plan members enrolled during the entire study period (January 2007–December 2008) who had at least 1 of the chronic conditions targeted by the CM programs (n = 6459). Adjusted Clinical Groups Predictive Modeling (ACG-PM) system risk scores in 2007 were used to predict those with the top 5% highest health care expenditures in 2008. Comparisons of model performance (ie, c statistic, sensitivity, specificity, positive predictive value) and identified population size were used to assess differences among 3 cut point selection approaches: (a) single arbitrary cut point, (b) single optimal cut point, and (c) condition-specific optimal cut points. The “optimal” cut points (ie, single and condition-specific) both outperformed the “arbitrary” selection process, yielding higher probabilities of correct prediction and sensitivities. The condition-specific optimal cut point approach also exhibited better performance than applying a single optimal cut point uniformly across the entire population regardless of condition (ie, a higher c statistic, specificity, and positive predictive value, although sensitivity was lower), while identifying a more manageable number of members for CM program outreach. CM programs can optimize targeting algorithms by utilizing evidence-based cut points that incorporate condition-specific variations in risk. By efficiently targeting and intervening with future high-cost members, health care costs can be reduced. (Population Health Management 2011;14:205–210)
Get full access to this article
View all available purchase options and get full access to this article.
References
1.
California HealthCare Foundation. Snapshot: Health Care Costs 101, 2008 Editionhttp://www.chcf.org/∼/media/Files/PDF/H/HealthCareCosts08.pdfMarch112008. 1. California HealthCare Foundation. Snapshot: Health Care Costs 101, 2008 Edition. Available at: http://www.chcf.org/∼/media/Files/PDF/H/HealthCareCosts08.pdf. Accessed March 11, 2008.
2.
Bodenheimer TFernandez A. High and rising health care costs. Part 4: Can costs be controlled while preserving quality?Ann Intern Med200514326-31. 2. Bodenheimer T, Fernandez A. High and rising health care costs. Part 4: Can costs be controlled while preserving quality? Ann Intern Med 2005;143:26–31.
3.
Baan CARuige JBStolk RP et al. Performance of a predictive model to identify undiagnosed diabetes in a health care settingDiabetes Care199922213-219. 3. Baan CA, Ruige JB, Stolk RP, et al. Performance of a predictive model to identify undiagnosed diabetes in a health care setting. Diabetes Care 1999;22:213–219.
4.
Sylvia MLShadmi EHsiao CJBoyd CMSchuster ABBoult C. Clinical features of high-risk older persons identified by predictive modelingDis Manag2006956-62. 4. Sylvia ML, Shadmi E, Hsiao CJ, Boyd CM, Schuster AB, Boult C. Clinical features of high-risk older persons identified by predictive modeling. Dis Manag 2006;9:56–62.
5.
Cousins MSShickle LMBander JA. An introduction to predictive modeling for disease management stratificationDis Manag20025157-167. 5. Cousins MS, Shickle LM, Bander JA. An introduction to predictive modeling for disease management stratification. Dis Manag 2002;5:157–167.
6.
Weir SAweh GClark RE. Case selection for a Medicaid chronic care management programHealth Care Financ Rev20083061-74. 6. Weir S, Aweh G, Clark RE. Case selection for a Medicaid chronic care management program. Health Care Financ Rev 2008;30:61–74.
7.
Weiner JAbrams CACG Software Documentation/Users Manual, Version 5.0Baltimore, MDJohns Hopkins University Bloomberg School of Public Health, Health Services Research & Development Center2001. 7. Weiner J, Abrams C. ACG Software Documentation/Users Manual, Version 5.0. Baltimore, MD: Johns Hopkins University Bloomberg School of Public Health, Health Services Research & Development Center; 2001.
8.
The Johns Hopkins University ACG Case-Mix SystemAbout ACGshttp://www.acg.jhsph.edu/html/AboutACGs.htmMarch302009. 8. The Johns Hopkins University ACG Case-Mix System. About ACGs. Available at: http://www.acg.jhsph.edu/html/AboutACGs.htm. Accessed March 30, 2009.
9.
Reid RJMacWilliam LVerhulst LRoos NAtkinson M. Performance of the ACG case-mix system in two Canadian provincesMed Care20013986-99. 9. Reid RJ, MacWilliam L, Verhulst L, Roos N, Atkinson M. Performance of the ACG case-mix system in two Canadian provinces. Med Care 2001;39:86–99.
10.
Reid RRoos NMacWillliam LFrohlich NBlack C. Assessing population health care need using a claims-based ACG morbidity measure: A validation analysis in the province of ManitobaHealth Serv Res2002371345-1364. 10. Reid R, Roos N, MacWillliam L, Frohlich N, Black C. Assessing population health care need using a claims-based ACG morbidity measure: A validation analysis in the province of Manitoba. Health Serv Res 2002;37:1345–1364.
11.
Adams EKBronstein JMRaskind–Hood C. Adjusted clinical groups: Predictive accuracy for Medicaid enrollees in three statesHealth Care Financ Rev20022443-61. 11. Adams EK, Bronstein JM, Raskind–Hood C. Adjusted clinical groups: Predictive accuracy for Medicaid enrollees in three states. Health Care Financ Rev 2002;24:43–61.
12.
Perkins NJSchisterman EF. The inconsistency of “optimal” cutpoints obtained using two criteria based on the receiver operating characteristic curveAm J Epidemiol2006163670-675. 12. Perkins NJ, Schisterman EF. The inconsistency of “optimal” cutpoints obtained using two criteria based on the receiver operating characteristic curve. Am J Epidemiol 2006; 163:670–675.
13.
Selby JVKarter AJAckerson LMFerrara ALiu J. Developing a prediction rule from automated clinical databases to identify high-risk patients in a large population with diabetesDiabetes Care2001241547-1555. 13. Selby JV, Karter AJ, Ackerson LM, Ferrara A, Liu J. Developing a prediction rule from automated clinical databases to identify high-risk patients in a large population with diabetes. Diabetes Care 2001;24:1547–1555.
14.
Sylvia MLGriswold MDunbar LBoyd CMPark MBoult C. Guided care: Cost and utilization outcomes in a pilot studyDis Manag20081129-36. 14. Sylvia ML, Griswold M, Dunbar L, Boyd CM, Park M, Boult C. Guided care: Cost and utilization outcomes in a pilot study. Dis Manag 2008;11:29–36.
15.
Gonen MAnalyzing Receiver Operating Characteristic Curves with SASCary, NCSAS Institute Inc.2007. 15. Gonen M. Analyzing Receiver Operating Characteristic Curves with SAS. Cary, NC: SAS Institute Inc.; 2007.
16.
Loesche WJTaylor GGiordano J et al. A logistic regression model for the decision to perform access surgeryJ Clin Periodontol199724171-179. 16. Loesche WJ, Taylor G, Giordano J, et al. A logistic regression model for the decision to perform access surgery. J Clin Periodontol 1997;24:171–179.
17.
Coffin MSukhatme S. Receiver operating characteristic studies and measurement errorsBiometrics199753823-837. 17. Coffin M, Sukhatme S. Receiver operating characteristic studies and measurement errors. Biometrics 1997;53:823–837.
18.
Rochester MAPashayan NMatthews FDoble AMcLoughlin J. Development and validation of risk score for predicting positive repeat prostate biopsy in patients with a previous negative biopsy in a UK populationBMC Urol200997. 18. Rochester MA, Pashayan N, Matthews F, Doble A, McLoughlin J. Development and validation of risk score for predicting positive repeat prostate biopsy in patients with a previous negative biopsy in a UK population. BMC Urol 2009;9:7.
19.
Klein SAlexander DAHutchinson JDSimpson JASimpson JMBell JS. The Aberdeen trauma screening index: An instrument to predict post-accident psychopathologyPsychol Med200232863-871. 19. Klein S, Alexander DA, Hutchinson JD, Simpson JA, Simpson JM, Bell JS. The Aberdeen trauma screening index: An instrument to predict post-accident psychopathology. Psychol Med 2002;32:863–871.
20.
Kristjansson BHill GMcDowell ILindsay J. Optimal cut-points when screening for more than one disease state: An example from the Canadian study of health and agingJ Clin Epidemiol1996491423-1428. 20. Kristjansson B, Hill G, McDowell I, Lindsay J. Optimal cut-points when screening for more than one disease state: An example from the Canadian study of health and aging. J Clin Epidemiol 1996;49:1423–1428.
Information & Authors
Information
Published In
Population Health Management
Volume 14 • Issue Number 4 • August 2011
Pages: 205 - 210
PubMed: 21241172
Copyright
Copyright 2011, Mary Ann Liebert, Inc.
History
Published online: 11 August 2011
Published in print: August 2011
Published ahead of print: 17 January 2011
Topics
- Big data
- Big data analytics
- Health economics
- Health management programs
- Health security
- Insurance
- Medicare
- Population health management
- Population health strategies and models
- Predictive analytics
- Public health care
- Public health organizations and programs
- Public Health Research and Practice
- Technology, engineering, and computational biology
Authors
Author Disclosure Statement
Ms. Murphy, Ms. Castro, and Dr. Sylvia disclosed no conflicts of interest.
Metrics & Citations
Metrics
Citations
Export Citation
Export citation
Select the format you want to export the citations of this publication.
View Options
Get Access
Access content
To read the fulltext, please use one of the options below to sign in or purchase access.⚠ Society Access
If you are a member of a society that has access to this content please log in via your society website and then return to this publication.