Research Article
No access
Published Online: 1 June 2017

Fair Prediction with Disparate Impact: A Study of Bias in Recidivism Prediction Instruments

Publication: Big Data
Volume 5, Issue Number 2

Abstract

Recidivism prediction instruments (RPIs) provide decision-makers with an assessment of the likelihood that a criminal defendant will reoffend at a future point in time. Although such instruments are gaining increasing popularity across the country, their use is attracting tremendous controversy. Much of the controversy concerns potential discriminatory bias in the risk assessments that are produced. This article discusses several fairness criteria that have recently been applied to assess the fairness of RPIs. We demonstrate that the criteria cannot all be simultaneously satisfied when recidivism prevalence differs across groups. We then show how disparate impact can arise when an RPI fails to satisfy the criterion of error rate balance.

Get full access to this article

View all available purchase options and get full access to this article.

References

1.
Model penal code: Sentencing. American Law Institute. 2016. Available at: https://www.ali.org/projects/show/sentencing (last accessed September 14, 2016).
2.
Blomberg T, Bales W, Mann K, et al. Validation of the compas risk assessment classification instrument. 2010. Available at: http://criminology.fsu.edu/wp-content/uploads/Validation-of-the-COMPAS-Risk-Assessment-Classification-Instrument.pdf (last accessed September 14, 2016).
3.
Casselman B, Barry-Jester AM, Goldstein D. Should prison sentences be based on crimes that haven't been committed yet? Available at: https://fivethirtyeight.com/features/prison-reform-risk-assessment (last accessed September 14, 2016).
4.
Angwin J, Larson J, Mattu S, Kirchner L. 2016. Machine bias: There's software used across the country to predict future criminals and it's biased against blacks. Available online at www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing (last accessed February 6, 2017).
5.
Northpointe. Compas risk & need assessment system: Selected questions posed by inquiring agencies. Available at: http://www.northpointeinc.com/files/technical_documents/Selected_Compas_Questions_Posed_by_Inquiring_Agencies.pdf (last accessed September 14, 2016).
6.
Flores AW, Bechtel K, Lowenkamp CT. False positives, false negatives, and false analyses: A rejoinder to “machine bias: There's software used across the country to predict future criminals. And it's biased against blacks. 2016. Available at: http://www.crj.org/page/-/publications/rejoinder7.11.pdf
7.
Dieterich W, Mendoza C, Brennan T. Compas risk scales: Demonstrating accuracy equity and predictive parity. 2016. Available at: http://go.volarisgroup.com/rs/430-MBX-989/images/ProPublica_Commentary_Final_070616.pdf (last accessed February 6, 2017).
8.
Angwin J, Larson J, Mattu S, Kirchner L. 2016. How we analyzed the compas recidivism algorithm. Available online at www.propublica.org/article/how-we-analyzed-the-compas-recidivism-algorithm (last accessed February 6, 2017).
9.
Administrative Office of the United States Courts. An overview of the federal post conviction risk assessment, September 2011.
10.
Singh JP. Predictive validity performance indicators in violence risk assessment: A methodological primer. Behav Sci Law. 2013;310:0 8–22.
11.
Skeem JL, Lowenkamp CT. Risk, race, & recidivism: Predictive bias and disparate impact. Criminology. 2016;54:680–712.
12.
Skeem JL, Monahan J, Lowenkamp CT. Gender, risk assessment, and sanctioning: The cost of treating women like men. Law Hum Behav. 2016;40:580–593.
13.
Hardt M, Price E, Srebro N, et al. Equality of opportunity in supervised learning. In: Advances in Neural Information Processing Systems. NIPS, 2016, pp. 3315–3323.
14.
Zliobaite I. On the relation between accuracy and fairness in binary classification. arXiv preprint arXiv:1505.05723, 2015.
15.
Dwork C, Hardt M, Pitassi T, et al. Fairness through awareness. In: Proceedings of the 3rd Innovations in Theoretical Computer Science Conference. ACM, 2012, pp. 214–226.
16.
Calders T, Verwer S. Three naive bayes approaches for discrimination-free classification. Data Mining Knowl Discov. 2010;210:277–292.
17.
Fish B, Kun J, Lelkes ÁD. A confidence-based approach for balancing fairness and accuracy. In: Proceedings of the 2016 SIAM International Conference on Data Mining. SIAM, 2016, pp. 144–152.
18.
Romei A, Ruggieri S. A multidisciplinary survey on discrimination analysis. T Knowl Eng Rev. 2014;290:582–638.
19.
Barocas S, Selbst AD. Big data's disparate impact. Cal Law Rev. 2016;104:67-1.
20.
Hannah-Moffat K. Actuarial sentencing: An unsettled? proposition. Justice Quart. 2013;300:270–296.
21.
Skeem J. Risk technology in sentencing: Testing the promises and perils (commentary on hannah-moffat, 2011). Justice Quart. 2013;300:297–303.
22.
Monahan J, Skeem JL. Risk assessment in criminal sentencing. Annu Rev Clin Psychol. 2016;12:489–513.
23.
Kleinberg J, Mullainathan S, Raghavan M. Inherent trade-offs in the fair determination of risk scores. arXiv preprint arXiv:1609.05807, 2016.
24.
Corbett-Davies S, Pierson E, Feller A, Goel S. A computer program used for bail and sentencing decisions was labeled biased against blacks. It's actually not that clear. Available at: https://www.washingtonpost.com/news/monkey-cage/wp/2016/10/17/can-an-Algorithm-be-racist-our-analysis-is-more-cautious-than-propublicas/?utm_term=.b7e706ae3362 (last accessed February 2, 2017).
25.
Ritter N. Predicting recidivism risk: New tool in Philadelphia shows great promise. Natl Inst Just J 271, 2013.
26.
PCS. Validation of risk scale. Technical report, Pennsylvania Commission on Sentencing, 2013.
27.
Cohen J. Statistical power analysis for the behavioral sciences (2nd edition). Mahwah, NJ: Lawrence Erlbaum Associates, 1988.
28.
Uniform crime report: Crime in the United States, 2015-offenses cleared. U.S. Department of Justice, 2016.
29.
Meehl PE. Clinical versus statistical prediction: A theoretical analysis and a review of the evidence. Minneapolis, MN: University of Minnesota Press, 1954.
30.
Dawes RM, Faust D, Meehl PE. Clinical versus actuarial judgment. Science. 1989;243:1668–1674.
31.
Grove WM, Zald DH, Lebow BS, et al. Clinical versus mechanical prediction: A meta-analysis. Psychol Assess. 2000;12:1-9.
32.
Kleinberg J, Lakkaraju H, Leskovec J, Ludwig J, Mullainathan S. Human decisions and machine predictions. Technical report, National Bureau of Economic Research. 2017.
33.
Anwar S, Fang H. Testing for racial prejudice in the parole board release process: Theory and evidence. Technical report, National Bureau of Economic Research. 2012.
34.
Sweeney LT, Haney C. The influence of race on sentencing: A meta-analytic review of experimental studies. Behav Sci Law. 1992;100:179–195.

References

Cite this article as: Chouldechova A (2017) Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big Data 5:2, 153–163, DOI: 10.1089/big.2016.0047.

Information & Authors

Information

Published In

cover image Big Data
Big Data
Volume 5Issue Number 2June 2017
Pages: 153 - 163
PubMed: 28632438

History

Published in print: June 2017
Published online: 1 June 2017

Permissions

Request permissions for this article.

Topics

Authors

Affiliations

Alexandra Chouldechova* [email protected]
H. John Heinz III College, Carnegie Mellon University, Pittsburgh, Pennsylvania.

Notes

*
Address correspondence to: Alexandra Chouldechova, Heinz College, Carnegie Mellon University, 4800 Forbes Avenue, Pittsburgh, PA 15213, E-mail: [email protected]

Author Disclosure Statement

No competing financial interests exist.

Metrics & Citations

Metrics

Citations

Export citation

Select the format you want to export the citations of this publication.

View Options

Get Access

Access content

To read the fulltext, please use one of the options below to sign in or purchase access.

Society Access

If you are a member of a society that has access to this content please log in via your society website and then return to this publication.

Restore your content access

Enter your email address to restore your content access:

Note: This functionality works only for purchases done as a guest. If you already have an account, log in to access the content to which you are entitled.

View options

PDF/EPUB

View PDF/ePub

Full Text

View Full Text

Media

Figures

Other

Tables

Share

Share

Copy the content Link

Share on social media

Back to Top