Resource-Aware Fair Conformal Prediction for Stu-dent Early Warning Systems
Keywords:
early warning systems, conformal prediction, algo-rithmic fairness, resource allocation, learning to defer, educational data miningAbstract
Early-warning systems in schools and universities increasingly attach distribution-free guarantees to their decisions, bounding within every demographic group how often at-risk students are missed and how often students on track are flagged. Equalized conformal decision layers achieve this by referring ambiguous students to counselors, but they treat counselor workload as a by-product: at a 10% error level they refer 32% to 56% of a cohort, and because empty prediction sets are referred as well, relaxing the error level can even increase the workload. We propose Resource-Aware Fair Conformal Prediction (RAFCP), which takes the review capacity as an input and returns the strongest equalized guarantee that this capacity can support. RAFCP resolves empty prediction sets automatically, which pre-serves the group-level miss and false-flag bounds and makes re-ferrals nested in the error level; it certifies the level with a boot-strap capacity planner that uses only cross-fitted training scores, so finite-sample validity is retained; and it reconciles the plan with the screened cohort so that exactly the budgeted number of students is referred. We further prove that equal error levels across groups are minimax-optimal under any monotone capacity constraint. On three public U.S. cohorts (Tennessee STAR, High School and Beyond, and the LSAC Bar Passage Study) with 30 stratified splits, RAFCP lowers the worst-group automated error at a 10% referral budget from 0.43, 0.38, and 0.58 under confidence-based deferral to 0.19, 0.28, and 0.22, keeps between-group gaps below 0.02, and outperforms each deferral baseline in at least 27 of 30 paired splits.
References
F. Zhou, “Moment-constrained side-force allocation for wind-assisted merchant ships: Hydrodynamic performance of retractable anti-leeway fins and flap rudders,” Innovation and Technology Studies, vol. 3, no. 1, pp. 27–41, 2026.
H. Lakkaraju, E. Aguiar, C. Shan, D. Miller, N. Bhanpuri, R. Ghani, and K. L. Addison, “A machine learning framework to identify students at risk of adverse academic outcomes,” in Proc. 21st ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, 2015, pp. 1909–1918, doi: 10.1145/2783258.2788620.
R. Yu, H. Lee, and R. F. Kizilcec, “Should college dropout prediction models include protected attributes?” in Proc. 8th ACM Conf. Learning @ Scale, 2021, pp. 91–100, doi: 10.1145/3430895.3460139.
R. S. Baker and A. Hawn, “Algorithmic bias in education,” Int. J. Artificial Intelligence in Education, vol. 32, no. 4, pp. 1052–1092, 2022, doi: 10.1007/s40593-021-00285-9.
J. Gardner, C. Brooks, and R. Baker, “Evaluating the fairness of predictive student models through slicing analysis,” in Proc. 9th Int. Conf. Learning Analytics & Knowledge, 2019, pp. 225–234, doi: 10.1145/3303772.3303791.
J. Kleinberg, S. Mullainathan, and M. Raghavan, “Inherent trade-offs in the fair determination of risk scores,” in Proc. 8th Innovations in Theoretical Computer Science Conf. (ITCS), LIPIcs, vol. 67, 2017, pp. 43:1–43:23, doi: 10.4230/LIPIcs.ITCS.2017.43.
G. Pleiss, M. Raghavan, F. Wu, J. Kleinberg, and K. Q. Weinberger, “On fairness and calibration,” in Advances in Neural Information Processing Systems, vol. 30, 2017, pp. 5680–5689.
M. Hardt, E. Price, and N. Srebro, “Equality of opportunity in supervised learning,” in Advances in Neural Information Processing Systems, vol. 29, 2016, pp. 3315–3323.
V. Vovk, A. Gammerman, and G. Shafer, Algorithmic Learning in a Random World. New York, NY, USA: Springer, 2005, doi: 10.1007/b106715.
X. Yue and R. Zhu, “An explainable hybrid machine learning framework for cash-flow-conditioned multi-horizon liquidity risk early warning: An SME-oriented benchmark study,” World Journal of Management Science, vol. 2, no. 1, pp. 63–72, 2024.
V. Vovk, “Conditional validity of inductive conformal predictors,” in Proc. Asian Conf. Machine Learning, PMLR, vol. 25, 2012, pp. 475–490.
M. Sadinle, J. Lei, and L. Wasserman, “Least ambiguous set-valued classifiers with bounded error levels,” J. American Statistical Association, vol. 114, no. 525, pp. 223–234, 2019, doi: 10.1080/01621459.2017.1395341.
Y. Romano, R. F. Barber, C. Sabatti, and E. J. Candès, “With malice toward none: Assessing uncertainty via equalized coverage,” Harvard Data Science Review, vol. 2, no. 2, 2020, doi: 10.1162/99608f92.03f00592.
E. Jones, S. Sagawa, P. W. Koh, A. Kumar, and P. Liang, “Selective classification can magnify disparities across groups,” in Proc. Int. Conf. Learning Representations (ICLR), 2021.
D. Teng, “TRIAD: Deadline- and reliability-aware scheduling for post-earthquake structural assessment workflows,” World Journal of Engineering Research, vol. 3, no. 6, pp. 55–66, 2025.
A. N. Angelopoulos, S. Bates, A. Fisch, L. Lei, and T. Schuster, “Conformal risk control,” in Proc. Int. Conf. Learning Representations (ICLR), 2024.
Y. Jin and E. J. Candès, “Selection by prediction with conformal p-values,” J. Machine Learning Research, vol. 24, no. 244, pp. 1–41, 2023.
D. Madras, T. Pitassi, and R. Zemel, “Predict responsibly: Improving fairness and accuracy by learning to defer,” in Advances in Neural Information Processing Systems, vol. 31, 2018, pp. 6147–6157.
H. Mozannar and D. Sontag, “Consistent estimators for learning to defer to an expert,” in Proc. 37th Int. Conf. Machine Learning, PMLR, vol. 119, 2020, pp. 7076–7087.
E. Straitouri, L. Wang, N. Okati, and M. Gomez Rodriguez, “Improving expert predictions with conformal prediction,” in Proc. 40th Int. Conf. Machine Learning, PMLR, vol. 202, 2023, pp. 32633–32653.
Q. Fang, “TRUE-EWS: Trustworthy early warning with equalized conformal guarantees for equitable student support in U.S. education,” Educational Research and Human Development, vol. 1, no. 2, pp. 44–54, 2024, doi: 10.61784/erhd4059.
M. Kull, T. Silva Filho, and P. Flach, “Beta calibration: A well-founded and easily implemented improvement on logistic calibration for binary classifiers,” in Proc. 20th Int. Conf. Artificial Intelligence and Statistics, PMLR, vol. 54, 2017, pp. 623–631.
J. Chen, X. Zhang, and X. Han, “Explainable identification of tax-related financial reporting anomalies in U.S. public companies: Cross-fitted tax-consistency residuals and Owen-value Shapley propagation,” World Journal of Economics and Business Research, vol. 1, no. 1, pp. 33–45, 2023.
A. B. Krueger, “Experimental estimates of education production functions,” Quarterly J. Economics, vol. 114, no. 2, pp. 497–532, 1999, doi: 10.1162/003355399556052.
C. E. Rouse, “Democratization or diversion? The effect of community colleges on educational attainment,” J. Business & Economic Statistics, vol. 13, no. 2, pp. 217–224, 1995.
C. Kleiber and A. Zeileis, Applied Econometrics with R. New York, NY, USA: Springer, 2008, doi: 10.1007/978-0-387-77318-6.
J. Wang, Y. Ge, J. Liu, and B. Jiang, “Early warning of liquidity stress in U.S. banks with seasonally aligned frailty hazards and delayed-feedback conformal alarms,” Journal of Trends in Finance and Economics, vol. 1, no. 1, pp. 85–100, 2024.
T. Le Quy, A. Roy, V. Iosifidis, W. Zhang, and E. Ntoutsi, “A survey on datasets for fairness-aware machine learning,” WIREs Data Mining and Knowledge Discovery, vol. 12, no. 3, Art. no. e1452, 2022, doi: 10.1002/widm.1452.
G. Ke, Q. Meng, T. Finley, T. Wang, W. Chen, W. Ma, Q. Ye, and T.-Y. Liu, “LightGBM: A highly efficient gradient boosting decision tree,” in Advances in Neural Information Processing Systems, vol. 30, 2017, pp. 3146–3154.
H. Weerts, M. Dudík, R. Edgar, A. Jalali, R. Lutz, and M. Madaio, “Fairlearn: Assessing and improving fairness of AI systems,” J. Machine Learning Research, vol. 24, no. 257, pp. 1–8, 2023.
J. Zou, E. Feng, P. Li, and G. Wang, “Horizon-aware group-conditional conformal prediction: Post-hoc shrinkage for sparse multi-horizon coverage auditing,” Machine Learning with Applications, vol. 26, art. 101038, 2026, doi: 10.1016/j.mlwa.2026.101038.
J. Chen et al., “KE-MLLM: A Knowledge-Enhanced Multi-Sensor Learning Framework for Explainable Fake Review Detection,” Applied Sciences, vol. 16, art. 2909, 2026, doi: 10.3390/app16062909.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Katherine Price, Daria Iyer, Julia Mitchell
This work is licensed under a Creative Commons Attribution 4.0 International License.
All articles published in the Journal of Social Sciences, Humanities and Innovation (JSSHI) are licensed under the Creative Commons Attribution 4.0 International License (CC BY 4.0).
This license permits unrestricted use, distribution, reproduction, and adaptation in any medium or format, provided that the original author(s) and source are properly credited. Users are allowed to copy, redistribute, remix, transform, and build upon the material for any purpose, including commercial use, as long as appropriate attribution is given and any changes made are indicated.
Authors retain copyright of their work while granting the journal the right of first publication. The journal supports open access principles and encourages the wide dissemination of scholarly research to promote knowledge sharing and academic collaboration.
For more information about the license, visit: https://creativecommons.org/licenses/by/4.0/
