Classifying COVID-19 hospitalizations in epidemiology cohort studies: The C4R study

Elizabeth C. Oelsner; Akshaya Krishnaswamy; Rafail Rustamov; Pallavi P. Balte; Tauqeer Ali; Norrina B. Allen; Howard F. Andrews; Pramod Anugu; Alexander Arynchyn; Lori A. Bateman; Jianwen Cai; Harry Chang; Lucas Chen; Mitchell S.V. Elkind; James S. Floyd; Kelley Pettee Gabriel; Sina A. Gharib; Jose D. Gutierrez; Karen Hinckley Stukovsky; Virginia J. Howard; Carmen R. Isasi; Lauren Jager; Ling Jin; Suzanne E. Judd; Alka M. Kanaya; Namratha R. Kandula; Maureen R. Kelly; Sadiya S. Khan; Anna Kucharska-Newton; Joyce S. Lee; Emily B. Levitan; Cora E. Lewis; Barry J. Make; Kimberly Malloy; Jennifer J. Manly; David Mauger; I. Min Yuan; Joanne M. Murabito; Charles G. Murphy; Arnita F. Norwood; George T. O’Connor; Victor E. Ortega; Ashmi A. Patel; Amber Pirzada; Elizabeth A. Regan; Kimberly B. Ring; Wayne D. Rosamond; David A. Schwartz; James M. Shikany; Daniela Sotres-Alvarez; Cheryl Tarlton; Janis Tse; Elman M. Urbina Meneses; Maya Vankineni; Sally E. Wenzel; Prescott G. Woodruff; Vanessa Xanthakis; Ji Hyun Yang; Neil A. Zakai; Ying Zhang; Wendy S. Post

doi:10.1371/journal.pone.0316198

Classifying COVID-19 hospitalizations in epidemiology cohort studies: The C4R study

Elizabeth C. Oelsner, Akshaya Krishnaswamy, Rafail Rustamov, Pallavi P. Balte, Tauqeer Ali, Norrina B. Allen, Howard F. Andrews, Pramod Anugu, Alexander Arynchyn, Lori A. Bateman, Jianwen Cai, Harry Chang, Lucas Chen, Mitchell S.V. Elkind, James S. Floyd, Kelley Pettee Gabriel, Sina A. Gharib, Jose D. Gutierrez, Karen Hinckley Stukovsky, Virginia J. HowardCarmen R. Isasi, Lauren Jager, Ling Jin, Suzanne E. Judd, Alka M. Kanaya, Namratha R. Kandula, Maureen R. Kelly, Sadiya S. Khan, Anna Kucharska-Newton, Joyce S. Lee, Emily B. Levitan, Cora E. Lewis, Barry J. Make, Kimberly Malloy, Jennifer J. Manly, David Mauger, I. Min Yuan, Joanne M. Murabito, Charles G. Murphy, Arnita F. Norwood, George T. O’Connor, Victor E. Ortega, Ashmi A. Patel, Amber Pirzada, Elizabeth A. Regan, Kimberly B. Ring, Wayne D. Rosamond, David A. Schwartz, James M. Shikany, Daniela Sotres-Alvarez, Cheryl Tarlton, Janis Tse, Elman M. Urbina Meneses, Maya Vankineni, Sally E. Wenzel, Prescott G. Woodruff, Vanessa Xanthakis, Ji Hyun Yang, Neil A. Zakai, Ying Zhang, Wendy S. Post

Research output: Contribution to journal › Article › peer-review

Abstract

Rationale Robust COVID-19 outcomes classification is important for ongoing epidemiology research on acute and post-acute COVID-19 conditions. Protocolized medical record review is an established method to validate endpoints for clinical trials and cardiovascular epidemiology cohorts; however, a protocol to adjudicate hospitalizations for COVID-19 among epidemiology cohorts was lacking. Objectives We developed a protocol to ascertain and adjudicate hospitalized COVID-19 across a meta-cohort of 14 US prospective cohort studies. This report describes the first three years of protocol implementation (October 1, 2020—October 1, 2023) and evaluates its repeatability and performance compared to classification by administrative codes. Methods The protocol was adapted from cohort approaches to clinical cardiovascular events ascertainment and adjudication. Potential COVID-19 hospitalizations and deaths were identified by self-/proxy-report and, in some cases, active surveillance. Medical records were requested from hospitals and adjudicated for COVID-19 outcomes by clinically trained personnel according to a standardized rubric. Inter-rater agreement was assessed. The sensitivity and specificity of discharge diagnosis codes was compared to adjudicated diagnoses. Measurements and main results The study obtained medical records for 1,167 potential COVID-19 hospitalizations, which underwent protocolized adjudication. Adjudication confirmed COVID-19 infection was present for 1,030 (88%) events, of which COVID-19 was not the cause of hospitalization for 78 (8%). Of 952 hospitalizations determined by adjudicators to be caused by COVID-19, 319 (34%) participants were critically ill and 210 (22%) died. Pneumonia was confirmed in 822 (86%) and acute kidney injury in 350 (37%); other cardiovascular and thrombotic complications were rare (2–5%). Interrater reliability among adjudicators was high (kappa = 0.85–1.00) except for myocardial infarction (kappa = 0.60). Compared to adjudication, sensitivity of discharge diagnosis codes was higher for pneumonia (84%) and pulmonary embolism (81%) than for other complications (48–70%). Conclusions Protocolized adjudication confirmed four out of five COVID-19 hospitalizations in a US meta-cohort and confirmed cases of pneumonia, pulmonary embolism, and other conditions that were not indicated by discharge diagnosis codes. These results highlight the importance of validating health outcomes for use in research on COVID-19 and post-COVID-19 conditions, and some limitations of claims-based data.

Original language	English (US)
Article number	e0316198
Journal	PloS one
Volume	20
Issue number	2 February
DOIs	https://doi.org/10.1371/journal.pone.0316198
State	Published - Feb 2025

All Science Journal Classification (ASJC) codes

General

Access to Document

10.1371/journal.pone.0316198

Cite this

Oelsner, E. C., Krishnaswamy, A., Rustamov, R., Balte, P. P., Ali, T., Allen, N. B., Andrews, H. F., Anugu, P., Arynchyn, A., Bateman, L. A., Cai, J., Chang, H., Chen, L., Elkind, M. S. V., Floyd, J. S., Gabriel, K. P., Gharib, S. A., Gutierrez, J. D., Stukovsky, K. H., ... Post, W. S. (2025). Classifying COVID-19 hospitalizations in epidemiology cohort studies: The C4R study. PloS one, 20(2 February), Article e0316198. https://doi.org/10.1371/journal.pone.0316198

@article{1bcd893e2aec4da181e447fd9710ef3a,

title = "Classifying COVID-19 hospitalizations in epidemiology cohort studies: The C4R study",

abstract = "Rationale Robust COVID-19 outcomes classification is important for ongoing epidemiology research on acute and post-acute COVID-19 conditions. Protocolized medical record review is an established method to validate endpoints for clinical trials and cardiovascular epidemiology cohorts; however, a protocol to adjudicate hospitalizations for COVID-19 among epidemiology cohorts was lacking. Objectives We developed a protocol to ascertain and adjudicate hospitalized COVID-19 across a meta-cohort of 14 US prospective cohort studies. This report describes the first three years of protocol implementation (October 1, 2020—October 1, 2023) and evaluates its repeatability and performance compared to classification by administrative codes. Methods The protocol was adapted from cohort approaches to clinical cardiovascular events ascertainment and adjudication. Potential COVID-19 hospitalizations and deaths were identified by self-/proxy-report and, in some cases, active surveillance. Medical records were requested from hospitals and adjudicated for COVID-19 outcomes by clinically trained personnel according to a standardized rubric. Inter-rater agreement was assessed. The sensitivity and specificity of discharge diagnosis codes was compared to adjudicated diagnoses. Measurements and main results The study obtained medical records for 1,167 potential COVID-19 hospitalizations, which underwent protocolized adjudication. Adjudication confirmed COVID-19 infection was present for 1,030 (88%) events, of which COVID-19 was not the cause of hospitalization for 78 (8%). Of 952 hospitalizations determined by adjudicators to be caused by COVID-19, 319 (34%) participants were critically ill and 210 (22%) died. Pneumonia was confirmed in 822 (86%) and acute kidney injury in 350 (37%); other cardiovascular and thrombotic complications were rare (2–5%). Interrater reliability among adjudicators was high (kappa = 0.85–1.00) except for myocardial infarction (kappa = 0.60). Compared to adjudication, sensitivity of discharge diagnosis codes was higher for pneumonia (84%) and pulmonary embolism (81%) than for other complications (48–70%). Conclusions Protocolized adjudication confirmed four out of five COVID-19 hospitalizations in a US meta-cohort and confirmed cases of pneumonia, pulmonary embolism, and other conditions that were not indicated by discharge diagnosis codes. These results highlight the importance of validating health outcomes for use in research on COVID-19 and post-COVID-19 conditions, and some limitations of claims-based data.",

author = "Oelsner, {Elizabeth C.} and Akshaya Krishnaswamy and Rafail Rustamov and Balte, {Pallavi P.} and Tauqeer Ali and Allen, {Norrina B.} and Andrews, {Howard F.} and Pramod Anugu and Alexander Arynchyn and Bateman, {Lori A.} and Jianwen Cai and Harry Chang and Lucas Chen and Elkind, {Mitchell S.V.} and Floyd, {James S.} and Gabriel, {Kelley Pettee} and Gharib, {Sina A.} and Gutierrez, {Jose D.} and Stukovsky, {Karen Hinckley} and Howard, {Virginia J.} and Isasi, {Carmen R.} and Lauren Jager and Ling Jin and Judd, {Suzanne E.} and Kanaya, {Alka M.} and Kandula, {Namratha R.} and Kelly, {Maureen R.} and Khan, {Sadiya S.} and Anna Kucharska-Newton and Lee, {Joyce S.} and Levitan, {Emily B.} and Lewis, {Cora E.} and Make, {Barry J.} and Kimberly Malloy and Manly, {Jennifer J.} and David Mauger and Yuan, {I. Min} and Murabito, {Joanne M.} and Murphy, {Charles G.} and Norwood, {Arnita F.} and O{\textquoteright}Connor, {George T.} and Ortega, {Victor E.} and Patel, {Ashmi A.} and Amber Pirzada and Regan, {Elizabeth A.} and Ring, {Kimberly B.} and Rosamond, {Wayne D.} and Schwartz, {David A.} and Shikany, {James M.} and Daniela Sotres-Alvarez and Cheryl Tarlton and Janis Tse and {Urbina Meneses}, {Elman M.} and Maya Vankineni and Wenzel, {Sally E.} and Woodruff, {Prescott G.} and Vanessa Xanthakis and Yang, {Ji Hyun} and Zakai, {Neil A.} and Ying Zhang and Post, {Wendy S.}",

note = "Publisher Copyright: {\textcopyright} 2025 Oelsner et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.",

year = "2025",

month = feb,

doi = "10.1371/journal.pone.0316198",

language = "English (US)",

volume = "20",

journal = "PloS one",

issn = "1932-6203",

publisher = "Public Library of Science",

number = "2 February",

}

Oelsner, EC, Krishnaswamy, A, Rustamov, R, Balte, PP, Ali, T, Allen, NB, Andrews, HF, Anugu, P, Arynchyn, A, Bateman, LA, Cai, J, Chang, H, Chen, L, Elkind, MSV, Floyd, JS, Gabriel, KP, Gharib, SA, Gutierrez, JD, Stukovsky, KH, Howard, VJ, Isasi, CR, Jager, L, Jin, L, Judd, SE, Kanaya, AM, Kandula, NR, Kelly, MR, Khan, SS, Kucharska-Newton, A, Lee, JS, Levitan, EB, Lewis, CE, Make, BJ, Malloy, K, Manly, JJ, Mauger, D, Yuan, IM, Murabito, JM, Murphy, CG, Norwood, AF, O’Connor, GT, Ortega, VE, Patel, AA, Pirzada, A, Regan, EA, Ring, KB, Rosamond, WD, Schwartz, DA, Shikany, JM, Sotres-Alvarez, D, Tarlton, C, Tse, J, Urbina Meneses, EM, Vankineni, M, Wenzel, SE, Woodruff, PG, Xanthakis, V, Yang, JH, Zakai, NA, Zhang, Y & Post, WS 2025, 'Classifying COVID-19 hospitalizations in epidemiology cohort studies: The C4R study', PloS one, vol. 20, no. 2 February, e0316198. https://doi.org/10.1371/journal.pone.0316198

TY - JOUR

T1 - Classifying COVID-19 hospitalizations in epidemiology cohort studies

T2 - The C4R study

AU - Oelsner, Elizabeth C.

AU - Krishnaswamy, Akshaya

AU - Rustamov, Rafail

AU - Balte, Pallavi P.

AU - Ali, Tauqeer

AU - Allen, Norrina B.

AU - Andrews, Howard F.

AU - Anugu, Pramod

AU - Arynchyn, Alexander

AU - Bateman, Lori A.

AU - Cai, Jianwen

AU - Chang, Harry

AU - Chen, Lucas

AU - Elkind, Mitchell S.V.

AU - Floyd, James S.

AU - Gabriel, Kelley Pettee

AU - Gharib, Sina A.

AU - Gutierrez, Jose D.

AU - Stukovsky, Karen Hinckley

AU - Howard, Virginia J.

AU - Isasi, Carmen R.

AU - Jager, Lauren

AU - Jin, Ling

AU - Judd, Suzanne E.

AU - Kanaya, Alka M.

AU - Kandula, Namratha R.

AU - Kelly, Maureen R.

AU - Khan, Sadiya S.

AU - Kucharska-Newton, Anna

AU - Lee, Joyce S.

AU - Levitan, Emily B.

AU - Lewis, Cora E.

AU - Make, Barry J.

AU - Malloy, Kimberly

AU - Manly, Jennifer J.

AU - Mauger, David

AU - Yuan, I. Min

AU - Murabito, Joanne M.

AU - Murphy, Charles G.

AU - Norwood, Arnita F.

AU - O’Connor, George T.

AU - Ortega, Victor E.

AU - Patel, Ashmi A.

AU - Pirzada, Amber

AU - Regan, Elizabeth A.

AU - Ring, Kimberly B.

AU - Rosamond, Wayne D.

AU - Schwartz, David A.

AU - Shikany, James M.

AU - Sotres-Alvarez, Daniela

AU - Tarlton, Cheryl

AU - Tse, Janis

AU - Urbina Meneses, Elman M.

AU - Vankineni, Maya

AU - Wenzel, Sally E.

AU - Woodruff, Prescott G.

AU - Xanthakis, Vanessa

AU - Yang, Ji Hyun

AU - Zakai, Neil A.

AU - Zhang, Ying

AU - Post, Wendy S.

N1 - Publisher Copyright: © 2025 Oelsner et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

PY - 2025/2

Y1 - 2025/2

N2 - Rationale Robust COVID-19 outcomes classification is important for ongoing epidemiology research on acute and post-acute COVID-19 conditions. Protocolized medical record review is an established method to validate endpoints for clinical trials and cardiovascular epidemiology cohorts; however, a protocol to adjudicate hospitalizations for COVID-19 among epidemiology cohorts was lacking. Objectives We developed a protocol to ascertain and adjudicate hospitalized COVID-19 across a meta-cohort of 14 US prospective cohort studies. This report describes the first three years of protocol implementation (October 1, 2020—October 1, 2023) and evaluates its repeatability and performance compared to classification by administrative codes. Methods The protocol was adapted from cohort approaches to clinical cardiovascular events ascertainment and adjudication. Potential COVID-19 hospitalizations and deaths were identified by self-/proxy-report and, in some cases, active surveillance. Medical records were requested from hospitals and adjudicated for COVID-19 outcomes by clinically trained personnel according to a standardized rubric. Inter-rater agreement was assessed. The sensitivity and specificity of discharge diagnosis codes was compared to adjudicated diagnoses. Measurements and main results The study obtained medical records for 1,167 potential COVID-19 hospitalizations, which underwent protocolized adjudication. Adjudication confirmed COVID-19 infection was present for 1,030 (88%) events, of which COVID-19 was not the cause of hospitalization for 78 (8%). Of 952 hospitalizations determined by adjudicators to be caused by COVID-19, 319 (34%) participants were critically ill and 210 (22%) died. Pneumonia was confirmed in 822 (86%) and acute kidney injury in 350 (37%); other cardiovascular and thrombotic complications were rare (2–5%). Interrater reliability among adjudicators was high (kappa = 0.85–1.00) except for myocardial infarction (kappa = 0.60). Compared to adjudication, sensitivity of discharge diagnosis codes was higher for pneumonia (84%) and pulmonary embolism (81%) than for other complications (48–70%). Conclusions Protocolized adjudication confirmed four out of five COVID-19 hospitalizations in a US meta-cohort and confirmed cases of pneumonia, pulmonary embolism, and other conditions that were not indicated by discharge diagnosis codes. These results highlight the importance of validating health outcomes for use in research on COVID-19 and post-COVID-19 conditions, and some limitations of claims-based data.

AB - Rationale Robust COVID-19 outcomes classification is important for ongoing epidemiology research on acute and post-acute COVID-19 conditions. Protocolized medical record review is an established method to validate endpoints for clinical trials and cardiovascular epidemiology cohorts; however, a protocol to adjudicate hospitalizations for COVID-19 among epidemiology cohorts was lacking. Objectives We developed a protocol to ascertain and adjudicate hospitalized COVID-19 across a meta-cohort of 14 US prospective cohort studies. This report describes the first three years of protocol implementation (October 1, 2020—October 1, 2023) and evaluates its repeatability and performance compared to classification by administrative codes. Methods The protocol was adapted from cohort approaches to clinical cardiovascular events ascertainment and adjudication. Potential COVID-19 hospitalizations and deaths were identified by self-/proxy-report and, in some cases, active surveillance. Medical records were requested from hospitals and adjudicated for COVID-19 outcomes by clinically trained personnel according to a standardized rubric. Inter-rater agreement was assessed. The sensitivity and specificity of discharge diagnosis codes was compared to adjudicated diagnoses. Measurements and main results The study obtained medical records for 1,167 potential COVID-19 hospitalizations, which underwent protocolized adjudication. Adjudication confirmed COVID-19 infection was present for 1,030 (88%) events, of which COVID-19 was not the cause of hospitalization for 78 (8%). Of 952 hospitalizations determined by adjudicators to be caused by COVID-19, 319 (34%) participants were critically ill and 210 (22%) died. Pneumonia was confirmed in 822 (86%) and acute kidney injury in 350 (37%); other cardiovascular and thrombotic complications were rare (2–5%). Interrater reliability among adjudicators was high (kappa = 0.85–1.00) except for myocardial infarction (kappa = 0.60). Compared to adjudication, sensitivity of discharge diagnosis codes was higher for pneumonia (84%) and pulmonary embolism (81%) than for other complications (48–70%). Conclusions Protocolized adjudication confirmed four out of five COVID-19 hospitalizations in a US meta-cohort and confirmed cases of pneumonia, pulmonary embolism, and other conditions that were not indicated by discharge diagnosis codes. These results highlight the importance of validating health outcomes for use in research on COVID-19 and post-COVID-19 conditions, and some limitations of claims-based data.

UR - http://www.scopus.com/inward/record.url?scp=85217476142&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85217476142&partnerID=8YFLogxK

U2 - 10.1371/journal.pone.0316198

DO - 10.1371/journal.pone.0316198

M3 - Article

C2 - 39928595

AN - SCOPUS:85217476142

SN - 1932-6203

VL - 20

JO - PloS one

JF - PloS one

IS - 2 February

M1 - e0316198

ER -

Classifying COVID-19 hospitalizations in epidemiology cohort studies: The C4R study

Abstract

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this