Reliability assessment of a hospital quality measure based on rates of adverse outcomes on nursing units.

Document Type

Article

Publication Date

12-1-2017

Identifier

DOI: 10.1177/0962280215618688

Abstract

The purpose of this study was to develop methods for assessing the reliability of scores on a widely disseminated hospital quality measure based on nursing unit fall rates. Poisson regression interactive multilevel modeling was adapted to account for clustering of units within hospitals. Three signal-noise reliability measures were computed. Squared correlations between the hospital score and true hospital fall rate averaged 0.52 ± 0.18 for total falls (0.68 ± 0.18 for injurious falls). Reliabilities on the other two measures averaged at least 0.70 but varied widely across hospitals. Parametric bootstrap data reflecting within-unit noise in falls were generated to evaluate percentile-ranked hospital scores as estimators of true hospital fall rate ranks. Spearman correlations between bootstrap hospital scores and true fall rates averaged 0.81 ± 0.01 (0.79 ± 0.01). Bias was negligible, but ranked hospital scores were imprecise, varying across bootstrap samples with average SD 11.8 (14.9) percentiles. Across bootstrap samples, hospital-measure scores fell in the same decile as the true fall rate in about 30% of cases. Findings underscore the importance of thoroughly assessing reliability of quality measurements before deciding how they will be used. Both the hospital measure and the reliability methods described can be adapted to other contexts involving clustered rates of adverse patient outcomes.

Journal Title

Statistical methods in medical research

Volume

26

Issue

6

First Page

2951

Last Page

2961

MeSH Keywords

Accidental Falls; Biostatistics; Health Personnel; Hospitals; Humans; Inpatients; Models, Statistical; Nursing Care; Patient Safety; Quality Assurance, Health Care; Quality of Health Care; Reproducibility of Results; Signal-To-Noise Ratio

Keywords

Health care quality; quality measurement; reliability

Link to Full Text
Library Record

Share

COinS