Katherine A. Werth, Hon. BS, The University of Tennessee at Chattanooga; Christopher J. L. Cunningham, PhD, The University of Tennessee at Chattanooga; Mukta Panda, MD MACP FRCP-London, University of Tennessee College of Medicine-Chattanooga
Purpose/Objectives
The information and data presented in this poster are associated with a larger longitudinal study of resilience in healthcare workers (HW). At the time of this submission, five months of data are collected; in this poster, we will share methodological and psychometric details pertaining to numerous widely used measures of resilience-related individual differences and psychological health and well-being that we adapted for use in longitudinal research. We will also demonstrate the utility of these shortened measures in retaining longitudinal participants while maintaining richness of data. We will also address the following research questions (RQ):
RQ1: Will shortened forms of common measures demonstrate adequate reliability?
RQ2: Do shortened measures present evidence of stability/change over time as expected?
Background
Individual scale and overall survey length are critical considerations when designing any evaluation, as they directly impact participant response rates and retention over time, as well as data quality (Galesic & Bosnjak, 2009; Hoerger, 2010). Respondents are more likely to stay engaged and respond fully to briefer surveys and overly long surveys can actually be detrimental to response quality and participant retention (e.g., Stanton et al., 2002a). Unfortunately, psychometric research suggests measurement reliability may be improved with longer multi-item scales (e.g., Hamby & Peterson, 2016). This paradox is an ongoing source of debate and frustration among researchers of psychosocial phenomena. Increasing research suggests that with careful design, short-form measures can demonstrate reliability and accuracy consistent with longer-form equivalents (e.g., Maloney et al., 2007). Extending from this background, we developed a set of short-form measures for use in repeated assessment of multiple resilience-related individual differences and psychological health and well-being outcomes in a diverse sample of HW.
Methods
Data were collected from a targeted, diverse sample of HW recruited through professional networks, identified by peers and supervisors as consistently demonstrating resilience and/or general excellence in their healthcare-related work. Of 60 invited HW, 54 agreed and completed the baseline survey, 50 completed the first follow-up, and 47 completed the second, third, and fourth follow-ups (average follow-up response rate of 88.42%). Data were gathered via an internet survey including items targeting demographic details, resilience-related individual differences, and health-related outcomes (Table 1 for details). Short-form versions of these measures were also developed by adapting established measures using existing factor loading information and evaluation of item content. These data were mostly gathered before vaccination was available for COVID-19, as part of a larger study of HW resilience that will be complete by late Summer 2021; all data for this presentation have been preliminarily analyzed for this submission. Additional person-level analyses will be included in the actual poster.
Findings
Descriptive statistics for all measures are in Table 1. Short-form measures revealed moderate to high average correlations (interform reliability/consistency) with their long-form equivalents, r = .56-.83 (RQ1; Table 2). Single-item measures had low to moderate average correlations with their long-form counterparts, r = .32-.50 (RQ1; Table 2). Correlations between data gathered at the initial short-form measure administration and subsequent timepoints indicates good stability and test-retest reliability for most measures (Table 3 and Figure 1). Average correlations between shortened scales and the seven-day COVID hospitalization average were negligible on all accounts, r = -.07-.06, but moderate relationships were observed at some time points (RQ2).
Discussion
These data show that shortened scales with two or more items demonstrate adequate interform reliability with their long-form versions and are viable options for use in research and evaluation contexts (particularly ones involving repeated measurements over time). In addition, single-item measures may not be adequate to capture a given construct, though in the case of this research the single-item measures were the only measures not adapted from longer-form measures, instead measures with reliability and validity information were located. In all shortened measures, test-retest reliability was significant. Although nonsignificant average correlations were found between repeated short-form scale scores and average COVID hospitalizations over time, interesting patterns of relationships between our measures and this indicator of COVID severity were observed at certain time points (additional post hoc analyses are still being completed with respect to this point and will be included in the ultimate poster). The consistency of participants’ responses on the measured variables may be due to the highly resilient nature of our sample, which was composed of HW who excelled in healthcare-related work and may not have been as impacted by overwhelming COVID cases as other HW.
Conclusions
Short-form scales with two or more items have evidence of good interform reliability and are a viable option for longitudinal studies and brief surveys to reduce participant fatigue. Further analyses upon completion of data collection will additionally help understand the utility of shortened scales and aid future researchers. Ultimately, we hope this research demonstrates to other researchers the benefits of reducing survey length in attempts to increase participant retention and attention.