Renzo Bianchi, Ph.D., Jay Verkuilen, Ph.D., and Irvin Sam Schonfeld, Ph.D., M.P.H.
Purpose/Objectives
The Occupational Depression Inventory (ODI) assesses work-ascribed depressive symptoms (Bianchi & Schonfeld, 2020). The ODI is designed to help occupational health specialists identify workers suffering from high levels of symptoms that they attribute to their jobs. The specialist can then direct suffering workers to clinicians and take steps to mitigate the stressful job conditions. While evidence indicates that the instrument is reliable and valid, it requires additional investigation. The purpose of this study was to investigate the psychometric and structural properties of the ODI relying on Mokken scale analysis (MSA).
Background
Referenced to the nine DSM-5 symptoms used to diagnose depression, the ODI grades the severity of depressive symptoms that individuals specifically attribute to their work and establishes provisional diagnoses of job-ascribed depression (Bianchi & Schonfeld, 2020). By contrast to standard depression symptom scales are cause-neutral. The ODI was developed in view of growing concerns about the depressogenic effects of intractable job stress (Madsen et al., 2017; McEwen, 2012; Schonfeld & Chang, 2017).
We examined the psychometric and structural properties of the ODI relying on MSA (Mokken, 1971; Molenaar, 1982; van der Ark, 2007, 2012), a method anchored in nonparametric Item Response Theory (IRT). The method has gained popularity in health and clinical research (e.g., Dima et al., 2018; Stochl et al., 2012). MSA, a probabilistic version of Guttman scaling that allows for measurement error, involves ordering hierarchically, i.e., by degree of “difficulty,” items measuring the same latent variable although in the context of the ODI, item “difficulty” reflects the idea that items have different probabilities of being endorsed because different symptoms have different levels of gravity (Meijer & Baneke, 2004). The fatigue item, for example, is likely to be “easier” to endorse than, say, the item referencing suicidal ideation. Fatigue is relatively common whereas suicidal thoughts are much more specific to depression and denote a more severe depressive symptom.
MSA, founded on the Loevinger’s homogeneity (H), or scalability, coefficient, considers items pairwise. Scalability reflects the extent to which endorsement of more “difficult” (more severe symptom) items is related to a higher probability of endorsing “easier” (less severe symptom) items (Dima et al., 2018). H has been viewed as reflecting the accuracy by which items within a scale order the respondents (Sijtsma & Molenaar, 2002). H has been generalized beyond binary items to polytomous ones.
We also inspected monotonicity and invariant item ordering (IIO), two other MSA features (Stochl et al., 2012). Monotonicity implies that, as one moves from the low end of a latent variable to its high end, the probability of endorsing an item should not decrease (Dima et al., 2018), allowing the investigator to order respondents on a latent continuum based on the sum score. IIO implies that items should keep the same order of “difficulty” at all levels of the latent variable (Dima et al., 2018). That the IIO assumption is realistic for health and clinical scales has not been fully elucidated (Meijer & Egberink, 2012).
We aim to clarify the extent which the ODI meets the requirements for scalability, monotonicity, and IIO for the purpose of allowing occupational health specialists to confidently employ the ODI.
To conserve space, literature cited is available upon request.
Methods
The sample, recruited online via Qualtrics, comprised 3,454 education staff members recruited in French K-12 schools (Mage = 45, SDage = 10; 83% women). All participants completed the ODI.
We conducted our MSA using the Mokken package version 3.0.3 (van der Ark, 2007, 2012) in R version 4.0.3 (R Core Team, 2020).
Findings
The ODI’s scalability was high, with the H coefficient substantially exceeding the 0.50 threshold for a strongly homogeneous scale. Item-level Hs were also high; no pairwise H was problematic. No monotonicity violation was detected. Item ordering met the standard for sufficient accuracy (HT = 0.56). We found a small number of IIO violations; none was serious. Total score reliability, as indexed by Guttman’s lambda-2 and the Molenaar-Sijtsma statistic, reached 0.90. The ODI?s suicidal ideation item acted as a sentinel item-its endorsement signaled that the endorser likely had a host of other symptoms.
Discussion
The ODI demonstrated strong scalability, monotonicity, sufficient IIO accuracy, and high total score reliability. The results are consistent with those of Bianchi and Schonfeld’s (2020) initial validation study of the ODI, in which the ODI displayed strong alpha and omega reliabilities, high factorial validity, and essential unidimensionality.
Although the sample was self-selected and limited to one occupational sector, it was large. We subjected the ODI to a rigorous MSA; measures of job-related distress have seldom been subject to MSA. MSA has properties that align better with the way symptom scales are used in practice (e.g., being focused on justifying a total score rather than on the latent variable).
Conclusions
Occupational health specialists can confidently use the ODI. But the ODI should be explored in additional occupational groups.