The Ephemeral Nature of Wording Effects

Fernando P. Ponce*, David Torres Irribarra, Alvaro Vergés, Victor B. Arias

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

1 Scopus citations

Abstract

This article explores the analysis and interpretation of wording effects associated with using direct and reverse items in psychological assessment. Previous research using bifactor models has suggested a substantive nature of this effect. The present study uses mixture modeling to systematically examine an alternative hypothesis and surpass recognized limitations in the bifactor modeling approach. In preliminary supplemental Studies S1 and S2, we examined the presence of participants who exhibited wording effects and evaluated their impact on the dimensionality of Rosenberg’s Self-Esteem and the Revised Life Orientation Test, confirming the ubiquity of wording effects in scales containing direct and reverse items. Then, after analyzing the data for both scales (n = 5,953), we found that, despite a significant association between wording factors (Study 1), a low proportion of participants simultaneously exhibited asymmetric responses in both scales (Study 2). Similarly, despite finding both longitudinal invariance and temporal stability of this effect in three waves (n = 3,712, Study 3), a small proportion of participants was identified with asymmetric responses over time (Study 4), reflected in lower transition parameters compared to the other patterns of profiles examined. In both cases, we illustrate how bifactor models capitalize on the responses of individuals who do not even exhibit wording effects, yielding spurious correlations suggesting a substantive nature of the wording effect. These findings support the notion of an ephemeral nature underlying wording effects. The discussion focuses on alternative hypotheses to understand these findings and emphasizes the utility of including reverse items in psychological assessment.

Original languageEnglish
Pages (from-to)1472-1494
Number of pages23
JournalJournal of Personality and Social Psychology
Volume125
Issue number6
DOIs
StatePublished - 29 Jun 2023
Externally publishedYes

Bibliographical note

Publisher Copyright:
© 2023 American Psychological Association

Keywords

  • careless responding
  • psychological assessment
  • response style
  • transient mechanisms
  • wording effects

Fingerprint

Dive into the research topics of 'The Ephemeral Nature of Wording Effects'. Together they form a unique fingerprint.

Cite this