Keywords
Correlated residuals
Response biases Aquiescencia
Residuales correlacionados
Sesgos de respuesta
How to Cite
Abstract
Background: In spite of the interest generated by controlling variance unrelated to the content in personality measures,
few studies have assessed the combined impact of acquiescence (ACQ) and correlated residuals in the factor-analytic
structural solutions that serve as a basis for item calibration. Method: This article compares three control procedures
in databases that simultaneously exhibit acquiescence (ACQ) and correlated residuals: the SIREN method (NavarroGonzalez, et al., 2024; ACQ control), the MORGANA method (Ferrando et al., 2022; 2023; correlated residuals
control), and a combined double control method. A ‘control’ procedure was also examined in which the presence of
both determinants was ignored. Results: The findings reveal significant differences between the three control methods,
with the ACQ control method and the combined method proving more effective. Moreover, when the residual variance
shifts to the factor loadings, it appears to be captured by the ACQ correction method. Conclusions: The findings
suggest using a mixed correction procedure in order to obtain less biased item parameter estimates when both sources
of unwanted variance are operating.
References
Aichholzer, J. (2014). Random intercept EFA of personality scales. Journal of Research in Personality, 53, 1-4. https://doi.org/10.1016/j. jrp.2014.07.001
Arias, V. B., Ponce, F. P., Garrido, L. E., Nieto-Cañaveras, M. D., Martínez- Molina, A., & Arias, B. (2024). Detecting non-content-based response styles in survey data: An application of mixture factor analysis. Behavior Research Methods, 56, 3242–3258. https://doi.org/10.3758/ s13428-023-02308-w
Baumgartner, H., & Steenkamp, J. B. E. (2001). Response styles in marketing research: A cross-national investigation. Journal of Marketing Research, 38(2), 143-156. https://doi.org/10.1509/jmkr.38.2.143.18840 Bentler, P. M., Jackson, D. N., & Messick, S. (1971). Identification of content and style: A two-dimensional interpretation of acquiescence. Psychological Bulletin, 76(3), 186-204. https://doi.org/10.1037/h0031474
Billiet, J. B & Davidov, E. (2008). Testing the stability of an acquiescence style factor behind two interrelated substantive variables in a panel design. Sociological Methods Research, 36(4), 542-562. https://doi. org/10.1177/0049124107313901
Billiet J. B. & McClendon M. K. (2000). Modeling acquiescence in measurement models for two balanced sets of items. Structural Equation Modeling: A Multidisciplinary Journal, 7(4), 608-628. http:// doi.org/1207/S15328007SEM0704_5
Chan, W., & Bentler, P. M. (1993). The covariance structure analysis of ipsative data. Sociological Methods & Research, 22(2), 214-247. https://doi.org/10.1177/0049124193022002003
Cheung, G. W., & Rensvold, R. B. (2000). Assessing extreme and acquiescence response sets in cross-cultural research using structural equations modeling. Journal of Cross-cultural Psychology, 31(2), 187- 212. https://doi.org/10.1177/0022022100031002003
Christensen, A. P., Garrido, L. E., & Golino, H. (2023). Unique variable analysis: A network psychometrics method to detect local dependence. Multivariate Behavioral Research, 58(6), 1165-1182. https://doi.org/10.1080/00273171.2023.2194606
Cronbach, L. J. (1946). Response sets and test validity. Educational and Psychological Measurement, 6(4), 475-494. https://doi.org/10.1177/001316444600600405
Danner, D., Aichholzer, J., & Rammstedt, B. (2015). Acquiescence in personality questionnaires: Relevance, domain specificity, and stability. Journal of Research in Personality, 57, 119-130. https://doi.org/10.1016/j.jrp.2015.05.004
de la Fuente, J., & Abad, F. J. (2020). Comparing methods for modeling acquiescence in multidimensional partially balanced scales. Psicothema 32(4), 590-597. http://doi.org/10.7334/psicothema2020.96
Ferrando, P. J., & Anguiano-Carrasco, C. (2010). Acquiescence and social desirability as item response determinants: An IRT-based study with the Marlowe–Crowne and the EPQ Lie scales. Personality and Individual Differences, 48(5), 596-600. https://doi.org/10.1016/j.paid.2009.12.013
Ferrando, P. J., Hernandez-Dorado, A., & Lorenzo-Seva, U. (2022). Detecting correlated residuals in exploratory factor analysis: New proposals and a comparison of procedures. Structural Equation Modeling: A Multidisciplinary Journal, 29(4), 630-638. https://doi.org/10.1080/10705511.2021.2004543
Ferrando, P. J., Hernández-Dorado, A., & Lorenzo-Seva, U. (2023). A simple two-step procedure for fitting fully unrestricted exploratory factor analytic solutions with correlated residuals. Structural Equation Modeling: A Multidisciplinary Journal, 31(3), 420–428. https://doi.org/10.1080/10705511.2023.2267181
Ferrando, P. J., & Lorenzo-Seva, U. (2010). Acquiescence as a source of bias and model and person misfit: A theoretical and empirical analysis. British Journal of Mathematical and Statistical Psychology, 63(2), 427- 448. https://doi.org/10.1348/000711009X470740
Ferrando, P. J., Lorenzo-Seva, U., & Chico, E. (2003). Unrestricte factor analytic procedures for assessing acquiescent responding in balanced, theoretically unidimensional personality scales. Multivariate Behavioral Research, 38(3), 353-374. https://doi.org/10.1207/ S15327906MBR3803_04
Ferrando, P. J., Lorenzo-Seva, U., & Chico, E. (2009). A general factor- analytic procedure for assessing response bias in questionnaire measures. Structural Equation Modeling: A Multidisciplinary Journal, 16(2), 364-381. https://doi.org/10.1080/10705510902751374
Ferrando, P. J., Morales-Vives, F., & Lorenzo-Seva, U. (2016). Assessing and controlling acquiescent responding when acquiescence and content are related: A comprehensive factor-analytic approach. Structural Equation Modeling: A Multidisciplinary Journal, 23(5), 713–725. https://doi.org/10.1080/10705511.2016.1185723
Hand, J., & Brazzell, C. O. (1965). Contamination in measures of acquiescence and social desirability. Psychological Reports, 16(3), 759- 760. https://doi.org/10.2466/pr0.1965.16.3.759
Harman, H. H. (1962). Modern Factor Analysis (2a ed.). University of Chicago Press. Hinz, A., Michalski, D., Schwarz, R., & Herzberg, P. Y. (2007). The acquiescence effect in responding to a questionnaire. GMS Psycho- Social Medicine, 4, 1-9. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2736523/pdf/PSM-04-07.pdf
Krosnick, J. A. (1999). Maximizing questionnaire quality. In J. P. Robinson, P. R. Shaver, & L. S. Wrightsman (Eds.), Measures of Political Attitudes
(pp. 37–57). Academic Press.
Lorenzo-Seva, U., & Ferrando, P.J. (2009). Acquiescent responding in partially balanced multidimensional scales. British Journal of Mathematical and Statistical Psychology, 62(2), 319-326. https://doi.org/10.1348/000711007X265164
Lorenzo-Seva, U., & Rodríguez-Fornells, A. (2006). Acquiescent responding in balanced multidimensional scales and exploratory factor analysis. Psychometrika, 71(4), 769-777. https://doi.org/10.1007/ s11336-004-1207-4
Maydeu-Olivares, A., & Coffman, D. L. (2006). Random intercept item factor analysis. Psychological Methods, 11(4), 344–362. https://doi.org/10.1037/1082-989X.11.4.344
Messick, S. (1995). Standards of validity and the validity of standards in performance asessment. Educational measurement: Issues and Practice, 14(4), 5-8. https://doi.org/10.1111/j.1745-3992.1995. tb00881.x
Montoya, A. K., & Edwards, M. C. (2021). The poor fit of model fit for selecting number of factors in exploratory factor analysis for scale evaluation. Educational and Psychological Measurement, 81(3), 413– 440. https://doi.org/10.1177/0013164420942899
Morales-Vives, F., Cosi, S., Lorenzo-Seva, U., & Vigil-Colet, A. (2019). The inventory of callous-unemotional traits and antisocial behavior (inca) for young people: Development and validation in a community sample. Frontiers in Psychology, 10, Article 713. https://doi. org/10.3389/fpsyg.2019.00713
Navarro-González, D., Lorenzo-Seva, U., & Vigil-Colet, A. (2016). How response bias affects the factorial structure of personality self- reports. Psicothema, 28(4), 465-470. http://www.doi.org/10.7334/ psicothema2016.113
Navarro-Gonzalez, D., Ferrando, P. J., Morales-Vives, F., Hernandez- Dorado, A., & Navarro-Gonzalez, M. D. (2023). Package ‘siren’ (version 1.0.4) [Software package]. https://cran.r-project.org/web/ packages/siren/index.html
Navarro-Gonzalez, D., Ferrando, P. J., Morales-Vives, F., & Hernandez- Dorado, A. (2024). SIREN: A hybrid CFA-EFA R package for controlling acquiescence in restricted factorial [Manuscript submitted for publication]. Department of Psychology, University of Rovira i Virgili.
Pan, J., Ip, E. H., & Dubé, L. (2017). An alternative to post hoc model modification in confirmatory factor analysis: The Bayesian lasso. Psychological methods, 22(4), 687-704. https://doi.org/10.1037/ met0000112
Park, M., & Wu, A. D. (2019). Item Response Tree Models to Investigate Acquiescence and Extreme Response Styles in Likert-Type Rating Scales. Educational and Psychological Measurement, 79(5), 911–930. https://doi.org/10.1177/0013164419829855
Podsakoff, P.M, MacKenzie, S.B., Lee, J., & Podsakoff, N.P. (2003). Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology, 885(879), 1010-1037. https://doi.org/10.1037/0021-9010.88.5.879
Primi, R., Santos, D., De Fruyt, F., & John, O. P. (2019). Comparison of classical and modern methods for measuring and correcting for acquiescence. British Journal of Mathematical and Statistical Psychology, 72(3), 447-465. https://doi.org/10.1111/bmsp.12168
Ray, J. J. (1979). Is the acquiescent response style not so mythical after
all? Some results from a successful balanced F scale. Journal of Personality Assessment, 43(6), 638–643. https://doi.org/10.1207/ s15327752jpa4306_14
Ray, J. J. (1983). Reviving the problem of acquiescent response bias. Journal of Social Psychology, 121(1), 81–96. https://doi.org/10.1080/00224545.1983.9924470
Revelle, M. W. (2015). Package ‘psych’. The Comprehensive R Archive Network, 337, Article 338. https://mirror.ibcp.fr/pub/CRAN/web/packages/psych/psych.pdf
Saris, W. E., Satorra, A., & Sörbom, D. (1987). The detection and correction of specification errors in structural equation models. Sociological Methodology, 105-129. https://doi.org/10.2307/271030
Saris, W. E., Satorra, A., & Van der Veld, W. M. (2010). Testing structural equation models or detection of misspecifications? Structural Equation Modeling, 16, 561–582. https://doi.org/https://doi. org/10.1080/10705510903203433
Savalei, V., & Falk, C. F. (2014). Recovering substantive factor loadings in the presence of acquiescence bias: A comparison of three approaches. Multivariate Behavioral Research, 49(5), 407-424. https:// doi.org/10.1080/00273171.2014.931800
Vigil-Colet, A., Morales-Vives, F., Camps, E., Tous, J., & Lorenzo-Seva,
U. (2013). Development and validation of the overall personality assessment scale (OPERAS). Psicothema, 25(1), 100-106. https://www.redalyc.org/pdf/727/72725690017.pdf
Vigil-Colet, A., Navarro-González, D., & Morales-Vives, F. (2020). To reverse or to not reverse Likert-type items: That is the question. Psicothema, 32(1), 108-114. https://doi.org/10.7334/ psicothema2019.286
Watson, D. (1992). Correcting for acquiescent response bias in the absence of a balanced scale: An application to class consciousness. Sociological Methods & Research, 21(1), 52-88. https:// doi.org/10.1177/0049124192021001003
Weijters, B.; Geuens, M.; Schillewaert, N. (2010). The individual consistency of acquiescence and extreme response style in self-report questionnaires. Applied Psychological Measurement, 34(2), 105 -121. https://doi.org/10.1177/0146621609338593