Palabras clave
Scale
Studies based on observational
methodology
Validity evidence
Reliability Calidad metodológica
Escala
Estudios basados en metodología
observacional
Evidencias de validez
Fiabilidad
Cómo citar
Resumen
Antecedentes: Hasta donde conocemos, hay escasa evidencia de validez de escalas para medir calidad metodológica
de estudios basados en metodología observacional (EBMO). Se presentan evidencias de validez de la Escala de Calidad
Metodológica (MQSOM) para EBMO en base a su estructura interna. Método: Se aplicó MQSOM a 650 artículos que
emplearon metodología observacional. Se calculó el coeficiente de correlación intraclase (CCI) para la fiabilidad inter
e intracodificadores. Se realizó un análisis paralelo mediante implementación óptima para estudiar su dimensionalidad.
Finalmente, se realizó un análisis factorial exploratorio con media muestra obtenida aleatoriamente, seguido de un
análisis factorial confirmatorio con la otra mitad. Resultados: La fiabilidad inter e intra codificador fueron adecuadas
(CCI > ,73). El análisis paralelo sugirió multidimensionalidad (UniCo = .41, ECV = .31). Se obtuvo una estructura
factorial de segundo orden (factor general: Calidad metodológica) con dos factores de primer orden (F1 Diseño y F2
Medición y análisis), RMSEA = 0.000, NNFI = 1, GFI = .98, AGFI = .97, con adecuados resultados en fiabilidad y
discriminación. Conclusiones: MQSOM es un instrumento breve (11 ítems), útil para profesionales de la intervención,
investigadores o comisiones de valoración, para diseñar, implementar o evaluar EBMO.
Citas
Anguera, M. T. (2003). Observational methods (General). In R. FernándezBallesteros (Ed.), Encyclopedia of psychological assessment, Vol. 2 (pp. 632-637). Sage.
Anguera, M. T., Blanco-Villaseñor, A., Losada, J. L., & Portell, M.(2018). Guidelines for designing and conducting a study that applies observational methodology. Anuario de Psicología, 48(1), 9-17. https://doi.org/10.1016/j.anpsic.2018.02.001
Anguera, M. T., Blanco-Villaseñor, A., Losada, J. L., & Sánchez-Algarra, P. (2020). Integración de elementos cualitativos y cuantitativos en metodología observacional [Integration of qualitative and quantitative elements in observational methodology]. Ámbitos: Revista Internacional de Comunicación, 49, 49-70. https://doi.org/10.12795/Ambitos.2020. i49.04
Anguera, M. T., Camerino, O., & Castañer, M. (2012). Mixed methods procedures and designs for research on sport, physical education and dance. In O. Camerino, M. Castañer, & M. T. Anguera (Ed.), Mixed methods research in the movement sciences: Case studies in sport, physical education and dance (pp. 3-27). Routledge.
Argudo-Iturriaga, F. M., Garcí-Marín, P., Borges-Hernández, P. J., & Ruíz-Lara, E. (2021). Effects of rules changes on water polo shooting performance according to the final classification: High, medium, and worst level. Journal of Physical Education and Sport, 21(1), 188-200. https://doi.org/10.7752/jpes.2021.01026
Bakeman, R. (1978). Untangling streams of behavior: Sequential analysis of observation data. In G. P. Sackett (Ed.), Observing behavior: Vol. 2. Data collection and analysis methods (pp. 63-78). University of Park Press.
Barbero-García, M. I. (1993). Psicometría II. Métodos de elaboración de escalas [Psychometrics II. Methods to develop scales]. UNED.
Barbero-García, M. I., Vila-Abad, E., & Holgado-Tello, F. P. (2015). Psicometría [Psychometrics]. Sanz y Torres.
Bartlett, M. S. (1954). A note on multiplying factors for various chi-squared approximations. Journal of the Royal Statistical Society, Series B, 16, 296-298. https://www.jstor.org/stable/2984057
Blanco-Villaseñor, A., Losada, J. L., & Anguera, M. T. (2003). Data analysis techniques in observational designs applied to the environment-behavior relation. Medio Ambiente y Comportamiento Humano, 4(2), 111-126. http://mach.webs.ull.es/PDFS/VOL4_2/VOL_4_2_d.pd
Chacón-Moscoso, S., Anguera, M. T., Sanduvete-Chaves, S., Losada, J. L., & Portell, M. (2019). Methodological quality checklist for studies based on observational methodology (MQCOM). Psicothema, 31(4), 458-464. https://doi.org/10.7334/psicothema2019.116
Chacón-Moscoso, S., Anguera, M. T., Sanduvete-Chaves, S., & SánchezMartín, M. (2014). Methodological convergence of program evaluation designs. Psicothema, 26(1), 91-96. https://doi.org/10.7334/7psicothema2013.144
Chacón-Moscoso, S., Sanduvete-Chaves, S., Anguera, M. T., Losada, J.L., Portell, M., & Lozano-Lozano, J. A. (2018). Preliminary checklist for reporting observational studies in sports areas: Content validity. Frontiers in Psychology, 9, Article 291. https://doi.org/10.3389/fpsyg.2018.00291
Chacón-Moscoso, S., Sanduvete-Chaves, S., Lozano-Lozano, J. A., Portell, M., & Anguera, M. T. (2021). From randomized control trial to mixed methods: A practical framework for program evaluation based on methodological quality. Anales de Psicología, 37(3), 599-608. https://doi.org/10.6018/analesps.470021
Chacón-Moscoso, S., Sanduvete-Chaves, S., Portell, M., & Anguera, M. T. (2013). Reporting a program evaluation: Needs, program plan, intervention, and decisions. International Journal of Clinical and Health Psychology, 13(1), 58-66. https://doi.org/10.1016/S1697-2600(13)70008-5
Cochran, W. G., & Chambers, S. P. (1965). The planning of observational studies of human populations. Journal of the Royal Statistical Society. Series A (General), 128(2), 234-266. https://doi.org/10.2307/2344179
Ferrando, P. J., & Lorenzo-Seva, U. (2017). Program FACTOR at 10: Origins, development, and future directions. Psicothema, 29(2), 236-241. https://doi.org/10.7334/psicothema2016.304
Ferrando, P. J., & Lorenzo-Seva, U. (2018). Assessing the quality and appropriateness of factor solutions and factor score estimates in exploratory item factor analysis. Educational and Psychological Measurement, 78, 762-780. https://doi.org/10.1177/0013164417719308
Field, A. (2018). Discovering statistics using IBM SPSS statistics. Sage. Harrison, R. L., Reilly, T. M., & Creswell, J. W. (2020). Methodological rigor in mixed methods: An application in management studies. Journal of Mixed Methods Research, 14(4), 4773-495. https://doi.org/10.1177/1558689819900585
Holgado-Tello, F. P., Chacón-Moscoso, S., Barbero-García, M. I., & VilaAbad, E. (2010). Polychoric versus Pearson correlations in exploratory and confirmatory factor analysis of ordinal variables. Quality & Quantity, 44, 153–166. https://doi.org/10.1007/s11135-008-9190-y
Holgado-Tello, F. P., Suárez-Falcón, J. C., & Morata-Ramírez, M. A. (2019). Modelos de ecuaciones estructurales, desde el “path analysis” al análisis multigrupo: una guía práctica con LISREL [Structural equation models, from path analysis to multigroup analysis: A practical guide with LISREL]. Sanz y Torres.
Hooper, D., Coughlan, J., & Mullen, M. (2008). Structural equation modelling: Guidelines for determining model fit. Electronic Journal of Business Research Methods, 6, 56–60. https://academic-publishing.org/index.php/ejbrm/article/view/1224
Kaiser, H., & Rice, J. (1974). Little Jiffy, Mark Iv. Educational and Psychological Measurement, 34(1), 111–117. https://doi.org/10.1177/001316447403400115
Kalkbrenner, M. T. (2023). Alpha, omega, and h internal consistency reliability estimates: Reviewing these options and when to use them. Counseling Outcome Research and Evaluation, 14(1), 77-88. https://doi.org/10.1080/21501378.2021.1940118
Lappi, O., Rinkkala, P., & Pekkanen, J. (2017). Systematic observation of an expert driver’s gaze strategy-an on-road case study. Frontiers in Psychology, 8, Article 620. https://doi.org/10.3389/fpsyg.2017.00620
Lapresa, D., Santesteban, G., Arana, J., Anguera, M. T., & Aragón, S. (2017). Observation system for analyzing individual boccia BC3. Journal of Developmental and Physical Disabilities, 29, 721-734. https://doi.org/10.1007/s10882-017-9552-2
Leech, N. L., & Onwuegbuzie, A. J. (2010). Guidelines for conducting and reporting mixed research in the field of counseling and beyond. Journal of Counseling & Development, 88(1), 61-66. https://doi.org/10.1002/j.1556-6678.2010.tb00151.x
Maciá, M. J., Sánchez-Sánchez, J., García-Córdoba, J. A., & Gallardo, A. M. (2021). Analysis of the safety and accessibility of sports facilities in secondary school. Cuadernos de Psicología del Deporte, 21(1), 242-257. https://revistas.um.es/cpd/article/view/395671/294531
Muñiz, J., & Fonseca-Pedrero, E. (2019). Ten steps for test development. Psicothema, 31(1), 7-16. https://doi.org/10.7334/psicothema2018.291
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T.C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson, A., Lalu,M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S., ... & Moher, D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. International Journal of Surgery, 88, Article 105906. https://doi.org/10.1016/j.ijsu.2021.105906.
Portell, M., Anguera, M. T., Chacón-Moscoso, S., & Sanduvete-Chaves,S. (2015). Guidelines for reporting evaluations based on observational methodology. Psicothema, 27(3), 283-289. https://doi.org/10.7334/psicothema2014.276
Portney, L., & Watkins, M. (2000). Foundations of clinical research: Applications to practice. Prentice Hall.
Sánchez-Meca, J. (2022). Revisiones sistemáticas y metaanálisis en educación: un tutorial [Systematic reviews and meta-analysis in education: A tutorial]. RiiTE Revista Interuniversitaria de Investigación en Tecnología Educativa, 13, 5-40. https://doi.org/10.6018/riite.545451
Sanduvete-Chaves, S., Lozano-Lozano, J. A., Chacón-Moscoso, S., & Holgado-Tello, F. P. (2018). Development of a work climate scale in emergency health services. Frontiers in Psychology, 9, Article 10. https://doi.org/10.3389/fpsyg.2018.00010
Serna, J., Muñoz, V., Hileno, R., Solsona, E., & Sáez de Ocáriz, U. (2017). T-Patterns initiated with ball screen or one-on-one in basketball. Journal of Sport Psychology 26(Suppl. 1), 81-86. https://www.redalyc.org/pdf/2351/235150578014.pdf
von Elm, E., Altman, D. G., Effer, M., Pocock, S. J., Gotzsche, P. C., & Vandenbroucke, J. P. (2007). The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: Guidelines for reporting observational studies. Bulletin of World Health Organization, 85(11), 867-872. https://doi.org/10.2471/blt.07.045120.