Palabras clave
Technology
Test
Assessment
Measurement TIC
Tecnología
Test
Evaluación
Medición
Cómo citar
Resumen
Antecedentes: La irrupción de la tecnología digital en las áreas de medición y evaluación psicológica y educativa expande el concepto clásico de test de lápiz y papel. Los modelos de evaluación construidos sobre la ubicuidad de los smartphones, las redes sociales o el desarrollo del software abren nuevas posibilidades para la evaluación. Método: El estudio se organiza en cuatro partes en cada una de las cuales se discuten las ventajas y limitaciones de una aplicación de la tecnología a la evaluación: la evaluación ambulatoria, las redes sociales, la gamificación y las pruebas de elección forzosa. Resultados: Los nuevos desarrollos resultan claramente relevantes en el ámbito de la medición y la evaluación psicológica y educativa. Entre otras ventajas, aportan una mayor validez ecológica al proceso evaluativo y eliminan el sesgo relacionado con la evaluación retrospectiva. Conclusiones: Algunas de estas nuevas aproximaciones llevan a un escenario multidisciplinar con una tradición aún por construir. La psicometría está obligada a integrarse en este nuevo espacio aportando una sólida experiencia en la medición de variables psicológicas. Se muestran los temas de debate y retos que ha de abordar el buen quehacer de la psicología en la incorporación de estas nuevas aproximaciones.
Citas
Adjerid, I., & Kelley, K. (2018). Big data in psychology: A framework for research advancement. The American Psychologist, 73(7), 899-917.
https://doi.org/10.1037/amp0000190
Aguado, D., Andrés, J. C., García Izquierdo, A. L., & Rodríguez, J. (2019). LinkedIn” Big Four”: Job performance validation in the ICT sector. Journal of Work and Organizational Psychology, 35, 53-64. https://doi.org/10.5093/jwop2019a7
Aguado, D., Rubio, V. J., & Lucía, B. (2011). The Risk Propensity Task (PTR): A proposal for a behavioral performance-based computer test for assessing risk propensity. Anales de Psicología, 27(3), 862-870.
Andrés, J. C., Aguado, D., & De Miguel, J. (2022). What’s behind LinkedIn? Measuring the LinkedIn big four dimensions through rubrics. Psychologist Papers, 43(1), 12-20. https://doi.org/10.23923/pap.psicol.2979
Armstrong, M. B., Landers, R. N., & Collmus, A. B. (2016). Gamifying recruitment, selection, training, and performance management:
Game-thinking in human resource management. In D. Davis & H. Gangadharbatla (Eds.), Emerging research and trends in gamification.
(pp.140-165). IGI Global.
Bangerter, A., Roulin, N., & König, C. J. (2012). Personnel selection as a signaling game. Journal of Applied Psychology, 97, 719-738.
https://doi.org/10.1037/a0026078
Bell, I. H., Lim, M. H., Rossell, S. L., & Thomas, N. (2017). Ecological momentary assessment and intervention in the treatment of psychotic
disorders: A systematic review. Psychiatric Services, 68(11), 1172-1181. https://doi.org/10.1176/appi.ps.201600523
Bennett, R. E. (2015). The changing nature of educational assessment. Review of Research in Education, 39(1), 370-407.
https://doi.org/10.3102/0091732X14554179
Bogarín, A,. Cerezo, R., & Romero, C. (2018). Discovering learning processes using Inductive Miner: A case study with Learning Management Systems (LMSs). Psicothema, 30(3), 322-329. https://doi.org/10.7334/psicothema2018.116.
Boyd, D., & Ellison N. (2007). Social network sites: Definition, history, and scholarship. Journal of Computer-Mediated Communication, 13(1), 210-230. https://doi.org/10.1111/j.1083-6101.2007.00393.x
Brown, A., & Maydeu-Olivares, A. (2011). Item response modeling of forced choice questionnaires. Educational and Psychological Measurement, 71(3), 460-502. https://doi.org/10.1177/0013164410375112
Caballero-Hernández, J. A., Palomo-Duarte, M., & Dodero, J. M. (2017). Skill assessment in learning experiences based on serious games: A systematic mapping study. Computers & Education, 113, 42-60. https://doi.org/10.1016/j.compedu.2017.05.008
Cattell, R. B. (1966). The data box: Its ordering of total resources in terms of possible relational systems. In R. B. Cattell (Ed.), Handbook of multivariate experimental psychology (pp. 67-128). Rand-McNally.
Cattell, R. B., & Warburton, F. W. (1967). Objective personality and motivation tests. University of Illinois Press.
Collmus, A. B., Armstrong, M. B., & Landers, R. N. (2016). Game-thinking within social media to recruit and select job candidates. In N. R. Landers & B. G. Schmidt (Eds.), Social media in employee selection and recruitment: Theory, practice, and current challenges (pp. 103-124). Springer International Publishing.
Delgado-Gómez, D., Carmona-Vázquez, C., Bayona, S., Ardoy-Cuadros, J., Aguado, D., Baca-García, E., & Lopez-Castroman, J. (2016). Improving impulsivity assessment using movement recognition: a pilot study. Behavior Research Methods, 48(4), 1575-1579. https://doi.org/10.3758/s13428-015-0668-y
Delgado-Gómez, D., Sújar, A., Ardoy-Cuadros, J., Bejarano-Gómez, A., Aguado, D., Miguelez-Fernandez, C., Blasco-Fontecilla, H., & Peñuelas-Calvo, I. (2020). Objective assessment of attention-deficit hyperactivity disorder (ADHD) using an infinite runner-based computer game: a pilot study. Brain Sciences, 10(10), Article 716. https://doi.org/10.3390/brainsci10100716
Donath, J. (2008). Signals in social supernets. Journal of Computer Mediated Communication, 13(1), 231-251. https://doi.org/10.1111/j.1083-6101.2007.00394.x
Ebner-Priemer, U. W., & Trull, T. J. (2009). Ambulatory assessment: An innovative and.promising approach for clinical psychology. European Psychologist, 14(2), 109-119. https://doi.org/10.1027/1016-9040.14.2.109
Elliot, S., Lawty-Jones, M., & Jackson, C. (1996). Effects of dissimulation on self-report and objective measures of personality. Personality and Individual Differences, 21, 335- 343. https://doi.org/10.1016/0191-8869(96)00080-3
Elosua, P. (2022). Impact of ICT on the assessment environment. Innovations through continuous improvement. Psychologist Papers, 43(1), 3-11. https://doi.org/10.23923/pap.psicol.2985
European Commission (2021). Proposal for a Regulation laying down harmonised rules on artificial intelligence. https://op.europa.eu/en/publication-detail/-/publication/e0649735-a372-11eb-9585-01aa75ed71a1/language-en/format-PDF/source-205836026
Espinoza-Cuadros, F., Fernández-Pozo, R., Toledano, D. T., Alcázar-Ramírez, J. D., López-Gonzalo, E., & Hernández-Gómez, L. A. (2016). Reviewing the connection between speech and obstructive sleep apnea. Biomedical engineering online, 15(1), 20. https://doi.org/10.1186/s12938-016-0138-5
Fernández, S., Stöcklin, M., Terrier, L., & Kim, S. (2021). Using available signals on LinkedIn for personality assessment. Journal of Research in Personality, 93, Article 104122. https://doi.org/10.1016/j.jrp.2021.104122
Fokkema, M., Iliescu, D., Greiff, S., & Ziegler, M. (2022). Machine Learning and Prediction in Psychological Assessment: Some Promises and Pitfalls. European Journal of Psychological Assessment, 38(3), 165-175. https://doi.org/10.1027/1015-5759/a000714
Fonseca Pedrero, E., Pérez-Álvarez, M., Al-Halabí, S., Inchausti, F., Muñiz, J., López-Navarro, E., Pérez de Albéniz, A., Lucas Molina, B., Debbané, M., Bobes-Bascarán, M. T., Gimeno-Peón, A., Prado-Abril, J., FernándezÁlvarez, J., Rodríguez-Testal, J. F., González Pando, D., Díez-Gómez, A., García Montes, J. M., García-Cerdán, L., Osma, J., Peris Baquero, Ó., …Marrero, R. J. (2021). Tratamientos psicológicos empíricamente apoyados para adultos: Una revisión selectiva [Evidence-based psychological treatments for adults: A selective review]. Psicothema, 33(2), 188-197. https://doi.org/10.7334/psicothema2020.426
Fonseca-Pedrero, E., Ródenas-Perea, G., Pérez-Albéniz, A., Al-Halabí, S., Pérez, M., & Muñiz, J. (2022). The time of ambulatory assessment.
Psychologist Papers, 43, 21-28. https://doi.org/10.23923/pap.psicol.2983
Frasca, K. J., & Edwards, M. R. (2017). Web‐based corporate, social and video recruitment media: Effects of media richness and source credibility on organizational attraction. International Journal of Selection and Assessment, 25(2), 125-137. https://dx.doi.org/10.1111/ijsa.12165
Funder, D. C. (2012). Accurate personality judgment. Current Directions in Psychological Science, 21, 177-182. https://doi.org/10.1177/0963721412445309
García-Garzón, E., Nieto, M. D., Garrido, L. E., & Abad, F. J. (2020). Bi-factor exploratory structural equation modeling done right: using the SLiDapp application. Psicothema, 32, 607-614. https://doi.org/10.7334/psicothema2020.179
Hall, M., Scherner, P. V., Kreidel, Y., & Rubel, J. A. (2021). A systematic review of momentary assessment designs for mood and anxiety symptoms. Frontiers in Psychology, 12, Article 642044. https://doi.org/10.3389/FPSYG.2021.642044
Heron, K. E., Everhart, R. S., McHale, S. M., & Smyth, J. M. (2017). Using mobile-technology-based ecological momentary assessment (EMA) methods with youth: A systematic review and recommendations. Journal of Pediatric Psychology, 42(10), 1087–1107. https://doi.org/10.1093/JPEPSY/JSX078
Hicks, L. E. (1970). Some properties of ipsative, normative, and forced choice normative measures. Psychological Bulletin, 74(3), 167-184.
https://doi.org/10.1037/h0029780
Hontangas, P. M., Leenen, I., & de la Torre, J. (2016). Traditional scores versus IRT estimates on forced choice tests based on a dominance model. Psicothema, 28(1), 76-82. https://doi.org/10.7334/psicothema2015.204
Jasper, F., & Ortner, T. M. (2014). The tendency to fall for distracting information while making judgments: development and validation of the Objective Heuristic Thinking Test. European Journal of Psychological Assessment, 30, 193-207. https://doi.org/10.1027/1015-5759/a000214
Jobin, A., Ienca, M., & Vayena, E. (2019). The global landscape of AI ethics guidelines. Nature Machine Intelligence, 1, 389-399.
https://doi.org/10.1038/s42256-019-0088-2
Judge, T. A., Rodell, J. B., Klinger, R. L., Simon, L. S., & Crawford, E. R. (2013). Hierarchical representations of the five-factor model of personality in predicting job performance: Integrating three organizing frameworks with two theoretical perspectives. Journal of Applied Psychology, 98(6), 875-925. https://doi.org/10.1037/a0033901
Kato, P. M., & de Klerk, S. (2017). Serious games for assessment: Welcome to the jungle. Journal of Applied Testing Technology, 18(S1), 1-6.
Kluemper, D. H., Rosen, P. A., & Mossholder, K. W. (2012). Social networking websites, personality ratings, and the organizational context: More than meets the eye? Journal of Applied Social Psychology, 42(1), 1143-1172. https://doi.org/10.1111/j.1559-1816.2011.00881.x
Knight, R. G., & Titov, N. (2009). Use of virtual reality tasks to assess prospective memory: Applicability and evidence. Brain Impairment, 10, 3-13. https://doi.org/10.1375/brim.10.1.3
König, C., Demetriou, A., Glock, P., Hiemstra, A., Iliescu, D., Ionescu, C., Langer, M., Liem, C., Linnenbürger, A., Siegel, R., & Vartholomaios, I. (2020). Some advice for psychologists who want to work with computer scientists on big data. Personnel Assessment and Decisions, 6, 17-23. https://doi.org/10.25035/pad.2020.01.002
Kreitchmann, R. S., Abad, F. J., & Sorrel, M. A. (2021). A genetic algorithm for optimal assembly of pairwise forced choice questionnaires. Behavior Research Methods, 54, 1476-1492. https://doi.org/10.3758/s13428-021-01677-4
Kreitchmann, R. S., Abad, F. J., Ponsoda, V., Nieto, M. D., & Morillo, D. (2019). Controlling for response biases in self-report scales: Forced choice vs. psychometric modeling of Likert items. Frontiers in Psychology, 10. Article 2309. https://doi.org/10.3389/fpsyg.2019.02309
Kreitchmann, R. S., Sorrel, M. A., & Abad, F. J. (2022). On bank assembly and block selection in multidimensional forced choice adaptive assessments. Educational and Psychological Measurement. Advance online publication. https://doi.org/10.1177/00131644221087986
Langenfeld, T., Burstein, J., & von Davier, A. A. (2022). Digital-First Learning and Assessment Systems for the 21st Century. Frontiers in Education, 7, Article 857604. https://doi.org/10.3389/feduc.2022.857604
Landers, R. N (2015). An introduction to game-based assessment: Frameworks for the measurement of knowledge, skills, abilities and other human characteristics using behaviors observed within videogames. International Journal of Gaming and Computer-Mediated Simulations, 7, 4–8.
Law, A. S., Logie, R. H., & Pearson, D. G. (2006). The impact of secondary tasks on multitasking in a virtual environment. Acta Psychologica, 122, 27–44. https://doi.org/10.1016/j.actpsy.2005.09.002
López-Mora, C., Carlo, G., Roos, J., Maiya, S., & González-Hernández, J. (2021). Perceived Attachment and Problematic Smartphone Use in Young People: Mediating Effects of Self-Regulation and Prosociality. Psicothema, 33(4), 564-570. https://doi.org/10.7334/psicothema2021.60
Lejuez, C. W., Read, J. P., Kahler, C., Richards, J. B., Ramsey, S. E., Stuart, G. L., Strong, D.R., & Brown, R. A. (2002). Evaluation of a behavioral
measure of risk taking: The Balloon Analogue Risk Task (BART). Journal of Experimental Psychology: Applied, 8, 75-84. https://doi.org/10.1037//1076-898x.8.2.75
Lumsden, J., Skinner, A., Woods, A. T., Lawrence, N. S., Munafò, M. (2016). The effects of gamelike features and test location on cognitive test performance and participant enjoyment. PeerJ, 4. Article e2184. https://doi.org/10.7717/peerj.2184
Maass, W., Parsons, J., Purao, S., Storey, V. C., & Woo, C. (2018). Data-driven meets theory-driven research in the era of big data: Opportunities and challenges for information systems research. Journal of the Association for Information Systems, 19, Article 12, https://doi.org/10.17705/1jais.00526
Mairesse, F., Walker, M. A., Mehl, M. R., & Moore, R. K. (2007). Using linguistic cues for the automatic recognition of personality in conversation and text. Journal of artificial intelligence research, 30, 457-500. https://www.aaai.org/Papers/JAIR/Vol30/JAIR-3012.pdf
Martínez, A., & Salgado, J. F. (2021). A meta-analysis of the faking resistance of forced choice personality inventories. Frontiers in Psychology, 12, Article 732241. https://doi.org/10.3389/fpsyg.2021.732241
Myin-Germeys, I., & Kuppens, P. (2021). The Open Handbook of Experience Sampling Methodology: A step-by-step guide to designing, conducting, and analyzing ESM studies. The center for Research on Experience sampling and Ambulatory methods Leuven (REAL).
Myin-Germeys, I., Kasanova, Z., Vaessen, T., Vachon, H., Kirtley, O., Viechtbauer, W., & Reininghaus, U. (2018). Experience sampling
methodology in mental health research: New insights and technical developments. World Psychiatry, 17(2), 123-132. https://doi.org/10.1002/wps.20513
Nieto, M. D., Abad, F. J., & Olea, J. (2018). Assessing the Big Five with bifactor computerized adaptive testing. Psychological Assessment, 30(12), 1678-1690. https://doi.org/10.1037/pas0000631
Nikolaou, I. (2014). Social networking web sites in job search and employee recruitment. International Journal of Selection and Assessment, 22(2), 179-189. https://doi.org/10.1111/ijsa.12067
Ortner, T. M., & Proyer, R. T. (2015). Objective personality tests. In T. M. Ortner & F. J. R. Van de Vijver (Eds.), Behavior-based assessment in psychology (pp. 133-149). Hogrefe.
Oswald, F. L. (2020). Future research directions for big data in psychology. In S. E. Woo, L. Tay, & R. W. Proctor (Eds.), Big data in psychological research. (pp. 427-441). American Psychological Association.
Palmier-Claus, J. E., Myin-Germeys, I., Barkus, E., Bentley, L., Udachina, A., Delespaul, P. A., Lewis, S. W., & Dunn, G. (2011). Experience sampling research in individuals with mental illness: reflections and guidance. Acta Psychiatrica Scandinavica, 123(1), 12-20. https://doi.org/10.1111/j.1600-0447.2010.01596.x
Parsons, T. D. (2012). Virtual simulations and the second life metaverse: paradigm shift in neuropsychological assessment. In N. Zagalo, L. Morgado, & Boa-Ventura, A. (Eds.), Virtual worlds and metaverse platforms: New communication and identity paradigms (pp. 234-250). IGI Global.
Pavlov, G., Shi, D., Maydeu-Olivares, A., & Fairchild, A. (2021). Item desirability matching in forced choice test construction. Personality and
Individual Differences, 183, Article 111114. https://doi.org/10.1016/j.paid.2021.111114
Powers, M. B., & Emmelkamp, P. M. (2008). Virtual reality exposure therapy for anxiety disorders: A meta-analysis. Journal of Anxiety Disorders, 22, 561-569. https://doi.org/10.1016/j.janxdis.2007.04.006
Roulin, N., & Bangerter, A. (2013). Social networking websites in personnel selection. Journal of Personnel Psychology 12(1), 143-151.
https://doi.org/10.1027/1866-5888/a000094
Roulin, N., & Levashina, J. (2019). LinkedIn as a new selection method: Psychometric properties and assessment approach. Personnel Psychology, 72(2), 187-211. https://doi.org/10.1111/peps.12296
Russell, S., & Norvig, P. (2021). Artificial intelligence: a modern approach. Pearson.
Ryan, A. M., & Derous, E. (2019). The unrealized potential of technology in selection assessment. Journal of Work and Organizational Psychology, 35(2), 85-92. https://doi.org/10.5093/jwop2019a10
Ryan, A. M., & Ployhart, R. E (2014). A century of selection. Annual Review of Psychology. 65, 693-717. https://doi.org/10.1146/annurev-psych-010213-115134
Salgado, J. F. (2016). A theoretical model of psychometric effects of faking on assessment procedures: Empirical findings and implications for personality at work: A theoretical model of faking psychometric effects. International Journal of Selection and Assessment, 24(3), 209-228. https://doi.org/10.1111/ijsa.12142
Salgado, J. F., Anderson, N., & Tauriz, G. (2015). The validity of ipsative and quasi-ipsative forced choice personality inventories for different occupational groups: A comprehensive meta-analysis. Journal of Occupational and Organizational Psychology, 88(4), 797-834.
https://doi.org/10.1111/joop.12098
Santamaría, P., & Sánchez-Sánchez, F. (2022). Open questions in the use of new technologies in psychological assessment. Psychologist Papers 43(1), 48-54. https://doi.org/10.23923/pap.psicol.2984
Schwartz, H. A., Eichstaedt, J. C., Kern, M. L., Dziurzynski, L., Ramones, S. M., Agrawal, M., ... & Ungar, L. H. (2013). Personality, gender, and age in the language of social media: The open-vocabulary approach. PloS one, 8(9), Article 73791. https://doi.org/10.1371/journal.pone.0073791
Schwab, K. (2017). The fourth industrial revolution. Crown Publishing Group. Sedano-Capdevila, A., Porras-Segovia, A., Bello, H. J., Baca-García, E., & Barrigon, M. L. (2021). Use of ecological momentary assessment to study suicidal thoughts and behavior: a systematic review. Current Psychiatry
Reports, 23(7), Article 41. https://doi.org/10.1007/s11920-021-01255-7 Spence, M. (1973). Job market signalling. Quarterly Journal of Economics, 87(3), 355-374.
Stark, S., Chernyshenko, O. S., & Drasgow, F. (2005). An IRT approach to constructing and scoring pairwise preference items involving stimuli on different dimensions: The multi-unidimensional pairwise-preference model. Applied Psychological Measurement, 29(3), 184-203. https://doi.org/10.1177/0146621604273988
Stone, A. A., & Shiffman, S. (1994). Ecological momentary assessment (EMA) in behavioral medicine. Annals of Behavioral Medicine, 16, 199–202. https://doi.org/10.1093/abm/16.3.199
Stoughton, J.W. (2016). Applicant reactions to social media in selection: Early returns and future directions. In N. R. Landers & B. G. Schmidt (Eds.), Social media in employee selection and recruitment: Theory, practice, and current challenges (pp. 249–263). Springer International Publishing.
Stoughton, J.W., Thompson, L. F., & Meade, A.W. (2015). Examining applicant reactions to the use of social networking websites in pre-employment screening. Journal of Business and Psychology, 30, 73-88. https://doi.org/10.1007/s10869-013-9333-6
Tausczik, Y. R., & Pennebaker, J. W. (2010). The psychological meaning of words: LIWC and computerized text analysis methods. Journal of language and social psychology, 29(1), 24-54. Trull, T. J., & Ebner-Priemer, U. (2013). Ambulatory Assessment. Annual
Review of Clinical Psychology, 9, 151-176. https://doi.org/10.1146/ANNUREV-CLINPSY-050212-185510
Trull, T. J., & Ebner-Priemer, U. W. (2020). Ambulatory assessment in psychopathology research: A review of recommended reporting guidelines and current practices. Journal of Abnormal Psychology, 129(1), 56–63. https://doi.org/10.1037/ABN0000473
Van Iddekinge, C. H., Lanivich, S. E., Roth, P. L., & Junco, E. (2016). Social media for selection? Validity and adverse impact potential of a Facebookbased assessment. Journal of Management, 42(7), 1811-1835. https://dx.doi.org/10.1177/0149206313515524
van Roekel, E., Keijsers, L., & Chung, J. M. (2019). A review of current ambulatory assessment studies in adolescent samples and practical
recommendations. Journal of Research on Adolescence, 29(3), 560-577. https://doi.org/10.1111/JORA.12471
von Davier, A. A., Mislevy, R. J., & Hao, J. (Ed.) (2021). Computational psychometrics: New methodologies for a new generation of digital learning and assessment into psychometrics. Springer. https://doi.org/10.1007/978-3-030-74394-9
Weiner, J., Herff, C., & Schultz, T. (2016). Speech-based detection of Alzheimer’s disease in conversational German. In International Speech Communications Association (Eds.), Understanding speech processing in human and machines (pp. 1938-1942). Curran Associates, Inc.
Woods, S. A., Ahmed, S., Nikolaou, I., Costa, A. C., & Anderson, N. R. (2020). Personnel selection in the digital age: A review of validity and applicant reactions, and future research challenges. European Journal of Work and Organizational Psychology, 29(1), 64-77.
https://doi.org/10.1080/1359432X.2019.1681401
Youyou, W., Kosinski, M., & Stillwell, D. (2015). Computer-based personality judgments are more accurate than those made by humans. Proceedings of the National Academy of Sciences, 112(4), 1036-1040. https://doi.org/10.1073/pnas.1418680112
Zide, J., Elman, B., & Shahani-Denning, C. (2014). LinkedIn and recruitment: How profiles differ across occupations. Employee relations, 36(5), 583-604. https://doi.org/10.1108/ER-07-2013-0086
Ziegler, M., Schmukle, S., Egloff, B., & Bühner, M. (2010). Investigating measures of achievement motivation(s). Journal of Individual Differences, 31, 15-21. https://doi.org/10.1027/1614-0001/a000002