Psychometric evaluation of the Cognitive Ability Assessment using Rasch analysis and exploratory factor analysis
DOI:
https://doi.org/10.33736/jcshd.10611.2026Keywords:
cognitive ability assessment, exploratory factor analysis, Malaysian undergraduates, psychometric properties, Rasch analysisAbstract
Despite the growing emphasis on employability, there is a lack of culturally relevant and psychometrically robust tools to assess the cognitive abilities of Malaysian undergraduates. This study examined the psychometric properties of the Cognitive Ability Assessment (CAA), a 50-item instrument designed to measure employability-related cognitive abilities in this population. A cross-sectional study with 278 students from a public university examined the Cognitive Ability Assessment (CAA) using Rasch modelling and Exploratory Factor Analysis (EFA) to assess item functioning, dimensionality, reliability, and construct validity. Rasch analysis showed acceptable item fit (infit MNSQ = 0.73–1.32) and a broad difficulty range (−4.60 to 5.16 logits), with unidimensionality supported for the Quantitative (46.1%), Fluid (40.0%), and Spatial (60.4%) domains, but Comprehension explained only 26.1% of variance, indicating multidimensionality. Differential item functioning identified 10 items with large DIF and 4 with intermediate DIF across academic programmes. EFA explained 30–48% of the variance after item refinement. Internal consistency was moderate (α = 0.54–0.67), with acceptable model fit for most domains (CFI = 0.94–0.97), except Fluid (CFI = 0.83). The findings suggest that the CAA shows promising measurement precision but needs refinement to achieve structural coherence and subgroup fairness. Future research should confirm its factor structure, test invariance across populations, and assess predictive validity for employment outcomes.
References
Arthur, W. J., Day, E. A., McNelly, T. L., & Edens, P. S. (2003). A meta-analysis of the criterion-related validity of assessment centre dimensions. Personnel Psychology, 56(1), 125–153. https://doi.org/10.1111/j.1744-6570.2003.tb00146.x
Bartram, D. (2005). The great eight competencies: A criterion-centric approach to validation. Journal of Applied Psychology, 90(6), 1185–1203. https://doi.org/10.1037/0021-9010.90.6.1185
Bond, T. G., & Fox, C. M. (2015). Applying the Rasch model: Fundamental measurement in the human sciences (3rd ed.). Routledge.
Brown, T. A. (2015). Confirmatory factor analysis for applied research (2nd ed.). Guilford Press.
Cheong, K. C., Hill, C., Fernandez-Chung, R., & Leong, Y. C. (2016). Employing the ‘unemployable': Employer perceptions of Malaysian graduates. Studies in Higher Education, 41(12), 2253–2270. https://doi.org/10.1080/03075079.2015.1034260
Chien, T. W., Hsu, S. Y., Chein, T., Guo, H. R., & Su, S. B. (2008). Using Rasch analysis to validate the revised PSQI to assess sleep disorders in Taiwan's hi-tech workers. Community Mental Health Journal, 44, 417–425. https://doi.org/10.1007/s10597-008-9144-9
Costello, A. B., & Osborne, J. W. (2005). Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment, Research & Evaluation, 10(7), 1–9. https://doi.org/10.7275/jyj1-4868
Creswell, J. W., & Creswell, J. D. (2018). Research design: Qualitative, quantitative, and mixed methods approaches (5th ed.). Sage Publications.
DeVellis, R. F. (2016). Scale development: Theory and applications (4th ed.). Sage.
Djiwandono, P. I. (2006). Cultural bias in language testing. TEFLIN Journal, 17(1), 81–88. https://doi.org/10.15639/teflinjournal.v17i1/85-93
Ercikan, K., Por, H. H., & Guo, H. (2023). Cross-cultural validity and comparability in assessments of complex constructs. In N. Foster & M. Piacentini (Eds.), Innovating assessments to measure and support complex skills (pp. 190–210). OECD Publishing. https://doi.org/10.1787/e5f3e341-en
Etikan, I., Musa, S. A., & Alkassim, R. S. (2016). Comparison of convenience sampling and purposive sampling. American Journal of Theoretical and Applied Statistics, 5(1), 1–4. https://doi.org/10.11648/j.ajtas.20160501.11
Fabrigar, L. R., & Wegener, D. T. (2012). Exploratory factor analysis. Oxford University Press.
Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4(3), 272–299. https://doi.org/10.1037/1082-989X.4.3.272
Flora, D. B., & Curran, P. J. (2004). An empirical evaluation of alternative methods of estimation for confirmatory factor analysis with ordinal data. Psychological Methods, 9(4), 466–491. https://doi.org/10.1037/1082-989X.9.4.466
Frey, C. B., & Osborne, M. A. (2017). The future of employment: How susceptible are jobs to computerisation? Technological Forecasting and Social Change, 114, 254–280. https://doi.org/10.1016/j.techfore.2016.08.019
Ismail, A. A., & Hassan, R. (2019). Technical competencies in digital technology towards Industrial Revolution 4.0. Journal of Technical Education and Training, 11(3), 55–62. https://doi.org/10.30880/jtet.2019.11.03.008
Kline, R. B. (2015). Principles and practice of structural equation modelling (4th ed.). Guilford Press.
Linacre, J. M. (1994). Sample size and item calibration stability. Rasch Measurement Transactions. https://www.rasch.org/rmt/rmt74m.htm
Linacre, J. M. (2026). A user's guide to Winsteps Ministep: Rasch-model computer programs. Winsteps.com. https://www.winsteps.com/a/Winsteps-Manual.pdf
Linacre, M. (2025). Winsteps now does CMLE and JMLE: Multiple-choice, rating scale and partial credit Rasch analysis. Winsteps.com. https://www.winsteps.com/winsteps.htm
Mustapha, R., & Abdullah, A. (2004). Malaysia transitions toward a knowledge-based economy. The Journal of Technology Studies, 30(3), 51–61. https://doi.org/10.21061/jots.v30i3.a.8
Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). McGraw-Hill.
Raven, J., Raven, J. C., & Court, J. H. (2000). Manual for Raven's Progressive Matrices and Vocabulary Scales. Harcourt Assessment.
Revelle, W. (2026). Psych: Procedures for psychological, psychometric, and personality research. https://CRAN.R-project.org/package=psych
Roslan, F. N., Mohd Fuzi, N., Idris, N., Che Hashim, H. I., Abd Razak, S. S., & Ong, S. Y. Y. (2024). Factors affecting graduate employability in Malaysian public university. International Journal of Academic Research in Business & Social Sciences, 14(12), 2382–2387. http://dx.doi.org/10.6007/IJARBSS/v14-i12/24198
Saleh, H., & Abdul Wahab, N. A. (2025). Employers' perspectives on Malaysian graduates' skills: A contemporary study. Journal of TVET and Technology Review, 3(1), 16–23. https://doi.org/10.30880/jttr.2025.03.01.002
Schmitt, N., & Kuljanin, G. (2008). Measurement invariance: Review of practice and implications. Human Resource Management Review, 18(4), 210–222. https://doi.org/10.1016/j.hrmr.2008.03.003
Schneider, W. J., & McGrew, K. S. (2018). The Cattell–Horn–Carroll theory of cognitive abilities. In D. P. Flanagan & E. M. McDonough (Eds.), Contemporary intellectual assessment: Theories, tests, and issues (4th ed., pp. 73–163). The Guilford Press.
Schwab, K. (2016). The fourth industrial revolution. World Economic Forum.
Sedgwick, P. (2014). Cross-sectional studies: Advantages and disadvantages. BMJ, 348, g2276. https://doi.org/10.1136/bmj.g2276
Smith, E. V. J. (2002). Detecting and evaluating the impact of multidimensionality using item fit statistics and principal component analysis of residuals. Journal of Applied Measurement, 3(2), 205–231.
Tabachnick, B. G., & Fidell, L. S. (2013). Using multivariate statistics (6th ed.). Pearson Education.
Tee, P. K., Wong, L. C., Dada, M., Song, B. L., & Ng, C. P. (2024). Demand for digital skills, skill gaps and graduate employability: Evidence from employers in Malaysia. F1000Research, 13, 389. https://doi.org/10.12688/f1000research.148514.1
Tennant, A., & Conaghan, P. G. (2007). The Rasch measurement model in rheumatology: What is it and why use it? When should it be applied, and what should one look for in a Rasch paper? Arthritis & Rheumatism, 57(8), 1358–1362. https://doi.org/10.1002/art.23108
Thurstone, L. L. (1947). Multiple-factor analysis: A development and expansion of the vectors of mind. University of Chicago Press.
Wang, T. (2010). Comparative evaluation of survey methods. In J. Sheth & N. Malhotra (Eds.), Wiley International Encyclopedia of Marketing. Wiley. https://doi.org/10.1002/9781444316568.wiem02043
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 UNIMAS Publisher
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Copyright Transfer Statement for Journal
1) In signing this statement, the author(s) grant UNIMAS Publisher an exclusive license to publish their original research papers. The author(s) also grant UNIMAS Publisher permission to reproduce, recreate, translate, extract or summarize, and to distribute and display in any forms, formats, and media. The author(s) can reuse their papers in their future printed work without first requiring permission from UNIMAS Publisher, provided that the author(s) acknowledge and reference publication in the Journal.
2) For open access articles, the author(s) agree that their articles published under UNIMAS Publisher are distributed under the terms of the CC-BY-NC-SA (Creative Commons Attribution-Non Commercial-Share Alike 4.0 International License) which permits unrestricted use, distribution, and reproduction in any medium, for non-commercial purposes, provided the original work of the author(s) is properly cited.
3) The author(s) is/are responsible to ensure his or her or their submitted work is original and does not infringe any existing copyright, trademark, patent, statutory right, or propriety right of others. Corresponding author(s) has (have) obtained permission from all co-authors prior to submission to the journal. Upon submission of the manuscript, the author(s) agree that no similar work has been or will be submitted or published elsewhere in any language. If submitted manuscript includes materials from others, the authors have obtained the permission from the copyright owners.
4) In signing this statement, the author(s) declare(s) that the researches in which they have conducted are in compliance with the current laws of the respective country and UNIMAS Journal Publication Ethics Policy. Any experimentation or research involving human or the use of animal samples must obtain approval from Human or Animal Ethics Committee in their respective institutions. The author(s) agree and understand that UNIMAS Publisher is not responsible for any compensational claims or failure caused by the author(s) in fulfilling the above-mentioned requirements. The author(s) must accept the responsibility for releasing their materials upon request by Chief Editor or UNIMAS Publisher.
5) The author(s) should have participated sufficiently in the work and ensured the appropriateness of the content of the article. The author(s) should also agree that he or she has no commercial attachments (e.g. patent or license arrangement, equity interest, consultancies, etc.) that might pose any conflict of interest with the submitted manuscript. The author(s) also agree to make any relevant materials and data available upon request by the editor or UNIMAS Publisher.
