International Journal of Cognitive Research in Science, Engineering and Education (IJCRSEE)

IJCRSEE Editorial Team; Siti Eneng Sururiyatul Mu'aziyah; Syech Zainal; Raya Agni; Supriyadi Supriyadi; Isnawati Isnawati

doi:10.23947/2334-8496-2025-13-3-737-748

Authors

Siti Eneng Sururiyatul Mu'aziyah Departement of Science Education, Universitas Trunojoyo Madura, Indonesia https://orcid.org/0009-0006-1834-2692
Syech Zainal Departement of Science Education, Universitas Tadulako, Indonesia https://orcid.org/0000-0003-2315-7206
Raya Agni Departement of Science Education, Universitas Tadulako, Indonesia https://orcid.org/0009-0005-1630-5583
Supriyadi Supriyadi Physics Study Program, Universitas Musamus, Indonesia https://orcid.org/0000-0002-5944-3055
Isnawati Isnawati Department of Physics Education, Universitas Pendidikan Indonesia, Indonesia https://orcid.org/0000-0003-2810-6678

DOI:

https://doi.org/10.23947/2334-8496-2025-13-3-737-748

Keywords:

Science Identity, PISA 2025 Framework, Rasch Model, STEM

Abstract

This study developed and validated a Science Identity Questionnaire for secondary school students based on the PISA 2025 framework using a Rasch model analysis. The questionnaire aimed to measure key aspects of science identity, including science capital, attitude toward science, and environmental awareness. A total of 496 students (200 males and 269 females) from various Indonesian provinces participated. The data were analyzed using the Rasch model to examine item validity, reliability, and demographic bias. The results indicated that the questionnaire fit the Rasch model excellently, confirming its validity, with Outfit and Infit Mean Square values within acceptable ranges. Reliability analysis yielded person and item reliabilities of 0.72 and 0.99, respectively. No significant bias was detected between male and female students or between urban and rural respondents. These findings support using this tool to assess science identity and its role in promoting long-term STEM engagement. This instrument can inform interventions that aim to foster inclusive science education.

Downloads

Download data is not yet available.

References

Baghaei, P., & Carstensen, C. H. (2013). Fitting the mixed Rasch model to a reading comprehension test: Identifying reader types. Practical Assessment, Research & Evaluation, 18(5), 1-12. https://eric.ed.gov/?id=EJ1005138

Belova, T., Islamov, A. E., Rozhnov, A. A., Zhdanov, S. P., Sokolova, E. I., & Tsomartova, D. A. (2024). Do gender and science success moderate the effects of science learning self-efficacy on science identity?. In Frontiers in Education (Vol. 9, p. 1409077). Frontiers Media SA. 9. https://doi.org/10.3389/feduc.2024.1409077 DOI: https://doi.org/10.3389/feduc.2024.1409077

Bhatti, H. A., Mehta, S., McNeil, R., Yao, S. Y., & Wilson, M. (2023). A scientific approach to assessment: Rasch measurement and the Four Building Blocks. In Advances in Applications of Rasch Measurement in Science Education (pp. 157-187). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-031-28776-3_7 DOI: https://doi.org/10.1007/978-3-031-28776-3_7

Staggs, M. M., & Brown, J. C. (2024). ‘Me hizo sentir como científica’: the expressed science identities of multilingual learners in high school biology classrooms. International Journal of Science Education, 46(2), 155-180. https://doi.org/10.1080/09500693.2023.2225230 DOI: https://doi.org/10.1080/09500693.2023.2225230

Budi, A. P., Purwanto, E., & Wangid, M. N. (2024). Development and validation of Indonesian youth career identity scale: the Rasch analysis. International Journal of Evaluation and Research in Education, 14(1), 415. https://doi.org/10.11591/ijere.v14i1.29802 DOI: https://doi.org/10.11591/ijere.v14i1.29802

Chou, Y. T., & Wang, W. C. (2010). Checking dimensionality in item response models with principal component analysis on standardized residuals. Educational and Psychological Measurement, 70(5), 717-731. https://doi.org/10.1177/0013164410379322 DOI: https://doi.org/10.1177/0013164410379322

Constantinou, F., (2024). Assessing Students’ Application Skills Through Contextualized Tasks: Toward a More Comprehensive Framework for Embedding Test Questions in Context. Practical Assessment, Research, and Evaluation 29(1), 10. https://doi.org/10.7275/pare.2103

Deta, U. A., A, S. K., Laila, L., Prahani, B. K., & Suprapto, N. (2024). PISA science framework 2018 vs 2025 and its impact in physics education: Literature review. Momentum: Physics Education Journal, 8(1), 95–107. https://doi.org/10.21067/mpej.v8i1.9215 DOI: https://doi.org/10.21067/mpej.v8i1.9215

Fischer, L., Rohm, T., Carstensen, C. H., & Gnambs, T. (2021). Linking of Rasch-Scaled Tests: Consequences of Limited Item Pools and Model Misfit. Frontiers in Psychology, 12, 633896. https://doi.org/10.3389/FPSYG.2021.633896 DOI: https://doi.org/10.3389/fpsyg.2021.633896

Gottfried, J., Ježek, S., Králová, M. & Řiháček, T., (2022). Autocorrelation Screening: A Potentially Efficient Method for Detecting Repetitive Response Patterns in Questionnaire Data. Practical Assessment, Research, and Evaluation 27(1), 2. https://doi.org/10.7275/vyxb-gt24

Gunawan, F. M., Suryana, D., & Rahmanadia, H. (2023). Analysis of Validity and Reliability of Self-Acceptance Scale Using Rasch Model. Jurnal Pendidikan, 24(2), 71–80. https://doi.org/10.33830/jp.v24i2.4604.2023 DOI: https://doi.org/10.33830/jp.v24i2.4604.2023

Hsu, P. L., & Hernández Frausto, S. I. (2024). What I see of you in me: how do high school students position scientists in a science internship?. International Journal of Science Education, 1-19. https://doi.org/10.1080/09500693.2024.2394709 DOI: https://doi.org/10.1080/09500693.2024.2394709

Huda, M., Ali, A. H., Za, T., Ahmad, R., Selamat, A. Z., Ibrahim, M. H., Anshari, M., & Glorino, M. (2024). Understanding of Digital Ethics for Information Trust: A Critical Insight into Gender Violence Anticipation. In: Communication Technology and Gender Violence. Signals and Communication Technology. Springer, Cham. https://doi.org/10.1007/978-3-031-45237-6_14 DOI: https://doi.org/10.1007/978-3-031-45237-6_14

Ivanova, M. G. & Michaelides, M. P., (2023). Measuring Test-Taking Effort on Constructed-Response Items with Item Response Time and Number of Actions. Practical Assessment, Research, and Evaluation 28(1), 15. https://doi.org/10.7275/pare.1921

Khine, M. S. (2020). Rasch Measurement. In Rasch Measurement. https://doi.org/10.1007/978-981-15-1800-3 DOI: https://doi.org/10.1007/978-981-15-1800-3

Latifah, M., Saripah, I., Suryana, D., & Sunarya, Y. (2024). Validity and Reliability of Self-Concept Instrument Using Rasch Model. Jurnal Kajian Bimbingan Dan Konseling, 9(1), 26–35. https://doi.org/10.17977/um001v9i12024p26-35 DOI: https://doi.org/10.17977/um001v9i12024p26-35

Latip, A. (2024). Pembelajaran IPA Berorientasi Pada Framework Strategic Vision and Direction for Science (SVDS) PISA 2024. Jurnal Kajian Pendidikan IPA, 4(1), 305–314. https://doi.org/10.52434/jkpi13579 DOI: https://doi.org/10.52434/jkpi13579

Liu, Y., Wang, J., & Chen, X. (2023). Cross-cultural adaptation of STEM motivation scales: A meta-validation approach. International Journal of STEM Education, 10(1), 35–48. https://doi.org/10.1186/s40594-023-00456-7

Lockhart, M. E., Kwok, O. M., & Yoon, M. (2024). Investigating science identity classifications of rural high school students: a person-centered approach. International Journal of Science Education, 46(16), 1671-1689. https://doi.org/10.1080/09500693.2023.2293681 DOI: https://doi.org/10.1080/09500693.2023.2293681

Mahasneh, A. M., Gazo, A. M., Abood, M. H., & Aldamat, O. A. (2024). Exploring science identity among Jordanian high school students: a case study. International Journal of Evaluation and Research in Education, 13(2). https://doi.org/10.11591/ijere.v13i2.27701 DOI: https://doi.org/10.11591/ijere.v13i2.27701

Morell, L., Bathia, S., Koo, B. W., Wilson, M., Gochyyev, P., & Smith, R. (2025). Developing and Gathering Validity Evidence for an Instrument to Measure How High School Students Identify as Researchers. Research in Science Education, 55(2), 359-382. https://doi.org/10.1007/s11165-024-10194-1 DOI: https://doi.org/10.1007/s11165-024-10194-1

Oktavianti, F., & Aini, F. Q. (2024). Rasch Model Analysis to Develop Assessment Instruments for Student’s Science Process Skills on Chemical Bonding Material. Hydrogen: Jurnal Kependidikan Kimia, 12(6), 1441-1459. https://doi.org/10.33394/hjkk.v12i6.13941 DOI: https://doi.org/10.33394/hjkk.v12i6.13941

Potgieter, M., Alexander, I., & Engelbrecht, P. (2022). Science identity and cross-national validation: Lessons from global science education. Science Education International, 33(4), 312–325. https://doi.org/10.33828/sei.v33.i4.6 DOI: https://doi.org/10.33828/sei.v33.i4.6

Qudratuddarsi, H., Hidayat, R., Raja Maamor Shah, R. L. Z. binti, Nasir, N., Imami, M. K. W., & Mat Nor, R. bin. (2022). Rasch Validation of Instrument Measuring Gen-Z Science, Technology, Engineering, and Mathematics (STEM) Application in Teaching during the Pandemic. International Journal of Learning, Teaching and Educational Research, 21(6), 104–121. https://doi.org/10.26803/ijlter.21.6.7 DOI: https://doi.org/10.26803/ijlter.21.6.7

Rodriguez, A. J., Tolbert, S., & Mark, S. L. (2024). Investigating science teacher educator identity through the politics of domestication and critical positional praxis. Journal of Research in Science Teaching, 61(3), 625-644. https://doi.org/10.1002/tea.21915 DOI: https://doi.org/10.1002/tea.21915

Sadler, T. D., & Zeidler, D. L. (2009). Scientific literacy, PISA, and socioscientific discourse: Assessment for progressive aims of science education. Journal of Research in Science Teaching, 46(8), 909–921. https://doi.org/10.1002/TEA.20327 DOI: https://doi.org/10.1002/tea.20327

Sari, R. L., & Adiati, R. P. (2023). Perceived Threat Appraisal of Covid-19 among Indonesian Employees: Response to Work From Home (WFH). In M. N. M. Warid, & M. A. A. S. Khan (Eds.), AIP Conference Proceedings (1 ed.). Article 050001 (AIP Conference Proceedings; Vol. 2739, No. 1). American Institute of Physics Inc.. https://doi.org/10.1063/5.0126686 DOI: https://doi.org/10.1063/5.0126686

Sawitri, D. R., Handoyo, S., Hasnida, Creed, P. A., Prihatsanti, U., Kristiana, I. F., Perdhana, M. S., Fajrianthi, Sari, R. L., Rahmawati, E., & Zahreni, S. (2024). The research resilience scale: development and initial validation. International Journal for Educational and Vocational Guidance, 24(3), 763-780. https://doi.org/10.1007/s10775-023-09582-2 DOI: https://doi.org/10.1007/s10775-023-09582-2

Soeharto, S. (2021). Development of a diagnostic assessment test to evaluate science misconceptions in terms of school grades: A Rasch measurement approach. Journal of Turkish Science Education, 18(3), 351-370. https://doi.org/10.36681/tused.2021.78 DOI: https://doi.org/10.36681/tused.2021.78

Soeharto, S., & Csapó, B. (2022). Assessing Indonesian student inductive reasoning: Rasch analysis. Thinking Skills and Creativity, 46, 101132. https://doi.org/10.1016/j.tsc.2022.101132 DOI: https://doi.org/10.1016/j.tsc.2022.101132

Sunni, F. M., Dawana, I. R., Arrahmaniyyah, R., Sunarti, T., Madlazim, M., Wasis, W., & Rizki, I. A. (2024). Research Trend of Rasch Model in the Development of Scientific Literacy Assessment Through Bibliometrics. Jurnal IPA Dan Pembelajaran IPA, 8(4), 413–429. https://doi.org/10.24815/jipi.v8i4.42307 DOI: https://doi.org/10.24815/jipi.v8i4.42307

Tavakol, M., & Dennick, R. (2011). Making sense of Cronbach’s alpha. International Journal of Medical Education, 2, 53–55. https://doi.org/10.5116/ijme.4dfb.8dfd DOI: https://doi.org/10.5116/ijme.4dfb.8dfd

Young, G., Srivastava, A., Estrada, M., Woodcock, A., Schultz, P. W., & Hernandez, P. R. (2025). Colliding identities? The role of multiple identities among historically underrepresented students pursuing scientific research careers. Self and Identity, 24(1-2), 81-105. https://doi.org/10.1080/15298868.2024.2447253 DOI: https://doi.org/10.1080/15298868.2024.2447253

Zhu, J. B., & Luo, Y. Z. (2024). The Prediction of Science Achievement with Science Identity and Science Learning Self-Efficacy among China’s Upper-Secondary Students. Journal of Baltic Science Education, 23(2), 390-410. https://doi.org/10.33225/jbse/24.23.390 DOI: https://doi.org/10.33225/jbse/24.23.390

Zwick, R., Thayer, D. T., & Lewis, C. (1999). An empirical Bayes approach to Mantel-Haenszel DIF analysis. Journal of Educational Measurement, 36(1), 1-28. https://doi.org/10.1111/j.1745-3984.1999.tb00543.x DOI: https://doi.org/10.1111/j.1745-3984.1999.tb00543.x