Keywords
assessment literacy
pedagogical competency
self-report questionnaire
teachers’ evaluation
How to Cite
Abstract
The article presents the results of an experimental study aimed at piloting a self-report questionnaire of teachers’ assessment competence in the Ukrainian educational context. The study is motivated by the need for reliable instruments to diagnose and develop teachers’ professional competence amidst school reform.
The chosen experimental procedure for piloting the self-report questionnaire aimed to establish its compliance with key psychometric requirements (internal consistency and structural validity) and its alignment with validated instruments used to evaluate teachers’ professional competence.
Data were collected via an online survey. The instrument consisted of 28 statements in a matrix format, with a 5-point frequency Likert scale. The items targeted the frequency with which teachers perform key actions related to assessing students’ learning outcomes. In total, 349 lower-secondary teachers completed the questionnaire; 224 complete responses were included in the statistical analysis. Data processing (descriptive statistics, reliability estimation, factor extraction) was carried out
in IBM SPSS Statistics 26 and STATISTICA 12.
The results indicate that the instrument is psychometrically reliable and substantively relevant. Content validity was ensured by aligning the items with national guidelines (the New Ukrainian School framework, the Professional Standard for Teachers, and Ministry of Education and Science recommendations on student assessment). Internal consistency reached Cronbach’s α = 0.89, evidencing sufficient homogeneity. Principal Component Analysis (PCA) showed a multidimensional structure encompassing several conceptually distinct domains – typical of tools that capture complex professional teaching competencies, including assessment competence.
The findings support the use of the questionnaire as a stand-alone self-assessment tool and as a component of multi-evidence systems for teacher evaluation and professional development. Future research will focus on the deeper validation of the instrument, the development of a short form, and the integration of the instrument with classroom observation protocols to support targeted teacher professional learning.
References
AERA, APA, & NCME. (2014). Standards for educational and psychological testing. American Educational Research Association.
Ball, S. J., Maguire, M., & Braun, A. (2012). How schools do policy: Policy enactments in secondary schools. Routledge. https://doi.org/10.4324/9780203153185
Black, P., & Wiliam, D. (2009). Developing the theory of formative assessment. Educational Assessment, Evaluation and Accountability, 21(1), 5–31. https://doi.org/10.1007/s11092-008-9068-5
Brookhart, S. M. (2011). Grading and learning: Practices that support student achievement. Solution Tree.
Brown, G. (2012). Teachers’ conceptions of assessment: Comparing primary and secondary teachers in New Zealand. Assessment Matters, 3, 45-70. https://doi.org/10.18296/am.0097
Brown, G. T. L. (2019). Teachers’ conceptions of assessment: Comparing teachers from New Zealand and other countries. In H. Andrade & G. Cizek (Eds.), Handbook of formative assessment in the disciplines (pp. 45–64). Routledge.
Coburn, C. E. (2005). Shaping teacher sensemaking: School leaders and the enactment of reading policy. Educational Policy, 19 (3), 476–509. https://doi.org/10.1177/0895904804272245
Costello, A. B., & Osborne, J. W. (2005). Best practices in exploratory factor analysis. PARE, 10 (7), 1–9.
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297-334. https://doi.org/10.1007/s11092-015-9233-6
DeLuca, C., LaPointe-McEwan, D., & Luhanga, U. (2016). Teacher assessment literacy: A review of international standards and measures. Educational Assessment, Evaluation and Accountability, 28(3), 251–272. https://doi.org/10.1007/s11092-015-9233-6
Epp, A. Subjektive Theorien von Lehrkräften über ungünstige Faktoren in der Bildungsbiografie von Schülerinnen und Schülern – Wie konstruieren Lehrkräfte den Übergang von der Schule in die Berufsausbildung? Zeitschrift für Erziehungswissenschaft, 21, 973–990 (2018). https://doi.org/10.1007/s11618-018-0814-2
Fives, H., & Buehl, M. M. (2016). Teachers’ Beliefs, in the Context of Policy Reform. Policy Insights from the Behavioral and Brain Sciences, 3(1), 114-121. https://doi.org/10.1177/2372732215623554
Fraillon, J., Ainley, J., Schulz, W., Friedman, T., & Duckworth, D. (2020). ICILS 2018 Technical Report. IEA.
Fullan, M. (2007). Change Theory as a Force for School Improvement. In Burger, J.M., Webber, C.F., Klinck, P. (Eds). Intelligent Leadership. Studies in Educational Leadership, vol 6. Dordrecht: Springer. https://doi.org/10.1007/978-1-4020-6022-9_3
Gardner, J. & Galanouli, D. (2016). Teachers’ perceptions of assessment. In The SAGE Handbook of Curriculum, Pedagogy and Assessment (pp. 710-724). https://doi.org/10.4135/9781473921405.n44
Groeben, N., Wahl, D., Schlee, J., & Scheele, B. (1988). Das Forschungsprogramm Subjektive Theorien. Einführung in die Psychologie des reflexiven Subjekts. Tübingen: Francke
Jolliffe, I. T. (2002). Principal component analysis (2nd Ed.). Springer.
Mertler, C. A. (2003). Classroom assessment: A practical guide for educators. Los Angeles, CA: Pyrczak.
Mertler, C.A. & Campbell, C. (2005). Measuring Teachers’ Knowledge & Application of Classroom Assessment. Development of the Assessment Literacy Inventory. Paper presented at the annual meeting of the American Educational Research Association, Montréal, Quebec, Canada April 11–15, 2005
Mullis, I. V. S., & Martin, M. O. (Eds.). (2017). TIMSS 2019 Assessment Frameworks. Retrieved from: http:// timssandpirls.bc.edu/timss2019/frameworks/
Nunnally, J. C. (1967). Psychometric theory. New York: McGraw Hill.
OECD. (2018). TALIS 2018 conceptual framework. OECD Publishing.
OECD. (2019). TALIS 2018 Results (Volume I):Teachers and school leaders as lifelong learners. OECD Publishing. https://doi.org/10.1787/1d0bc92a-en
Pajares, M. F. (1992). Teachers’ Beliefs and Educational Research: Cleaning Up a Messy Construct. Review of
Educational Research, 62 (3), 307-332. https://doi.org/10.3102/00346543062003307
Pastore, S. (2023). Teacher assessment literacy: a systematic review. Frontiers in Education, 8. https://doi.org/10.3389/feduc.2023.1217167
Pastore, S., & Andrade, H. L. (2019). Teacher assessment literacy: a three-dimensional model. Teaching and Teacher Education, 84, 128–138. https://doi.org/10.1016/j.tate.2019.05.003
Podsakoff, P. M., MacKenzie, S. B., & Podsakoff, N. P. (2024). Common method bias: It’s bad, it’s complex, and it’s still unresolved. Annual Review of Organizational Psychology and Organizational Behavior, 11, 385–410. https://doi.org/10.1146/annurev-orgpsych-121422-081217
Podsakoff, P. M., MacKenzie, S. B., Lee, J.-Y., & Podsakoff, N. P. (2003). Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology, 88 (5), 879–903. https://doi.org/10.1037/0021-9010.88.5.879
Pohlmann, S. (2009): Der Übergang am Ende der Grundschulzeit. Zur Formation der Übergangs-empfehlung aus Sicht der Lehrkräfte. Münster: Waxm
Popham, W. J. (2011). Assessment literacy overlooked: a teacher educator’s confession. The Teacher Educator, 46(4), 265–273. https://doi.org/10.1080/08878730.2011.605048
Spillane, J. P., Reiser, B. J., & Reimer, T. (2002). Policy implementation and cognition: Reframing and refocusing implementation research. Review of Educational Research, 72(3), 387–431. https://doi.org/10.3102/00346543072003387
Streiner, D. L. (2008). Health measurement scales a practical guide to their development and use (4th Ed.). Oxford: Oxford University Press. https://doi.org/10.1093/acprof:oso/9780199231881.001.0001
Tsang, S., Royse, C.F., & Terkawi, A.S. (2017). Guidelines for developing, translating, and validating a questionnaire in perioperative and pain medicine. Saudi Journal of Anaesthesia, 11, 80-89. https://doi.org/10.4103/sja.SJA_203_17
Wolters, C. A., & Won, S. (2017). Validity and the Use of Self-Report Questionnaires to Assess Self-Regulated Learning. In Schunk, D. H., & Greene, J. A. (Eds.). H andbook of Self-Regulation of Learning and Performance. New York: Routledge. https://doi.org/10.4324/9781315697048
Xu, Y., & Brown, G. T. L. (2016). Teacher assessment literacy in practice: a reconceptualization. Teaching and Teacher Education, 58, 149–162. https://doi.org/10.1016/j.tate.2016.05.010
Zhang, Z., & Burry-Stock, J. A. (2003). Classroom assessment practices and teachers’ self-perceived assessment skills. Applied Measurement in Education, 16 (4), 323–342. https://doi.org/10.1207/S15324818AME1604_4
Гривко, А. В. (2024). Уявлення вчителів про окремі види робіт для поточного (формувального) оцінювання наскрізних результатів навчання учнів. Український педагогічний журнал, 3, 66–78, https://doi.org/10.32405/2411-1317-2024-3-66-78
Жук, Ю.О., & Ващенко, Л.С. (2024) Оцінювання педагогічної діяльності вчителів: пілотне дослідження. Український педагогічний журнал, 3, 52-65. https://doi.org/10.32405/2411-1317-2024-3-52-65
AERA, APA, & NCME. (2014). Standards for educational and psychological testing. American Educational Research Association (in English).
Ball, S. J., Maguire, M., & Braun, A. (2012). How schools do policy: Policy enactments in secondary schools. Routledge. https://doi.org/10.4324/9780203153185 (in English).
Black, P., & Wiliam, D. (2009). Developing the theory of formative assessment. Educational Assessment, Evaluation and Accountability, 21(1), 5–31. https://doi.org/10.1007/s11092-008-9068-5 (in English).
Brookhart, S. M. (2011). Grading and learning: Practices that support student achievement. Solution Tree (in English).
Brown, G. (2012). Teachers’ conceptions of assessment: Comparing primary and secondary teachers in New Zealand. Assessment Matters, 3, 45-70. https://doi.org/10.18296/am.0097 (in English).
Brown, G. T. L. (2019). Teachers’ conceptions of assessment: Comparing teachers from New Zealand and other countries. In H. Andrade & G. Cizek (Eds.), Handbook of formative assessment in the disciplines (pp. 45–64). Routledge. https://doi.org/10.4324/9781315161840-4 (in English).
Coburn, C. E. (2005). Shaping teacher sensemaking: School leaders and the enactment of reading policy. Educational Policy, 19 (3), 476–509. https://doi.org/10.1177/0895904804272245 (in English).
Costello, A. B., & Osborne, J. W. (2005). Best practices in exploratory factor analysis. PARE, 10 (7), 1–9 (in English).
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297-334. https://doi.org/10.1007/BF02310555 (in English).
DeLuca, C., LaPointe-McEwan, D., & Luhanga, U. (2016). Teacher assessment literacy: A review of international standards and measures. Educational Assessment, Evaluation and Accountability, 28 (3), 251–272. https://doi.org/10.1007/s11092-015-9233-6 (in English).
Epp, A. Subjektive Theorien von Lehrkräften über ungünstige Faktoren in der Bildungsbiografie von Schülerinnen und Schülern – Wie konstruieren Lehrkräfte den Übergang von der Schule in die Berufsausbildung? Zeitschrift für Erziehungswissenschaft, 21, 973–990 (2018). https://doi.org/10.1007/s11618-018-0814-2 (in German).
Fives, H., & Buehl, M. M. (2016). Teachers’ Beliefs, in the Context of Policy Reform. Policy Insights from the Behavioral and Brain Sciences, 3 (1), 114-121. https://doi.org/10.1177/2372732215623554 (in English).
Fraillon, J., Ainley, J., Schulz, W., Friedman, T., & Duckworth, D. (2020). ICILS 2018 Technical Report. IEA (in English).
Fullan, M. (2007). Change Theory as a Force for School Improvement. In Burger, J.M., Webber, C.F., Klinck, P. (Eds). Intelligent Leadership. Studies in Educational Leadership, vol 6. Dordrecht: Springer. https://doi.org/10.1007/978-1-4020-6022-9_3 (in English).
Gardner, J. & Galanouli, D. (2016). Teachers’ perceptions of assessment. In The SAGE Handbook of Curriculum, Pedagogy and Assessment (pp. 710-724). https://doi.org/10.4135/9781473921405.n44 (in English).
Groeben, N., Wahl, D., Schlee, J., & Scheele, B. (1988). Das Forschungsprogramm Subjektive Theorien. Einführung in die Psychologie des reflexiven Subjekts. Tübingen: Francke (in German).
Hryvko, A. V. (2024). Teachers’ perceptions of certain types of work for the current (formative) assessment of students’ cross-cutting learning outcomes. Ukrainian Educational Journal, 3, 66-78, https://doi.org/10.32405/2411-1317-2024-3-66-78 (in Ukrainian).
Jolliffe, I. T. (2002). Principal component analysis (2nd Ed.). Springer (in English).
Mertler, C. A. (2003). Classroom assessment: A practical guide for educators. Los Angeles, CA: Pyrczak (in English).
Mertler, C.A. & Campbell, C. (2005). Measuring Teachers’ Knowledge & Application of Classroom Assessment. Development of the Assessment Literacy Inventory. Paper presented at the annual meeting of the American Educational Research Association, Montréal, Quebec, Canada April 11–15, 2005(in English).
Mullis, I. V. S., & Martin, M. O. (Eds.). (2017). TIMSS 2019 Assessment Frameworks. Retrieved from Boston College, TIMSS & PIRLS International Study Center. http://timssandpirls.bc.edu/timss2019/frameworks/ (in English).
Nunnally, J. C. (1967). Psychometric theory. New York: McGraw Hill (in English).
OECD. (2018). TALIS 2018 conceptual framework. OECD Publishing (in English).
OECD. (2019). TALIS 2018 Results (Volume I): Teachers and school leaders as lifelong learners. OECD Publishing. https://doi.org/10.1787/1d0bc92a-en (in English).
Pajares, M. F. (1992). Teachers’ Beliefs and Educational Research: Cleaning Up a Messy Construct. Review of Educational Research, 62 (3), 307-332. https://doi.org/10.3102/00346543062003307 (in English).
Pastore, S. (2023). Teacher assessment literacy: a systematic review. Frontiers in Education, 8. https://doi.org/10.3389/feduc.2023.1217167 (in English).
Pastore, S., & Andrade, H. L. (2019). Teacher assessment literacy: a three-dimensional model. Teaching and Teacher Education, 84, 128–138. https://doi.org/10.1016/j.tate.2019.05.003 (in English).
Podsakoff, P. M., MacKenzie, S. B., & Podsakoff, N. P. (2024). Common method bias: It’s bad, it’s complex, and it’s still unresolved. Annual Review of Organizational Psychology and Organizational Behavior, 11, 385–410. https://doi.org/10.1146/annurev-orgpsych-121422-081217 (in English).
Podsakoff, P. M., MacKenzie, S. B., Lee, J.-Y., & Podsakoff, N. P. (2003). Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology, 88(5), 879–903. https://doi.org/10.1037/0021-9010.88.5.879 (in English).
Pohlmann, S. (2009): Der Übergang am Ende der Grundschulzeit. Zur Formation der Übergangs-empfehlung aus Sicht der Lehrkräfte. Münster: Waxm (in German).
Popham, W. J. (2011). Assessment literacy overlooked: a teacher educator’s confession. The Teacher Educator, 46(4), 265–273. https://doi.org/10.1080/08878730.2011.605048 (in English).
Spillane, J. P., Reiser, B. J., & Reimer, T. (2002). Policy implementation and cognition: Reframing and refocusing implementation research. Review of Educational Research, 72 (3), 387–431. https://doi.org/10.3102/00346543072003387 (in English).
Streiner, D. L. (2008). Health measurement scales a practical guide to their development and use (4th Ed.). Oxford: Oxford University Press. https://doi.org/10.1093/acprof:oso/9780199231881.001.0001 (in English).
Tsang, S., Royse, C.F., & Terkawi, A.S. (2017). Guidelines for developing, translating, and validating a questionnaire in perioperative and pain medicine. Saudi Journal of Anaesthesia, 11, 80-89. https://doi.org/10.4103/sja.SJA_203_17 (in English).
Wolters, C. A., & Won, S. (2017). Validity and the Use of Self-Report Questionnaires to Assess Self-Regulated Learning. In Schunk, D. H., & Greene, J. A. (Eds.). H andbook of Self-Regulation of Learning and Performance. New York: Routledge. https://doi.org/10.4324/9781315697048 (in English).
Xu, Y., & Brown, G. T. L. (2016). Teacher assessment literacy in practice: a reconceptualization. Teaching and Teacher Education, 58, 149–162. https://doi.org/10.1016/j.tate.2016.05.010 (in English).
Zhang, Z., & Burry-Stock, J. A. (2003). Classroom assessment practices and teachers’ self-perceived assessment skills. Applied Measurement in Education, 16 (4), 323–342. https://doi.org/10.1207/S15324818AME1604_4 (in English).
Zhuk, Y., & Vashchenko, L. (2024). Evaluation of teachers’ pedagogical activity: pilot study. Ukrainian Educational Journal, 3, 52-65. https://doi.org/10.32405/2411-1317-2024-3-52-65 (in Ukrainian).
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.