Rethinking Diagnostic Assessment Through Culturally Embedded AI for Meaningful Learning

Abstract

Advances in artificial intelligence have accelerated the transformation of educational assessment, but most innovations remain focused on automating scoring rather than generating meaningful evidence of learning. This study aims to develop and validate the Meaningful Diagnostic Assessment Framework (MDAF), which integrates culturally contextualised diagnostic tasks, AI-based analysis, and pedagogical interpretation to generate meaningful evidence of learning in mathematics education. The study employed a controlled quasi-experimental design involving 178 junior high school students in Indonesia. Data were analysed using Structural Equation Modelling (SEM), mediation analysis, and tests of agreement between AI-generated diagnoses and expert assessments. The results indicate positive associations between the MDAF and symbolic mathematical performance and student engagement. Cultural Embeddedness Perception partially mediated these relationships, while AI-generated diagnoses showed high agreement with expert assessments. These findings suggest that AI's primary value in assessment lies not in automatically generating scores, but in transforming student responses into meaningful evidence of learning that supports pedagogical decision-making. This study offers a new paradigm for diagnostic assessment by shifting the focus from “assessment as measurement” to “assessment as the generation of meaningful learning evidence,” thereby providing a conceptual foundation for developing digital assessments that are more adaptive, transparent, and oriented toward the learning process.