Identifying Student Profiles within Online Judge Systems using Explainable Artificial Intelligence

Authors

  • Mrs A N RAMAMANI Author
  • T V N KONDALA RAO Author

Keywords:

MultiInstance Learning and classical Machine  Learning formulations, Explainable Artificial  Intelligence is contemplated, humanunderstandable feedback

Abstract

Online Judge (OJ) systems are typically 
considered within programming-related courses 
as they yield fast and objective assessments of the 
code developed by the students. Such an 
evaluation generally provides a single decision 
based on a rubric, most commonly whether the 
submission successfully accomplished the 
assignment. Nevertheless, since in an educational 
context such information may be deemed 
insufficient, it would be beneficial for both the 
student and the instructor to receive additional 
feedback about the overall development of the 
task. This work aims to tackle this limitation by 
considering the further exploitation of the 
information gathered by the OJ and automatically 
inferring feedback for both the student and the 
instructor. More precisely, we consider the use of 
learning-based schemes—particularly, MultiInstance Learning and classical Machine 
Learning formulations—to model student 
behaviour. Besides, Explainable Artificial 
Intelligence is contemplated to provide humanunderstandable feedback. The proposal has been
evaluated considering a case of study comprising 
2,500 submissions from roughly 90 different 
students from a programming-related course in a 
Computer Science degree. The results obtained 
validate the proposal: the model is capable of 
significantly predicting the user outcome (either 
passing or failing the assignment) solely based on 
the behavioural pattern inferred by the 
submissions provided to the OJ. Moreover, the 
proposal is able to identify prone-to-fail student 
groups and profiles as well as other relevant 
information, which eventually serves as feedback 
to both the student and the instructor. 

Downloads

Published

18-06-2024

How to Cite

Identifying Student Profiles within Online Judge Systems using Explainable Artificial Intelligence. (2024). International Journal of Engineering Research and Science & Technology, 20(2), 1178-1188. https://ijerst.org/index.php/ijerst/article/view/382