Identifying Student Profiles within Online Judge Systems using Explainable Artificial Intelligence
Keywords:
MultiInstance Learning and classical Machine Learning formulations, Explainable Artificial Intelligence is contemplated, humanunderstandable feedbackAbstract
Online Judge (OJ) systems are typically
considered within programming-related courses
as they yield fast and objective assessments of the
code developed by the students. Such an
evaluation generally provides a single decision
based on a rubric, most commonly whether the
submission successfully accomplished the
assignment. Nevertheless, since in an educational
context such information may be deemed
insufficient, it would be beneficial for both the
student and the instructor to receive additional
feedback about the overall development of the
task. This work aims to tackle this limitation by
considering the further exploitation of the
information gathered by the OJ and automatically
inferring feedback for both the student and the
instructor. More precisely, we consider the use of
learning-based schemes—particularly, MultiInstance Learning and classical Machine
Learning formulations—to model student
behaviour. Besides, Explainable Artificial
Intelligence is contemplated to provide humanunderstandable feedback. The proposal has been
evaluated considering a case of study comprising
2,500 submissions from roughly 90 different
students from a programming-related course in a
Computer Science degree. The results obtained
validate the proposal: the model is capable of
significantly predicting the user outcome (either
passing or failing the assignment) solely based on
the behavioural pattern inferred by the
submissions provided to the OJ. Moreover, the
proposal is able to identify prone-to-fail student
groups and profiles as well as other relevant
information, which eventually serves as feedback
to both the student and the instructor.
Downloads
Published
Issue
Section
License

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.













