Track: Undergraduate Student Paper Competition

Abstract

A pre-requisite for exact sciences students is to master basic mathematical skills. Since 2016, to reduce the drop-out level of such students, Ben-Gurion University of the Negev offers a "Mathematics Proficiency" online – self-evaluation and training course. The course was developed by the Mathematics department in collaboration with the University’s Academic Development and Teaching Improvement Unit. The goal of this study was to classify key learning patterns and characteristics and relate those to student’s achievements at the end of the first semester of studies in the faculties of engineering and natural sciences (STEM in general), mainly via process mining and statistical analysis.

Process mining combines data mining with process modeling and analysis to discover, monitor and improve organizational processes by extracting knowledge from event logs available in the databases of organization’s information systems. In this research we utilized three leading process mining tools: Disco, ProM and Celonis, combined with R for statistical analyzes. Our analysis relied on the event and self-evaluation logs of the students' online course, without disclosing their personal identity.

After defining the examined population and the research method, an extensive data cleaning was carried out. We had over 25.5 thousand records reflecting 6265 students participating during 3 academic years of activities. Then, process mining tools were applied for modeling as well as identifying behavior patterns, those displayed over 1,100 variants, reflecting student behavior and diversity, Learning those patterns with process mining tools led to definition of 5 main learning patterns. Finally, the effect of the behavioral patterns during the course was examined through students’ success, particularly on the average grade in the first semester of studies.

The results show that 72% of the students did not use the on-line course despite recommendation, but from those who did use it, approximately half have performed most of the recommended tests (as per personal recommendation).  Statistical analysis indicated that there was no distinct difference between students that studied up to 2 self-evaluation tests before abandoned and students that did not use the on-line course at all. However, significant correlation was found between persistence learning pattern and students’ average. Students who have conducted most of the recommended tests (over 75%), achieved on the average, final score higher by 5.4 points (in 0-100 scale) than students who did not perform any tests at all.

Farther statistical analysis showed that the student's behavioral pattern in the Math Proficiency online course was related to the different student characteristics. It was checked by chi-square test for independence in every learning pattern, and gender, faculty, academic year, department and student type – all found with distinct dependence to student behavior.

This study found significant relation between students’ behavior on the Math Proficiency online course and their success (reflected by semester’s average score) and students’ characteristics. While this may help in recommending behavior to students and assessing their success prior to academic studies , it is important to note that there are many variables that influence the student's learning process during the semester. Others can be explored in further follow-up works