Abstract

Usage of Learning Management Systems (LMS) has become more widespread with the disruption of face-to-face educations after the covid-19 pandemic worldwide. There are several software products, which are usually named as LMS to enable and support distance education. However selection a suitable LMS is a complex Multiple Criteria Decision Making (MCDM) problem that requires consideration of many criteria and inputs from different parties like students, academicians etc. Usability evaluation of LMS can be considered as one of the critical steps in deciding which LMS system to be adapted. There are several studies related to usability evaluation of LMS in the literature, but utilization of MCDM methods and real life case studies are rare. Based on this motivation, usability evaluation of the SAKAI system that is in use at the Industrial Engineering Department of Dokuz Eylul University is performed. Axiomatic Design Procedure (ADP) is considered as a suitable MCDM method for usability evaluation as it allows an easy approach to data fusion and setting performance targets for decision-makers. A MCDM model is created based on the analyses of the related literature and information gathered from the academicians. Afterwards, a questionnaire is developed based on several statistical rules and directed to three types of system users, namely lecturers, graduate and undergraduate students in order to collect data about usability factors and their importance. After detailed statistical analyses on the gathered data and weighting criteria, ADP is performed to evaluate usability of the LMS. It is concluded that the proposed ADP based approach is easy to apply to practical circumstances and able to quantify usability of the existing SAKAI system. Usability of the SAKAI-LMS is found satisfactory based on the proposed ADP based evaluation. 

Keywords

Learning Management Systems, Usability Evaluation, Axiomatic Design, Distance Learning, Decision Making

Biographies

Adil Baykasoglu received his B.Sc., M.Sc. and Ph.D. degrees from Mechanical and Industrial Engineering areas in Turkey (Gaziantep) and England (Nottingham). He is presently a full Professor and chair at the Industrial Engineering Department at the Dokuz Eylul University. He has published numerous academic papers, 3 books and edited several conference books on operational research, computational intelligence, engineering management, and manufacturing systems design. He is also an active editor and referee for many scientific journals was published by Springer Nature.

Burcu Felekoglu received her B.Sc. and M.Sc. degrees in industrial engineering area in Turkey (Izmir) and Ph.D. degree in Manufacturing and Management division of engineering area in England (Cambridge). She is presently an Assistant Professor and Vice Chair at the Industrial Engineering Department at the Dokuz Eylul University. Her primary research areas include new product development management, innovation and technology management, and ergonomics. 

Ceylin Unal received her B.Sc. degree from Industrial Engineering in Turkey (Izmir) and currently studying on her M.Sc. degree also from Industrial Engineering in Turkey (Izmir). She is currently working on software usability as part of her master's thesis. She also provides consultancy to the e-commerce systems of a company in the private sector.