User Experience in Learning Management System: Edmodo versus Google Classroom

. Nowadays, Learning Management System (LMS) is widely used to support the learning process. LMS becomes a medium of interaction between students and teachers. Poor usability and user experience (UX) of an LMS can make students spend more time learning the system than learning the educational content. Therefore, usability and user experience evaluation are important for e-learning. Edmodo and Google Classroom are two widely used LMS. This study involved students as inspectors to compare the user experience of Edmodo versus Google Classroom using Technique for User Experience Evaluation in E-learning (TUXEL). Inspectors divided into two groups: one group inspected Edmodo and another group inspected Google Classroom. After completing the inspection, the inspectors were asked to fill three questionnaires: usability inspection, general LMS / pedagogical usability inspection, and user experience. The data then processed and analyzed per category per dimension. The results of the overall general usability and pedagogical usability inspections show fewer problematic aspects in Edmodo (22 items) than Google Classroom (30 items). However, Google Classroom seems to be preferred by students because it is simpler, practical and, more comfortable. Edmodo is recommended for distance learning since it has more features. While Google Classroom is more suitable to be used as a support or complement for a learning course.


Introduction
The use of information technology in education emerged the e-learning (electronic learning) term.It covers a broad spectrum of activities, from its utilization for supporting learning to fully online learning [1].E-learning is defined as instructions delivered through digital devices such as computers or mobile devices to support learning [2].E-learning includes the use of the internet and other important technologies to produce learning materials, teach students, and also organize courses [3].An E-learning system that is specially designed to provide educational content, activities, and support for managing learning programs is called Learning Management Systems (LMS) [4].
Today's LMS leads to social and collaborative learning.The collaborative method emphasizes social learning and involves students sharing knowledge and doing tasks collaboratively [5].Social learning creates informal learning: students can build a network, share, collaborate, and exchange ideas to solve a problem [6].Edmodo and Google Classroom are the LMS that implement collaborative methods.Even though Edmodo and Google Classroom do not explicitly mention themselves as LMS, based on the characteristics of their services, many observers and researchers [7][8][9][10] categorize them as LMS.
Edmodo is one of the social learning networks to connect students with instructors (teachers).Edmodo's network allows teachers to share content, distribute quizzes, assignments, and manage communication with students, colleagues, and parents.In Edmodo, instructors can discuss in online classes, provide polls to check students' understanding, and give badges to students based on behavior or performance.Edmodo offers features that are: accelerating learning goals, activating free admin accounts, badge awards for individual students, building community networks, making polls for students, measuring student progress, connecting students, administrators/teachers, and parents, Online class discussions, personalize with the application.
Google Classroom is an online collaboration platform for teachers and students designed to enhance the learning experience.Teachers can create online classrooms, ask students to join classes, and create and give assignments.Students and teachers can communicate about assignments, teachers can also monitor the progress of their students.The features offered by Google Classroom are providing classrooms without paper, access to other Google products such as Gmail, Google Drive, Google Form, and collaboration between teachers and students outside the classroom, easy to manage, save money, and for teachers to better track student progress.
LMS becomes a medium of interaction between students and teachers so their interaction will be easy and intuitive [11].The low quality of interaction in e-learning will hamper the development of effective learning with technology [12].Poor usability and user experience (UX) of an LMS can make students spend more time learning the system than learning the educational content [11].Therefore, usability and user experience evaluation is important for e-learning.
E-learning is very different from other interactive systems.The meaning of usability and user experience, in general, is not sufficient to cover the user experience in the elearning context [13].It is important to evaluate the pedagogical design of e-learning systems [14] [11].So, the user experience evaluation method, in general, has not been able to evaluate e-learning.A special method for evaluating user experience on elearning is needed.Technique for User Experience Evaluation in E-learning (TUXEL) is a technique provided to evaluate the usability and user experience of an LMS [15].TUXEL is questionnaire-based so that it is not only easily implemented at a low cost but also allows collecting perceptions from the user's point of view [15].It has come with a developed version: TUXEL 2.0, which is a guided inspection approach.TUXEL 2.0 has three evaluation stages: (1) usability inspection, (2) pedagogical usability inspection, and (3) user experience evaluation.
The study was conducted at a vocational high school in Malang Indonesia as there was a specific need to use the LMS on online learning during the COVID-19 pandemic and industrial work practice.In their second year, students are required to carry out industrial work practices for six months, however, learning in schools continues in parallel.The duty of work practices limits the student's active hours of learning at school.These time limitations raise the need for teachers and students towards LMS to communicate and interact outside the physical classroom.
This study evaluates students' experiences of their interactions with Edmodo and Google Classroom.Based on the results of the evaluation, it will be concluded which LMS is felt most positive by the students.The results of the evaluation can be useful for teachers or the school for consideration in adopting which LMS is the most appropriate to be applied to support learning

Technique for User eXperience Evaluation in e-Learning (TUXEL) 2.0
The development of TUXEL begins with defining three dimensions for TUXEL: usability, pedagogical usability, and user experience.The study of the initial version of TUXEL evidenced problems regarding the difficulty of students in filling out the questionnaire.The participants had trouble remembering the function, aspect, or element of the LMS.Based on those problems, TUXEL 2.0 was developed.It is designed to gather deeper usability and user experience problems of LMS based on students' viewpoints [15].It is questionnaire-based so that it is easily implemented at a low cost and fast.It also does not require prior user testing and is more intuitive for inspectors in finding problems [15].
The TUXEL 2.0 has a preliminary stage and three main stages.The preliminary stages are task definition and checklist matching.The three main stages, which include three dimensions of TUXEL, are (1) usability inspection, (2) general LMS / pedagogical usability inspection, and (3) user experience evaluation.The usability dimension is divided into 4 categories: (1) login, (2) general interface, (3) assignment, (4) assessment.Questionnaires were given to the participants regarding these 4 categories.The "login" category questions are related to the clarity of instructions and feedback while users are logging in.The questions of the "general interface" category, related to the usefulness of the user interface, such as the navigation structure, the terminology used, and the information setups.The questions of the "assignments" category are intended to evaluate specific aspects related to the task, such as the clarity of instructions, the level of user control, and the feedback from the LMS.The "assessments" category poses questions related to the instructional feedback and goal.
The usability dimension is divided into 4 categories: (1) login, (2) general interface, (3) assignment, (4) assessment.Questionnaires were given to the participants regarding these 4 categories.The "login" category questions are related to the clarity of instructions and feedback while users are logging in.The questions of the "general interface" category, related to the usefulness of the user interface, such as the navigation structure, the terminology used, and the information setups.The questions of the "assignments" category are intended to evaluate specific aspects related to the task, such as the clarity of instructions, the level of user control, and the feedback from the LMS.The "assessments" category poses questions related to the instructional feedback and goal.
The pedagogical usability dimension is also divided into 4 categories: (1) help and documentation, (2) LMS learnability, (3) learning through the LMS, (4) LMS flexibility.The first category, "help and documentation" relates to whether the LMS provides assistance and whether the learning material is comprehensive and forwardlooking.The second category, "LMS learnability" is related to the ease of the system to learn.The third category, "learning through the LMS" is about evaluating the extent to which students can learn through the.The fourth category, "LMS flexibility" is to evaluate the extent to which students can choose the learning material they want.

Methodology
The study begins with the preliminary stages: defining inspection tasks and matching the tasks with TUXEL 2.0 categories.Furthermore, the evaluation involved students as inspectors.The inspectors were vocational high school students in Malang, Indonesia who had been carrying out industrial work practices.The inspectors was asked to perform determined tasks.There were two groups of inspectors, group one inspected Edmodo and another group inspected Google Classroom.There are 12 people in each group, so there are 24 evaluators in total.Groups of ten to twelve participants gave statistically significant results in user experience evaluation [17].The data was gathered through questionnaires.There were three questionnaires: usability inspection, general LMS / pedagogical usability inspection, and user experience evaluation.The data are then processed and analyzed per category per dimension.

Task Definition and Checklist Matching
The preliminary stage was determining the tasks that must be carried out by students as inspectors.The tasks were determined by analyzing features on Edmodo and Google Classroom so that the comparison will be equal.The task determination also refers to the four categories of TUXEL 2.0: general interface, user login, assignments, and assessments.Each task should represent TUXEL 2.0 categories as can be seen in table 1.

General Usability Inspection
The general usability inspection is analyzing the usability of the LMS.Students as inspectors performed the tasks given and inspected each task.Based on the problems list provided by TUXEL 2.0 (can be read further in reference [18]), the inspectors checked if there was any problem that occurred with each task.When the inspectors found a problem, they write down the problem ID, specify the location or type of task, and define the recurrence level of the problem (the problem is repeating, repeating throughout the system, or repeating on several tasks on the report table provided by TUXEL 2.0 (see the "Usability Inspection -Problem Reporting Table" on reference [18]).If the problem repeats for several tasks, the task must also be defined.

Pedagogical Usability Inspection
Pedagogical usability is the usability that is related to the type of learning material that leads users to reach their learning achievements, so it will have implications for the learning platform [19].At this stage, it is expected that an LMS could support the delivery of learning material as well as facilitate the learning process.TUXEL 2.0 provides the "General LMS Evaluation Checklist" (see on reference [18]) to evaluate the pedagogical usability.The checklist was adapted into Bahasa Indonesia and filled by inspectors.Inspectors marked items that correspond to problems encountered when operating the LMS and explained the details of the problem in the comments section.

User Experience Evaluation
The last step is evaluating the user experience of the LMS using a seven-point semantic differential scale questionnaire.The aim is to gather student perceptions regarding student experiences when using LMS.TUXEL 2.0 adapts 12 of the 26 UEQ [16] pairs of adjectives: (1) unlikable/pleasing, (2) uncomfortable/comfortable, (3) does not meet expectations/meets expectations, (4) obstructive/supportive, (5) inefficient/efficient, (6) impractical/practical, (7) conventional/innovative, (8) uncreative/creative, (9) complicated/easy, (10) confusing /clear, (11) not interesting/interesting, ( 12) demotivating/motivating.The inspectors rated their experience on LMS by choosing one of seven-point that comes closest to the adjective that portrayed their feelings.After filling out the scale, the inspector gave responses to criticism or advice.Those seven-point semantic differential scale data were converted to a score range of -3 to +3, thereafter the median score was calculated.

Analysis
Data from the inspectors were collected, processed, and analyzed.The general usability inspection data were analyzed by counting the number of problems per task and item ID, examining the problem description, and checking whether the inspector assessed the problem as recurring.The pedagogical usability inspection data were analyzed for each aspect by counting the problems that had been check-marked and looking at the keywords of the comments given.Meanwhile, the data collected from the user experience form were processed by calculating the median for each pair of adjectives and examining the keywords of the comments given.For each dimension (the general usability, pedagogical usability, and user experience), the calculation results were linked with the keywords of the comments given.Thereafter the problem concluded from each dimension.Finally, the problems of each dimension were linked to drawing a complete conclusion related to the learning experience of the LMSs.

Results
Table 2 shows the number of problematic items and table 3 shows descriptions of the problematic items found on the inspection of the general usability and pedagogical usability dimensions.

General Usability of Edmodo versus Google Classroom
On the overall general usability dimension, Edmodo has fewer problematic aspects (9 items) than Google Classroom (12 items).In both LMS, problems are found in the login, general interface, assignments, and assessments categories.Out of the four items in the login category, one problematic item was found in Edmodo, and three problems were found in Google Classroom.Out of the 12 items in the general interface category, six problematic items were found in Edmodo, and five problems were found in Google Classroom.Out of the seven items in the assignment category, two problematic items were found in both LMSs.Out of the three items in the assessment category, no problematic items were found in Edmodo and two problems were found in Google Classroom.
The problem with logging in to Edmodo was discovered by students who felt confused when choosing which column is required because there was no sign for the required column (LF2).Meanwhile, the problem with logging in to Google Classroom is that when logging in, students were redirected to the Google Account login page which is considered confusing by students (LF4, LF3, and LF1).In the General Interface category, students reported difficulties in both LMSs to find navigation, menus, and options (N1, N2).Students were also struggling to understand icons, buttons, and labels (L4).In Edmodo, students found unnecessary content and/or information (VD2), yet important information is not visible (VD3).
The problem in the assignment category is that students felt that the instructions or instructions are not clear (L2) on assignments in Edmodo and there is no information on how to fill in the correct way (LF4) on assignments in Google Classroom.Students also found that Edmodo does not accommodate action cancellation, while Google Classroom does not give success or failure messages when submitting assignments (LF1).In the assessment category, there is no problematic item was found in Edmodo, whereas there are two problems were found in Google Classroom are related to no immediate feedback provided for a quiz (IF1) and no additional material or link when the student finished the task (IF2).

Pedagogical Usability of Edmodo versus Google Classroom
On the overall pedagogical usability dimension, Edmodo has fewer problematic aspects (13 items) than Google Classroom (24 items).In both LMS, problems are found in the help and documentation, LMS learnability, learning through the LMS, and LMS flexibility categories.Out of the three items in the help and documentation category, all items were found problematic in both LMSs.Out of the five items in the LMS learnability category, all items are problematic in Edmodo and only three problematic items were found in Google Classroom.Out of the six items in the learning through the LMS category, two problems were found in Edmodo but all items were found problematic in Google Classroom.Out of the seven items in the assessment category, three problematic items were found in Edmodo and six problems were found in Google Classroom.
The problem with the help and documentation category in both LMSs are manual or online help is not provided (HD1); manual or online help is hard to find (HD2); and instruction provided is complex (HD3).In the LMS learnability category, problems were found related to the ease with which LMS was learned by the user.Both LMS were reported that not intuitive so students are difficult to operate them (L1), students need other help to complete tasks (L3) and students will forget how to operate LMS after a long time (L6).However, Google Classroom is considered easier and quicker (L7 and N3) to perform tasks rather than Edmodo.
The problem with the learning through the LMS category is both LMSs do not provide features for students to communicate with students or teachers (CL2), and view the activities and grades of other students (CL3) on the LMS.On top of that, Google Classroom does not provide services to carry out group activities (CL1), file sharing from students to other students (CL4), no question and answer forum between students (CL5) and no progress tracking in the completion of the course (OO2).On the LMS flexibility category, both LMSs reported that students are unable to choose a course, order and pace to study (FL1), consult difficulties in learning (FL2) and customize the interfaces of the LMSs (P1).On top of that, Google Classroom does not provide additional feedback with the teacher/instructor (FI3), shortcuts to increase productivity

User Experience of Edmodo versus Google Classroom
Figure 1 shows the overall median calculation for user experience in both LMSs which tend to be neutral and positive (scores 0 to 3).Almost all of the keywords show Google Classroom has more positive scores than Edmodo, except for the last keyword i.e. "creative".Edmodo has a neutral median score (0) of the "pleasant" keyword whereas Google Classroom has a positive score (2).In the "comfortable" keyword, the median score for Edmodo is quite positive (1) while Google Classroom score is positive (2).In the "efficient" keyword, the median score of Edmodo and Google Classroom both positive (2).In "practical" keywords, Edmodo's median score is between positive/very positive (2.5), slightly below Google Classroom which is very positive (3).Followed by the "easy" keyword, Edmodo is rated between neutral-fairly positive (0.5), far below Google Classroom which is positive (2).In the "event" keyword, the median score for Edmodo is neutral (0), almost the same as Google Classroom which is between neutral/fairly positive (0.5).Furthermore, the "meets expectations" keyword on Edmodo is neutral (0) while Google Classroom is quite positive (1).In the "supportive" keyword, the median score of both Edmodo and Google Classroom is positive (2).The "interesting" keyword, Edmodo has a median between quite positive/positive (1.5) and Google Classroom is positive (2).In the "motivate to learn" keyword, the median score p-ISSN: 2540-9433; e-ISSN: 2540-9824 of Edmodo is quite positive/positive (1.5) while the median of Google Classroom is positive (2).The "inventive" keyword shows the similarity of the median score between and Google Classroom, which is positive (2).For the "creative" keyword, the median score of Edmodo is very positive (3), above the Google Classroom score which is positive (2).

Conclusion and Recommendation
The results of the overall general usability and pedagogical usability inspections show fewer problematic aspects in Edmodo (22 problematic items) than Google Classroom (30 problematic items).This is because Edmodo provides more complete features than Google Classroom.Edmodo features are more complete for the assessment, learning through the LMS and LMS flexibility categories.Edmodo provides scoring along with immediate and relevant feedback on students' assessments.Edmodo also provides space for group activities and discussion forums between students and teachers.However, the simplicity of Google Classroom seems to be preferred by students.Although it does not provide as many features as Edmodo, students find Google Classroom more fun, comfortable, practical, interesting and motivating to learn and fulfill students' expectations of an LMS, rather than Edmodo.
The problems encountered in both LMS are related to the ease of use.On the general interface category, students reported difficulty finding navigation, menus or options on the LMS.The user experience measurement also shows that the "confusing/evident" keyword is calculated as neutral.Joyce and Nielsen [20] argue that teenagers, of which high school students are included, are over-confidence, but they perform worse than adults in performance on the web.Due to their lower reading levels, impatience, and underdeveloped research skills, thereby reducing their successes in doing the tasks.Therefore, usability is thus as important for students as well as for any other user group.
Edmodo claims to be an LMS with a variety of complete questions/evaluations.Google Classroom has the advantage of integrating GDrive, GMail, and other technologies.In addition, Google Classroom provides features that provide freedom to fail during exploration.So that, Edmodo is recommended for distance learning since it has more complete features, especially for assessment.Online learning with Edmodo allows students to receive feedback from learning activities and communicate to students or teachers.While Google Classroom is more suitable to be used as a support or complement for a learning course as it is simpler and more practical.Google Classroom makes it easy for students to get learning material from the teacher or submit assignments.

Table 1 .
The tasks for Edmodo and Google Classroom inspection.

Table 2 .
Number of problematic items found on General Usability and Pedagogical Usability Dimension Hanifah Muslimah et al. , User Experience in Learning Management System...201 p-ISSN: 2540-9433; e-ISSN: 2540-9824

Table 3 .
Problems Classification and Description per Item's ID