Education in the Knowledge Society 23 (2022)

Comparison of the Level of Collaborative Learning in a Distance Course

Comparación del nivel de aprendizaje colaborativo en un curso a distancia

William Reyes^a

^aUniversidad Autónoma de Yucatán, Mérida, México.

https://orcid.org/0000-0002-3443-6385 wreyes@correo.uady.mx

ABSTRACT

In this research, collaborative learning was compared in four versions of a distance course, identified as V1, V2, V3, and V4, each designed with various learning strategies. This study aimed to achieve the following objectives: 1) Compare the level of collaborative learning obtained from a distance course over time. 2) Compare the level of collaborative learning obtained in the different versions of a distance course. 3) Identify the elements that influenced collaborative learning over time. 4) Identify if the modifications in the strategies carried out in the distance course impacted the level of collaborative learning. The study design was of the non-experimental evolutionary type, with a trend and group comparison. The instrument was administered at the end of each version of the course. This instrument was named Constructivist On-Line Learning Environment Survey, or COLLES, whose dimensions measure collaborative learning in a virtual learning environment. The results showed that both Interactivity and Relevance dimensions had an upward trend during the duration of the research, while in comparing the versions of the course, V3 had the highest scores in the six dimensions of collaborative learning, in addition to this. The version had a significant impact on Interactivity, Relevance, and Peer Support dimensions. In general, all versions of the course had satisfactory results at the level of collaborative learning. However, it is concluded that, due to its structure and strategies used, V3 is the one that favored collaborative learning the most.

Keywords Comparative study; Learning strategy; Collaborative learning; Distance teaching.

RESUMEN

En esta investigación se comparó el nivel de aprendizaje colaborativo en cuatro versiones de un curso a distancia, identificadas como V1, V2, V3 y V4, cada una diseñada con variadas estrategias de aprendizaje. La investigación tuvo los siguientes objetivos: 1) Comparar el nivel de aprendizaje colaborativo obtenido de un curso a distancia a través del tiempo. 2) Comparar el nivel de aprendizaje colaborativo obtenido en las distintas versiones de un curso a distancia. 3) Identificar los elementos que influyeron en el aprendizaje colaborativo a través del tiempo. 4) Identificar si las modificaciones en las estrategias realizadas en el curso a distancia tuvo algún impacto en nivel de aprendizaje colaborativo. El diseño del estudio fue del tipo no experimental evolutivo, de tendencia y comparación de grupos, y para ello se administró al final de cada versión del curso el instrumento Encuesta sobre el entorno de aprendizaje constructivista en línea, COLLES, por sus siglas en inglés, cuyas dimensiones miden el aprendizaje colaborativo en un entorno virtual de aprendizaje. En los resultados se encontró que las dimensiones Interactividad y Relevancia tuvieron una tendencia al alza durante el tiempo que duró la investigación. Por otra parte, al comparar las versiones del curso, V3 tuvo las mayores puntuaciones en las seis dimensiones del aprendizaje colaborativo, además esta versión tuvo un impacto significativo en las dimensiones de Interactividad, Relevancia y Apoyo de los Compañeros. En general, todas las versiones del curso tuvieron resultados satisfactorios en el nivel de aprendizaje colaborativo, sin embargo, se concluye que, por su estructura y estrategias empleadas, V3 es la que más favoreció al aprendizaje colaborativo.

Palabras clave Estudio comparativo; Estrategias de aprendizaje; Aprendizaje colaborativo; Enseñanza a distancia.

1. Introduction

The upswing and expansion of distance education are evident. Especially in higher education, largely because learning is subjectively dependent on the personal context of each student, unlike face-to-face education, where the work rhythm is defined by the class sequence or the teacher (Clark, 2020). In addition to this, new generations of students perceive the internet as a natural way to learn (Choudhury & Pattnaik, 2020). Other characteristics would be the course design (Lee, Yoon, & Lee, 2009; Olasina, 2019) the quality of the materials used in classes (Vershitskaya et al., 2020; Carter & Hagood, 2019; Rengel, Pascual, Íñiguez, Martín, & Vasallo, 2019; Edmundson, 2006) and the set of skills that teachers might adopt to develop e-learning on their students (Semradova & Hubackova, 2016); also, with the implementations of new innovative strategies on this modality such as the flipped classroom (Strelan, Osborn, & Palmer, 2020), gamification (Torres-Toukoumidis, Ramírez-Montoya, & Romero-Rodríguez, 2019) or the active teaching methods (Pološki & Aleksic, 2020), e-learning becomes a suitable modality for distance courses, just as society is nowadays demanding (García-Peñalvo, 2020).

However, diverse challenges might appear for those institutions and teachers who develop courses, workshops, and programs on e-learning environments; some of those examples would be to ensure that students are learning effectively (Miguel, Caballé, & Xhafa, 2017; Ahmed, Hussain, & Farid, 2018; Poondej & Lerdpornkulrat, 2019), to provide them with an immediate and high-quality based feedback (Al-Hamad, Alhamad, & Al-Omari, 2020) and higher support by the teachers and tutors (Hiliger et al., 2020); in other words, greater interaction and support among students, with their teachers and tutors, to promote dialogue, debate, organization of ideas and; furthermore, that they would assume their responsibility to learn (Mercado, 2015). Although various approaches and strategies exist, it is in collaborative learning that the training needs in distance education can be attended to today.

1.1. Collaborative learning in distance education

Although collaborative learning has been often used in real-life interactional contexts (Gros, 2005), it is on distance education where it has become more relevant, especially by incorporating Information and Communication Technologies (ICT) and computer-based systems because these are tools that allow the achievement of a “shared, coordinated and interdependent process, in which students work together to reach a common goal on a virtual environment” (Guiter & Pérez-Mateo, 2013, p. 24). However, achieving collaborative learning in distance education implies challenges on the design of strategies; for example, Azhari et. al (2020) pointed out that the strategies with high levels of interactivity such as the desire to share ideas, recommend solutions and answer questions from their peers were those in which they generated and maintained expectations and interest of the students, generally because of design innovation, due to the surprise factor or the disruption of the conventionalism found on learning activities; unlike strategies designed as homework where technology did not provide innovative or differentiating elements for the students.

In other research, Sahani (2018) identified that the most valued technology-based collaborative activities were those that innovatively used e-learning and digital resources, involving students and incorporating them into realistic and relevant learning contexts. Other investigations, such as those by Syed-Mohamad, Pardi, Zainal and Ismail (2006), Neroni, Meijs, Gijselaers and Kirschner (2019), Fraternali and Herrera (2019), and Vershitskaya, Mikhaylova, Gilmanshina, Dorozhkin and Epaneshnikov (2020), agreed that collaborative learning requires innovative and empowering elements to generate both desire and interest on students to get involved voluntarily; especially when it comes to a virtual environment. This implies that teachers already possess sufficient competencies to design effective collaborative environments (Loes, 2019), as well as enough creativity to innovate and stimulate their students to learn (García-Peñalvo, 2021). As can be seen, collaborative learning in distance education is an amalgam of innovation in the use of e-learning technologies and the correct choice of strategies so that students could successfully interact and participate voluntarily while achieving training.

1.2. Context and objectives of the research

The Autonomous University of Yucatan (UADY by its acronym in Spanish), located in the southeast of Mexico, has offered a course and educational programs in distance modality since 2000 (UADY, 2013), using a learning management system Moodle, in addition to its educational model allows the design and development of blended and distance courses (UADY, 2012). The distance course used for this study has been taught since 2013. During that time, various learning strategies were implemented, initially following a conventional type such as online tasks and discussion forums, progressing through more elaborate ones such as individual and group challenges, to the most developed, and advanced types based on gamification and key interactions such as badges, points, narratives, and commissions, among others. With these changes, four versions of the course were defined, to which various analyzes have been carried out (Reyes & Quiñonez, 2020), however for this research, the interest was focused on the level of collaborative learning comparing the different versions of the course, the reason for which the following objectives were determined:

O1. Compare the level of collaborative learning obtained from a distance course over time.

O2. Compare the level of collaborative learning obtained in the different versions of a distance course.

O3. Identify the elements that influenced collaborative learning over time.

O4. Identify if the modifications in the strategies carried out in the distance course had any impact on the level of collaborative learning.

2. Methodology

2.1. Design

The type of study carried out was a non-experimental evolutionary, trend, and group comparison (Stockemer, 2019), administering a standardized survey to compare collaborative learning of the distance course between the years 2013 and 2019.

2.1.1 Characteristics and evolution of the distance course

The distance learning course of this research is taught in the UADY degree program in education, it is optional and it is offered in the intensive summer period in June and July, three hours a day (60 hours in total). Table 1 describes the general elements of the distance course.

The academic program of the course is the same from the beginning of the research, as well the general objective, evaluation criteria, learning units, the products to be delivered, as well as the same learning management system where it is taught. To identify each version of the distance course, a key was assigned: V1, V2, V3, and V4. Table 2 presents a general description of each version of the course and the years in which they were implemented.

In V1 the working scheme was through the sequence of units and topics, all the resources were available for the accomplishment of the tasks, both individually and in teams, some of those tasks were through the discussion forums to debate and exchange ideas on the topics. In the end, the students handed in a portfolio of evidence with the best materials they had designed during the course. The general scheme of work in V1 is presented in Figure 1.

This version of the distance learning course was developed on the Moodle platform, the resources were mainly hyperlinks to videos and readings. The tasks were geared towards goal achievement rather than collaborative learning. When the assignments were completed and the portfolio of evidence handed in, the course ended and the final grades were given.

In V2, some characteristics of gamification were implemented (Bai, Foon, & Huang, 2020), for example, using a less academic and more playful language (Sheldon, 2012), as well as the design of activities more oriented to games and video games (Kapp, 2012), so tasks were changed for individual and group challenges and had prerequisites to continue (see Figure 2). Discussion forums were incorporated to resolve doubts and the instructor followed up on the students’ progress so that everyone had the same opportunities to progress through the challenges (Reyes & Góngora, 2016).

V2 was also developed on the Moodle platform, the same program was used, however, the focus of this version was on challenges, and resources available were geared towards to solve them. As in V1, a portfolio of evidence with the best works was handed in to complete the course.

In V3 it was done through storytelling, allowing students, in teams, to assume a role and be immersed in learning experiences through gamification (Kapp, 2012). The challenges were transformed into chapters, again using the prerequisites for the teams to continue with history (see Figure 3). Resources were exchanged for more up-to-date ones, and there was greater follow-up through forums and messages to answer questions during the course.

In this version of the course, both individual and group badges were implemented, considered as rewards (Werbach & Hunter, 2015) for carrying out activities in the context of the course in a relevant way, such as completing a chapter before the deadline, delivering materials with characteristics beyond what was requested, among others. The development of this course was also carried out in Moodle, incorporating multimedia elements such as audios to explain the story and the indications for the delivery of materials, as well as the use of images and graphics to indicate the way forward on the platform. The portfolio of evidence and course completion remained the same as in previous versions.

For V4, the story was changed to a more complex one with subplots while the chapters were maintained. More mechanics such as the PBL Triad were added: Points, Badges and Leaderboards (Werbach & Hunter, 2015) and the virtual escape room, also the Commissions were incorporated based on this textual type “allows overcoming almost routine practices in-school activity” (Fajre & Aranciba, 2000, p. 134), is more appropriate due to how the instructions to be followed by the students are presented. These mechanics gave a greater variety of activities to carry out; when accomplishing all the commissions the next chapter was unlocked (see Figure 4), the resources that were used to accomplish the commissions were similar to those of V3.

As mentioned, the changes in the distance course were in teaching and learning strategies; therefore, to contrast the differences that existed in the different versions of the distance course concerning the activities; Table 3 shows an example of the indications that were made in the different versions of the distance course.

As can be seen in the example, the theme and objective were the same, the product was similar for the four versions, however, each version had different indications, while in V1 it is a conventional task, in the other versions they evolved in their approach and intentionality.

2.1.2. Participants

Between 2013 and 2019, 158 undergraduate students from the same Faculty participated in the course, their participation was voluntary as it is an optional course. It was established as a requirement that they have completed 50% or more of the curriculum to have sufficient skills to be fit for the course, and have acquired previous experiences with subjects in distance learning. Table 4 shows the distribution of students by year.

2.1.3. Instrument

The instrument selected was the COLLES [Constructivist On-Line Learning Environment Survey] (Taylor & Maor, 2000) that allows evaluating the quality of a distance learning environment from the perspective of collaborative learning (Dougiamas & Taylor, 2002). The instrument consists of 24 items divided into six dimensions (see annex), which refer to “the perceptions of the students about the existence of a virtual classroom environment that helps them to reconstruct themselves as reflective and collaborative” (Taylor & Maor, 2000, p. 4). Table 5 presents the instrument specifications.

To answer each item, the student selected an option according to a Likert scale consisting of five alternatives.:

“1= Almost Never – 2= Seldom – 3= Sometimes – 4= Often – 5= Almost always”

This instrument is found by default in the Moodle platform as an activity called a predefined survey, so it was used in all versions of the course without any modification on the items or dimensions.

Regarding reliability and validity, similar studies using this instrument verified through internal validity tests that it has an α coefficient of .80 (Rivero, 2018; Azhari et al., 2020); In the case of reliability, Baker (2005) points out that the dimensions on this instrument are useful for teachers and researchers, and that it is easy to understand for students of any educational level. On the other hand, in a study conducted by Sthapornnanon, Sakulbumrungsil, Theeraroungchaisri and Watcharadamrongkun (2009), the scores obtained from this instrument coincided with those of other instruments, obtaining more precise results on the perception of students towards collaborative learning on a virtual environment. Finally, the studies by Pearson (2005), Pearson and Trinidad (2005), and Trinidad, Aldridge and Fraser (2005) separately used the dimensions whose results were satisfactory in distance learning environments. We can conclude that the evidence shown by these studies validates this instrument and its dimensions. Also, it has been used in international studies on universities worldwide with similar results, indicating the instrument used for this research as reliable.

2.1.4. Data processing

Students answered the online survey after completing the last activity; on V1 it was enabled manually, on V2, V3, and V4 the access was automatic once that they had finished the last challenge or chapter respectively. The responses were stored on the platform database and downloaded in CSV format. Each downloaded file was assigned the year of the course where it belonged. All files with the answers were integrated into a single database, identifying each record of the year of the course to which it belonged. The items, in the six dimensions and the years in the course versions, were grouped in the database. The means of the scores for each dimension were obtained to compare them between each year and know their trend. Non-parametric tests were also performed comparing means between groups and the ANOVA of a Kruskal Wallis factor. In all tests, the significance value considered was .05. The integration of the database and its processing was carried out with the statistical programs SPSS version 24 and Jamovi version 1.1.9.0.

2.1.5. Control variables

Due to the nature of this study, variables that could affect the results were identified; These control variables were assigned inclusion and exclusion criteria to keep them constant, eliminating their effect during the duration of the investigation and for it to be reliable and valid (Schjoedt & Sangboon, 2015).

2.2. Limitations and delimitations of the study

Although the identified variables were controlled to reduce or eliminate their effect on the research, it is recognized that other strange variables affected the results, for example, not having a statistical procedure to determine the sample due to the voluntary participation of students in the course. Also, there was no control on the number of participants enrolled in the course due to the needs of the institution, especially in V2, which made the number of participants per year highly variable.

On the other hand, in the studies where this instrument was used, it was administered twice: at the beginning and at the end of each unit to compare both scores, this procedure is known as the “preferred and actual survey” (Syed-Mohamad et al., 2006, p. 187; Sthapornnanon et al., 2009, p. 5; Rivero, 2018, p. 199; Azhari et al., 2020, p. 274); in this research, it was intentionally administered just on one occasion: at the end of the course in each group to know the trend over time and comparison between versions of the course.

3. Results

After processing the collected data and obtaining the means of the scores, as well as the statistical tests mentioned in the methodology section, the results obtained in this investigation are presented as it follows.

3.1. Dimensional trend

By contrasting the dimensions in each of the years in which the course was taught, the scores that can be seen in Figure 5 were obtained.

According to the previous figure, an upward trend in the dimensions of Interactivity is distinguished (from 3.20 in 2013 to 4.25 in 2019) and Peer Support (from 3.50 in 2013 to 4.06 in 2019); in the other dimensions the trends were irregular and it was not possible to establish a trend per se; however, the highest scores are observed in 2016, decreasing in 2017 and 2018, increasing again in 2019; As seen in the Figure 5, there are differences from one year to another in the same version of the course.

By grouping the scores by course version in the six dimensions, the graphs are shown in Figure 6 were obtained.

Comparing each of the dimensions, it was found that V3 had the highest scores and V1 the lowest. In general, the results show that the scores of the course versions in all dimensions are high.

3.2. Course versions and their effect on dimensions

Once the scores were obtained, a K-W test was performed, this was to identify significant differences in the dimensions according to each version of the course. Table 7 shows the following results:

According to the scores obtained, there are significant differences in the dimensions of Interactivity, Relevance, and Peer Support. From the above, the versions of the course were compared in pairs. Table 8 shows these comparisons in each of the dimensions.

In the case of the Interactivity dimension, the difference found was between V1 with V3 and V1 with V4, in the other comparisons there is equality in the results. Regarding the Relevance dimension, the differences found were between V1 with V4 and V3 with V4. Finally, in the Peer Support dimension, the difference found was between V1 with V3 and V1 with V4 of the distance course.

4. Discussion

In the trend analysis, it was found that the Interactivity and Relevance dimensions increased, in the other dimensions the trend was irregular, this prevents comparing the level of collaborative learning through time. However, scores on the dimensions of this research are high when compared to other studies with Pearson and Trinidad (2005), Syed-Mohamad et al. (2006), Sthapornnanon et al. (2009), and Azhari et al. (2020), therefore, is an indicator that the distance learning course is adequately designed for collaborative work in virtuality.

On the other hand, when comparing the versions of the distance course, V3 had the highest scores in all six dimensions, besides, in the multiple comparison test, significant differences were found in the dimensions Interactivity, Relevance, and Peer Support, suggesting that the strategies employed involved students more in participating in the activities, working actively in the team and valuing what they learned for their professional training (Taylor & Maor, 2000; Dougiamas & Taylor, 2002).

It is noteworthy that V4, despite using a more complex narrative, with more multimedia resources, more slogans, applying the PBL triad, with more elements on the screen and with an instructional design consistent with the UADY Educational Model where the course was implemented (UADY, 2012; Reyes & Quiñonez, 2020), was not the version with the highest ratings, and that in the comparison tests it was not significant for this version if we take into account that the design of V4 should have been the result of the experiences acquired and the improvement in the elaboration of strategies throughout seven years of work in the same course. There is a possibility that the narrative used in this version distracted the students rather than helping them to focus on the activities, as the comments found in the forums and messages showed a marked interest in what would happen in the next chapter; this is not necessarily a bad thing, as Fanfarelli (2020) considers this to be an input for students to get involved in the learning process and perform their tasks.

Another aspect that is considered relevant in the investigation is that it confirmed the internal reliability of the COLLES survey was corroborated, which when making the corresponding statistical analysis, a coefficient of α = .924 was obtained, which coincides with all the studies analyzed, confirming the reliability of the instrument.

The results of this research contribute to affirming that the dimensions of collaborative learning are reliable predictors of student academic achievement, as in the studies by Azhari et al. (2020) concerning interactivity in virtual environments, that of Sahani (2018) about the relevance of learning using technological resources, and that of Neroni et al. (2019) about peer support.

Each version contributed elements of collaborative learning because the strategies used were intentionally aimed at significantly improving the experiences of students in virtual environments, which coincides with Hurlbut (2018) in his analysis of distance education as an enriching, collaborative environment full of new technology-driven learning experiences. These strategies allowed to maintain the attention and commitment of the students towards collaborative learning during the process, following Fanfarelli (2020) about the challenges facing distance education as an increasingly viable alternative for higher education.

In general, all the versions of the distance course had satisfactory results in all dimensions, besides, it allowed exploring various strategies that, are important for students to feel comfortable and stay experiencing fun and excitement while learning collaboratively (Zainuddin, Wah, Shujahat, & Perera 2020).

5. Conclusions and Recommendations

Although all versions of the course obtained satisfactory results for collaborative learning, the highest scores and the significant differences in the multiple comparisons were for V3, a distance course that was designed with strategies that encouraged collaboration, primarily through overall narrative and chapter advancement that involved the student with his or her team to achieve common goals which impacted on the dimensions of Interactivity, Relevance, and Peer Support.

This research provides important elements for the design of distance courses with a collaborative approach; innovative strategies, enhanced by ICTs, that generate interest and commitment from students, mainly for learning and collaboration, should be considered. From the evidence presented, it is recommended to use strategies focused on collaborative learning such as team building and that teams progress by fulfilling the tasks assigned as chapters in a unique narrative, giving rewards such as badges to students for the completion of activities that contribute to the course and enrich it; this confirms what the literature says about game-playing strategies concerning using at least three of them throughout a course since using less does not generate students' motivation and commitment to the course; on the contrary, it is risky to implement many strategies because, as with V4, it was a good course but did not significantly impact on collaborative learning.

Finally, it is recommended to analyze the course program, as well as the technical and administrative conditions existing in the institution to carry out the necessary modifications; but most importantly, the teachers desire to make the changes in the design and development of the tasks and activities to orientate them to strategies such as gamification.

As presented in this research, it is an arduous but constant path, which required years of experience to perfect the strategies used in the distance course to achieve high levels of collaborative learning.

6. Gratitude and fundings

This work is part of the "Gamification strategies to strengthen innovative practices and experiences in eLearning", a project funded by the Program for Professional Teaching Development, for the Superior Type (PRODEP for its acronym in Spanish) within the framework of the call "Support for the incorporation of new Time Teachers Complete (PTC for its acronym in Spanish)”.

Also; I would like to thank the School of Education of the Autonomous University of Yucatan for the use of its administrative facilities, human and material resources to carry out this research. Finally, to all students of the Bachelor of Education who participated actively and enthusiastically.

7. References

Ahmed, M., Hussain, S., & Farid, S. (2018). Factors Influencing the Adoption of e-Learning in an Open and Distance Learning Institution of Pakistan. The Electronic Journal of e-Learning, 16(2), 148–158.

Al-Hamad, N., Alhamad, A. & Al-Omari, F. (2020). Smart devices employment in teaching and learning: reality and challenges in Jordan universities. Smart Learning Environments, 7(1), 1–15. https://doi.org/10.1186/s40561-020-0115-0

Azhari, F., Jasmi, N., Abd, M., Jofrry, S., Lee, K., & Ming, L. (2020). Students’ Perceptions about Social Constructivist Learning Environment in e-learning. Indian Journal of Pharmaceutical Education and Research, 54(2), 271–278. https://doi.org/10.5530/ijper.54.2.31

Bai, S., Foon, K., & Huang, B. (2020). Does gamification improve student learning outcome? Evidence from a meta-analysis and synthesis of qualitative data in educational contexts. Educational Research Review, 30, 1–20. https://doi.org/10.1016/j.edurev.2020.100322

Baker, J. (2005). Constructivist Online Learning Environment Survey. In R. Reynolds (Ed.), Handbook of Research on Electronic Surveys and Measurements (pp. 299–301). Idea Group Inc. https://doi.org/10.4018/9781591407928.ch036

Carter, C., & Hagood, D. (2019). Activity Monitor Gaming and the Next Generation Science Standards: Students Engaging with Data, Measurement Limitations, and Personal Relevance. Journal of Science Education and Technology, 28, 589–601. https://doi.org/10.1007/s10956-019-09789-5

Choudhury, S., & Pattnaik, S. (2020). Emerging themes in e-learning: A review from Stakeholers'. Compuers & Education, 144(103657). https://doi.org/10.1016/j.compedu.2019.103657

Clark, T. (2020). Distance education. In E. Iadanza (Ed.) Clinical Engineering Handbook (2^nd ed.) (pp. 410–415). Academic Press. https://doi.org/10.1016/B978-0-12-813467-2.00063-8

Dougiamas, M. & Taylor, P. (2002). Interpretive analysis of an internet-based course constructed using a new courseware tool called Moodle. https://bit.ly/37oEOk9

Edmundson, A. (2006). Globalized e-learning cultural challenges. IGI Global. https://doi.org/10.4018/978-1-59904-301-2

Fajre, C., & Aranciba, V. (2000). La consigna: un manual de instrucciones para leer la escuela. Didáctica. Lengua y Literatura, 121–138.

Fanfarelli, J. (2020). Impact of narrative and badging on learning and engagement in a psychology learning game. British Journal of Educational Technology, 51(2), 387–419. https://doi.org/10.1111/bjet.12838

Fraternali, P., & Herrera, S. (2019). Model Driven Development of Gamified Applications. Journal of Web Engineering, 18(7), 655–694. https://doi.org/10.13052/jwe1540-9589.1874

Gagné, R., Wager, W., Golas, K., & Keller, J. (2005). Principles of Instructional Design (5^th Ed.). Thomson Wadsworth Leaning. https://doi.org/10.1002/pfi.4140440211

García-Peñalvo, F. J. (2020). Modelo de referencia para la enseñanza no presencial en universidades presenciales. Campus Virtuales, 9(1), 41–56.

García-Peñalvo, F. J. (2021). Avoiding the Dark Side of Digital Transformation in Teaching. An Institutional Reference Framework for eLearning in Higher Education. Sustainability, 13(4), Article 2023. https://doi.org/10.3390/su13042023

Gros, B. (2005). El aprendizaje colaborativo a través de la red: límites y posibilidades. Aula de innovación educativa, (162), 44–50.

Guiter, M. & Pérez-Mateo, M. (2013). Networked collaboration: towards a definition of online collaborative learning. Theory of Education: Education and Culture in the Information Society, 14(1), 10–30. https://doi.org/10.14201/eks.9440

Hilliger, I., Ortiz-Rojas, M., Pesántez-Cabrera, P., Scheihing, E., Tsai, Y.-S., Muñoz-Merino, P. J., Broos, T., Whitelock-Wainwright, A., & Pérez-Sanagustín, M. (2020). Identifying needs for learning analytics adoption in Latin American universities: A mixed-methods approach. The Internet and Higher Education, 45, Article 100726. https://doi.org/10.1016/j.iheduc.2020.100726

Hurlbut, A. (2018). Online vs. traditional learning in teacher education: a comparison of student progress. American Journal of Distance Education, 32(4), 248–266. https://doi.org/10.1080/08923647.2018.1509265

Kapp, K. (2012). The Gamificaction of learning and instruction. Pfeiffer.

Lee, B., Yoon, J., & Lee, I. (2009). Learners’ acceptance of e-learning in South Korea: Theories and results. Computers & Education, 53, 1320–1329. https://doi.org/10.1016/j.compedu.2009.06.014

Loes, C. (2019). Applied Learning through Collaborative Educational Experiences. New Directions for Higher Education, 2019(188), 13–21. https://doi.org/10.1002/he.20341

Mercado, R. (2015). El Aprendizaje Colaborativo a Distancia en México. in J. Zubieta & C. Rama, La Educación a Distancia en México (pp. 97–114). UNAM. https://doi.org/10.13140/RG.2.1.1440.9360

Miguel, J., Caballé, S., & Xhafa, F. (2017). Trustworthiness for secure collaborative learning. In J. Miguel, S. Caballé y F. Xhafa (Eds.), Intelligent Data Analysis for e-Learning (pp. 25–48). Academic Press. https://doi.org/10.1016/B978-0-12-804535-0.00003-4

Neroni, J., Meijs, C., Gijselaers, H., & Kirschner, P. (2019). Learning strategies and academic performance in distance education. Learning and Individual Differences, 73, 1–7. https://doi.org/10.1016/j.lindif.2019.04.007

Olasina, G. (2019). Human and social factors affecting the decision of students to accept e-learning. Interactive Learning Enviroment, 27(3), 363–376. https://doi.org/10.1080/10494820.2018.1474233

Pearson, J. (2005). Evaluatin e-learning enviroments in initial teacher education programmes using the online learning enviroment survey (OLES). 8th IFIP World Conference on Computers in Education, WCCE 2005 (pp. 1–9). Emerald Group Publishing LTtd.

Pearson, J., & Trinidad, S. (2005). OLES: an instrument for refining the design of e-learning environments. Journal of Computer Assisted learning, 21(6), 396–404. https://doi.org/10.1111/j.1365-2729.2005.00146.x

Pološki, N., & Aleksic, A. (2020). Are Active Teaching Methods Suitable for All Generation Y students?—Creativity as a Needed Ingredient and the Role of Learning Style. Education Sciences, 10(4), 1–14. https://doi.org/10.3390/educsci10040087

Poondej, C., & Lerdpornkulrat, T. (2019). Gamification in e-learning: A Moodle implementation and its effect on student engagement and performance. Interactive Technology and Smart Education, 17(1), 56–66. https://doi.org/10.1108/ITSE-06-2019-0030

Rengel, R., Pascual, E., Íñiguez, I., Martín, M., & Vasallo, B. (2019). Experiences on the Design, Creation, and Analysis of Multimedia Content to Promote Active Learning. Journal of Science Education and Technology, 28, 445–451. https://doi.org/10.1007/s10956-019-09777-9

Reyes, W., & Góngora, G. (2016). Retos en lugar de tareas: una propuesta de trabajo en línea con estudiantes de nivel superior. TEXTOS. Revista Internacional de Aprendizaje y Cibersociedad, 19(1). https://doi.org/10.37467/gka-revciber.v19.879

Reyes, W., & Quiñonez, S. (2020). Gamification in distance education: experiences in a university educational model. Revista Apertura, 12(2), 1–17. https://doi.org/10.32870/Ap.v12n2.1849

Rivero, M. (2018). Student’s perception about quality of a blended Learning environment with moodle assistance. Pixel-Bit. Revista de medios y educación, (53), 193–205. https://doi.org/10.12795/pixelbit.2018.i53.13

Sahni, S. (2018). Integrating Assessments in Innovative Learning Process. In H. Fardoun, K. Downing, & M. Mok (Eds.), The Future of Higher Education in the Middle East and Africa (pp. 105–124). Springer. https://doi.org/10.1007/978-3-319-64656-5_10

Schjoedt, L., & Sangboon, K. (2015). Control Variables: Problematic Issues and Best Practices. In K. Strang (Ed.) The Palgrave Handbook of Research Design in Business and Management. Palgrave Macmillan. https://doi.org/10.1057/9781137484956_15

Semradova, I., & Hubackova, S. (2016). Teacher responsibility in distance education. Procedia - Social and Behavioral Sciences, 217, 544–550. https://doi.org/10.1016/j.sbspro.2016.02.042

Sheldon, L. (2012). The multiplayer classroom. Cengage Learning.

Sthapornnanon, N., Sakulbumrungsil, R., Theeraroungchaisri, A., & Watcharadamrongkun, S. (2009). Social Constructivist Learning Environment in an Online Professional. American Journal of Pharmaceutical Education, 73(1), 1–9. https://doi.org/10.5688/aj730110

Stockemer, D. (2019). Quantitative Methods for the Social Sciences. Springer. https://doi.org/10.1007/978-3-319-99118-4

Strelan, P., Osborn, A., & Palmer, E. (2020). The flipped classroom: A meta-analysis of effects on student performance across disciplines and education levels. Educational Research Review, 30, 1–22. https://doi.org/10.1016/j.edurev.2020.100314

Syed-Mohamad, S., Pardi, K., Zainal, N., & Ismail, Z. (2006). Expanding nursing education through e-learning: A case study in Malaysia. In H. Park, P. Murray, & C. Delaney (Eds.), Studies in Health Technology and Informatics (Vol. 122, pp. 186–189). IMIA.

Taylor, P., & Maor, D. (2000). Assessing the efficacy of online teaching with the Constructivist On-Line Learning Environment Survey. In A. Herrmann and M.M. Kulski (Eds.), Flexible Futures in Tertiary Teaching. Proceedings of the 9th Annual Teaching Learning Forum, 2-4 February 2000. Curtin University of Technology.

Torres-Toukoumidis, A., Ramírez-Montoya, M., & Romero-Rodríguez, L. (2019). Assessment and evaluation of games-based learning (gbl) in e-learning contexts. Education in the Knowledge Society, 19(4), 109–128. https://doi.org/10.14201/eks2018194109128

Trinidad, S., Aldridge, J., & Fraser, B. (2005). Development, validation and use of the Online Learning Environment Survey. AJET. Australasian Journal of Educational Tecnology, 21(1), 60–81. https://doi.org/10.14742/ajet.1343

UADY. (2012). Modelo Educativo para la Formación Integral. https://bit.ly/3Erovik

UADY. (2013). Plan de Desarrollo UADY Virtual 2013 - 2020. https://bit.ly/3xCVSNM

Vershitskaya, E., Mikhaylova, A., Gilmanshina, S., Dorozhkin, E., & Epaneshnikov, V. (2020). Present-day management of universities in Russia: Prospects and challenges of e-learning. Education and Information Technologies, 25, 611–621. https://doi.org/10.1007/s10639-019-09978-0

Werbach, K., & Hunter, D. (2015). Gamification Toolkit: Dynamics, Mechanics, and Components for the Win. Wharton Digital Press.

Zainuddin, Z., Wah, S., Shujahat, M., & Perera, C. (2020). The impact of gamification on learning and instruction: A systematic review of empirical evidence. Educational Research Review, 30, 1–23. https://doi.org/10.1016/j.edurev.2020.100326

Annex

Items from the COLLES survey in this research

Relevance

1. My learning focuses on issues that interest me.

2. What I learn is important for my professional practice.

3. I learn how to improve my professional practice.

4. What I learn connects well with my professional practice.

Reflection

5. I think critically about how I learn.

6. I think critically about my own ideas.

7. I think critically about other students' ideas.

8. I think critically about ideas in the readings.

Interactivity

9. I explain my ideas to other students.

10. I ask other students to explain their ideas.

11. Other students ask me to explain my ideas.

12. Other students respond to my ideas.

Tutor support

13. The tutor stimulates my thinking.

14. The tutor encourages me to participate.

15. The tutor model good discourse.

16. The tutor model critical self-reflection.

Peer Support

17. Other students encourage my participation.

18. Other students praise my contribution.

19. Other students value my contribution.

20. Other students empathize with my struggle to learn.

Interpretation

21. I make good sense of other students' messages.

22. Other students make good sense of my messages.

23. I make good sense of the tutor's messages.

24. The tutor makes good sense of my messages.