The Use of Apps Based on Augmented Reality as a Tool for Teaching STEM Subjects in Compulsory Secondary Education

Firstly, we tested if there was a difference in significance between the pretests of both groups (control and experimental), and then, we analysed the pretest and post-test performances of the experimental and control groups. In addition, we analysed where there were different significances in pre-test-post-test performance in both groups. The level of significance used in all contrasts is analysed under 5%. To interpret its value, we add its effect size (d). To quantify the magnitude of the difference between two medians, values between .10 and .29 are considered small differences, .30 to .49 as moderate differences and values greater than .50 as large (Cohen, 2013).

5. Results

First, we investigated how the use of AR apps affects student learning. For this purpose, we verified that both groups (control and experimental) did not present significant differences in the pretest. Then, we proceeded to analyse the results in the post-test. For this purpose, we used the nonparametric Mann-Whitney U test. Table 3 shows the pretest results of both control and experimental groups in each case. The results show no significant differences between the two groups in all cases in the pre-test.

After verifying the contrast statistics in the pretest, we tested the impact of using AR apps with the nonparametric Wilcoxon test. The results are summarised in Table 4.

As we can see, the control group showed no statistically significant improvement in most cases, except for case 1. However, the improvement in using AR apps presented a more substantial effect size. In the same line, the enhancement happened in most cases formatted by the experimental group. The experimental group presented better post-test scores in cases 1, 2 and 4. Case 3 was close to being significant. Moreover, the differences between pre-tests and post-tests were substantial, even in case 3. The results indicated significant differences between the pre-test and post-test; the post-test is the one that achieves a higher score in most cases in the experimental group.

Second, we evaluated the satisfaction with the learning resources used in the four cases. The results show a high level of satisfaction. Notably, in all four cases studied, students found the apps very easy to use, with scores above 4. For learning success and duration of the activity, all scores were close to 4. Ease of use and overall satisfaction were above 4, exceeding 4.5, with only case 2 scoring below 4.

Interestingly, case 3 presented the lowest scores. More details are shown in figure 8. Generally, students positively evaluate using learning resources in teaching and learning.

Figure 8. Scores of the considered variables for the different cases.

6. Discussion

This article investigates the importance of using AR apps for teaching STEM subjects in the 8th, 9th and 10th grades of secondary education. Syawaludin et al. (2019) state that STEM topics are generally abstract and challenging to understand. AR-based apps are an alternative way to concretise abstract concepts covered in secondary education (Cheng & Tsai, 2013; Paembonan & Ikhsan, 2021; Syawaludin et al., 2019). On the other hand, AR apps offer advantages such as ease of use, increased satisfaction, increased motivation and reduced students’ cognitive load (Sırakaya & Alsancak Sırakaya, 2020). Thus, we aim to evaluate the learning activities designed in AR-based STEM for secondary school students and their associated benefits.

Our first research question targeted the impact of using augmented reality applications (AR apps) on student learning. The findings from this study suggest that learning activities with augmented reality perform better than activities that only traditionally use textbooks. Research development in Indonesia supports our results with students in 8th grade who have better learning outcomes after the intervention using AR APPs in their science classes (Astuti et al., 2019). Similarly, applying a qualitative study, Karagozlu et al. (2019) suggest that attending science classes with students in 7th grade positively affects their learning. AR apps improve science knowledge in university students (Criollo-C et al., 2022; Mariscal et al., 2020) and in primary education students (Çetin & Türkan, 2022). Garzón et al. (2019) found through a meta-analysis of 61 studies that AR apps have a medium effect on learning effectiveness.

Our second research question was targeted to determine the level of satisfaction with using augmented reality applications (AR apps) in students’ learning. Our results show that students evaluate these apps satisfactorily. This agrees with other studies, such as Salvador-Herranz et al. (2013), in which the students manifested that using AR apps helped them improve their learning of science issues. Likewise, students expressed that the classes with these applications help retain knowledge (Cai et al., 2013). Karagozlu et al. (2019) found that students consider using these apps as an excellent methodology to be applied in their classes, as they believe that they provide them with benefits such as improving their attention and raising interest in science subjects.

Similarly, Liou et al. (2017) found that these applications are easy to use because students are often familiar with their handling. In terms of satisfaction with its use, our results are consistent and in line with other investigations (Karagozlu et al., 2019; Sáez-López et al., 2019; Salvador-Herranz et al., 2013; Wojciechowski & Cellary, 2013). However, it must be recognised that these apps are also often time-consuming (Laine et al., 2016), and this is reflected in our findings, which show that of the five categories, this was the lowest score.

Our findings reinforce previous research on the importance of AR apps in the teaching and learning process. Furthermore, our results corroborate the appreciation of student satisfaction when teachers accompany their classes with these applications (Maas & Hughes, 2020).

Despite our study’s positive results, the sample size restricts the generalisation of the results. However, most studies on this topic are conducted with the same sample size (Sırakaya & Alsancak Sırakaya, 2020). Another limitation is that our results are based on quantitative research. The results could be strengthened if accompanied by qualitative methodology, so we suggest applying a mixed methodological approach for future research.

7. Conclusions

Overall, apps help teach specific contents of the STEM curriculum in secondary education. The different AR experiences proved that using the mobile phone as a central element in using AR in the classroom is very appropriate.

The use of AR-based experiences is fascinating from the point of view of developing key competencies since it allows the integration of many of them in the same activity. Furthermore, some of the experiences were carried out in groups of students under lockdown due to COVID-19, who could carry out the expertise entirely or in an adapted way in real-time from home.

Practically, all of the AR experiences carried out have been rated by teachers and students as very motivating, with many students continuing to work with home applications involving their families.

One of the teacher's limitations is the lack of the required hardware, especially the lack of tablets in educational centres. However, in many cases, it is efficiently solvable thanks to the personal smartphones of the students themselves. One factor to consider if we want to use AR apps is time, which teachers should consider when using this resource in their classes.

Several apps can serve more than one subject, which shows that the management of this technology can effectively overcome compartmentalised teaching and highlight the relationship between the different STEM disciplines.

Funding

Educational Research Project, "Improving scientific competence through augmented reality (EDUCYL2020-02)", funded by the Consejería de Educación de la Junta de Castilla y León through the Dirección General de Innovación y Formación del Profesorado. Grant “Educación para el Cambio Climático y la Sostenibilidad, un estudio longitudinal de aprendizaje intergeneracional” PID2020-114358RB-I00 funded by MCIN/AEI/ 10.13039/501100011033 ERDF A way of making Europe. Grant “Impactos climáticos en la educacion y medidas de adaptación basadas en la naturaleza en la educación. Eduheat” TED2021-130300B-C21 funded by MCIN/AEI /10.13039/501100011033 and by the European Union NextGenerationEU/ PRTR

Authors’ contributions

Enzo Ferrari: Data curation, Formal analysis, Methodology, Software, Validation, Visualization.
Pablo Herrero-Teijón: Data curation, Formal análisis, Investigation, Methodology, Supervision, Validation, Visualization.
Laura Delgado Martín: Conceptualization.
Santiago Andrés Sánchez: Investigation.
Anne-Marie Ballegeer: Conceptualization, Validation, Writing – review & editing
Diego Corrochano Fernández: Conceptualization.
Miguel Ángel Fuertes: Writing – original draft.
Patricia González Ocejo: Supervision.
Camilo Ruiz Méndez: Conceptualization, Writing – original draft, Writing – review & editing.

References

Allen, M. J., & Yen, W. M. (1979). Introduction to Measurement Theory. Wadsworth. Inc.

Álvarez, S. (2016). El trabajo interdisciplinar en el aula de matemáticas. Diseño de una experiencia. [Trabajo fin de máster, Universidad de Salamanca].

Arici, F., Yildirim, P., Caliklar, Ş., & Yilmaz, R. M. (2019). Research trends in the use of augmented reality in science education: Content and bibliometric mapping analysis. Computers & Education, 142, Article 103647. https://doi.org/10.1016/j.compedu.2019.103647

Astuti, F. N., Suranto, S., & Masykuri, M. (2019). Augmented reality for teaching science: Students’ problem solving skill, motivation, and learning outcomes. JPBI (Jurnal Pendidikan Biologi Indonesia), 5(2), 305-312. https://doi.org/10.22219/jpbi.v5i2.8455

Blázquez, A. (2017). Realidad aumentada en educación. Universidad Politécnica de Madrid. Gabinete de Tele-Educación. Vicerrectorado de Servicios Tecnológicos de la Universidad Politécnica de Madrid.

Brown, J. S., Collins, A., & Duguid, P. (1989). Situated cognition and the culture of learning. Educational Researcher, 18(1), 32–42. https://doi.org/10.3102/0013189X018001032

Buitrago-Pulido, R. D. (2015). Incidencia de la realidad aumentada sobre el estilo cognitivo: caso para el estudio de las matemáticas. Educación y Educadores, 18(1), 27-41. https://doi.org/10.5294/edu.2015.18.1.2

Cabero Almenara, J. (2020). Aprendiendo del tiempo de la COVID-19. Revista Electrónica Educare (Educare Electronic Journal), 24(Suplemento Especial), 1-3. https://doi.org/10.15359/ree.24-s.2

Cabero Almenara, J., & Barroso Osuna, J. M. (2016). Ecosistema de aprendizaje de realidad aumentada: posibilidades educativas. TCE: Tecnología, Ciencia y Educación, 5, 141-154. https://doi.org/10.51302/tce.2016.101

Cabero Almenara, J., & Valencia, R. (2021). And COVID-19 transformed the educational system: reflections and experiences to learn. International Journal of Educational Research and Innovation (IJERI), (15), 218-228. https://doi.org/10.46661/ijeri.5246

Cai, S., Chiang, F. K., & Wang, X. (2013). Using the augmented reality 3D technique for a convex imaging experiment in a physics course. International Journal of Engineering Education, 29(4), 856–865.

Çetin, H., & Türkan, A. (2022). The Effect of Augmented Reality based applications on achievement and attitude towards science course in distance education process. Education and Information Technologies, 27(2), 1397–1415. https://doi.org/10.1007/s10639-021-10625-w

Chang, H. Y., Binali, T., Liang, J. C., Chiou, G. L., Cheng, K. H., Lee, S. W. Y., & Tsai, C. C. (2022). Ten years of augmented reality in education: A meta-analysis of (quasi-) experimental studies to investigate the impact. Computers & Education, 191, 104641. https://doi.org/10.1016/j.compedu.2022.104641

Chen, Y., Wang, Q., Chen, H., Song, X., Tang, H., & Tian, M. (2019). An overview of augmented reality technology. Journal of Physics: Conference Series, 1237(2), Article 022082. https://doi.org/10.1088/1742-6596/1237/2/022082

Cheng, K. H., & Tsai, C. C. (2013). Affordances of Augmented Reality in Science Learning: Suggestions for Future Research. Journal of Science Education and Technology, 22(4), 449–462. https://doi.org/10.1007/s10956-012-9405-9

Cohen, J. (2013). Statistical power analysis for the behavioral sciences (2nd ed.) Academic press. https://doi.org/10.4324/9780203771587

Criollo-C, S., Altamirano-Suarez, E., Jaramillo-Villacís, L., Vidal-Pacheco, K., Guerrero-Arias, A., & Luján-Mora, S. (2022). Sustainable Teaching and Learning through a Mobile Application: A Case Study. Sustainability, 14(11), Article 6663. https://doi.org/10.3390/su14116663

Dunleavy, M., & Dede, C. (2014). Augmented Reality Teaching and Learning. In J. M. Spector, M. D. Merrill, J. Elen, & M. J. Bishop (Eds.), Handbook of Research on Educational Communications and Technology (pp. 735–745). Springer New York. https://doi.org/10.1007/978-1-4614-3185-5_59

Ebel, R. L., & Frisbie, D. A. (1972). Essentials of educational measurement. Prentice Hall.

Estapa, A., & Nadolny, L. (2015). The effect of augmented reality enhanced mathematics lesson on student achievement and motivation. Journal of STEM Education, 16, 40-48.

Fernández-Enríquez, R., & Delgado-Martín, L. (2020). Augmented Reality as a Didactic Resource for Teaching Mathematics. Applied Sciences, 10(7), Article 2560. https://doi.org/10.3390/app10072560

Finch, W. H., & French, B. F. (2018). Educational and Psychological Measurement. Routledge. https://doi.org/10.4324/9781315650951

Quicios García, M. P., Herrero Teijón, P., & González Ocejo, P. (2020). La eficacia de la enseñanza a distancia durante el confinamiento por COVID-19 en las zonas de baja cobertura de la España vaciada. El caso de Salamanca. Enseñanza & Teaching: Revista Interuniversitaria de Didáctica, 38(2), 67-85. https://doi.org/10.14201/et20203826785

Garzón, J., Kinshuk, Baldiris, S., Gutiérrez, J., & Pavón, J. (2020). How do pedagogical approaches affect the impact of augmented reality on education? A meta-analysis and research synthesis. Educational Research Review, 31, Article 100334. https://doi.org/10.1016/j.edurev.2020.100334

Garzón, J., Pavón, J., & Baldiris, S. (2019). Systematic review and meta-analysis of augmented reality in educational settings. Virtual Reality, 23(4), 447–459. https://doi.org/10.1007/s10055-019-00379-9

Giasiranis, S., & Sofos, L. (2016). Production and evaluation of educational material using augmented reality for teaching the module of “Representation of the information on computers” in junior high school. Creative Education, 7, 1270-1291. https://doi.org/10.4236/ce.2016.79134

Gopalan, M., Rosinger, K., & Ahn, J. B. (2020). Use of quasi-experimental research designs in education research: Growth, promise, and challenges. Review of Research in Education, 44(1), 218-243. https://doi.org/10.3102/0091732X20903302

Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2019). Multivariate data analysis (8^th Ed.). Annabel Ainscow.

Hedrick, T. E., Bickman, L., & Rog, D. J. (1993). Applied research design. A practical guide. Sage. https://doi.org/10.4135/9781412983457

Johnson, L., Adams Becker, S., Cummins, M., Estrada, V., Freeman, A., & Hall, C. (2016). The NMC Horizon Report: 2016 Higher Education Edition. The New Media Consortium.

Karagozlu, D., Kosarenko, N. N., Efimova, O. V., & Zubov, V. V. (2019). Identifying students’ attitudes regarding augmented reality applications in science classes. International Journal of Emerging Technologies in Learning, 14(22), 45–55. https://doi.org/10.3991/ijet.v14i22.11750

Laine, T. H., Nygren, E., Dirin, A., & Suk, H. J. (2016). Science Spots AR: a platform for science learning games with augmented reality. Educational Technology Research and Development, 64(3), 507–531. https://doi.org/10.1007/s11423-015-9419-0

Liou, H. H., Yang, S. J. H., Chen, S. Y., & Tarng, W. (2017). The influences of the 2D image-based augmented reality and virtual reality on student learning. Educational Technology and Society, 20(3), 110–121.

Maas, M. J., & Hughes, J. M. (2020). Virtual, augmented and mixed reality in K–12 education: A review of the literature. Technology, Pedagogy and Education, 29(2), 231–249. https://doi.org/10.1080/1475939X.2020.1737210

Magnusson, D. (1967). Test theory. Addison Wesley.

Mariscal, G., Jiménez, E., Vivas-Urias, M. D., Redondo-Duarte, S., & Moreno-Pérez, S. (2020). Virtual Reality Simulation-Based Learning. Education in the Knowledge Society, 21(11), 1–15. https://doi.org/10.14201/eks.20809

Martín-Gutiérrez, J., Mora, C. E., Añorbe-Díaz, B., & González-Marrero, A. (2017). Virtual technologies trends in education. Eurasia Journal of Mathematics, Science and Technology Education, 13(2), 469-486. https://doi.org/10.12973/eurasia.2017.00626a

Paembonan, T. L., & Ikhsan, J. (2021). Supporting Students’ Basic Science Process Skills by Augmented Reality Learning Media. Journal of Educational Science and Technology (EST), 7(2), 188-196. https://doi.org/10.26858/est.v0i0.19448

Pedrinaci, E., Caamaño, A., Cañal, P., & de Pro, A. (2012). 11 ideas clave: El desarrollo de la competencia científica. Graó.

Prendes Espinosa, C. (2015). Realidad aumentada y educación: Análisis de experiencias prácticas. Píxel-Bit. Revista de Medios y Educación, 46, 187-203. https://doi.org/10.12795/pixelbit.2015.i46.12

Price, L. R., (2017). Psychometric methods: Theory into practice. The Guilford Press.

Reinoso Ortiz, R. (2012). Posibilidades de la realidad aumentada en educación. Tendencias emergentes en educación con TIC. In J. Hernández Ortega, M. Pennesi, D. Sobrino López, & A. Vázquez Gutiérrez (Eds.), Tendencias emergentes en educación con TIC (pp. 175-197). Espiral.

Sáez-López, J. M., Sevillano-García, M. L. & Pascual-Sevillano, M. A. (2019) Aplicación del juego ubicuo con realidad aumentada en Educación Primaria. Comunicar. Revista Científica de Educomunicación, 27(61), 71-82. https://doi.org/10.3916/C61-2019-06

Salvador-Herranz, G., Pérez-López, D., Ortega, M., Soto, E., Alcañiz, M., & Contero, M. (2013). Manipulating Virtual Objects with Your Hands: A Case Study on Applying Desktop Augmented Reality at the Primary School. In 2013 46th Hawaii International Conference on System Sciences (Wailea, HI, USA, 7-10 January 2013) (pp. 31-39). https://doi.org/10.1109/HICSS.2013.390

Schleicher, A. (2020). The Impact of COVID-19 on Education: Insights from "Education at a Glance 2020". OECD Publishing.

Sinha, M., & Srivastava, M. (2022). Augmented Reality-Enabled Instagram Game Filters: Key to Engaging Customers. Journal of Promotion Management, 28(4), 467–486. https://doi.org/10.1080/10496491.2021.2008577

Sırakaya, M., & Alsancak Sırakaya, D. (2022). Augmented reality in STEM education: a systematic review. Interactive Learning Environments, 30(8), 1556–1569. https://doi.org/10.1080/10494820.2020.1722713

Squire, K. (2006). From content to context: Videogames as designed experience. Educational Researcher, 35(8), 19–29. https://doi.org/10.3102/0013189x035008019

Stanley, J. C., & Campbell, D. (1973). Diseños experimentales y cuasiexperimentales en la investigación social. Rand McNal.

Syawaludin, A., Gunarhadi, & Rintayati, P. (2019). Enhancing elementary school students’ abstract reasoning in science learning through augmented reality-based interactive multimedia. Jurnal Pendidikan IPA Indonesia, 8(2), 288–297. https://doi.org/10.15294/jpii.v8i2.19249

Villalustre Martínez, L., del Moral Pérez, M. E., & Neira Piñeiro, M. R. (2019). Percepción docente sobre la Realidad Aumentada en la Enseñanza de Ciencias en Primaria. Análisis DAFO. Revista Eureka sobre Enseñanza y Divulgación de las Ciencias, 16(3), 3301-3320. https://doi.org/10.25267/Rev_Eureka_ensen_divulg_cienc.2019.v16.i3.3301

Wojciechowski, R., & Cellary, W. (2013). Evaluation of learners’ attitude toward learning in ARIES augmented reality environments. Computers and Education, 68, 570–585. https://doi.org/10.1016/j.compedu.2013.02.014

Annexes

Evidence of item analysis and reliability

We calculated item indices typical of Classical Test Theory (CTT) as Magnusson (1967) recommended. CTT involve the item difficulty (P), Item discrimination (D) and Cronbach’s alpha (α) if the item is deleted. Item difficulty (P) is calculated as the proportion of correct responses on each item. According to Allen and Yen (1979), values between 0.30 and 0.70 provide helpful information about knowledge measurement. Moreover, values outside of this range, less than .3, are considered extremely difficult, and above .7 is moderately easy. Item Discrimination (D) is how an item differentiates those who know more from those who know less. D is based on the proportion of correct answers from the high group and the total number of correct answers from both the high and low groups. According to Ebel and Frisbie (1972, p. 232) and Price (2017), when D presents values above .4 are very good items, .30 to .39, reasonably good but possibly subject to improvement. Values between .20 to .29 Marginal items, usually needing improvement below. Finally, values less than .19 are Poor items to be rejected or improved by revision. An alternative technique for estimating item discrimination is the correlation (r_pbs) between the item response and the total score on the scale (Finch & French, 2018). The reliability for every item is measured with Cronbach's alpha (if the item is deleted). This should have values close to or above .7. Average variance extracted should have values close to .4 with indicators loadings above .5.

According to the results in Table S.1, we can see that the most challenging item was item 6 (P = 0.282), and the easiest were items 5 and 9 (P = 0.846 and P = 0.821 respectively). Most items had P values within the 0.3–0.7 range that Allen and Yen (1978) suggested were recommended.1. Based upon D, we would conclude that the best discriminating item was 5. However, according to r_pbs was item 1. On the other hand, the scale had an appropriate reliability (α=.717), and the average variance extracted was .371.

The results in Table S.2 show that the most challenging items were items 7 (P = 0.469), and the most straightforward items were items 1 and 4 (P = 0.688). All items had P values within the range of 0.3-0.7. According to D and r_pbs, we may conclude that the best discriminant item was item 5. This was adequate regarding reliability (α for total scale is 0.808.), and the average variance extracted is .475.

Table S.3 shows that the most challenging was item 6 (P = 0.444), and the most straightforward items were items 5 to 9 (P = 0.926). However, we decided to keep them because they have crucial content validity. Using D and r_pbs, we concluded that the best discriminant item was item 5. Concerning the reliability, this was satisfactory (α=.710), and the average variance extracted is .392.

We can see from the results in Table S.4 that the most challenging item was item 6 (P = 0.481), and the most straightforward item was item 3 (P = 0.926). Using D, we can conclude that the most discriminating items were items 5 and 6. However, according to r_pbs, item 1 was the best discriminant. With reliability concerns, this was below .7. However, it is adequate for exploratory studies (Hair et al., 2019) such as ours. The average variance extracted was appropriate, with .388.

_______________________________

(*) Autor de correspondencia / Corresponding author