One picture or a thousand words? Influence of question length and illustration support on the success and skip rates on online tests
Abstract The growing popularity of automatically graded online tests, either as an evaluation or self-assessment tool in online or blended education, demands a review of how these questions are designed and delivered to their intended audience. This paper analyzes the results of over 20,000 pre-university mock online quizzes designed to train the students for the Spanish university admission test (known as “Pruebas de Acceso a la Universidad” or “Selectividad”) in the technical drawing subject, corresponding to the June and September intakes of 2009 and 2015. The influence of two key aspects on the questions success and skip rates is assessed: (a) the presence or absence of illustration support and (b) the length of the question as a proxy of reading comprehension difficulty. The results support that the presence of an accompanying illustration in the questions result in fewer skipped questions and mode successful answers, while the length of the question has the opposite effect. The performance difference in the 6-year span is also discussed, showing a slight decline over time in the pass rates while the skip rates remain stable. When comparing both two intakes, corresponding to different academic profiles of students that passed the June exam and those who did not, the success ratio is unsurprisingly lower for the students in the second intake. These findings should help improving the design of online quizzes, including more visual content and/or rephrasing the questions to be more concise, to fit the requirements of students educated in a more visual environment of multimedia technologies.
- Referencias
- Cómo citar
- Del mismo autor
- Métricas
Cleveland, W. S. (1993). Visualizing Data (1 edition). Murray Hill, N.J.?: Summit, N.J: Hobart Press.
Codd, E. F. (1990). The relational model for database management: version 2. Reading, Mass: Addison-Wesley.
Generalitat de Catalunya. (2009). Posa’t a prova. Retrieved June 14, 2016, from aplicacions.universitats.gencat.cat/posat/
Instituto Nacional de Estadística. (2009, December 17). Pruebas de Acceso a la Universidad. Año 2009. Instituto Nacional de Estadística. Retrieved from http://www.ine.es/prensa/np581.pdf
Ministerio de Educación, Cultura y Deporte. (2015). Datos generales de las Pruebas de Acceso a la Universidad (PAU), año 2015. Retrieved from http://www.mecd.gob.es/educacion-mecd/dms/mecd/educacion-mecd/areas-educacion/universidades/estadisticas-informes/estadisticas/estadistica-de-las-pruebas-de-acceso-a-la-universidad/anno-2015/datos-generales-de-las-pau_2015.xls
Peng, R. D. (2011). Reproducible Research in Computational Science. Science, 334(6060), 1226. https://doi.org/10.1126/science.1213847
R Core Team. (2016). R: A Language and Environment for Statistical Computing. Vienna, Austria. Retrieved from http://www.R-project.org/
Wickham, H. (2007). Reshaping Data with the {reshape} Package. Journal of Statistical Software, 21(12), 1–20. doi:https://doi.org/10.18637/jss.v021.i12
Wickham, H. (2009). ggplot2: Elegant Graphics for Data Analysis. New York: Springer-Verlag. doi:https://doi.org/10.1007/978-0-387-98141-3
Wickham, H. (2010). A Layered Grammar of Graphics. Journal of Computational and Graphical Statistics, 19(1), 3–28. https://doi.org/10.1198/jcgs.2009.07098
Wickham, H. (2011). The Split-Apply-Combine Strategy for Data Analysis. Journal of Statistical Software, 40(1), 1–29. doi:https://doi.org/10.18637/jss.v040.i01
Wickham, H. (2014). Tidy Data. Journal of Statistical Software, 59(10), 1–23. doi:https://doi.org/10.18637/jss.v059.i10
Wickham, H. (2016a). stringr: Simple, Consistent Wrappers for Common String Operations. Retrieved from https://CRAN.R-project.org/package=stringr
Wickham, H. (2016b). tidyr: Easily
Tidy Data with `spread()` and `gather()` Functions. Retrieved from https://CRAN.R-project.org/package=tidyr
Wickham, H., & Francois, R. (2016). dplyr: A Grammar of Data Manipulation. Retrieved from https://CRAN.R-project.org/package=dplyr
Wilkinson, L. (2005). The Grammar of Graphics. New York: Springer. doi:10.1007/0-387-28695-0
Wilson, G., Aruliah, D. A., Brown, C. T., Hong, N. P. C., Davis, M., Guy, R. T., … Wilson, P. (2014). Best Practices for Scientific Computing. PLOS Biol, 12(1), e1001745. https://doi.org/10.1371/journal.pbio.1001745
Wing, J. M. (2006). Computational Thinking. Communications of the ACM, 49(3), 33–35. https://doi.org/10.1145/1118178.1118215
Codd, E. F. (1990). The relational model for database management: version 2. Reading, Mass: Addison-Wesley.
Generalitat de Catalunya. (2009). Posa’t a prova. Retrieved June 14, 2016, from aplicacions.universitats.gencat.cat/posat/
Instituto Nacional de Estadística. (2009, December 17). Pruebas de Acceso a la Universidad. Año 2009. Instituto Nacional de Estadística. Retrieved from http://www.ine.es/prensa/np581.pdf
Ministerio de Educación, Cultura y Deporte. (2015). Datos generales de las Pruebas de Acceso a la Universidad (PAU), año 2015. Retrieved from http://www.mecd.gob.es/educacion-mecd/dms/mecd/educacion-mecd/areas-educacion/universidades/estadisticas-informes/estadisticas/estadistica-de-las-pruebas-de-acceso-a-la-universidad/anno-2015/datos-generales-de-las-pau_2015.xls
Peng, R. D. (2011). Reproducible Research in Computational Science. Science, 334(6060), 1226. https://doi.org/10.1126/science.1213847
R Core Team. (2016). R: A Language and Environment for Statistical Computing. Vienna, Austria. Retrieved from http://www.R-project.org/
Wickham, H. (2007). Reshaping Data with the {reshape} Package. Journal of Statistical Software, 21(12), 1–20. doi:https://doi.org/10.18637/jss.v021.i12
Wickham, H. (2009). ggplot2: Elegant Graphics for Data Analysis. New York: Springer-Verlag. doi:https://doi.org/10.1007/978-0-387-98141-3
Wickham, H. (2010). A Layered Grammar of Graphics. Journal of Computational and Graphical Statistics, 19(1), 3–28. https://doi.org/10.1198/jcgs.2009.07098
Wickham, H. (2011). The Split-Apply-Combine Strategy for Data Analysis. Journal of Statistical Software, 40(1), 1–29. doi:https://doi.org/10.18637/jss.v040.i01
Wickham, H. (2014). Tidy Data. Journal of Statistical Software, 59(10), 1–23. doi:https://doi.org/10.18637/jss.v059.i10
Wickham, H. (2016a). stringr: Simple, Consistent Wrappers for Common String Operations. Retrieved from https://CRAN.R-project.org/package=stringr
Wickham, H. (2016b). tidyr: Easily
Tidy Data with `spread()` and `gather()` Functions. Retrieved from https://CRAN.R-project.org/package=tidyr
Wickham, H., & Francois, R. (2016). dplyr: A Grammar of Data Manipulation. Retrieved from https://CRAN.R-project.org/package=dplyr
Wilkinson, L. (2005). The Grammar of Graphics. New York: Springer. doi:10.1007/0-387-28695-0
Wilson, G., Aruliah, D. A., Brown, C. T., Hong, N. P. C., Davis, M., Guy, R. T., … Wilson, P. (2014). Best Practices for Scientific Computing. PLOS Biol, 12(1), e1001745. https://doi.org/10.1371/journal.pbio.1001745
Wing, J. M. (2006). Computational Thinking. Communications of the ACM, 49(3), 33–35. https://doi.org/10.1145/1118178.1118215
Redondo, E., Regot, J., Fonseca, D., Valls, F., & Giménez, L. (2016). One picture or a thousand words? Influence of question length and illustration support on the success and skip rates on online tests. Education in The Knowledge Society, 17(4), 111–128. https://doi.org/10.14201/eks2016174111128
Most read articles by the same author(s)
- David Fonseca Escudero, Ernest Redondo Domínguez, Francesc Valls, Motivation and Academic Improvement Using Augmented Reality for 3D Architectural Visualization , Education in The Knowledge Society: Vol. 17 No. 1 (2016)
Downloads
Download data is not yet available.
+
−