Developing multimodal conversational agents for an enhanced e-learning experience
Abstract Conversational agents have become a strong alternative to enhance educational systems with intelligent communicative capabilities, provide motivation and engagement, and increment significant learning and helping in the acquisition of meta-cognitive skills. In this paper, we present Geranium, a multimodal conversational agent that helps children to appreciate and protect their environment. The system, which integrates an interactive chatbot, has been developed by means of a modular and scalable framework that eases building pedagogic conversational agents that can interact with the students using speech and natural language.
- Referencias
- Cómo citar
- Del mismo autor
- Métricas
Abu-Shawar, B., Atwell, E. Fostering language learner autonomy via adaptive conversation tutors. Proc. of Corpus Linguistics, 2007, pp. 1-8.
Aleven, V., Ogan, A., Popescu, O., Torrey, C., Koedinger, K. Evaluating the Effectiveness of a Tutorial Dialog System for Self-Explanation. Proc. of 7th Int. Conference on Intelligent Tutoring Systems (ITS’04), 2004, pp. 443- 454
Bickmore, T-W. Relational Agents: Effecting Change through Human- Computer Relationships. PhD Thesis, Media Arts & Sciences, Massachusetts Institute of Technology, 2003
Bos, J., Klein, E., Lemon, O., Oka, T. The Verbmobil prototype system - a software engineering perspective. Journal of Natural Language Engineering. 5(1), 1999, 95-112
Cassell, J., Sullivan, J., Prevost, S., Churchill, E-F. Embodied Dialog systems, The MIT Press, 2001
Cavazza, M., de la Camara, R-S., Turunen, M. How Was Your Day? a Companion ECA. Proc. of AAMAS’10 Conference, 2010, pp. 1629-1630.
Chou, C-Y., Chan, T-W., Lin, C-J. Redefining the Learning Companion: the Past, Present and Future of Educational Agents. Computers & Education, 40, 2003, 255-269
Corchado, J., Tapia, D., Bajo, J.: A multi-agent architecture for distributed services and applications. Computational Intelligence 24(2), 2008, 77–107
De Rosis, F., Cavalluzzi, A., Mazzotta, I., Noviell,i N. Can embodied dialog systems induce empathy in users? Proc. of AISB'05 Virtual Social Characters Symposium, 2005, pp. 1-8
Dominguez, K. VoiceXML 31 Success Secrets - 31 Most Asked Questions On VoiceXML - What You Need To Know. Emereo Publishing, 2014
Dowding, J., Clancey, W-J., Graham, J. Are You Talking to Me? Dialogue Systems Supporting Mixed Teams of Humans and Robots. Proc. of AIAA Fall Symposium "Annually Informed Performance: Integrating Machine Listing and Auditory Presentation in Robotic Systems, 2006, pp. 22-27
Edlund, J., Gustafson, J., Heldner, M., Hjalmarsson, A. Towards human- like spoken dialog systems. Speech Communication, 50(8-9), 2008, 630- 645
Forbus, K-D., Whalley, P-B., Evrett, J-O., Ureel, L., Brokowski, M., Baher, J., Kuehne, S-E. CyclePad: An articulate virtual laboratory forengineering thermodynamics. Artificial Intelligence, 114(1-2), 1999, 297- 347
Fryer, L., Carpenter, R. Bots as Language Learning Tools. Language Learning and Technology, 10(3), 2006, 8-14
Gorostiza, J-F., Salichs, M-A. End-user programming of a social robot by dialog. Robotics and Autonomous Systems, 59(12), 2011, 1102–1114.
Graesser, A-C., Chipman, P., Haynes, B-C., Olney, A. AutoTutor: An Intelligent Tutoring System with Mixed-initiative Dialog. IEEE Trans. in Education, 48, 2005, 612-618
Graesser, A-C., Person, N-K., Harter, D. Teaching Tactics and Dialog in AutoTutor. International Journal of Artificial Intelligence in Education. 12, 2001, 23-39
Gratch, J., Rickel, J., Andre, J., Badler, N., Cassell, J., Petajan, E. Creating Interactive Virtual Humans: Some Assembly Required. Proc. of IEEE Conference on Intelligent Systems, 2002, pp. 54-63
Grigoriadou, M., Tsaganou, G., Cavoura, T. Dialog-Based Reflective System for Historical Text Comprehension. Proc. of Workshop on Learner Modelling for Reflection at Artificial Intelligence in Education, 2003, pp. 238-247
Griol, D., Molina, J., Sanchis, A., Callejas, Z. A Proposal to Create Learning Environments in Virtual Worlds Integrating Advanced Educative Resources. JUCS Journal 18(18), 2012, 2516–2541
Griol, D., Molina, J., Sanchis, A., Callejas, Z. Bringing together commercial and academic perspectives for the development of intelligent AmI interfaces. Journal of Ambient Intelligence and Smart Environments, 4(3), 2012, 183-207
Heffernan, N-T. Web-Based Evaluations Showing both Cognitive and Motivational Benefits of the Ms. Lindquist Tutor. Proc. of Int. Conference on Artificial Intelligence in Education, 2003, pp. 115-122
Kerly, A., Ellis, R., Bull, S. Dialog systems in E-Learning. Proc. of AI’08, 2008, pp. 169-182
Kerly, A., Ellis, R., Bull, S. CALMsystem: A Dialog system for Learner Modelling. Knowledge Based Systems, 21(3), 2008, 238-246
Kerly, A., Bull, S. Children's Interactions with Inspectable and Negotiated Learner Models. Proc. of Int. Conf. on Intelligent Tutoring Systems, 2008, pp. 132-141.
Kumar, R., Rose, C-P. Architecture for Building Dialog systems that Support Collaborative Learning. IEEE Trans. Learn. Technol. 4(1), 2011, 21-34
Latham, A., Crockett, K-A., McLean, D., Edmonds, B. A conversational intelligent tutoring system to automatically predict learning styles. Computers & Education. 59(1), 2012, 95-109
Li, S., Wrede, B. Why and how to model multi-modal interaction for a mobile robot companion. Proc. of AAAI Spring Symposium on Interaction Challenges for Intelligent Assistant, 2007, pp. 71-79
Litman, D-J., Silliman, S. ITSPOKE: An Intelligent Tutoring Spoken Dialog System. Proc. of Human Language Technology Conference: North American Chapter of the Association for Computational Linguistics, 2004, pp. 5-8
López-Cózar, R., Araki, M. Spoken, Multilingual and Multimodal Dialog Systems. Development and Assessment. John Wiley and Sons, 2005.
Marsella, S-C., Johnson, W-L., Labore, C-M. Interactive pedagogical drama for health interventions. Artificial Intelligence in Education: Shaping the Future of Learning through Intelligent Technologies, 2003, pp. 341-348
Matsui, T., Asoh, H., Asano, F., Fry, J., Hara, I., Motomura, Y., Itoh, K. Spoken Language Interface of the Jijo-2 Office Robot. Springer Tracts in Advanced Robotics, 6/2003, 2003, 307-317
Mctear, M-F. Spoken dialog technology. Springer, 2004
Mctear, M.F., Callejas, Z. Voice Application Development for Android Packt Publishing, 2013
Mostow, J. Why and How Our Automated Reading Tutor Listens. Proc of Int. Symposium on Automatic Detection of Errors in Pronunciation Training (ISADEPT), 2012, pp. 43-52
Nakano, M., Hasegawa, Y., Funakoshi, K., Takeuchi, J., Torii, T., Nakadai, K., Kanda, N., Komatani, K., Okuno, H-G., Tsujino, H. A multi- expert model for dialogue and behavior control of conversational robots and agents. Knowledge-Based Systems. 24(2), 2011, 248-256
Pieraccini, R. The Voice in the Machine: Building Computers that Understand Speech. The MIT Press, 2012
Pieraccini, R., Suendermann, D., Dayanidhi, K., Liscombe, J. Are we there yet? Research in Commercial Spoken Dialog Systems, Lecture Notes in Computer Science, 5729, 2009, 3-13
Pinzón, C., Bajo, J., de Paz, J., Corchado, J. S-MAS: An adaptive hierarchical distributed multi-agent architecture for blocking malicious SOAP messages within Web Services environments. Expert Systems with Applications 38(5), 2011, 5486-5499
Pon-Barry, H., Schultz, K., Bratt, E-O., Clark, B., Peters, S. Responding to student uncertainty in spoken tutorial dialog systems. International Journal of Artificial Intelligence in Education. 16, 2006, 171-194
Roda, C., Angehrn, A., Nabeth, T. Dialog systems for Advanced Learning: Applications and Research. In: Proc. of BotShow’01 Conference, 2001, pp. 1-7
Rodríguez, S., de Paz, Y., Bajo, J., Corchado, J.M. Social-based Planning Model for Multi-agent Systems. Expert Systems with Applications 38(10) , 2011, 13005–13023
Rosé, C-P., Moore, J-D., VanLehn, K., Allbritton, A. A Comparative Evaluation of Socratic versus Didactic Tutoring. Proc of Cognitive Sciences Society, 2001, pp. 869-874
Sidner, C-L., Kidd, C-D., Lee, C., Lesh, N. Where to look: a study of human-robot engagement. Proc. of 9th Int. Conference on Intelligent user interfaces (IUI’04), 2004, pp. 78-84
Vaquero, C., Saz, O., Lleida, E., Marcos, J., Canalís, C. Vocaliza: An application for computer-aided speech therapy in Spanish language. Proc. of IV Jornadas en Tecnología del Habla, 2006, pp. 321–326
Wang, N., Johnson, L-W. The Politeness Effect in an intelligent foreign language tutoring system. Proc. of Intelligent Tutoring Systems (ITS’08), 2008, pp. 270 - 280
Wang, Y., Wang, W., Huang, C. Enhanced Semantic Question Answering System for e-Learning Environment. Proc of AINAW’07 Conference, 2007, pp. 1023-1028
Williams, J., Young, S. Partially Observable Markov Decision Processes for Spoken Dialog Systems. Computer Speech and Language, 21(2), 2007, 393–422
Aleven, V., Ogan, A., Popescu, O., Torrey, C., Koedinger, K. Evaluating the Effectiveness of a Tutorial Dialog System for Self-Explanation. Proc. of 7th Int. Conference on Intelligent Tutoring Systems (ITS’04), 2004, pp. 443- 454
Bickmore, T-W. Relational Agents: Effecting Change through Human- Computer Relationships. PhD Thesis, Media Arts & Sciences, Massachusetts Institute of Technology, 2003
Bos, J., Klein, E., Lemon, O., Oka, T. The Verbmobil prototype system - a software engineering perspective. Journal of Natural Language Engineering. 5(1), 1999, 95-112
Cassell, J., Sullivan, J., Prevost, S., Churchill, E-F. Embodied Dialog systems, The MIT Press, 2001
Cavazza, M., de la Camara, R-S., Turunen, M. How Was Your Day? a Companion ECA. Proc. of AAMAS’10 Conference, 2010, pp. 1629-1630.
Chou, C-Y., Chan, T-W., Lin, C-J. Redefining the Learning Companion: the Past, Present and Future of Educational Agents. Computers & Education, 40, 2003, 255-269
Corchado, J., Tapia, D., Bajo, J.: A multi-agent architecture for distributed services and applications. Computational Intelligence 24(2), 2008, 77–107
De Rosis, F., Cavalluzzi, A., Mazzotta, I., Noviell,i N. Can embodied dialog systems induce empathy in users? Proc. of AISB'05 Virtual Social Characters Symposium, 2005, pp. 1-8
Dominguez, K. VoiceXML 31 Success Secrets - 31 Most Asked Questions On VoiceXML - What You Need To Know. Emereo Publishing, 2014
Dowding, J., Clancey, W-J., Graham, J. Are You Talking to Me? Dialogue Systems Supporting Mixed Teams of Humans and Robots. Proc. of AIAA Fall Symposium "Annually Informed Performance: Integrating Machine Listing and Auditory Presentation in Robotic Systems, 2006, pp. 22-27
Edlund, J., Gustafson, J., Heldner, M., Hjalmarsson, A. Towards human- like spoken dialog systems. Speech Communication, 50(8-9), 2008, 630- 645
Forbus, K-D., Whalley, P-B., Evrett, J-O., Ureel, L., Brokowski, M., Baher, J., Kuehne, S-E. CyclePad: An articulate virtual laboratory forengineering thermodynamics. Artificial Intelligence, 114(1-2), 1999, 297- 347
Fryer, L., Carpenter, R. Bots as Language Learning Tools. Language Learning and Technology, 10(3), 2006, 8-14
Gorostiza, J-F., Salichs, M-A. End-user programming of a social robot by dialog. Robotics and Autonomous Systems, 59(12), 2011, 1102–1114.
Graesser, A-C., Chipman, P., Haynes, B-C., Olney, A. AutoTutor: An Intelligent Tutoring System with Mixed-initiative Dialog. IEEE Trans. in Education, 48, 2005, 612-618
Graesser, A-C., Person, N-K., Harter, D. Teaching Tactics and Dialog in AutoTutor. International Journal of Artificial Intelligence in Education. 12, 2001, 23-39
Gratch, J., Rickel, J., Andre, J., Badler, N., Cassell, J., Petajan, E. Creating Interactive Virtual Humans: Some Assembly Required. Proc. of IEEE Conference on Intelligent Systems, 2002, pp. 54-63
Grigoriadou, M., Tsaganou, G., Cavoura, T. Dialog-Based Reflective System for Historical Text Comprehension. Proc. of Workshop on Learner Modelling for Reflection at Artificial Intelligence in Education, 2003, pp. 238-247
Griol, D., Molina, J., Sanchis, A., Callejas, Z. A Proposal to Create Learning Environments in Virtual Worlds Integrating Advanced Educative Resources. JUCS Journal 18(18), 2012, 2516–2541
Griol, D., Molina, J., Sanchis, A., Callejas, Z. Bringing together commercial and academic perspectives for the development of intelligent AmI interfaces. Journal of Ambient Intelligence and Smart Environments, 4(3), 2012, 183-207
Heffernan, N-T. Web-Based Evaluations Showing both Cognitive and Motivational Benefits of the Ms. Lindquist Tutor. Proc. of Int. Conference on Artificial Intelligence in Education, 2003, pp. 115-122
Kerly, A., Ellis, R., Bull, S. Dialog systems in E-Learning. Proc. of AI’08, 2008, pp. 169-182
Kerly, A., Ellis, R., Bull, S. CALMsystem: A Dialog system for Learner Modelling. Knowledge Based Systems, 21(3), 2008, 238-246
Kerly, A., Bull, S. Children's Interactions with Inspectable and Negotiated Learner Models. Proc. of Int. Conf. on Intelligent Tutoring Systems, 2008, pp. 132-141.
Kumar, R., Rose, C-P. Architecture for Building Dialog systems that Support Collaborative Learning. IEEE Trans. Learn. Technol. 4(1), 2011, 21-34
Latham, A., Crockett, K-A., McLean, D., Edmonds, B. A conversational intelligent tutoring system to automatically predict learning styles. Computers & Education. 59(1), 2012, 95-109
Li, S., Wrede, B. Why and how to model multi-modal interaction for a mobile robot companion. Proc. of AAAI Spring Symposium on Interaction Challenges for Intelligent Assistant, 2007, pp. 71-79
Litman, D-J., Silliman, S. ITSPOKE: An Intelligent Tutoring Spoken Dialog System. Proc. of Human Language Technology Conference: North American Chapter of the Association for Computational Linguistics, 2004, pp. 5-8
López-Cózar, R., Araki, M. Spoken, Multilingual and Multimodal Dialog Systems. Development and Assessment. John Wiley and Sons, 2005.
Marsella, S-C., Johnson, W-L., Labore, C-M. Interactive pedagogical drama for health interventions. Artificial Intelligence in Education: Shaping the Future of Learning through Intelligent Technologies, 2003, pp. 341-348
Matsui, T., Asoh, H., Asano, F., Fry, J., Hara, I., Motomura, Y., Itoh, K. Spoken Language Interface of the Jijo-2 Office Robot. Springer Tracts in Advanced Robotics, 6/2003, 2003, 307-317
Mctear, M-F. Spoken dialog technology. Springer, 2004
Mctear, M.F., Callejas, Z. Voice Application Development for Android Packt Publishing, 2013
Mostow, J. Why and How Our Automated Reading Tutor Listens. Proc of Int. Symposium on Automatic Detection of Errors in Pronunciation Training (ISADEPT), 2012, pp. 43-52
Nakano, M., Hasegawa, Y., Funakoshi, K., Takeuchi, J., Torii, T., Nakadai, K., Kanda, N., Komatani, K., Okuno, H-G., Tsujino, H. A multi- expert model for dialogue and behavior control of conversational robots and agents. Knowledge-Based Systems. 24(2), 2011, 248-256
Pieraccini, R. The Voice in the Machine: Building Computers that Understand Speech. The MIT Press, 2012
Pieraccini, R., Suendermann, D., Dayanidhi, K., Liscombe, J. Are we there yet? Research in Commercial Spoken Dialog Systems, Lecture Notes in Computer Science, 5729, 2009, 3-13
Pinzón, C., Bajo, J., de Paz, J., Corchado, J. S-MAS: An adaptive hierarchical distributed multi-agent architecture for blocking malicious SOAP messages within Web Services environments. Expert Systems with Applications 38(5), 2011, 5486-5499
Pon-Barry, H., Schultz, K., Bratt, E-O., Clark, B., Peters, S. Responding to student uncertainty in spoken tutorial dialog systems. International Journal of Artificial Intelligence in Education. 16, 2006, 171-194
Roda, C., Angehrn, A., Nabeth, T. Dialog systems for Advanced Learning: Applications and Research. In: Proc. of BotShow’01 Conference, 2001, pp. 1-7
Rodríguez, S., de Paz, Y., Bajo, J., Corchado, J.M. Social-based Planning Model for Multi-agent Systems. Expert Systems with Applications 38(10) , 2011, 13005–13023
Rosé, C-P., Moore, J-D., VanLehn, K., Allbritton, A. A Comparative Evaluation of Socratic versus Didactic Tutoring. Proc of Cognitive Sciences Society, 2001, pp. 869-874
Sidner, C-L., Kidd, C-D., Lee, C., Lesh, N. Where to look: a study of human-robot engagement. Proc. of 9th Int. Conference on Intelligent user interfaces (IUI’04), 2004, pp. 78-84
Vaquero, C., Saz, O., Lleida, E., Marcos, J., Canalís, C. Vocaliza: An application for computer-aided speech therapy in Spanish language. Proc. of IV Jornadas en Tecnología del Habla, 2006, pp. 321–326
Wang, N., Johnson, L-W. The Politeness Effect in an intelligent foreign language tutoring system. Proc. of Intelligent Tutoring Systems (ITS’08), 2008, pp. 270 - 280
Wang, Y., Wang, W., Huang, C. Enhanced Semantic Question Answering System for e-Learning Environment. Proc of AINAW’07 Conference, 2007, pp. 1023-1028
Williams, J., Young, S. Partially Observable Markov Decision Processes for Spoken Dialog Systems. Computer Speech and Language, 21(2), 2007, 393–422
Griol, D., Molina, J. M., & Sanchís De Miguel, A. (2014). Developing multimodal conversational agents for an enhanced e-learning experience. ADCAIJ: Advances in Distributed Computing and Artificial Intelligence Journal, 3(1), 13–26. https://doi.org/10.14201/ADCAIJ2014381326
Most read articles by the same author(s)
- David Griol, José Molina, Measuring the differences between human-human and human-machine dialogs , ADCAIJ: Advances in Distributed Computing and Artificial Intelligence Journal: Vol. 4 No. 2 (2015)
- Ana Cristina Bicharra, Nayat Sanchez-Pi, Luis Correia, José Manuel Molina, Multi-agent simulations for emergency situations in an airport scenario , ADCAIJ: Advances in Distributed Computing and Artificial Intelligence Journal: Vol. 1 No. 3 (2012)
- David Griol, Jose M. Molina, A proposal to manage multi-task dialogs in conversational interfaces , ADCAIJ: Advances in Distributed Computing and Artificial Intelligence Journal: Vol. 5 No. 2 (2016)
- David Griol, Jose Manuel Molina, Simulating heterogeneous user behaviors to interact with conversational interfaces , ADCAIJ: Advances in Distributed Computing and Artificial Intelligence Journal: Vol. 5 No. 4 (2016)
- David Griol, Jose Manuel Molina, From VoiceXML to multimodal mobile Apps: development of practical conversational interfaces , ADCAIJ: Advances in Distributed Computing and Artificial Intelligence Journal: Vol. 5 No. 3 (2016)
- David Griol, Jesús García-Herrero, José Manuel Molina, Combining heterogeneous inputs for the development of adaptive and multimodal interaction systems , ADCAIJ: Advances in Distributed Computing and Artificial Intelligence Journal: Vol. 2 No. 3 (2013)
Downloads
Download data is not yet available.
+
−