A Serious Game for the Daily Care of Autistic Children in Times of Covid-19

Un juego serio para el cuidado diario de niños autistas en tiempos de Covid-19

Vitor Oikawa^a, Cibelle Albuquerque de la Higuera Amato^b, Valéria Farinazzo Martins^c(*)

^a Faculdade de Computação e Informática, Universidade Presbiteriana Mackenzie, São Paulo, Brasil.

https://orcid.org/0000-0001-7344-3986vitor9998@gmail.com

^b Programa de Pós-Graduação em Ciências do Desenvolvimento Humano, Universidade Presbiteriana Mackenzie, São Paulo, Brasil.

https://orcid.org/0000-0003-2422-6998cibelle.amato@mackenzie.br

^c Faculdade de Computação e Informática / Programa de Pós-Graduação em Computação Aplicada, Universidade Presbiteriana Mackenzie, São Paulo, Brasil.

https://orcid.org/0000-0002-5058-6017valeria.farinazzo@mackenzie.br

ABSTRACT

Autism (autism spectrum disorder – ASD) is characterized by changes in social communication, restricted interests, and repetitive behaviors. Most autistic children need different types of intervention, whether behavioral, linguistic, social, or occupational, among others. Since the emergence of the virus that caused the COVID-19 pandemic, the lives of millions of people have been drastically impacted: schools closed, commercial establishments closed, lockdown, and remote work. Children with ASD have experienced a series of changes in their lives, in addition to those mentioned, since they had their appointments reduced or canceled and, after a few months, changed in the online version. Masks became mandatory for many months until greater flexibility was obtained, which still faces ups and downs. These children had to adapt to living in a different environment than they lived in long ago. For example, wearing masks makes it difficult to understand people’s words. This project presents a serious game to train the child’s ability to understand the words now said by people wearing a face protection mask without the possibility of the visual resource of facial mimicry. Nine speech therapists and fourteen diagnostic children with ASD validated the game. The children tested the game during one of the online speech therapy sessions. Both groups answered a satisfaction questionnaire the speech therapist carried out, which was adapted to their characteristics. The evaluation results with both groups provide evidence that the game was well accepted.

Keywords
autism spectrum disorder, serious game, Covid-19, games.

RESUMEN

El autismo (trastorno del espectro autista-TEA) se caracteriza por cambios en la comunicación social, intereses restringidos y comportamientos repetitivos. La mayoría de los niños autistas necesitan diferentes tipos de intervención, ya sea conductual, lingüística, social, ocupacional, entre otras. Desde la aparición del virus que provocó la pandemia de la Covid-19, la vida de millones de personas se ha visto impactada drásticamente: escuelas cerradas, establecimientos comerciales cerrados, lockdown, trabajo remoto. Los niños con TEA han experimentado una serie de cambios en sus vidas, además de los mencionados, desde que vieron reducidas o canceladas sus citas y, al cabo de unos meses, cambiadas en la versión online. El uso de mascarillas se hizo obligatorio durante muchos meses, hasta lograr una mayor flexibilidad, que aún enfrenta altibajos. Estos niños tuvieron que adaptarse a vivir en un ambiente diferente al que vivían hace mucho tiempo. Por ejemplo, les cuesta entender lo que dice la gente debido al uso de mascarillas. Este proyecto presenta un juego serio con el objetivo de entrenar la capacidad del niño para comprender las palabras que ahora dicen las personas que usan una máscara de protección facial, sin la posibilidad del recurso visual de la mímica facial. Nueve logopedas y catorce niños diagnosticados con TEA validaron el juego. Los niños probaron el juego durante una de las sesiones de terapia del habla en línea. Ambos grupos respondieron a un cuestionario de satisfacción realizado por el logopeda, adaptado a sus características. Los resultados de la evaluación con ambos grupos evidencian que el juego tuvo buena aceptación.

Palabras clave
autismo, videojuegos serios, Covid-19, videojuegos.

1. Introduction

COVID-19 is a virus discovered in 2019 that has caused quite complicated situations in people worldwide compared to conditions of war and the Spanish flu. In addition to the physical health problems themselves, it has triggered psychological disorders and structural changes in the world economy and how people interact (Barbosa et al., 2020).

Since the emergence of the COVID-19 pandemic, a series of changes in society’s life has been necessary for general, particularly social isolation (Fundação Oswaldo Cruz, 2020; García-Peñalvo et al., 2020) and all the consequences it brought us. For example, parents had to be with their children under extreme levels of stress in a confined space with limited reinforcers for 24 hours a day, 7 days a week (Degli Espinosa et al., 2020). In this pandemic situation, caring for children with autism could be challenging for families and caregivers. Autistic Spectrum Disorder (ASD) is defined as a change in development and behavior. Therefore, in general, these children have interventions for several hours a week at home with special therapists or in dedicated hospitals and institutes. However, during this moment, the sessions of therapists were reduced or canceled and, after a few months, changed to the online version (Narzisi, 2020).

At that moment, due to contagion containment measures, preventive actions were required to combat this pandemic and reduce the transmission rate, with control and protection measures such as hand hygiene, masks, and social isolation (Fernandes & Nohama, 2020). However, the use of masks made communication between people more difficult as the possibility of looking at the speaker’s lips and facial mimic is lost and due to the interference and distortion of the speech sound caused by the mask.

The new ‘normal’ defined during the COVID-19 pandemic forced us to re-assess how people with special needs are able to learn, live together, play, and have the support of health professionals in their daily lives, such as those with ASD.

In this context, in which the whole society faced difficulties in experiencing the pandemic, there were children and adolescents with ASD. ASD is a neurodevelopmental disorder, beginning in childhood, whose main characteristics are difficulties in language/communication, social interaction, and behavior (American Psychiatric Association, 2013). For these children and adolescents, the whole scenario resulting from the COVID-19 pandemic was challenging to understand (Fernandes & Nohama, 2020). As a result, several activities were abruptly altered for these children and adolescents at the beginning of the pandemic (and have not yet been wholly re-established). Among them, it is possible to highlight: the face-to-face mode classes were suspended or started to operate on a student rotation basis, the speech therapy sessions were suspended and resumed in the online mode, and the need to wear a mask to go out home, among others.

While it was inconvenient for these children and adolescents to wear a mask, it interfered with their ability to communicate as there were no visual cues that could be used to facilitate the communication process.

The social impact expected for this project consisted of making a digital serious game available to assist in therapeutic intervention in speech therapy sessions for better care of children and adolescents with autism before the COVID-19 pandemic.

According to Scotini, Orsati, and de Macedo (2021) and Sánchez (2021), technology is essential to help educate people with autism in developing their communication skills. According to Narzisi (2020), “Serious games can be useful to improve social cognition and to recognize facial emotions, emotional gestures, and emotional situations in children with ASD. Serious games could be an educational alternative to video games or the internet tout-court”.

The present study aims to: 1. validate a serious game of motivational role of performing activities within Speech Therapy sessions and how this game can be used to assist in therapeutic intervention, especially for initial training in the use of the mask and sound perception, during the pandemic phase; 2. This will be applied to autistic children in speech therapy sessions, and these children’s opinions about the game will be collected through their speech therapists.

This work was developed during the first year of the COVID-19 pandemic. Even after the end of the pandemic period, it can be used for other purposes that go beyond the confinement period, as it is more about training the social cognition and oral communication of children with ASD.

The article is organized as follows: Section 2 presents the theoretical basis to facilitate understanding the other sections of the work; Section 3 shows the entire game development process; Section 4 presents the evaluation methodology; and Section 5 provides test results and discussions. Finally, in Section 6, the conclusions of the work are presented.

2. Theoretical foundations

2.1. Autistic Spectrum Disorder

ASD is among the mental health problems that most hinder child development (American Psychiatric Association, 2013). A global developmental disorder that appears in the first three years of life. In addition, it affects normal brain development related to social and communication skills. The disease is marked by three fundamental characteristics: inability to interact socially, difficulty in the mastery of language to communicate or deal with symbolic games, and a pattern of restrictive and repetitive behavior (Campanário, 2008).

With this, it is possible to perceive the importance of special care for children; since discovering ASD early, it is possible to improve their personal and social development. The CDC (Centers for Disease Control and Prevention) in the United State has been extremely important in disseminating epidemiological data on ASD. According to studies available on its website: About 1 in 44 children have been identified with ASD. It is reported to occur in all racial, ethnic, and socioeconomic groups, and ASD is more than 4 times more common among boys than among girls (Maenner et al., 2021). A study carried out with individuals with ASD during the pandemic shows that this was a challenging period for the parents and caregivers of these children. The results revealed that family members had great difficulties managing activities and the most severe behavior problems (Colizzi, 2020). Behavior problems were related to the change in routine caused by the social isolation imposed by the COVID-19 pandemic (Parens et al., 2020). Regarding the pandemic, there is also a study that points out that the social isolation imposed by COVID-19 favored people with ASD because it reduced daily stress, especially in activities that required social demands, which were drastically reduced (Lugo-Marín et al., 2021). Recently, it has been observed that information technologies have been used to assist numerous practices in the health area in activities such as diagnosis, therapy, management, and education. These activities require changes and the development of new skills by health professionals in the areas involved (Fernandes et al., 2014).

Piscalho and Veiga-Simão (2014) indicate that a game can play an essential role in developing the child in general. It provides reflection and interrelationship between objects and events, helping the child expand the imagination and improve social, communicative, and autonomy competencies.

Considering that children with autism have difficulties responding under the control of the correct stimuli, it is important to teach playing in a structured and clean context of competing stimuli (Ferreira de Carvalho et al., 2016; Strain, 1990). Faced with a more systematic teaching, it becomes necessary to incorporate incidental education of play. Thus, the child initiates an interaction with an object or an activity that he is interested in; during the session, the professional takes advantage of this natural situation and proposes a demand, uniting teaching with play in a natural way (Ferreira de Carvalho et al., 2016).

It is known that play is fundamental for these children and can help teach academic skills, such as visual-motor orientation games, such as fitting and puzzle games (Figueiredo et al., 2021). Lorenzo et al. (2020) verified that there is growing research interest in subjects related to education, Virtual Reality, and ASD. On the other hand, it is also known that many ASD children have a high interest in computers and electronic games (Aguiar et al., 2018). A study conducted during the pandemic points out that the time of use of electronic devices by people with ASD doubled in this period. Before the pandemic, the average time was 3.3 hours on weekdays; during the pandemic, it was 6.9 hours a week. This allowed these children to have fewer opportunities to engage in social situations. Parents observed that they chose activities with less emotional and cognitive effort. The study highlights that these individuals took negative risks during this period, as their social interactivity worsened and their isolation further increased (Cardy et al., 2021). Offering games aimed at children with ASD is optimizing the time these children spend entertained with electronic devices in learning and interaction opportunities. Research conducted in 2019 (Carreño-León et al., 2019) studied considerations for designing user interfaces in apps for children with ASD. The authors emphasize that knowing the existing proposals for the design of user interfaces provides a basis for design processes when designing applications for children with ASD generates advantages at the time of the project. Focusing efforts on important points such as motivating communication skills and boosting social interaction is possible. The conclusion of the study is that the use of technology aimed at children with ASD can be positive. They cite positive factors such as reducing social isolation and improving social interaction, communication, fun, and teamwork, thus strengthening social and communication skills, among other skills.

2.2. Minimal Pairs

Language alterations in children with ASD are well-known and studied; they are important as a diagnostic criterion and prognostic factor. The vast majority of studies focus on the functional, pragmatic aspects of language because that is where the greatest difficulties and changes are. However, there are few specific studies of the phonetic-phonological aspects of the language of these children. Even though there is no guarantee that speech disorders are specifically related to ASD, they are communicative and social barriers and, therefore, must be considered and analyzed. The significant increase in the incidence of people with ASD added to the advances in proposals and public policies aimed at inclusion, has allowed children with ASD to be much more present in the school space. In this context, aspects related to the schooling process, such as the acquisition of reading and writing, began to be further investigated. A study (Clendon et al., 2021) points out that there are several reasons why children with ASD may have specific difficulties in the teaching-learning process. They emphasize that literacy difficulties are frequent due to specific characteristics of autism, changes in oral language, or associated cognitive characteristics. In this learning path, the development of phonological awareness is essential. Phonological awareness is a complex skill that involves the ability to consciously think, reflect, and manipulate speech sounds (Gonçalves et al., 2013). Phonological awareness is part of the phonological processing of oral language. Good performance and acquisition of oral language become fundamental in developing phonological awareness. Early in literacy, oral language and phonological awareness skills are interrelated (Donicht et al., 2019). Studies highlight the importance of phonological awareness in the written language acquisition process (Hipfner-Boucher et al., 2014; Santos & Befi-Lopes, 2012; Wertzner et al., 2014). Phonological processing comprises mental information processing operations based on the phonological structure of oral language. The understanding that speech can be segmented and the ability to manipulate these segments. For example, they are gradually developed following the child’s awareness of the language’s sound system, that is, of words, syllables, and phonemes as identifiable units (Capovilla & Capovilla, 2000). The process of elaboration of the contrast established by the proposal of minimal pairs is part of this path. Torres et al. (2018) analyze the phonetic performance of 25 children with ASD at various school levels with the use of a specific protocol. They observed that the phonological difficulties of the children studied are not negligible and imply an important obstacle in their oral communication. Despite the repercussions of the school and the social performance of these children. Recent studies have investigated the literacy process of children with ASD. Many factors may be related to the literacy learning difficulties presented by children with ASD. Among them are characteristics of the autism spectrum, deficits in oral language or social communication (Clendon et al., 2021). The development of phonemic and phonological skills is fundamental for the development of phonological awareness.

In the minimal contrast therapy method, also known as minimal pairs, pairs of words are selected that are distinguished by a single consonant or vowel but which are produced as homonyms by the child (Pagliarin, & Keske-Soares, 2007). As an example, according to Freitas and Alves (2016), it is possible to say that there is a minimal pair when two phonic sequences are distinguished by a single phoneme, for example, the Portuguese words “tom” (tone) and “dom” (gift). Here, there is a minimal difference characterized by loudness. The /t/ is classified as an occlusive consonant segment (depending on the meeting of the posterior part of the tongue with the soft palate), alveolar (depending on whose sound is articulated at the meeting of the tip of the tongue with the dental alveoli), voiceless. The /d/ segment is classified as occlusive, alveolar, voiced. Therefore, the contrast is considered in an identical environment since the difference is found through a single sound, in the same place, in the two sound sequences. The mentioned example makes clear the distinction between the phonemes /t/ and /d/. Table 1 provides examples of minimal pairs.

The method of minimal pairs has degrees of difficulty. When the sound that distinguishes a minimal pair is presented at the beginning of the word, it is called easy; in the middle, it is medium; and at the end, it is difficult. There are several reasons for replacing phonemes; only two will be treated: the substitutions involving the spelling of voiced and voiceless phonemes. Some pairs of phonemes have a characteristic of differentiating themselves by the sound trait; some are voiced, and others are voiceless. The phonemes / p /, / t /, / k /, / f /, and / s /are considered voiceless since they do not present vibration of the vocal folds when produced. In turn, the phonemes / b /, / d /, / v /, and / z /are performed with vocal fold vibration; therefore, they are considered voiced phonemes. The soundtrack corresponds to an important distinction between the pairs of these sets of phonemes: / p / x / b /; / t / x / d /; / f / x / v /; / s / x / z /.

Based on the voiceless/voiced substitutions, the studied minimal pairs analyzed should be related to the environment in which the game was created in its respective phase, as the Brazilian Portuguese words “pão” (bread) and “mão” (hand), “torta” (pie) and “porta” (door) belong to the bakery stage; “truta” (trout) and “fruta” (fruit), “uva” ( grape) and “luva” (glove) belong to the supermarket stage; “bola” (ball) and “cola” (glue), “caneta” (pen) and “careta” (grimace) belong to the school stage.

2.3. Related works

Using minimal pairs is a common method in speech therapy sessions. Some studies were found in the literature using this method in speech therapy sessions, such as the Kera puzzle game (Figueiredo et al., 2021). This game makes use of minimal pairs, working as an auxiliary tool by professionals in the speech therapy field in the intervention of children with phonological disorders.

Autism has always been a constant theme in the speech therapy field, according to the systematic review by Fernandes and Nohama (2020). As a result, some speech therapist intervention works in childhood autism were found in the literature, such as Barbosa et al. (2020). This work aims to analyze the effects of clinical listening on the parental discourse of children with autism on speech therapy teamwork at a Child and Youth Psychosocial Care Center.

The study by Scotini, Orsati, and Macedo (2021) aims to evaluate the accessibility of different applications available for training children with ASD. The study was carried out in three stages: 1. identification of applications available for people with ASD; 2. Characterization of applications according to the skills trained, and 3. Evaluation and classification of applications based on a specific guideline for accessibility for people with ASD. Among the features most found in the applications are simple interfaces, avoidance of distracting stimuli, and clear instructions. The results presented by the authors highlighted the importance of the list of applications, the characteristics contained in the accessibility guideline, and their ranking. They serve as a basis for users with ASD, family members, teachers, and health professionals to search for an application.

In her work, Sánchez (2021) analyzes and evaluates 50 educational digital applications used to develop communication between students diagnosed with autism. The results showed 8 apps with excellent quality and 17 with an adequate score, with their recommended use for educational interventions. A list of these selected applications was generated as a useful resource for teachers and families.

The work brought by Rehman et al. (2021), in the context of the new “normal”, focuses on identifying features of several highly rated mobile applications designed to assist people with ASD.

The work by da Silva et al. (2020) brings an objective discussion about the use of Digital Information and Communication Technologies in the literacy and learning process of people with autism. A systematic review of the literature was carried out only with national publications (Brazil) between 2014 and 2019. The results indicate that TDICs bring positive contributions to the literacy and learning process but suggest a lack of research in the area.

In the literature, no studies have been found that relate these two themes mentioned above, minimal pairs and autism.

3. Game Development

The game developed is aimed at being used as an additional tool by professionals in the Speech Therapy field for the intervention of ASD children, in training their ability to understand the words now spoken by people with the use of a face protection mask, without the possibility of the visual appeal of the face, such as facial mimic. This game was developed by combining the approaches of interactive software development, top-down and bottom-up, generating refined prototypes until an acceptable version is reached according to the requirements initially identified (Sommerville, 2011). In this type of approach, the project’s success depends on the designers knowing who their users are, that the team acts together, and that the various versions are created quickly and tested. The stage of game development is set out below. This approach was used in this project to allow the game to be implemented in parts and tested by experts in an agile way.

3.1. Requirement Analysis

The elicitation of functional and non-functional requirements took place through several meetings with the specialist in Speech Therapy, the developer, and the specialist in Computing. Thus, considering the target audience as children between 8 and 11 years old, the following functional requirements were considered:

• The game must provide instruction before each stage. This is essential so that children do not feel lost. It is desirable that these instructions be carried out by text and by pre-recorded voice.
• The user can collect a tip (corresponding to remembering the objective of that moment) in each phase. These tips should be in an easily accessible place for the child.
• The game should present three everyday scenarios to the user: a bakery, a supermarket, and a school. Familiar environments are important for people with ASD in order to train social cognition.
• Each scenario must have four phases. The first phase will have a minimal pair related to the context of the scenario. The second phase will have the same minimal pair but with a greater degree of difficulty. The third phase will have a different minimal pair related to the context of the scenario. Finally, the fourth phase will have the same minimal pair but with a higher degree of difficulty. This difficulty progression strategy becomes challenging and does not demotivate the child.
• The player must lose life when colliding with the enemy or if he collects a word out of the requested order. This strategy allows the child to understand that they need to improve in this aspect.
• The game must distinguish the order of words requested, generating a loss of life if this order is not respected.
• The player must collect the words in the order in which they were requested to pass the level. This strategy allows the child to understand how they can progress in the game.
• More lives should be available at each new phase due to the increased difficulty.

The non-functional requirements of the game are:

• The game must not have an internet connection dependency.
• The game must be run on a Windows operating system. The game must be played on a desktop.
• The game is a single-user.
• The game’s usability requirements are:
  • The game must have a design suitable for the target audience (colors, figures, text, buttons, and messages).
  • The game must have audio to identify the figures suitable for the treatment.
  • The game should be easy for a child with autism.
  • The game must be intuitive. In other words, the child must know how to play without the need for a manual, although there are instructions and tips if necessary.
• The game should be motivating for children and adolescents with autism.
• The game cannot have objects that take the user’s focus away. Background elements, ambient music, and other distractors must be analyzed so as not to compromise the child’s focus.

3.2. Design

The game was designed to run on desktop/notebook devices with Windows operating system. The programming language used was GDscript, a Godot Engine language. The following are some key parts of the project:

• Creation of scenarios and figures. To develop an environment that was motivating and inviting for autistic children in the specific age group and the context of a pandemic, it was necessary to think about the most common environments that a child attends. As a result, a bakery, a supermarket, and a school were obtained. Then it was necessary to look for images for the background to avoid confusing the child (without overloading written or visual information). These images were treated in such a way as to remove the background, improve the quality, blur the background, and perfect the images.
• Regarding the images of the objects, there was a study of possible minimal pairs that would make sense in each proposed environment. First, all images were obtained from the internet (Creative Commons license) and edited. Then, these images have undergone validation of the scenarios and objects edited by a professional in the speech therapy field.
• Creation of audios. The audios responsible for guiding the child to play (hunting the minimal pairs in a specific order) were recorded using the mask to give more realism to the game.
• Gameplay creation. Regarding the gameplay itself, it was necessary to think about funnily integrating the minimal pairs, arriving at the idea of a platform game with several different and different difficulty levels. Each day-to-day scenario contains, in each phase, minimal pairs belonging to that context. Before starting each phase, an instruction asks for the minimal pairs and a specific order related to the scenario.
• Minimal pairs. Each phase has an instruction with the respective minimal pair, increasing the difficulty in each phase passed. An example would be: “Take me bread, hand, hand, bread, in that order”.

3.3. Implementation

For the research, a digital serious game will be used with some phases developed for this purpose in three different scenarios, as already mentioned (a bakery, a supermarket, and a school). The game starts on a screen demonstrating the basic controls of the game (Fig. 1), and then a new screen appears with a voice instructing the user to pick up objects in a certain order (Fig. 2). Then, the child has to collect them in the order he was asked, in a scenario where he needs to go up/down and jump over obstacles. The child is represented by the boy’s avatar with a mask. If he collects the wrong word or collides with an enemy, he will lose points (hearts), as shown in Fig. 3. In the same way, the child will earn stars if they get the right words. There are difficulty levels for each scenario presented in the game.

Each scenario has four phases, so when the child wins all phases, a screen will appear congratulating him (Fig. 4 and Fig. 5).

4. Evaluation methodology

According to the data provided below, the game was evaluated, involving a group of professionals in speech therapy and children with ASD.

4.1. Participants

Study participants will be divided into two groups. There are nine speech therapists in the first group, whose inclusion criterion consists of exercising their profession and working with autistic children and adolescents. Fourteen children and adolescents are in the second group, aged between 8 and 11 years, whose inclusion criterion is to attend speech therapy sessions, be autistic with a medical report, and be considered able, by the specialist, to use the game. Exclusion criteria will be those not considered suitable by the speech therapist.

Failure to participate in this research (using the game) does not prevent the participants from carrying out activities planned for the speech therapy session.

The recruitment of both groups will be done for convenience since one of the project members is a speech therapist and already has contact with groups of speech therapists and children in sessions.

4.2. Local

Due to the pandemic, the research will take place online. The group of speech therapists will evaluate/validate the game at their place of convenience. For children and adolescents, the use of the game will occur in virtual speech therapy sessions that are already taking place. The research will take place in an appropriate place (of convenience for both groups) to guarantee the participants’ confidentiality and privacy.

4.3. Procedures

At first, speech therapists received the executable file for installing the game on their computers. After that, they will be free to test the game for as long as they wish, but the expectation is that they will be able to navigate the game in 30 minutes. After that, they will be asked to answer a questionnaire about the game they evaluated (5 minutes). This questionnaire was validated by a group of speech therapists specialized in ASD. The questionnaire was created with the following questions:

• Is the game motivating for ASD children and adolescents?
• Is the design adequate (Colors, Figures, Texts, Buttons and Messages)
• Is the figure identification audio suitable for the treatment?
• Is the game easy for an ASD child?
• Is the game intuitive?
• Is the game playful?
• Do you believe the game could be used for intervention in speech therapy sessions?

At the appropriate time within the session, the speech therapist applied the game so that ASD children could use it for a time not exceeding 30 minutes in a single session. Finally, the children’s impressions about the use of the game will be captured by a simple questionnaire that the children will answer (with the help of the speech therapist). The questionnaire is:

• Is the game cool?
• Is the game fun?
• Is it an easy-understand game?
• Would you play it again?
• Did you understand the voice even with a mask?
• Is the avatar cool?
• Say things you liked about the game.
• Say things you did not like about the game.

As the speech therapist will conduct the questionnaire, she is free to use the question in another way to be more efficient.

No information that could identify the participants was used. After analyzing the results, all those responsible for the participants will receive the return of the study through registered e-mail.

5. Results and discussions

The methodology used to evaluate the game was composed of two parts: evaluation with speech therapists and evaluation with children, as described below.

5.1. Evaluation with speech therapists

The test questionnaire was answered by nine participants, who were professionals in the speech therapy field. The responses were recorded between November 9th and 13th, 2020, and were summarized in Table 2.

After the child finished the game, the speech therapists were invited to use the game and give their opinions about the user interface, design, visual and sound effects to validate its playful, motivating, and functional character in its use in Speech Therapy sessions. Thus, a questionnaire was made available on Google Forms, containing ten multiple-choice questions, using the 5-point Likert scale.

5.2. Evaluation with children

Fourteen children aged between eight and eleven years participated in the evaluation. After using the game for an average of 30 minutes, the children were asked to give their opinion on the game, using a questionnaire with ten multiple-choice questions (3-point Likert scale) and two open-ended questions. The results of the questionnaire are presented in Table 3 and the following text.

In addition to the multiple-choice questions, the children were asked questions about what they liked most about the game; among the answers were: “Everything,” “To play,” “avatar,” “to pick things up,” and “Much cool.” Regarding what they liked least, they said, “Nothing,” “the noise,” “I wanted to play every day,” and “it’s fast.” It can be seen from the children’s responses that the game had a very positive evaluation concerning satisfaction with its use and its visual elements. Already, regarding the sound elements, we realized that because the game is audible primarily, it even disturbed some children a little, something familiar because autistic children are more sensitive to sounds than the average population. So, based on the results, small changes in the audio would be something to consider. Regarding future sessions, open-ended questions will also be asked to speech therapists, asking for suggestions and recommendations.

5.3. Results obtained and other related discoveries

According to the authors (Meghan et al., 2018), numerous studies confirm the positive impact and effectiveness of using games to practice various skills in specialized therapy for children with autism spectrum disorder (ASD). Other authors (Simonoff et al., 2008) consider using games essential for developing social skills in children with ASD. Social skills are directly related to the ability to understand the speech and facial expressions of the interlocutor. This competence was compromised during the mandatory use of facial masks imposed by the COVID-19 pandemic. Children with ASD exhibit impairment in producing facial expressions adapted to social contexts. A research group (Grossard et al., 2019) conducted a study on the proposal of a serious game to teach facial expressions to children with ASD. During the literature search, they found very few serious games focused on producing facial expressions adapted to a specific social context. Other studies emphasize that offering a digital tool that aids in the social integration and inclusion of children with ASD into their environment is more natural and, consequently, enhances their skills and independence. (García-Guillén et al., 2016).

Kerns et al (2015) highlight the need for serious games specifically targeted for communicative contexts. The authors believe that communication is essential for any type of activity. Communicative difficulties or even the absence of speech can result in negative consequences for children with ASD, such as low educational levels, depression, low income, and loss of confidence or self-esteem. A literature review conducted by Tsikinas and Xinogalos (2019) found that most studies for individuals with autism aim to improve social and communication skills. Moreover, most studies on the effects of serious games on individuals with ASD had a positive impact. According to the authors, learning involving serious games is considered very promising for individuals with ASD.

In the context of ASD treatment, serious games can be developed to meet the therapeutic needs of children, adapting to their abilities and challenges. Serious Games’ benefits in the intervention for individuals with ASD include improving social skills, engagement and motivation, treatment individualization, immediate feedback, and task-based learning. However, it is essential to note that serious games should be used as part of a comprehensive therapeutic approach, integrating with other forms of intervention to meet the individual needs of each patient with ASD. Ongoing research in this area is crucial to optimize the therapeutic potential of serious games and improve the quality of life for those affected by ASD. The incorporation of serious games into the therapeutic arsenal has emerged as an effective strategy to enhance the quality of life and development of children and adolescents with ASD (Mihova et al, 2023).

5.4. Serious games and their benefits for treatment

With rapid technological progress, especially smartphones, scientists have developed innovative applications, such as serious games for mobile devices, to assist children with autism spectrum disorder. However, the design of these applications has not always considered crucial aspects, such as usability and accessibility. Usability and, especially, accessibility are essential differentiators for the quality and efficiency of serious games (Jaramillo-Alcázar et al, 2022). There seems to be a lack of clinical evidence that children with neurodevelopmental disorders can benefit from the application of serious games (Kokol et al, 2020). However, as early as Dakin and Frith (2005) noted, visual media had a universal appeal to children. Still, for those with autism spectrum disorder, the attraction might be even more intense due to distinctive visual characteristics that play a significant role in the information assimilation process.

In recent years, the implementation of computerized intervention programs, commonly referred to as serious games, has been observed to enhance socio-cognitive skills in children within the Autism Spectrum. These serious games are an economically viable solution to bridge the gap between the growing demand for evidence-based interventions and the constraints of access to specialized autism services (Casale et al., 2015). Among all Information and Communication Technology (ICT) tools, serious games are the most promising for instructing children with autism spectrum disorder (Kirst et al, 2022).

In summary, serious games emerge as a promising therapeutic tool for children with autism spectrum disorder. The ability of these games to combine entertainment and education offers an engaging medium to enhance specific social and cognitive skills essential for the development of these children. However, future initiatives must consider not only therapeutic efficacy but also usability, accessibility, and the integration of playful features to ensure a holistic and practical approach to therapy for children with ASD. Continuous improvement and research in this intersection between technology and treatment can improve these individuals’ quality of life and holistic development.

6. Conclusions

The COVID-19 pandemic has brought profound societal changes, whether in their social or professional context. COVID-19 is changing the routine of young children with ASD, and they are called to respect rules and habits that ‘are not always understandable for them (i.e., disinfecting your hands, using masks, staying at home, homeschooling, etc.) (Narzisi, 2020).

This article presented a serious game to assist speech therapy sessions with autistic children before COVID-19. All phases of the game’s development cycle were given, and tests were performed with speech therapists and children belonging to the target audience during speech therapy sessions. Both tests confirmed that the game has good usability and is a fascinating resource for therapeutic intervention, especially in the initial training for using a mask and sound perception during the pandemic.

Regarding the specific needs of children with ASD, as described in detail in the methodology, the game provided the anticipation of actions, visual support, realistic figures, everyday environments, the possibility of working with communication, and conversational skills in social situations.

It is worth noting that although the WHO declared, on May 5, 2023, the end of the Public Health Emergency of International Importance (ESPII) regarding COVID-19, the research continues to have social relevance. Training children with ASD to pay attention to speech without using facial expressions helps this audience to have better interaction in means of communication that do not allow the use of facial expressions, such as a telephone call.

As future work, it is possible to create more phases with different day-to-day environments, increasing difficulty levels of the minimal pairs required, and improvements in interactions with children, such as developing a virtual robot to interact in place of the audio.

Authors’ contributions

Valéria Farinazzo Martins: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Supervision, Validation, Visualization, Writing–original draft, Writing–review and editing.

Vitor Oiakawa: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Software, Visualization, Writing–original draft, Writing–review and editing.

Cibelle Albuquerque de la Higuera Amato: Conceptualization, Investigation, Methodology, Supervision, Validation, Writing–original draft, Writing–review and editing.

References

Aguiar, E. C., Gomes, V. O., Sarinho, V. T., & de Santana-Bahia-Brasil, F. (2018). Quiztea-uma proposta de desenvolvimento de quiz digitais para indivíduos portadores do transtorno do espectro autista. In XVII Brazilian Symposium on Computer Games and Digital Entertainment (SBGames) (pp. 276-281). Sociedade Brasileira de Computação. https://doi.org/10.5753/sbcas.2019.6262.

American Psychiatric Association. (2013). Diagnostic and Statistical Manual of Mental Disorders - DSM-5 (5th ed.). American Psychiatric Association. https://doi.org/10.1176/appi.books.9780890425596.

Barbosa, A. M., Figueiredo, A. V., Viegas, M. A. S., & Batista, R. L. N. F. F. (2020). Os impactos da pandemia COVID-19 na vida das pessoas com transtorno do espectro autista. Revista da Seção Judiciária do Rio de Janeiro, 24(48), 91-105. https://doi.org/10.30749/2177-8337.v24n48p91-105.

Cardy, R. E., Dupuis, A., Anagnostou, E., Ziolkowski, J., Biddiss, E. A., Monga, S., Brian, J., Penner, M., & Kushki, A. (2021). Characterizing changes in screen time during the COVID-19 pandemic school closures in Canada and its perceived impact on children with autism spectrum disorder. Frontiers in Psychiatry, 12, 702774. https://doi.org/10.3389/fpsyt.2021.702774.

Campanário, I. S. (2008). Espelho, espelho meu: a psicanálise eo tratamento precoce do autismo e outras psicopatologias graves. Ágalma.

Capovilla, A. G., & Capovilla, F. C. (2000). Efeitos do treino de consciência fonológica em crianças com baixo nível sócio-econômico. Psicologia: Reflexão e Crítica, 13, 7-24. https://doi.org/10.1590/S0102-79722000000100003.

Carreño-León, M., Leyva-Carrillo, A., Carreño-León, M. A., & Sandoval-Bringas, J. A. (2019). Consideraciones para el diseño de interfaces de usuario en aplicaciones para niños con autismo. In F. J. Álvarez Rodríguez, M. A. Carreño-León, & J. A. Sandoval Bringas (Eds.), Avances sobre Reflexiones, Aplicaciones y Ttcnologías inclusivas (pp. 70-74). Consejo Nacional de Acreditación en Informática y Computación A.C.

Casale, M. B., Mittal, A., Whalen, C., Stahmer, A., Quiocho, J., & Vejnoska, S. F. (2015). One size does not fit all: A smarter way to develop computer assisted interventions for children with asd. International Journal of Serious Games, 2(2). https://doi.org/10.17083/ijsg.v2i2.63.

Clendon, S., Paynter, J., Walker, S., Bowen, R., & Westerveld, M. F. (2021). Emergent literacy assessment in children with autism spectrum disorder who have limited verbal communication skills: A tutorial. Language, Speech, and Hearing Services in Schools, 52(1), 165-180. https://doi.org/10.1044/2020_LSHSS-20-00030.

Colizzi, M., Sironi, E., Antonini, F., Ciceri, M. L., Bovo, C., & Zoccante, L. (2020). Psychosocial and behavioral impact of COVID-19 in autism spectrum disorder: An online parent survey. Brain Sciences, 10(6), 341. https://doi.org/10.3390/brainsci10060341.

Dakin, S., & Frith, U. (2005). Vagaries of visual perception in autism. Neuron, 48(3), 497-507. https://doi.org/10.1016/j.neuron.2005.10.018.

da Silva, J. A., de Carvalho, M. E., Caiado, R. V. R., & Barros, I. B. R. (2020). As tecnologias digitais da informação e comunicação como mediadoras na alfabetização de pessoas com transtorno do espectro do autismo: uma revisão sistemática da literatura. Texto Livre: Linguagem e Tecnologia, 13(1), 45-64. https://doi.org/10.17851/1983-3652.13.1.45-64.

Degli Espinosa, F., Metko, A., Raimondi, M., Impenna, M., & Scognamiglio, E. (2020). A model of support for families of children with autism living in the COVID-19 lockdown: Lessons from Italy. Behavior Analysis in Practice, 13(3), 550-558. https://doi.org/10.1007/s40617-020-00438-7.

Donicht, G., Ceron, M. I., & Keske-Soares, M. (2019). Erros ortográficos e habilidades de consciência fonológica em crianças com desenvolvimento fonológico típico e atípico. In CoDAS (Vol. 31, e20170212). Sociedade Brasileira de Fonoaudiologia. https://doi.org/10.1590/2317-1782/20182018212.

Fernandes, M., & Nohama, P. (2020). Jogos Digitais para Pessoas com Transtornos do Espectro do Autismo (TEA): Uma Revisão Sistemática. Revista Iberoamericana de Tecnología en Educación y Educación en Tecnología, (26), 72-80. https://doi.org/10.24215/18509959.26.e8.

Fernandes, F. G., Santos, S. C., de Oliveira, L. C., Rodrigues, M. L., & Vita, S. S. B. V. (2014). Realidade virtual e aumentada aplicada em reabilitação fisioterapêutica utilizando o sensor kinect e dispositivos móveis. Universidade Federal de Uberaba-UNIUBE, Minas Gerais.

Ferreira de Carvalho, O. M., & D’Oliveira de Paula Nunes, L. R. (2016). Possibilidades do uso de jogos digitais com criança autista: Estudo de caso. In V. L. P. dos Santos Caminha, J. Yoneda Huguenin, L. M. de Assis, & P. Pires Alves (Eds.), Autismo: Vivências e caminos (p. 77-90). Blucher, https://doi.org/10.5151/9788580391329-10.

Figueiredo, K. O., Cardenuto, R. R., Amato, C. A., & Martins, V. F. (2021). Kera Puzzle: jogo digital educacional para apoio à intervenção fonoaudiológica. Revista Ibérica de Sistemas e Tecnologias de Informação, (E41), 503-515.

Freitas, A. M. A., & Alves, G. C. (2016). Consciência fonológica: a utilização de jogo digital como estratégia para o ensino de Língua Portuguesa. Letras & Letras, 32(2), 83-108. https://doi.org/10.14393/LL63-v32n2a2016-5.

Fundação Oswaldo Cruz. (2020). Saúde mental e atenção psicossocial na pandemia Covid-19. Último acesso em: setembro, 2020. https://d66z.short.gy/6P1YK3

García Guillén, S., Garrote Rojas, D., & Jiménez Fernández, S. (2016). Uso de las TIC en el Trastorno de Espectro Autista: aplicaciones. EDMETIC, 5(2), 134–157. https://doi.org/10.21071/edmetic.v5i2.5780.

García-Peñalvo, F. J., Corell, A., Abella-García, V., & Grande, M. (2020). Online assessment in higher education in the time of COVID-19. Education in the Knowledge Society, 21(12), 1-12. https://doi.org/10.14201/eks.23013.

Gonçalves, T. S., Neves, T. A. P., Nicolielo, A. P., Crenitte, P. A. P., & Lopes-Herrera, S. A. (2013). Phonological awareness in children from public schools and particularly during the process of literacy. Audiology-Communication Research, 18(2), 78-84. https://doi.org/10.1590/S2317-64312013000200005.

Grossard, C., Hun, S., Dapogny, A., Juillet, E., Hamel, F., Jean-Marie, H., Bourgeois, J., Pellerin, H., Foulon, P., Serret, S., Grynszpan1, O., Bailly, K., & Cohen, D. (2019). Teaching facial expression production in autism: The serious game JEMImE. Creative Education, 10(11), 2347-2366. https://doi.org/10.4236/ce.2019.1011167.

Hipfner-Boucher, K., Milburn, T., Weitzman, E., Greenberg, J., Pelletier, J., & Girolametto, L. (2014). Relationships between preschoolers’ oral language and phonological awareness. First Language, 34(2), 178-197. https://doi.org/10.1177/0142723714525945.

Jaramillo-Alcázar, A., Arias, J., Albornoz, I., Alvarado, A., & Luján-Mora, S. (2022). Method for the development of accessible mobile Serious Games for children with Autism Spectrum Disorder. International Journal of Environmental Research and Public Health, 19(7), 3844. https://doi.org/10.3390/ijerph19073844.

Kerns, C. M., Kendall, P. C., Zickgraf, H., Franklin, M. E., Miller, J., & Herrington, J. (2015). Not to be overshadowed or overlooked: Functional impairments associated with comorbid anxiety disorders in youth with ASD. Behavior Therapy, 46(1), 29-39. https://doi.org/10.1016/j.beth.2014.03.005.

Kirst, S., Diehm, R., Bögl, K., Wilde-Etzold, S., Bach, C., Noterdaeme, M., Poustka, L., Ziegler, M., & Dziobek, I. (2022). Fostering socio-emotional competencies in children on the autism spectrum using a parent-assisted serious game: A multicenter randomized controlled trial. Behaviour Research and Therapy, 152, Article 104068. https://doi.org/10.1016/j.brat.2022.104068

Kokol, P., Vošner, H. B., Završnik, J., Vermeulen, J., Shohieb, S., & Peinemann, F. (2020). Serious game-based intervention for children with developmental disabilities. Current pediatric reviews, 16(1), 26-32. https://doi.org/10.2174/1573396315666190808115238.

Lorenzo, G., Lorenzo-Lledó, A., Lledó Carreres, A., & Pérez-Vázquez, E. (2020). Approach from a bibliometric perspective of the educational application of virtual reality in people with autism spectrum disorder. Knowledge Society, 21, 14. https://doi.org/10.14201/eks.19522.

Lugo-Marín, J., Gisbert-Gustemps, L., Setien-Ramos, I., Español-Martín, G., Ibañez-Jimenez, P., Forner-Puntonet, M., Arteaga-Henríquez, G., Soriano-Día, A., Duque-Yemail, J. D., & Ramos-Quiroga, J. A. (2021). COVID-19 pandemic effects in people with Autism Spectrum Disorder and their caregivers: Evaluation of social distancing and lockdown impact on mental health and general status. Research in Autism Spectrum Disorders, 83, Article 101757. https://doi.org/10.1016/j.rasd.2021.101757.

Matthew J. Maenner, Shaw, K. A., Bakian, A. V., Bilder, D. A., Durkin, M. S., Esler, A., Sarah M. Furnier, Hallas, L., Hall-Lande, J., Hudson, A., Hughes, M. M., Patrick, M., Pierce, K., Poynter, J. N., Salinas, A., Shenouda, J., Vehorn, A., Warren, Z., Constantino, J. N., DiRienzo, M., Fitzgerald, R. T., Grzybowski, A., Spivey, M. H., Pettygrove, S., Zahorodny, W., Ali, A., Andrews, J. G., Baroud, T., Gutierrez, J., Hewitt, A., Lee, L.-C., Lopez, M., Mancilla, K. C., McArthur, D., Schwenk, Y. D., Washington, A., Williams, S., & Cogswell, M. E. (2021). Prevalence and Characteristics of Autism Spectrum Disorder Among Children Aged 8 Years — Autism and Developmental Disabilities Monitoring Network, 11 Sites, United States, 2018. Surveillance Summaries, 70(11), 1-16. https://doi.org/10.15585/mmwr.ss7011a1

Meghan, N., Qian, Y., Massolo, M., & Croen, L. A. (2018). Psychiatric and medical conditions in transition-aged individuals with ASD. Pediatrics, 141(Supplement_4), S335-S345. https://doi.org/10.1542/peds.2016-4300K.

Mihova, P., Stankova, M., Andonov, F., & Stoyanov, S. (2023). The Use of Serious Games for Developing Social and Communication Skills in Children with Autism Spectrum Disorders. In C. P. Lim, A. Vaidya, Y. W. Chen, V. Jain, & L. C. Jain, (Eds.) Artificial Intelligence and Machine Learning for Healthcare: Vol. 2: Emerging Methodologies and Trends (pp. 181-196). Springer International Publishing. https://doi.org/10.1007/978-3-031-11170-9_7.

Narzisi, A. (2020). Handle the autism spectrum condition during Coronavirus (COVID-19) stay at home period: Ten tips for helping parents and caregivers of young children. Brain Sciences, 10(4), 207. https://doi.org/10.3390/brainsci10040207.

Pagliarin, K. C., & Keske-Soares, M. (2007). Abordagem contrastiva na terapia dos desvios fonológicos: considerações teóricas. Revista CEFAC, 9, 330-338. https://doi.org/10.1590/S1516-18462007000300006.

Parens, E., Matthews, L., & Appelbaum, P. S. (2020). Polygenic risk scores, prediction of psychiatric disorders, and the health of all of us. The Lancet Psychiatry, 7(6), 481. https://doi.org/10.1016/S2215-0366(20)30185-1.

Piscalho, I. A. D., & Veiga-Simão, A. M. (2014). Promoção da autorregulação da aprendizagem das crianças: proposta de instrumento de apoio à prática pedagógica. Nuances: estudos sobre Educação, 25(3), 170-190. https://doi.org/10.14572/nuances.v25i3.3163.

Rehman, I. U., Sobnath, D., Nasralla, M. M., Winnett, M., Anwar, A., Asif, W., & Sherazi, H. H. R. (2021). Features of mobile apps for people with autism in a post COVID-19 scenario: Current status and recommendations for apps using AI. Diagnostics, 11(10), 1923. https://doi.org/10.3390/diagnostics11101923.

Sánchez, M. C. (2021). Análisis y evaluación de aplicaciones para desarrollar la comunicación en el alumnado con trastorno del espectro autista. EDUTEC. Revista Electrónica de Tecnología Educativa, (75), 168-187. https://doi.org/10.21556/edutec.2021.75.1681.

Santos, M. T. M. D., & Befi-Lopes, D. M. (2012). Vocabulary, phonological awareness and rapid naming: contributions for spelling and written production. Jornal da Sociedade Brasileira de Fonoaudiologia, 24, 269-275. https://doi.org/10.1590/S2179-64912012000300013.

Scotini, C. Y., Orsati, F. T., & de Macedo, E. C. (2021). Levantamento e avaliação da acessibilidade de aplicativos para apoio ao aprendizado de crianças com diagnóstico de transtorno do espectro autista. Cadernos de Pós-Graduação em Distúrbios do Desenvolvimento, 21(1), 39-67. https://doi.org/10.5935/cadernosdisturbios.v21n1p39-67.

Simonoff, E., Pickles, A., Charman, T., Chandler, S., Loucas, T., & Baird, G. (2008). Psychiatric disorders in children with autism spectrum disorders: prevalence, comorbidity, and associated factors in a population-derived sample. Journal of the American Academy of Child & Adolescent Psychiatry, 47(8), 921-929. https://doi.org/10.5935/cadernosdisturbios.v21n1p39-67.

Strain, P. S. (1990). LRE for preschool children with handicaps: What we know, what we should be doing. Journal of early intervention, 14(4), 291-296. https://doi.org/10.1177/105381519001400401.

Sommerville, I. (2011). Software Engineering (9th ed.). Addison-Wesley.

Torres, J. C., León, H., & Figueroa, M. (2018). Desempeño fonético-fonológico en niños con Trastorno del Espectro Autista (TEA) de pre-kínder a tercero básico. Revista Chilena de Fonoaudiología, 17, 1-18. https://doi.org/10.5354/0719-4692.2018.51611.

Tsikinas, S., & Xinogalos, S. (2019). Studying the effects of computer serious games on people with intellectual disabilities or autism spectrum disorder: A systematic literature review. Journal of Computer Assisted Learning, 35(1), 61-73. https://doi.org/10.1111/jcal.12311.

Wertzner, H. F., Pulga, M. J., & Pagan-Neves, L. D. O. (2014). Metaphonological skills among children with speech sound disorder: the influence of age and disorder severity. Audiology-Communication Research, 19, 243-251. https://doi.org/10.1590/S2317-64312014000300007.

_______________________________

(*) Autor de correspondencia / Corresponding author