Validation of a Cyberbullying Serious Game Using Game Analytics

Bullying is a serious social problem at schools, very prevalent independently of culture and country, and particularly acute for teenagers. With the irruption of always-on communications technology, the problem, now termed cyberbullying, is no longer restricted to school premises and hours. There are many different approaches to address cyberbullying, such as school buddies, educational videos, or involving police in counseling; however, awareness continues to be insufficient. We have developed Conectado, a serious game to be used in the classroom to increase awareness on bullying and cyberbullying in schools. While playing the game, students gain a first-hand immersive experience of the problem and the associated emotions, fostering awareness and empathy with victims. This paper describes Conectado and presents its validation with actual students using game analytics.


I. INTRODUCTION
B ULLYING is a serious social problem, and it is present in all countries to a greater or lesser extent [1], [2]. Approximately 40 to 55 percent of students around the world are involved in some form of bullying and/or cyberbullying, either as a victims, aggressors or observers; and 20 to 50 percent of them say that they have been victimized [3], [4], [5]. The prevalence of bullying and cyberbullying varies significantly between countries, for example some studies indicate that the percentage of victimization in the United States and Asia is 55 percent, compared to other Central and South American countries (22), Canada (25), Oceania (25) and Europe (30 percent) [3].
In the last few years, the problem of bullying has grown significantly. The emergence of new technologies, and particularly the expansion of Internet access and social networks, together with the early use of these technologies by children and young adults, has led to the appearance of another form of bullying known as cyberbullying. Cyberbullying is not restricted to school hours and premises and can occur at any time and place [5], [6], [7].
Bullying victims can be defined as those that are targets of: a) aggressive behaviors, both physical and verbal and both direct and indirect; b) behaviors that isolate the individual from the group and/or; c) behaviors that create feelings such as powerlessness and fear [8], [9], [10], [11]. Cyberbullying appears when bullying is carried out through new technologies such as the Internet, mobile phones or different media: e-mail, forums, social networks, text messages, chat rooms, or online video games [1], [12], [13].
Among the most common effects of bullying and cyberbullying are problems associated with attention, behavior, and emotional regulation, which usually interfere with the victims' ability to learn and adapt in schools [5], [14], [15]. Important psychological disorders and even suicides have also been documented [3], [13]. Bullying is a complex problem that requires multiple perspectives and initiatives to deal with, and many educational centers do not know how to handle it effectively. There are many different approaches and tools to address this problem, including school buddies [16], awareness campaigns, training talks, educational videos, and video games, among others. However, more research is needed on the topic, as not all approaches are readily available, and many of them are not scalable due to different factors such as cost or specific requirements; in particular, some require one or more specialists to carry out the intervention or to train teachers. In addition, most of the interventions do not consider the parents, who also need to be provided with resources to better educate their children at home and to provide them with strategies to identify and address bullying.
In this paper we present Conectado, a serious game designed for use in school classrooms to increase awareness on bullying and cyberbullying. By playing this game, we give the player the opportunity to experience the problem of bullying and cyberbullying and associated emotions from the perspective of the victim, in a safe environment, increasing empathy with the victim and learning strategies to defend against harasment. We also describe the experimental validation of Conectado at schools using game analytics techniques.
The rest of the paper is structured as follows: Section II reviews the current solutions against bullying and cyberbullying; Section III presents the game Conectado; Section IV describes the use of game analytics to validate serious games; The Section V describes the methodology used; the results of which are presented in Section VI; and finally, Section VII summarizes the main conclusions and outlines future work.

II. CURRENT APPROACHES
Bullying and cyberbullying are problems that affect a significant percentage of students and their families. Parents, teachers and students often do not know how to act or even how to clearly identify the problem. We briefly review below some of the current approaches, projects and tools that focus on preventing and acting on bullying and cyberbullying.
There are numerous projects and web platforms focused on increasing awareness about this social problem, its consequences and what signals help to identify it. These approaches provide resources and help to prevent both bullying and cyberbullying. Some of the most relevant initiatives and projects addressing these problems include the 1. Cyberbullying Research Center (cyberbullying.org) is a U.S.-based website dedicated to providing up-to-date information on the nature, extent, causes and consequences of cyberbullying among adolescents. The platform also explores and addresses other risky behaviors among adolescents such as malpractice in social networks, sexting and other issues related to digital citizenship. 2. Pantallas Amigas (www.pantallasamigas.net) is a Spanish and Portuguese-language initiative whose main mission is to promote the safe and healthy use of new technologies as well as the promotion of responsible digital citizenship in childhood and adolescence by having carried out different interventions and campaigns in collaboration with different entities. 3. StopBullying (www.stopbullying.gov), administered by the U.S. Department of Health and Human Services, is a website that provides information about harassment, cyberbullying, its risks and how to prevent and respond to these problems The most common way to combat bullying is through campaigns and projects organized by associations, foundations and, as in the case of Spain, by regional governments, such as: game designed to reduce bullying in the classroom. The video game presents a 3D scenario where the user can take the role of victim or observer. SchoolLife is designed to work in groups, where several people participate directly and the rest can give their opinion on what is happening in a chat provided by the game. While free, the premium version can also generate reports. After studying current existing resources, we found that none of them focus on the student's emotions to encourage passive observers (and potential bully accomplices) to take active roles and help the victim. Most of the prevention resources that exist so far focus on teaching what the problem is, its consequences, and how to proceed in these cases -but do not provide a simulated first-hand experience. In addition, there is a lack of studies to validate the impact of these projects. Of the videogames mentioned, we have only found studies with users for the Cybereduca project, where 176 young people participated in the experiments, with 83 being part of the control group. The results were positive and the intervention yielded both a significant decrease in victimization behaviours (0.6 points out of 7 between pre and post) and significant increase in various positive social behaviours in the experimental group (0.57 points out of 7), as compared to the control group.
In response, we have developed Conectado, a serious game to increase awareness on bullying and cyberbullying in schools through experience and emotion. The game places players in the role of a victim, creating a common experience in the class and focusing on conveying feelings such as helplessness and frustration.
In Conectado, the player plays the role of victim, while in many other games the player is an observer of the bullying or cyberbullying situation. It is focused on conveying feelings and increasing empathy with the victims, while many other resources focus on imparting knowledge through definitions and ingame lessons, which are less attractive to learners. There are already studies on how video games can increase empathy and affect players' motivation and attention [20], [21].
Another advantage is that Conectado is made with Unity, which allows the game to be deployed to many platforms and run in any of the devices commonly found in high schools. Our experiments were carried out on Linux and Windows computers (as found in the computer labs of the participating schools); but tablets and other devices are also supported.
This paper focuses on testing the effectiveness of the game in changing the player's awareness and feelings, through data collection in experiments with real students, and analysis of the data that is generated during the game and in pre-and post-tests.

III. CONECTADO, A SERIOUS GAME AGAINST BULLYING
Conectado is a video game that has been designed and developed with the goal of raising awareness on bullying and cyberbullying through experience and emotions. As a tool, it is intended to help educational professionals to increase the interactivity and emotional engagement of their classes. Its vocabulary and conversations are targeted at students between 12 and 17 years old. Within the videogame, players are placed in the role of a student suffering cyberbullying by schoolmates (see Figs. 1a and 1c).
The game provides a safe environment that allows players to experience firsthand some of the feelings experienced by victims of bullying and cyberbullying, as well as to reflect on the consequences of bullying-related action and inaction by increasing player's awareness of the problem. The game follows the classical adventure videogame structure where the primary means of interaction with the game is to choose dialogue options, which alter the story and can lead to different endings.
In the video game, the player lives the life of a highschool student arriving at a new school. During the 5 days in the game, more and more classmates turn against and harass the player, following the lead of the classroom bully (pictured in Fig. 1a). Each of these game-days, the player-protagonist moves from home to school; spends the day at school; and returns home with the protagonist's parents. At the end of each day the protagonist experiences a nightmare, a mini-game that is designed to be impossible to overcome and is intended to frustrate and stress the player, echoing the feelings of their character. Through the different scenarios, players can talk to the characters in the game, such as classmates. Conversations are navigated by choosing between different dialogue options. Each player's choices can alter the flow of the story, decreasing the level of confrontation and affecting future dialogues and events, as well as the ending that is reached on the 5th day. However, the bullying situation cannot be totally avoided or resolved until the actual ending. The three different endings vary according to the protagonists' level of friendship with classmates, whether players have asked the teacher for help or not, and the relationship between the protagonist and the ingame parents. The videogame is designed so that players feel powerless because they cannot solve the problem; and seeks to make them understand that they need to ask for help, because they will otherwise be unable to solve the (cyber)bullying situation on their own.
A secondary goal of the game is to teach strategies that reduce the risk of becoming a victim, an abuser or an enabling bystander. For example, by informing players about the risks and consequences of mocking or making derogatory comments on social networks; or uploading images that can be easily accessed by anyone (see Figs. 1b and 1c). Another advantage of the video game is that it reflects on the social aspect of the problem, since cyberbullying affects not only those directly involved, but also the degree of bullying-awareness of the social environment [3], [22]; and this social aspect can be effectively represented in a language and channel that teenagers understand as their own.
The game provides a common experience for all students that can be used to initiate later discussions under the supervision of a professional such as a teacher or guidance counselor. The aim is to make players reflect on their common game experience and better understand the ultimate consequences of their actions. In addition, the game sheds light on important aspects such as asking for help (presented as an act of courage as opposed to snitching), communicating with parents [14], or avoiding being an observer/collaborator who looks the other way; since passive bystanders are often involuntary but necessary participants in harassment processes.
As depicted in Fig. 1a, the game is currently in Spanish, although it has been designed to be easy to translate into other languages. The design of the game and the simulated scenarios is based on different multiple experiences, projects and studies [23], [24] which should be common across multiple countries and cultures. The game is played in first-person mode, so that the player is not represented in the game by an avatar which facilitates feeling identified with the protagonist. The game also considers the player's gender, which the user is requested to select at the very start. Dialogues and small parts of the plot change according to the chosen gender (Spanish grammar is more gender-aware than English), increasing the players' sense of identification.
The video game works by engaging the player's empathy, confronting the player with feelings such as impotence, inferiority, frustration and loneliness. This is done through minigames in the form of nightmares that cannot be overcome; and through dialogues where the player must choose among multiple answers, none of which include violent responses or lead to a lasting solution. Only at the end of the game can such a solution be reached; and, depending on player choices, it may be far from satisfactory. This gameplay is designed to persuade players that bullying and cyberbullying can only be effectively stopped by asking for help and, importantly, offering help, as these are serious social problems that victims cannot solve alone.
The implementation of the game is based on an educational design that encompasses the key points of serious game design as described in [25]: 1. The pedagogical objectives must have clear instructional objectives. 2. The simulation of the domain must represent it reliably in the gameplay, in order to teach and raise awareness at the true target domain. 3. The progression and problems the player will encounter must be defined. 4. The game must exhibit marked aesthetics that make it attractive for the player. 5. The interactions of the player with the simulation must be clearly defined. 6. The conditions of use, how, when, where and by whom the video game will be used, must be clearly specified. The most common duration of a Spanish high school lecture is 55 minutes, so the game is designed to last from 30 to 40 minutes depending on the speed and reading comprehension of the players. This leaves at least 15 minutes remaining that teachers can use to start a discussion as mentioned earlier in this section.

IV. GAME ANALYTICS, VALIDATING, AND IMPROVING SERIOUS GAMES
For instructors, it is important that educational tools provide control and situational awareness of the classroom; generally, such tools should allow professionals to view student progress and understand if the tools are serving their purpose. In the case of serious games, the necessary data collection and analysis is part of Game Learning Analytics, which can be defined as the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs, in particular applied to games [26], [27], [28].
Collected data from players' interactions with the game provide insights not only regarding players' improvement and learning, but also into the actual use of the game, including those parts that do not achieve their intended goals; and information on where players have problems with game mechanics or educational objectives. This allows developers to improve the serious game as an educational tool, and enables instructors to better adapt it for classroom use.
In this project, data is collected via a tracker module within the game which tracks in-game player actions and sends them to a server for storage and analysis. The actions that are being tracked are referred to as traces, and are analyzed in real time by the server, which then generates visualization dashboards that provide overviews to stakeholders such as teachers and instructors. The dashboards can then be queried for further details on particular students [27]. The data traces are formatted using the xAPI (Experience API) format, and in particular the xAPI application profile for serious games [29]. Use of this standard is intended to facilitate the development and the combination of tools required to build analytics systems [30]; and to comply both with data portability and with an adequate experimental design with the privacy data requirements imposed by the EU GDPR [31].
For this game, the visualizations generated for the teacher display the in-game locations of players, their relationship with classmates, their in-story decisions, and, for players that have finished the game, the specific ending that has been reached. In addition, data provide the information needed to understand how players are interacting with the game, including, for example, how much time they have spent in each scene [32]. This can be useful to know if, when and where players are getting stuck in the game. This information is also useful for developers, which can use it to locate difficulties for players, and to improve the adaptation of the video game as an educational tool within the classroom. For example, it allows gameplay time to be adjusted to fit within a specific class slot.
To organize and guide our analysis, we pose the following research questions: Is the serious game an appropriate learning tool for creating a change in cyberbullying and bullying awareness among early teenagers? Does the video game fit the typical duration of a class? Does the game have different effects depending on the gender of the student? Is the game valid for the entire target age range? Is the video game perceived as engaging? Does the video game create empathy towards victims?
To address these questions, the following research hypothesis are established, and will be tested and analyzed in Section V: 1. H1: The video game fits within the duration of a standard 1-hour class slot, leaving at least 15 minutes for guided reflection with the tutor. 2. H2: The intervention with the videogame increases awareness. 3. H3: The video game creates empathy towards victims. 4. H4: The video game is perceived as such and players like it even though it is not intended to be enjoyable.

V. METHODOLOGY
The effectiveness of the video game Conectado has been experimentally validated in three different educational centers in Spain, where a total of 257 students aged between 12 and 17 years old have participated. In one-hour sessions, students were asked to complete pre-post questionnaires and, between both, play the game.
The use of the Conectado videogame has been evaluated and accepted by participating schools' educational boards as a complementary and voluntary educational activity. All three highschools have signed the corresponding informed-consent forms, which specified that all collected data would be pseudonymized and not traceable back to individual students. Students were assigned random tokens, and only schools could, if they chose, retain a correspondence between tokens and students [33].
The single group pre-and post-test design without a control group was chosen for several reasons: 1. Feasibility in actual schools: This evaluation has been done on schools and finding educational centers that agree to participate in the validation is difficult as schools must be willing to modify their planning and must have the required infrastructure for deployment to perform these tests. 2. Choice of control treatment: schools that have participated used other resources and prevention programs such as mentors and professional talks (e.g., police program). Therefore, it was difficult to find an additional resource that brings something new and can be used in all centers, constituting a control group against which the Conectado video game can be compared. Additionally, the possibility that the control group would not use any tool was ruled out by schools. From an experimental point of view, such an experiment would also introduce risks, as controlgroup students would object to being left out. 3. Fairness and time constraints: schools are reluctant to divide their courses into two to use one part as a control group. Instead, they prefer to apply the resource, in this case the Conectado video game, to all their students in a class equally and at the same time. The obvious alternative, counter-balanced groups, requires significant additional time. There are some limitations associated with the single group pre-and post-test design [34], [35]. We have carried out experiments in 3 different schools, at different times, and with different staff, classrooms and previous interventions. There was no free time between the tests and the gameplay, and users neither leave the classroom nor comment about the intervention with each other. The previous characteristics of the different sessions minimize some of the problems of this kind of experiments (e.g., history, maturation). In relation to the test effects, we added other indicators of the usefulness of the intervention: in the free-text part of the post-test, a question asks players whether they considered that they had learned anything by playing. The answers to this question were compared with the increase in awareness assessed by the pre-post. Therefore, if data showed an increase in awareness in the prepost and players explicitly mentioned that they had learned new aspects about cyberbullying, then it appears reasonable to state that the increase was not only due to the questionnaire.

A. Materials
The pre-test questionnaire, prior to students playing the video game, consists of two parts. The first part asks for the player's age and gender. The second part consists of 18 items in which each player is asked if they consider a certain action as bullying or cyberbullying, rating it on a 7-point Likert scale with responses ranging from low (rated as 1) to high awareness (7).
The 18 questions about what actions respondents consider to be harassment or cyberbullying are drawn from the adaptation of different questionnaires such as CUVE3 [36], ECIP-Q, EBIP-Q [37] and the Cyberbullying Test [38], which are used to assess violence and different types of bullying and cyberbullying within schools. These tests have been previously validated both for bullying and cyberbullying and adapted to Spain demography.
The aim of these 18 adapted questions is to assess the player's perception of what actions they consider bullying or cyberbullying before they play the video game. The effectiveness of the game is valuated as the increase of the awareness based on the changes between both tests.
The post-test is performed by each user when the game is over, or when the time set to play is over, i.e., some players may not finish the game before the post-test. This post-game questionnaire consists of several sections: 1. The first section is identical to the second part of the pre-game questionnaire. The analysis of the data compares the pre-and post-test results, to determine if there is a change in the perception of students regarding acts that can be considered harassment and/or cyberbullying. 2. The second section is the Cyberbullying Test, which is used to assess the level of cyberbullying within each class. It is composed of 45 items: 15 to assess the observer level, 15 to assess the offender level and 15 to assess the victimization level. 3. The third and final section tries to evaluate what social networks students use and how often. 4. In addition, the questionnaire contains three free-text questions about the video game: whether the player has felt identified with a game character, the player's opinion about the game (and any suggestions for future versions), and the whether the player believes to have learnt anything about bullying or cyberbullying from playing the video game. The data collected from both questionnaires and traces of game interactions are only identified by the aforementioned random four-letter tokens. Before a session starts, a list of valid random tokens is generated and stored in the server, linked to the session where they will be considered valid. This ensures that the game and questionnaires cannot be used without entering a valid session-specific token; while linking each game-play session with its pre-post questionnaires.

B. Sessions
We initially ran a formative evaluation with 64 students in two high-schools to elicit initial feedback and check the applicability of the initial version of the game in one hour. With that initial feedback we produced a new version of the game that was shorter but had the same educational approach-specifically, we removed the repetion of one of the school days which was considered as reiterative by students. We consider this as only a minor change from the educational design, and therefore all educational game outputs have been considered together, regardless of version (short or long) being played.
Experiments have been carried out in different sessions of around 15 to 25 students, depending on the size of the computer classrooms in the different educational centers. Students were informed that they were going to play a video game about school life at the time of the session, but received no additional explanations on educational details or intent.
At the beginning of each session, students are provided with the token printed on a piece of paper, and use it to enter the game. Once the token is entered and validated, the game launches the pre-test that players must fill in, and does not allow playing until pre-test answers have been received. Once the game finishes, players are requested to fill in the post-test.

VI. RESULTS
Data gathered in the different sessions, as we will see below, are not only used to evaluate the effectiveness of the game, but also to improve it and to provide near real-time information to the teacher who uses the tool during the class.
We analyze data from three points of view: finding possible points of improvement, identifying the most common errors, and assessing the degree to which the game is adapted to the typical 1-hour duration of classes in Spanish educational centers. The purpose of evaluating the data from several viewpoints is to prove the usefulness of using game analytics techniques for game validation, improvement, and enhancing teacher awareness. With the data from player interactions with the game (traces) sent by the tracker to the server, and the age and gender of the players, we evaluate the in-class use of the game and how the information provided to the teachers through visualizations helps them to understand the development of each session, for example to locate players that may need help. Finally, with the results obtained from the pre-post questionnaires, we check the effect of the game on its players.

A. Improving the Game
We begin this analysis with a short overview of collected analytics. Out of the original 257 data sets, each corresponding to a play-through by a student, 27 data sets did not reach the server, and 36 resulted from students running the game multiple times, either because they closed the game in the middle of the session or because of errors within the game. We therefore discarded 63 data sets when analyzing the time that takes to complete the game. Analyzing the first valid collected sets from the first sessions (64 players), we found that the 35 players who finished the game took longer than expected (n ¼ 35, M ¼ 41.07 min; SD ¼ 5.68 minÞ. A consequence of this high duration is that, within this subgroup, a quarter of those players who had no problems with the game did not finish it. To prevent excessive game duration, we created a slightly shorter version to be used in the following sessions: version 2. With this new version, in the remaining 193 data sets, the server received 147 play-throughs and 90 percent of the players completed the game, and required shorter average times to do so (n ¼ 132, M ¼ 32.97 min; SD ¼ 6.23 minÞ than those that used the first version.
The first version of the game therefore did not satisfy hypothesis "H1: The video game fits within the duration of a standard 1-hour class slot, leaving at least 15 minutes for guided reflection with the tutor". This was solved in version 2 of the game, which was shortened to fit into the available time. The difference between playing times for both versions can be seen in Fig. 3a.
It is also observed that the younger the player, the longer the time needed to complete the game. This may be caused by the player's reading speed. Women also take longer to complete the game than men. This can be seen in Table I for the short version of the videogame.
The traces helped to easily find various game problems, which we missed during beta-testing. For instance, when the game was used in window mode, users often closed the entire game when attempting to close the in-built "social network" window, mistaking the game's controls for those of the internal window (see top-right corners of Figs. 1b and 1c). We also found that several sequences of in-game decisions could prevent players from reaching the very end of the game. Another observation is that most players who finished the game (74.4 percent) arrived at the best possible end, while only 8.9 percent arrived at a bad end and 16.7 percent reached a neutral end.

B. Gaining Insights From Analysis
In this case the use of analytics allows session supervisors, such as teachers, to monitor the deployment of the game in the classroom and receive information on the actions of players within the game. For example, they can see the scenes accessed by the players, know whether they have completed the game, access the level of friendship of players with the different ingame characters, and check the in-game progress at any moment. Analytics also provide a built-in inactivity indicator, where a player is considered to be inactive when no game traces are received after 30 seconds, alerting the supervisors.

C. Validating the Game
To check the effect of the game on the players, 223 data sets were used. 27 data sets did not reach the server and other 7 users did not complete the post-test, and were therefore excluded from this validation. The average age of the remaining data sets was 14.20 (N ¼ 223, SD ¼ 1.26Þ, where 121 were women (45.7) and 102 men (54.3 percent). The age distributions is depicted in Table II, where we can see that most of the players were between 13 and 15 years old.
The result from pre-and post-test is the mean of the means of each of the 18 common items of pre-post test.
The average of the 18 7-point Likert items that have been used to validate the game in the pre-test was 5.72 (N ¼ 223; SD ¼ 1.26Þ compared to 6.38 (N ¼ 223, SD ¼ 1.11Þ in the post-test (see Fig. 3b). A paired Wilcoxon test yields a statistically-significant effect (p À value < 0.001Þ. We also note that female generally present greater awareness values both before and after playing the game. The awareness value increases in both genders (see Fig. 3c). However, when segmenting by age and gender groups, we observe that the game is not effective with 17-year-olds, and has a less pronounced effect in 16-year-olds (see Table II).
The difference of 0.66 points over 7 of the 18 common prepost questions indicates that the game increases awareness in those players aged 12 to 16, fulfilling hypothesis "H2: The intervention with the videogame increases awareness". Although we found that it is not suitable for players aged 17.
It is necessary to emphasize the high level of awareness of the players at the beginning of the experiment, this is because the centers where it has been tested had already undertaken previous anti-bullying interventions, including awareness talks (but not using games) as described in the Methodology section.

D. Player Reflection and Feedback
The last part of the post-test contains three optional free-text questions that request the players' opinion about the game as described in Section V-A, including what they would change for future versions, and whether they identify themselves with any character of the game. Responses for these free-text questions were coded using a key based on preliminary analysis, initially composed of $ 25 binary categories. Many of the categories only made sense for specific questions, while others were generic (for example, "TooLong" was a question-specific category, while "PositiveSentiment" is generic). Two of the authors coded all 223 response rows independently, and then compared the codings, measuring agreement for each of the 3 questions using the average of the Jaccard Index over all responses. Initial agreement was 0.96 for Q1, 0.72 for Q2, and 0.75 for Q3. A single consensus encoding was then generated by both encoders by agreeing on classifications for only those answers that were initially encoded differently. All results in this section are based on the consensus encoding. Finally, the encoded answers were analyzed as binary vectors (with a 0 or 1 for each possible category), and explored using both spreadsheets and the R statistics and analysis platform.
As a first observation, there are 150 responses to the question "Do you feel identified with any of the characters that appear in the game (main character, Maria, Guille, Jose, Ana, Alex, Alison)?", 183 to the question "What did you think of the game's look, and what would you change?", and 180 to the question "Do you think you have learned something?"; that is, most users wrote a free-text response to at least one of these optional questions.
In general, most students reported not feeling represented by the characters: 87.89 percent replied "no" or equivalent, or  left an empty answer, and only 12.11 percent of all students mentioned actual game characters. From those that did mentioned a character, only 1 mentioned the bully, and the rest is 'divided between the victim (18.52 percent) and the surrounding classmates (77.78 percent).
When asked whether they liked the game, and what, if anything, they would like to change, answers were mostly positive (65.47 percent), while 17.94 percent left the answer empty, only 2.24 percent answered that they did not like the game, and the remaining 14.35 percent of comments were neither positive nor negative. 21 percent of those answering that they liked the game mentioned that they found it interesting, fun and/or entertaining. 19.13 percent of those that answered wrote that they would not change anything in the game, while 10.93 percent would make changes in the plot, and 8.74 percent would add or modify the choices in player dialogues. Only 4.93 percent of those who answered considered the game monotonous, and less than 3 percent considered its duration to be either too long or too short. In view of these comments, the video game has been positively received by a large majority of players; few have been bored, although there are things to improve; and in general, the game has been entertaining and liked, in addition to getting players to reflect on the feelings of victims of bullying and cyberbullying.
Analyzing the answers to the question "What did you think of the game's look, and what would you change?", we conclude that "H4: The video game is perceived as such, and players like it even though it is not intended to be enjoyable" is satisfied as 80 percent of the opinions about the game are positive and 22 percent of these mentioned that the game is fun or interesting. There are, however, users who did not like the game (less than 10 percent).
The last free-text question asked students whether they had learnt anything from the game, and if so, what. 63.23 percent said that they had learnt something, 19.28 percent left the question blank, and 10.31 percent answered that they had not learnt anything. From those that left non-empty answers, 9.23 percent mentioned already knowing about the problem and how to approach it. 17.95 percent mentioned that the game helped them to empathize more with the victim. Multiple responses describe player emotions, even though they were not explicitly requested in the questions. For example, 5.38 percent of players which did not leave all answers blank freely mentioned that they found the game to be believable/realistic in its plot, while 3.59 percent express desire to have an option to physically confront the bully (this option was deliberately excluded in the educational game design). These positive comments add even more value to what has already been statistically proven: the video game fulfils its main objective by helping to raise awareness among players, making sure that, within a safe environment, they are temporarily put in the place of victims of bullying and cyberbullying. In addition, the comments also reflect the player's understanding of the importance of not ignoring victims (i.e., neutral observer, bystanders), helping them, and talking to parents and professionals in education. Many users comment that the game helps to better understand victims and their feelings. These results validate hypothesis "H3: Video game creates empathy towards victims".
By considering responses as binary vectors and performing complete-linkage hierarchical clustering based on their Jaccard distances [39], we can isolate a few interesting clusters of answers: 32 (14 percent) of students did not fill in any answer, or only answered with a single positive comment in the question on whether they liked the game. This cluster is marked in red in Fig. 4. A further 8 (4 percent) replied "no" to the first question, and then left the other answers empty. The cluster is marked in dark green in Fig. 4. 28 (13 percent) replied that they liked the game, had learnt something, and did not feel explicitly represented by any character. Some of them make additional comments regarding small improvements or things that they have learnt, but their answers tend to be short. This cluster is marked in green in Fig. 4. A very similar cluster of 25 (11 percent) students is marked in cyan in Fig. 4, with the only significant difference being the difference between answering "no" in the 1st question versus leaving it empty. 20 (9 percent) felt represented by one or more characters. Most of this cluster feels positively about the game itself, and all of them report having learnt something, although only 5 explicitly mention what (mostly reporting empathy, the importance of helping bullied colleagues, and how to react if bullied). The cluster is marked in blue in Fig. 4. 21 (9 percent) did not feel represented, but liked both the game and outcome, and reported that they would either not change anything or only minor changes in some dialogues. The cluster is marked in magenta in Fig. 4. 3 (1 percent) of users simply did not like the game and considered not to have learnt anything. However, 13 (6 percent) of those that reported not having learnt anything did like the game, and indeed often offered multiple suggestions on how to improve it. This cluster is marked in purple in Fig. 4.

VII. DISCUSSION AND FUTURE WORK
This article describes the goals, methodology and main results of a set of experiments to validate the video game Conectado. From our analysis of the data and results, there is evidence that the video game fulfills its main objective, that is, to raise awareness about cyberbullying and to make players empathize with the victims (the videogame will be used by teachers to promote active discussion with the students in the classroom). Players also consider the tool as a video game that they enjoyed playing. Results also suggest that the ages at which it is most effective are 12 to 15 years.
During analysis, we quickly observed that the first version of the game was too long, and after releasing an updated version, we concluded that the use of analytics certainly helped to refine the game, as well as to validate its effects on players. The players themselves say that the video game helped them to better understand the victims. Furthermore, the experiments have also shown that video games are a valuable complement to other solutions that currently exist, with positive results even in centers that have already undertaken other interventions and campaigns to prevent bullying and cyberbullying.
Analysing the data from each school separately, we still obtained a positive difference between the results of the pretest and the post-test for every school. There is no significative difference between schools, even though each has different conditions, and had used different previous interventions on bullying and cyberbullying. This proves that cyberbullying awareness is still increased with the intervention, even when external variables change such as researchers, classrooms, teams, times, schedules and even previous intervention programs used in the schools.
We consider the game to be effective as 63 percent of players mention that the game has taught them new aspects of bullying, and about 20 percent of them mention that the videogame has made them reflect and be aware of the importance of helping the victims or identify themselves with the victims. There is also a positive difference between the preand post-test results that assess the awareness level (0.66 points out of 7).
We have developed a tool that helps young people aged 12 to 16 to empathize with victims of bullying and cyberbullying, and helps them reflect on their feelings and problems (H2 and H3). This tool is a video game that entertains and is positively received by most players (H4); and that fits within the time constraints of a typical class, to be used as a common experience for the group (H1).
Future lines of work are discussed below. A first line involves designing and executing experiments with teachers and students of Education Sciences, asking them to play the game and provide detailed feedback. The goal of these experiments will be to improve the game's value as a "classroom tool" to motivate classroom discussion on cyberbullying, from the teacher's point of view. We are interested in knowing how useful the video game is perceived by teachers, whether they would apply it in their class, and why and how to improve the game as an educational tool for classroom deployment.
Also, with the data collected, for later experiments, and since we now know the average time that players spend in each game scene, alerts can be displayed in real-time to indicate when players spend, for example, more than 20 percent of the average time in any scene. This allows us to alert teachers in the classroom about players that are getting stuck in the game, and potentially help them before the end of the gameplay session.
A second line is to carry out large-scale experiments in educational institutions. Although we consider that the current sample of students is representative, especially given the difficulty in finding centers equipped with the necessary material and availability, the aim is to make an experiment involving even more students to check whether classroom use of Conectado is feasible even in only modestly-equipped centers. This would provide a better model of acceptance among schools, teachers and educators.
Although the results obtained are considered as very satisfactory, there are aspects in which the game can be improved to create a better final version. We are interested in improving the videogame itself, including both its game mechanics and ease of translation into other languages. For example, we plan to create a tool to modify the story that the students will play, by selecting among different pre-built scenes, or from a larger set of situations, so that the teacher can adapt it to the context of the class and what has been experienced in the center related to situations of harassment and cyberbullying. The decision-making also can be improved by providing a larger number of options, together with suitable in-game consequences of choosing them. Finally, we intend to study the possibility of adding a mechanism to estimate player affinity to the different roles (victim, witness and bully) without explicitly asking the players, making the tool not only useful to raise awareness, but also to automatically infer player attitudes towards bullying and cyberbullying.
Dan Cristian Rotaru received the bachelor's and master's degrees in computer science from the Complutense University of Madrid, Madrid, Spain, in 2014 and 2016, respectively. Currently, he is working toward the Ph.D. degree in computer science. He has been part of eUCM since 2014, as a Contract Researcher for projects eMadrid and H2020 RAGE. His research interests include the study of educational video games and their application in the field of medicine, as well as the study of learning analytics techniques to improve their efficacy and their validation as an educational tool.
Cristina Alonso Fern andez received the bachelor's degree in computer science and in mathematics from the Complutense University of Madrid, Madrid, Spain, in 2016, and the master's degree in data mining and business intelligence from the same university in 2017. Since September 2016, she has been a part of eUCM, as a Researcher for part of the H2020 Beaconing Project. Her research interests include educational video games and application of data analysis and data mining for their improvement. Due to her work in beaconing, learning analytics techniques and standards are one of her main research areas. Iv an Mart ınez-Ortiz received the bachelor's degree in computer science (first in the Dean's List "premio extraordinario"), and the master's and Ph.D. degrees in computer science from the Complutense University of Madrid (UCM). He is an Associate Professor with the Department of Software Engineering and Artificial Intelligence (DSIA), UCM. He is also an Assistant to the Vice-rector of technology with UCM and has been the Vice-dean for innovation in the computer science studies. He has been a Lecturer with Computer Science School, UCM and with Computer Science School, Centro de Estudios Superiores Felipe II, Aranjuez, Spain. His research interests include e-learning technologies and the integration of educational modeling languages, serious games, and e-learning standardization.
Baltasar Fern andez-Manj on received the Ph.D. degree in physics from Universidad Complutense de Madrid (UCM), Madrid, Spain, in 1996. He is a Full Professor of computer science with UCM and the Director of the e-UCM e-learning research group. He has the Honorary Complutense-Telefonica chair on Digital Education and Serious Games. His research interest is focused on the applications of ICT in education and in serious games and educational simulations applied to different domains (e.g., medicine, education). He is also working in game learning analytics and the application of e-learning standards to the integration of those technologies in e-learning systems.