Virtual Museums in Arts Education. Results of a Pilot Project in Primary School Settings

Aims: The study presents the results of a pilot project in which virtual museums were used for teaching modern painters to sixth-grade primary school students. Study Design: Experimental study with one experimental and two control groups. Place and Duration of Study: A total of 132 students participated in the study coming from 6 primary schools located in Athens, Greece. The duration of the projects was between January to February 2017. Methodology: The virtual museums were developed by the students using the program Artsteps. For comparing the learning outcomes, two additional groups of students were formed. To the first, the teaching was conventional, while in the second the teaching was supported by multimedia presentations. Research data was collected using questionnaires and evaluation sheets. Results: From the analysis of the results it became evident that students that developed the virtual museums surpassed, in most cases, the other groups of students. The views of students for their work with virtual museums were highly positive. Conclusion: The results lead to the need for further investigation of the matter.


INTRODUCTION
We live in an era where the rapid technological developments resulted in the expansion of knowledge and to a significant change of views on how we educate students. However, certain subject matters, such as the arts education, have been sidelined to such an extent, that their teaching focuses solely on providing information [1]. So, it is becoming increasingly imperative to reconsider how we teach such subjects, and to establish new -and technologically enhancedteaching methods. Indeed, arts' teachers are now fusing the arts with ICT [2]. Several studies indicate the connection between visual arts and virtual reality environments, such as Second Life [3,4,5], stressing the fact that teachers have the opportunity to use ICT for engaging and motivating students, as well as for encouraging cooperation between them [6,7]. Virtual museums also have an intrinsic educational potential [8,9], and the related educational activities enable innovative teaching methods [9]. So, in visual arts, as well as in other teaching subjects, teachers use virtual museums as alternative learning environments [5].
Based on the above, it was quite logical to wonder whether the educational potential of virtual museums can be exploited for teaching subjects related to the visual arts. The basic research question that emerges is whether -and to what extent-the use of virtual museums in teaching arts in primary school students can lead to better learning outcomes compared to conventional teaching methods. For this reason, a pilot project was designed and implemented that had, as a target group, sixth-grade primary school students (ages 11 to 12). The organization, methodology, and the results of this project are presented and analyzed in the coming sections.

Arts Education
Arts play an important role, which also has an educational value, as they build cultures, help to create a society full of imagination and creativity, and help children and adults to develop their cultural identity [10]. Therefore, students' engagement with the arts is of major importance and it is the school's duty to maintain and foster this engagement. The relevant literature suggests that because of the uniqueness of arts education, compared to other teaching subjects, it cannot be substituted by any other teaching subject because it is directed to specific functions of the human mind [11].
Τhe teaching of arts to students has benefits not only regarding knowledge acquisition but also affects their overall mental and emotional development [11,12]. It can be argued that the most important of all, is that arts education offers to students the opportunity to express their understanding of their own lives, to discover its meaning and to understand its uniqueness [13]. Development of critical thinking and of problemsolving skills has also been reported [14]. In addition, students appear to cooperate better and able to tackle a given problem from multiple perspectives, as well as to solve problems by applying inductive thinking and building on existing knowledge [15].
While the arts education is included in primary, secondary and tertiary education, its importance is diminishing, as noted by several studies [2,16]. For example, primary school teachers are not self-confident when teaching arts to their students and, as a result, they marginalize the teaching of this course [17,18]. Besides the issue of low self-esteem of teachers in teaching arts courses, there are other issues that have a negative impact. The sheer volume of the teaching subjects that teachers have to manage on a daily basis, the time-consuming preparation for the teaching of visual arts, the emphasis on "important" courses (language, mathematics, science) are but a few of them [19].

Virtual Museums
As early as the mid-90s the majority of researchers widely accepted the educational potential of museum in terms of their importance for the cognitive, emotional, and social development of students [20]. The technological developments have led to the development of what is called "virtual museum". The term refers to a digital creation organized on a permanent or temporary basis, in the service of society and its development, open to the public, in which the tangible and intangible heritage of humanity and the environment is maintained, studied, communicated, and exposed [21].
Regarding education, learning in a museum (real or virtual) is not considered just a cognitive process. It is also a product of socio-cultural interactions. That is because students-visitors have the opportunity to build new cognitive schemas based on their previous experiences and perceptions, through the exploration and interaction with a wealth of objects, in an environment liberated from the constraints of the classroom [22].
Visiting a virtual museum is an experience as complex as the visit in a real museum; both options are derived from the free will of the visitor [23]. This action is part of the learning process, as the environment of a digital museum provides educational experiences to students who develop their abilities and skills, offers insights and information, is interactive, and it is based on the free will of the stakeholders [24]. Consequently, as a place of learning, communication, and entertainment, a virtual museum can act as a catalyst, affecting the behavior of its visitors/ students.
The use of virtual museums is quite common in countries with high levels of ICT integration in education [25]. For example, almost two-thirds of the European countries have launched initiatives/ procedures for the use of virtual museums in the curriculum [26].
A certain type of virtual museums are the ones that are based on 3D virtual worlds. Interesting applications have been developed in recent years, bridging arts and 3D virtual worlds [27,28]. Research has highlighted the positive effects that emanate from the use of 3D virtual worlds in the development of thinking processes related to art planning, learning, and assessment [29,30].
On the other hand, while teachers are gradually considering the use of virtual museums for the teaching of various teaching/learning subjects, the lack of relevant literature suggests that research has not yet shifted in the direction of examining their usefulness in arts education [31].

RESEARCH RATIONALE AND METHODOLOGY
As already mentioned, the main purpose of the study was to examine what learning outcomes can be achieved through the development of virtual museums, focusing on visual arts and primary school students. The following research hypotheses were formed: H1: Students who develop their own virtual museums achieve better learning outcomes, compared to students that are taught conventionally.
H2: The sustainability of knowledge is also better. H3: Students form positive attitudes and perceptions regarding the use of virtual museums as part of their teaching.
To investigate the above, a pilot program was designed with one experimental and two control groups, as it will be further elaborated in the coming paragraphs. Three important exponents of modern art and modern painting were selected as the teaching subject: Gauguin, Matisse, and Klee. Thus, three teaching units were formed (the study of Gauguin, Matisse, and Klee), together with an introductory one (introduction to modern art).
A collaborative teaching method was applied because the literature suggests that when students work in groups using ICT tools, increased levels of communication and interaction are noted and the benefits of using technology are maximized [32,33]. Therefore, students were divided into groups of four, having at their disposal a common computer, with which they worked collaboratively in order to develop virtual museums using the online program, Artsteps (http://www.artsteps.com/). ArtSteps is a freely available application that allows the development of 3D virtual museums. The use of the application is quite easy; users design their own museum's rooms, place the exhibits (e.g., paintings), and add information to them (web links, text, audio, and video). The design of the environment is such, that allows students to engage in activities that encourage active experimentation, decision-making, critical, and creative thinking [34]. In addition, it helps students to explore the subject from multiple perspectives, to assimilate and to transform the information, and, finally, to compose their own understanding of the subject they are studying. The transformation of knowledge is achieved in a playful and attractive way. Art Steps allows complex activities and tasks to take place and promotes interdisciplinary and multidisciplinary innovative activities, which are difficult to materialize using conventional means and teaching methods. Thus, the principles of constructivism, of experiential, and of collaborative learning are applied, providing the necessary theoretical teaching framework [35].
More specifically, constructivism supports the notion that students' active participation in the learning process plays an important role and that they construct their knowledge on the basis of what they already know and by making connections between old and new information [36]. Consequently, the project's basic idea was students, with the help and guidance of their teachers, to gradually develop their own virtual museums. During this process, it was hypothesized that students will obtain an overall view of modern painting, learn about the three artists, their works, their technical characteristics, and, finally, they will be able to compare these artists. The development of the virtual museums was done in stages, on the basis of Ott's and Pozzi's [9] ideas. These researchers suggested that the successful integration of ICT into arts education relies on the following: (a) personalization of the content, (b) researchbased learning, (c) interdisciplinary activities, and (d) collaborative learning.
The stages were: Preliminary stage (two teaching hours). During this stage, students learned how to use the application. Stage 1 (four teaching hours). Students chose the structure of the halls of the virtual museum, as well as the necessary colors. Following that, they searched the Internet for information regarding modern art, assessed the information they gathered, and decided what to keep. The relevant info was placed in one of the virtual museum's halls. Stages 2-4 (four teaching hours each). Students searched the Internet for information regarding the three artists, assessed the information they gathered, and decided what to keep. As in the previous stage, the relevant info was placed in three of the virtual museum's halls, together with representative paintings of each artist. Stage 5 (two teaching hours). The groups presented their museums to the whole class, followed by discussions for each artist and comments for the virtual museums.
Students, during the above stages, were absolutely free to work at their own pace, select what they like or not, and decide what is relevant to their projects (Fig. 1). In other words, they had total control over their learning process. It should be noted that the teachers had an advisory role.
To allow the interpretation of the importance of the learning outcomes, two more groups of students participated in the study, that were taught using different teaching methods. In both groups the same artists were examined, as in the experimental group. The duration of the teaching of each unit was also the same (with the exception of the preliminary stage). The first teaching method was conventional; students used conventional teaching material (e.g., notes, exercises, activities) that was written for this purpose. The teaching method was purely teacher-centered; students did not work in groups. The second method was also conventional, but this time, students worked in groups. In addition, the teaching was enhanced with audiovisual presentations and in-classroom activities (e.g., drawing paintings similar to the ones of the artists that were studied; writing and presenting short essays regarding the artists, expressing their thoughts about their drawing styles). During teaching, students were free to collaborate and discuss with each other, as in the experimental group.
For collecting research data, the following tools were used, common to all groups: (a) pre-test, to test the initial knowledge of students, (b) delayed post-test, which was given about fifteen days after the end of the project, to examine the sustainability knowledge, and (c) evaluation sheets, which were administered immediately

Fig. 1. Students work
following the end of each lesson (four in total, one for modern painting and one for each artist), in order to capture the immediate learning outcomes of the teachings. The above tests, included mainly closed-ended questions (rightwrong, multiple choice, and fill-in-the-blanks), while open-ended questions were about a third of the questions. Finally, a short questionnaire was devised (9 Likert-type questions, 4 open-ended) and was administered to students that developed the virtual museums after the end of the project, in order to record their views and attitudes about the whole project.
The pilot program was implemented in six primary schools in Athens, Greece, lasted roughly for two months (January to February 2017), and the initial sample size was 142 sixthgrade students. Prior to the implementation of the project, students' parents were informed of the project's purposes and settings and their written consent was requested (and given) for the participation of their children.

RESULTS AND DISCUSSION
A number of students had to be excluded from the study because they were absent in one or more session. The final sample size was 132 students, divided into three groups (Group0conventional teaching, Group1-contemporary teaching, Group2-virtual museums). For the analysis of the results, scores were computed on the basis of the number of correct answers in each evaluation sheet (including pre-and delayed post-tests). Mean scores per group of participants and per test are presented in Table 1.
One-way ANOVA tests were to be conducted to compare the scores of the three groups in all tests, in order to determine if they had any significant differences. Prior to conducting these tests, it was checked whether the assumptions of ANOVA testing were violated. Some minor issues regarding the normality of data were found. On the other hand, the literature suggests that ANOVA is robust to moderate deviations from normality (the absolute values of the skewness and kurtosis for the data not to be more than double their respective standard errors) and the false positive rate is not affected very much by this violation [37,38]. Since in the above cases the violations were minor, they were considered as acceptable deviations. Since all the other assumptions were met (equal number of participants in all groups per grade, no outliers, and no violation of the homogeneity of variance), the analysis was conducted ( Table 2).
Post-hoc comparisons were conducted using the Tuckey HSD test on all possible pairwise contrasts (except for the pre-test because no statistically significant differences were noted there). The results of these tests are presented in Table 3.
Taken together, these results suggest that: All groups had the same initial knowledge level since they did not have statistically significant differences in the pre-tests. Consequently, any differences observed in the participants' knowledge acquisition after the interventions can be attributed to the different teaching methods that were followed. Students in Group 2 (virtual museums) outperformed students in Group0 (conventional teaching) in all cases. Students in Group2 outperformed students in Group1 (contemporary teaching) in four cases (including the delayed post-test), while the no differences were observed in one case. Students in Group 1 outperformed students in Group0 in four cases (including the delayed post-test), while in one case the results were not statistically significantly different.  The data analysis, as presented above, confirms H1 as well as H2. The learning outcomes when students develop their own virtual museums in arts education are better compared to other teaching methods, and the retention of knowledge is also better.
The questionnaire administered to Group 2 at the end of the project recorded students views for the project. From their answers, it became evident that their views were highly positive, as presented in Table 4. Thus, H3 was confirmed.
Indicative students' responses: It was fun because we were able to develop our own museum. It was fun because I was able to place whatever I wanted on the walls of the museum. I had great fun. I had a great time and I learned a lot at the same time. I really liked that I built the museum together with my classmates.
I had never done something like this before, it was great. I learned a lot about modern art and the artists and would like to learn more.

DISCUSSION
The main research hypothesis was that when students develop their own virtual museums, this leads to better learning outcomes compared to conventional teaching methods. Art education was selected because, at school, it is a neglected subject matter [10]. Virtual museums were selected because it is believed that, through them, the understanding of visual arts' concepts is promoted [39]. To compare the learning outcomes, three different teaching methods were used. The data analyses revealed that in most of the evaluation sheets (including the delayed post-test), the group that developed virtual museums fared better compared to the groups that were taught conventionally. Therefore, the main research hypothesis was accepted. The results can be attributed to a number of reasons, related to the software that was used and with the teaching method.
The introduction of virtual museums in teaching, although an unprecedented experience for students, did not disorientate them and did not disturb the classroom's climate. Instead, a pleasant learning environment was developed, through which students achieved better learning outcomes compared to the other groups. The questionnaire's relevant questions confirm the good climate that prevailed during lessons.  Another fact that has to be taken under consideration is that students during the phases of teaching where the virtual museums were developed, worked on their own, without the teacher's guidance. The fact that satisfactory learning outcomes were achieved, seems to confirm the views of other researchers, who believe that students with a high degree of autonomy can achieve good performance [40,41]. Moreover, students worked in groups, thus, the results confirmed the opinions of those who believe that good results are achieved through teamwork [33]. Rahn and Kjaergaard [42], argue that collaboration among students produces better learning outcomes because cooperation and discussions among the group members, allows a deeper understanding of the phenomena that are studied. The confirmation of satisfactory levels of cooperation comes from students' responses to the relevant question in the questionnaire. Collaboration seems to have been fostered because students were actively engaged in experimentation, cooperation, and competition [43].
The playful/fun character of teaching seems to have played a major role, by providing more incentives for learning. The pleasant learning environment that is developed during students' engagement with virtual museums is highlighted by Carrozzino and Bergamasco [44] and also by Styliani, Fotis, Kostas, and Petros, [45]. This, in turn, leads to increased incentives for learning [46].
In addition, in the questionnaire, the majority of students stated that they liked the various teaching components and that they would like other subjects to be taught the same way. The attractiveness of digital learning tools is also identified in other studies [47] and it is a strong indication of students' endorsement of this alternative way of teaching. In particular, Artsteps, besides being an easy-to-use piece of software, it is freely available, and can be easy accessed from computers in schools. Its main characteristic is that, because it is a 3D environment, it enables the sense of presence, which is also important [48].
Students did not experience any problems during the development of their virtual museums and they became familiar with the controls very quickly. The good relationship that young children have with technology is well documented [49]. It can be argued that computer use is compatible with students' experiences and skills [50]. It is, therefore, logical that no problems were reported. Finally, it should be emphasized that in the delayed post-test Group2 exceeded -by far-the other groups, which leads to the conclusion that with the use of virtual environments better retention of knowledge is achieved, as suggested by North and North [51].
As a final note, an encouraging element that has to be commented is that the present study was based on a pilot project in a field that is not thoroughly tested and documented. Thus, flaws and problems were expected. Given that pretty good results were achieved, it is almost certain that a more well-organized effort is expected to have even better results.

CONCLUSION
Although the study's results, as presented in the preceding sections, are interesting, there are research limitations that must be taken into account. The sample, although sufficient for statistical analysis, was fairly limited both numerically and geographically. It is therefore quite difficult to generalize the results. The teaching was limited to a few relevant artists. The inclusion of more artists/lessons would have allowed the development of a more comprehensive project. Alas, time restrictions imposed by the schools did not allow this to happen. The use of more and diverse research tools, for example, interviews and observations, would have allowed an in-depth understanding of the research problem. Finally, as with any study, students, may not have been completely honest in their responses about their impressions from that was used, confusing the questionnaire with some form of evaluation. Future research can include a larger variety of art's concepts and artists. Also, different ages and/or different tools can be tested. Finally, future work can use and compare other teaching methods.
In conclusion, the need for changing the way we teach art to our students is almost self-evident. The development of virtual museums from students is a pretty interesting alternative pathway, which is worth further investigation.