Badges Are Back! - Fostering Self-Assessment during Personalised Learning in Making and Digital Fabrication

. Badges have been used in education and leisure activities for a long time. Digital badges in particular have been reported to become a gamification element fostering students’ engagement and motivation. In our work, we explore the use of badges during making and digital fabrication as a way to foster personalised learning paths within the eCraft2Learn learning ecosystem. The badge system description and visual representations are provided here, alongside the underlying pedagogical justification.


Introduction
Personalisation is a broader concept than individualization or differentiationpersonalised learning means that the learning process is driven by the learners' interests, needs and timeline. This means also adjusting the pace and the approach taking into account the learners' specificities. Thanks to new digital technologies, learning can be done anytime, anywhere or any place. In personalised learning, learners' previous knowledge, competence, life and work skills both in formal and informal settings should be considered. Within this frame, learners do have the opportunity to learn in ways that are preferable for themselves and also set their learning objectives for their own learning. In this sense, in personalised learning the teacher is acting more as a coach, rather than an instructor. This means that learner's selfregulation is a fundamental skill and not only the cognitive dimension, but all dimensions of the learner should be in focus (emotional, social, life experience etc.). (see Fullan, 2009;Patrick, 2013). Miliband (2016) has put forward five key elements that should be brought with personalised learning: • knowing the strengths and weaknesses of individual students, • develop competences and confidence in the learner through teaching and learning strategies based on individual needs • create curriculum choice that engages and respects students • put forward a radical approach to school organization • get the community, local institutions and social services We propose an approach to foster personalised learning through the use of badges during making and digital fabrication activities in order to provide a suitable solution to the first two challenges listed above. With our approach we do not only aim at assisting the students understand their strengths and weaknesses but also assisting them to develop their confidence during the learning experience. Furthermore, we also aim at fostering in students the capability to actively reflect on their learning process. This cannot be achieved if badges are used in the usual way, i.e. the teacher directly assigns the badges to the students on the basis of the outcomes of their activities and results during the implementation of their projects. Going beyond this, we propose in this paper to use badges within a self-assessment framework, where the students are requested to provide the evaluation of their own skills as they have developed during the practical activities. The self-assessment is composed of a score (in a scale from 1 to 5) and an textual justification for the score. The teacher reviews this self-evaluation and if necessary revise it, explaining to the students the reasons for the modifications. The teacher evaluations are converted into graphical scores in the form of badges.

Related Work
Personalised learning has been fostered by a number of pedagogical approaches as well as tool. For instance, McLoughlin and Lee (2010) discuss the feasibility of applying the social web tools (web 2.0) to foster personalised learning and self-regulation, through social media tools, such as Twitter™, urging for a pedagogical change to facilitate the integration of such social software tools. Peirce et al. (2008) propose an adaptive educational game architecture to foster personalised learning through gaming, avoiding the one-size-fits-all approach present in many educational games. Within the gaming approach, digital badges have also been introduced as a tool for boosting student's motivation, providing recognition of one's credentials as well as evidencing achievement (Gibson et al., 2015). With this in focus, we propose a badge system that is based on the students' self-assessment. Our system is developed as part of the eCraft2Learn learning ecosystem, as a tool for fostering personalised learning.

Badge System for Personalised Learning
The eCraft2Learn project (https://project.ecraft2learn.eu/) is a H2020 European project dedicated to develop an innovative learning ecosystem able to integrate making and digital fabrication, the essence of the recent maker movement, in a pedagogically sustained framework. The project's target are 13-17 years old students. The project can be summarised by mentioning the following main aspects (Alimisis et al., 2018): -a project-based oriented pedagogy able to foster engagement, motivation, selfawareness, team-working, both abstract and practical abilities, in students, with a specific attention to STEAM teaching-learning purposes; -a Unified User Interface (UUI) integrating open source or at least free software capable to support all the phases of development from ideation to evaluation and sharing of the produced artefacts; -a hardware architecture based on a low-cost, fully featuring board, like Raspberry PI3, used as developing system and running the UUI, and a do-it-yourself electronic board like Arduino to connect small components like motors, lights and sensors; -a 3D printing service to provide 3D printing capabilities able to enrich the production of artefacts with self-designed parts and to empower creativity; -a rich repository of open educational resources to support teachers and students, both at a personal and team level, in using the provided tools, promoting an autonomous learning progression and an easy way to overcome basic technical problems; -a series of use cases to be used to get started with the system, to show how to fully exploit its educational potential.
Though the learning path is partly guided by a set of five stages that we have identified during the project design (ideation, planning, creation, programming, sharing; see Figure 1, see Suero ), these stages do not form a linear sequence of step which have to be strictly orderly followed by the learners but they can use them as a general reference, adapting their path according to what they need or they want to deepen during the development. This is a first form of personalisation that the learning ecosystem allows and promotes.  In spite of its relative simplicity, an Arduino board equipped with actuators and sensors can provide the basis for meaningful robotic experiences: in fact, students must refer to physical phenomena and uncertainties, which are present in a real environment in contrast with the usual ideal essence typical of virtual environments. This means that they have to sense some physical aspect around their artefact and program accordingly its actions, something which solicits to have a better understanding of what is about, to refine solutions during the development to obtain the expected behaviours and to reflect on the reached solutions at personal level and within the team.
During small-scale pilot testing planned within the project, we observed that some teams decided to be challenged in building a real mobile robot. In this case, for time restrictions, they were provided with a simple Arduino-based robot kit that may include motors, LEDs, distance and light sensors, but even in this situation, they had the opportunity to personalise their artefact with some 3D-printed extension. In the end, we think that the ecraft2Learn ecosystem may represent an effective synthesis between educational robotics and the maker movement, providing a full spectrum of state-ofthe-art tools and a meaningful teaching-learning methodology.
As briefly introduced above, the entire eCraft2Learn ecosystem has been developed with the aim of making students the protagonists of the learning process, taking into account their interests and attitudes; for this purpose, it focuses on the concept of personalised learning. It is important to notice that the learning ecosystem has been designed to encourage collaborative group work in order to foster the multidisciplinary development of tasks as well as 21 st century skills such as collaboration, problemsolving and creativity. In order to actively observe their own learning progression and, at the same time, to receive a sort of immediate recognition and gratification of their efforts, the learning ecosystem offers integrated evaluation and badge system based on students' self-assessment. The design of the badge system followed the same principle of personalisation aiming at being intuitive to use. In our work, we use the selfassessment definition as "... a wide variety of mechanisms and techniques through which the students describe (i.e., assess) and possibly assign merit or worth to (i.e., evaluate) the qualities of their own learning processes and products" (Panadero et al., 2016). Self-assessment is concerned with the ability to correctly evaluate one's own work, acknowledging both strengths and weaknesses (an important aspect of personalised learning); as such, it is a key ability that students should acquire. In fact, the numerous studies on the impact of the self-assessment on learning show that the practice has positive effects both on results and on self-esteem (Panadero et al., 2013;Andrade et al., 2009;McMillan et al., 2008); moreover, they show a positive correlation with the ability of self-regulated learning (Andrade, 2010). However, Panadero et al. (2017) underline that the effectiveness of the self-assessment practice depends on how it is taught; it is not sufficient to ask the students to give a "grade" to their work, but it is fundamental to make them develop an awareness about the critical issues of subjects and about the chance for further improvement.

Self-assessment
The badge system contains a self-assessment mechanism through which the students can evaluate their progress. Taking into consideration the suggestions of key teacher informants, we identified six categories for self-assessment, corresponding to the various aspects of a digital fabrication project development: building electronic circuits, 3D design, programming, presentation of the project outcomes, quality of the teamwork and originality. Parameters provided for each of these categories guide the students to achieve a reasoned and objective evaluation of their work (Figure 2).
The system provides default parameters to use for self-evaluation; however, the teachers are able to provide their own parameters as well. Ideally, students and teachers should decide and set together the parameters, thus sharing the responsibility to represent the most relevant and meaningful aspects of the learning process. When asking the key teacher informants their opinion regarding the default parameters that define each category, we understood that flexibility was needed in terms of allowing the teachers to modify the categories and their parameters in order to tune the evaluation to the aspects that they considered relevant to be self-reflected and self-evaluated by the students.
The self-assessment system requires that the students first think about which parameters have been achieved; then, they are asked to assign to each of the six categories a score between 1 and 5 alongside a brief justification for their score. In addition, after each activity the students are asked what they learnt new, and what they found too difficult or they didn't completely understand. For this reason, even if reference parameters are available, the score that the students assign to their work is not necessarily tied to their actual achievements. That is, the artefact created by the students in their project might not work perfectly, but the student could anyway self-assess a good mark, because they have learned other important skills; what is important is that students provide justification to their considerations. For this purpose, the interface has text boxes where students should summarise the rationale behind the numerical scores that they assign to their work.
The information obtained from these rubrics is precious to the teacher: on the one hand it delivers data on the efficiency of the self-assessment process, and on the other hand it allows the teacher to judge if the learning activity was properly calibrated, offering a comprehensive vision of the strong and weak points of their students. The teacher can confirm the student produced score or assign a different one in the self-evaluation interface, provided that the teacher justifies the change. The teacher modifications and justifications are visible to the students. This transparent evaluation from the teacher and this cyclic process of "self-evaluation ⟷ evaluation ⟷ assessment ⟷ assessment Figure 2. The self-assessment mechanism showing the parameters for the Programming category evaluation of the self-evaluation" between the teacher and the students are key factors to develop the critical thinking of the students about their learning and the effectiveness of their learning strategies. Nevertheless, this cyclic process, the cognition and metacognition involved in it could result in overburdening the learning process. Therefore, in order to support the learners and keep them engaged in the learning process, we rely on gamification exploiting the quest for more badges and for "more stars" in each badge as a motivating and compelling feature.

Improving motivation through gamification
The use of game design elements in a non-game context is called gamification (Deterding et al., 2011). Since 2013, there has been a wide consensus on the gamification practice in the educational environment, especially at the higher degrees of education: badges are an example of game mechanics, along with points, levels, progress bars, leaderboards, virtual currency, and avatars. As several studies report, the use of these mechanics have a great potential in increasing students' engagement and in motivating them to learn and train new skills (Dicheva et al., 2015, Barata et al., 2013. In particular badges are proved to be effective in triggering competitive motivation (Pirker et al., 2014), in improving learners' participation (Dominguez et al., 2013) and in enhancing learning, time management and carefulness (Hakulinen et al., 2014).
In the eCraft2Learn digital platform of the learning ecosystem, the self-evaluation interface that we are presenting in this paper as a gamification process has been embodied into a set of badges. Indeed, the scores assigned by the teacher are elaborated by the eCraft2Learn system and converted into badges; they can also refer to the learning analytics system (Toivonen et al., 2018) in order to provide a more objective evaluation. Badges give students an immediate and easy to understand feedback about their progress in each category. The basic idea is outlined in Figure 3. This graphical interface at the moment serves only as a working prototype. The badge interface will be (re-)designed by professional graphical experts in order to be in line with the rest of the graphical appearance of the eCraft2Learn digital platform and to be highly appealing and attractive for the students.
The badge system we designed have a twofold goal. On the one hand, it shifts the attention of the students from the practical activities they have to carry on to complete the project assigned to them, to the skills they are supposed to learn during this activity. This is achieved by the careful choice of the six categories for self-assessment which were defined with the help of key teacher informants involved in the eCraft2Learn project. Each of these six categories visually prompt the students to actively reflect on the fundamental skill they are supposed to assimilate abstracting their thinking from the technical/practical problems at hand. As one can notice, the six categories are divided into technical skills specific to digital fabrication and making activities (such as circuit building or 3D design) and the cross-skills to be acquired (such as presentation capabilities and quality of the teamwork).

Figure 3. The scores confirmed by the teacher are converted into graphical badges
On the other hand, the badge system motivates the students to complete their digital fabrication projects and the practical activities associated with the projects 1 . The more activities the students complete, and more carefully they implement them, the more badges they will earn. Moreover, in order to earn more badges, they have to finish up their work and critically revise and evaluate it, speculating on the achievements of their own learning. This process aims at fostering a reasoning on the outcomes of their learning. Additionally, the students are encouraged to develop (and express) a fair selfevaluation, because the teacher is evaluating both the technical achievements and the self-evaluation capabilities of the students.

Conclusion
In this paper we presented a badge system with an underline gamification aspect. The badge system is based on a student's self-evaluation mechanism in order to create a cyclic evaluation process between students and teachers. The proposed badge system with its corresponding self-assessment mechanism focuses on supporting students' selfevaluation process (i.e. to award a grade to one's own work), as well as fostering personalised learning through self-reflection, supporting also self-regulation. The system is expected to foster personalisation of the learning path while assisting students to develop motivation and engagement on their digital fabrication activities.