OpenVM MOOCs: a design for self-regulated learning

This paper presents the design of two miniMOOCs (Massive Open Online Courses) Autonomy-driven learning and Self-Regulated Learning (SRL), developed in the Open Virtual Mobility MOOC. The design is based on two concepts: microlearning and SRL. All the MOOC elements were designed to support SRL. From a descriptive and quantitative approach, the research aims to explore participants' satisfaction and their perception on the supporting roles of the miniMOOC for SRL. Data was collected through an online survey through which 375 answers were collected. Results show the general satisfaction of learners with the MOOC design. As for the SRL design gamification and visualization of pathways were more relevant for certain SRL phases such as monitoring learning. OER and related quizzes were useful for all SRL phases. Quizzes especially supported self-reflection. Open Badges supported the forethought phase of SRL. Social elements such as forums and eportfolios obtained the lowest rates in relation to support of SRL. The paper concludes with recommendations for educational research and MOOCs design.


INTRODUCTION: HOW TO SUPPORT VIRTUAL MOBILITY SKILLS IN MOOCs
In the last years, the concept of «Virtual Mobility» has receiving a growing attention from educational policy makers and institutions, because it has the potential to make more accessible and effective students and teachers' mobility in Higher Education (Poce, 2020) Teresevičienė and colleagues (2011) define virtual mobility as an activity or a form of learning, research and communication and collaboration, based on the cooperation between higher education institutions from different backgrounds and cultures working and studying together through an ICT supported learning environment. The main purposes of virtual mobility are knowledge exchange and the improvement of intercultural competences. Despite the potential advantages, Virtual Mobility programs require participants to have good levels of transversal and digital skills to successfully be completed (Poulová et al, 2009). Rajagopal and Firssova (2018) recently identified 8 transversal knowledge and skills necessary to be involved in a Virtual Mobility experience, by applying a group concept mapping methodology and involving 49 experts in the domains of virtual mobility and/or open education with experience in higher education as university professors or education management and support: (1) Intercultural Skills; (2) Collaborative learning; (3) Autonomy-driven learning; (4) Networked Learning; (5) Media and Digital Literacy; (6) Active Self-Regulated Learning; (7) Open-mindedness; (8) Knowledge of Virtual Mobility and Open Education. Based on this classification a Massive Open Online Course (MOOC) named OpenVM MOOC was designed to support participants in a virtual mobility experience to develop pivotal skills to achieve their learning goals. The OpenVM MOOC was developed in the Erasmus+ strategic partnership Open Virtual Mobility 1 (2017-2020). It is an online learning environment for achievement, assessment, and recognition of virtual mobility skills. MOOCs are now being considered and applied by many institutions around the world as a valid internationalization instrument (Knight, 2014). Having said that, engagement and completion rates belong to major challenges in the design of MOOCs. Literature has identified different factors related to MOOCs dropout which can be classified into intertangled categories: 1) Course design (Colman, 2013) which includes little interaction with other learners and instructors, too little personalisation of instruction (Gütl et al., 2014) and course length (Jordan, 2015); 2) Learners personal features which includes motivational aspects, accomplishment perceptions (Khalil & Ebner, 2014), engagement (Kizilcec et al., 2013) and students' lack of self-regulated learning (SRL) skills (Pérez-Alvarez et al., 2016).
MOOCs designers should consider together course design and students' personal disposition toward learning.
In the project OpenVirtual Mobility micro-learning was adopted as an approach for MOOC design which can support students' SRL skills, as personal disposition toward learning. The microlearning design was introduced in previous work by  and in this current article we are presenting its design and assessment in terms of SRL.
The article describes the case study of two miniMOOCs (Massive Open Online Courses) Autonomy-driven learning and SRL, which are two out of eight MOOCs designed and implemented in the Open Virtual Mobility (OpenVM) MOOC, specifically aimed at supporting SRL related skills.
In the introductory part of this article, the concepts of SRL will be discussed. Then, the approach applied to the design of the MOOCs will be described. Results from the pilot evaluation conducted with students from diverse universities in Europe will be presented to provide recommendations for future research with focus on SRL as a design approach in MOOCs.

SELF-REGULATED LEARNING IN MOOCs
SRL was described by Zimmerman (2002) as a cyclical process composed of three phases and six sub phases (see Table 1).
1) "Forethought phase", in which metacognitive tasks are performed before learning. It is described with two sets of processes: a) task analysis and b) the self-motivation beliefs (Littlejohn et al, 2016). Although motivation tends to decrease during the MOOC duration, research has observed that students who were intrinsically motivated for learning and were aimed at achieving accreditation badges experienced little motivation decrease (Haug et al., 2014) 2) "Performance phase", which includes c) metacognitive tasks and d) self-observation. Littlejohn et al. (2016) observed that highly self-regulated learners deployed more flexible and non-linear strategies in MOOCs than those with lower SRL skills. Another recent research has observed that students with higher level of SRL skills self-control their learning by monitoring performance in MOOC assignments (Alonso-Mencía et al., 2020).
3) "Self-reflection phase", which takes place at the end of learning and is described as a process of e) self-judgment and f) self-reaction. Littlejohn et al. (2016) observed that highly SRL participants used tasks as formative assessment while others used to only refer to summative assessment tasks. The literature review by Pérez-Alvarez et al, (2016) showed that several MOOCs does not provide concrete information on how their design support SRL skills. They observed that goalsetting, strategic planning, self-motivation, self-monitoring and self-evaluation are the metacognitive processes mostly enhanced in MOOCs. These processes are usually supported through visualization of learning objects, personalized learning paths, and social contexts for collaborative learning. Lee et al., 2019 provide observations about how SRL metacognitive skills have been enhanced in MOOCs (Table 1).
In the OpenVM miniMOOCs some of these core elements -such as hints, pathways, quizzes, forums, ePortfolios and badges-were introduced as affordances for SRL within a micro-learning design. The present research will investigate learners' perception of these elements and their impact on students SRL strategies.

DESIGNING SELF-REGULATED LEARNING IN OPENVM MOOCS
The Open Virtual Mobility (VM) MOOC is aimed at developing Virtual Mobility Skills in higher education students. The OpenVM MOOC hosts a set of eight miniMOOCs, including SRL and Autonomy-driven Learning miniMOOCs, which have been piloted to investigate how their learning designs can enhance SRL in MOOCs. The concept of miniMOOCs is based on the taxonomy of MOOCs by Pilli & Admiraal (2016) who distinguished eight types of MOOCs, including miniMOOCSs, which are smaller, shorter, and cover less content and fewer skills than traditional MOOCs. As opposed to traditional MOOCs which tend to last four to twelve weeks, miniMOOCs are much shorter and may be split into shorter miniMOOCs to enhance the microlearning process (Lackner et al., 2015). Each OpenVM miniMOOC is divided into three shorter parts, called subMOOCs, each subMOOC related to one of the three levels, i. e. foundations, intermediate and advances (see Fig. 1). With each miniMOOC broken down into three smaller subMOOCs for one of the three levels, learning pathways can be designed by each learner in an individual way, depending on initial and intended skill level. To assess the initial level of a given skill, each miniMOOC starts with a pre-assessment. Based on the results of the preassessment, one of the three levels is proposed to the learner, who can decide to follow the recommended pathway or take an own decision. A granular system of micro-credentials based on Open Badges makes the learning outcomes, i. e. virtual mobility skills, visible and actionable to learners. The visualisation of skills which can be developed and micro-credentials which can be earned aims at supporting learners in selecting miniMOOCs and subMOOCs which best match their individual needs and support learning in a personally meaningful way. Furthermore, micro-learning activities in each subMOOC are aligned with structured microcontent in the form of micro-OER, which are OERs suitable to support micro-learning (Sun et al., 2015). The content in the two miniMOOCs is divided into diverse microtasks with OER-based contents in diverse formats, mainly short pieces of text, short (micro) videos and forums for collaborative tasks . The micro-learning design of the OpenVM miniMOOCs arises another key challenge for the design and successful learner experience. Since miniMOOC participants need to have a clear map of the possible learning pathways and their convenience for their own learning needs, the quality assurance framework for the OpenVM MOOC has highlighted the need to focus on the pedagogical design of the set of miniMOOCs both for facilitating students personalisation through the diverse learning pathways and for promoting students' SRL skills Tur et al., 2018;Tur et al., 2019). The OpenVM miniMOOCs include several elements aimed at supporting each of the three SRL phases (see Table 2). The design in terms of SRL learning received internal assessment by experts twice, as a theoretical design and for the pre-pilot stage, as documented in previous work Tur et al. 2018;Tur et al., 2019). To support students' metacognitive tasks in the forethought phase, each miniMOOC includes a welcoming message with the presentation of the miniMOOC, initial self-assessment (preassessment), description of the aims and learning outcomes and available Open Badges. In order to facilitate learners strategies during the performance phase, each miniMOOC includes an overview of learning contents including OERs, and other guides such as information about the learning path and possible further steps in the miniMOOC pathway, the design of the learning activities, tools for group formation and social learning options. Finally, to support the self-assessment phase, each miniMOOC specifies the assessment activities such as test-based e-assessment, e-portfolio, peer-assessment tasks, and discussion forums. Also, all quizzes offer quick feedback and results visualization to facilitate students' self-control, monitoring and reflecting processes. Furthermore, the OpenVM miniMOOCs design is embedded in the meaningful gamification approach which aims to enhance learner engagement in learning activities through less emphasis on external rewards and more emphasis on learner control and ownership (Buchem & Carlino, 2019). The concept of meaningful gamification builds on research which shows that external rewards, such as scoring-based gamification, tend to enhance short-term motivation and may even have negative effects on self-regulation (Nicholson, 2012). Some of the key elements of meaningful gamification include the user profile with attributes relevant for learning pathways, group formation and visualisation of skills and contributions with digital credentials; possibilities to choose learning pathways such as different miniMOOCs and subMOOCs including the recommendations based on preassessment of skills; the visualisation of progress in skill development such as number of subMOOCs taken, e-assessments completed and digital credentials earned; and the use of plugins to support meaningful gamification of the learning design (Buchem & Carlino, 2019). In this way, the learning design approach to OpenVM miniMOOCs utilises a range of pedagogical approaches including SRL and meaningful gamification and focuses not on the hierarchical sequence of content presentation but encourages learners to become active co-designers of own learning pathways.

Research questions
The present research was aimed at understanding how the OpenVM MOOCs learners assessed the quality of the MOOC main components and to which extent the MOOC supported participants SRL skills. The work is aimed at answering the following research questions: 1. How students generally evaluated the two MOOCs Autonomy Driven Learning and SRL? 2. How students evaluated the learning design of the two MOOCs in terms of SRL?

Participants
A total of 857 students participated in the two OpenVM miniMOOCs analyzed in this research.
We received a total of 375 answers to the questionnaire from participants.

Data collection and data analysis
Data were collected through a questionnaire submitted after a subMOOC completion. Since this research is aimed at investigating the impact of two specific MOOCs (Autonomy Driven Learning and SRL) on students SRL skills, only students' assessment for these two MOOCs are considered. A total of 375 answers were analyzed out of 1393 total answers received at the MOOC assessment questionnaire. The OpenVM Evaluation Questionnaire is organised in eight sections. In all the sections participants were required to express their level of agreement with a set of statements related to specific MOOC design elements on a Likert scale from 1 (strongly disagree) to 5 (totally agree). OpenVM Evaluation Questionnaire was structured as follow: 1) Personal details: age, gender, affiliation, and role. In this section participants are required to say which of the eight mini-MOOCs they are assessing; 2) Questions regarding the overall MOOC design: learning experience, quality of content instruction and support for learning; 3) Questions regarding digital credentials and meaningful gamification: quality of design, motivation, engagement and possibilities of choice; 4) Questions regarding technical aspects: use and usability; 5) Questions regarding the foundation level of a mini-MOOC: duration, language, content, use of multimedia; 6) Questions regarding the intermediate level of a mini-MOOC: extending questions from the foundation level by questions related to the matching tool and group formation activity, which are specific design elements used at this level; 7) Questions regarding the advanced level of a mini-MOOC: extending questions from the foundation and intermediate levels by questions related to e-portfolio and peerassessment activities, which are specific design elements at this level; 8) Questions related to the investigation about the extent to which MOOCs supported SRL.
In this section participants are required to answer also to open-ended questions. Within the first 7 sections, transverse questions regarding the MOOCs learning design (e.g. "the length of videos was good for me") and the MOOC support for SRL (e.g. "I learn what I expected from the MOOC") were inserted. Descriptive statistics (average, standard deviation, frequencies) were calculated in order to answer to the above-mentioned research questions.

Learners' general evaluation of the two MOOCs Autonomy Driven Learning and SRL
Comparing the average scores of the two MOOCs, "autonomy-driven learning" received higher evaluation compared to the "SRL" MOOC ( Figure 2). However, the general results can be considered satisfactory because average scores are tendentially higher than 3.5 (we used a Likert Scale from 1 to 5, where the median is 3). The Gamification average score in the MOOC SRL is an exception since it received an average score of 3,3. In both the MOOCs, the most appreciated features are 1. Technical features; 2. General evaluation; and the 3. OpenVM bagdes. Figure 2 A comparison of the average scores provided by participants to the six dimensions investigated within "autonomy-driven learning" and "SRL" MOOCs.
Comparing the average scores of the general assessment for the three levels within each MOOC (Figure 3)

Learners perception of MOOCs affordances for SRL skills
The average scores obtained for the 8 general statements regarding the MOOCs support for SRL are presented in Figure 7. The average scores are always higher than 3,6 to the 7 positive statements. The highest scores were obtained for the statements "I think this way of working is helpful for learning" (3,97), "The MOOC provides support for self-regulating the learning path" (3,91), and "The technological environment is friendly and usable" (3,90). On the other hand, the statement "I could choose my own learning pathway" obtained the lower average score (3,62). The only negative statement "In some places would have liked more information" obtained an average score of 3,2 which suggests a general nor agreement or disagreement with that statement. The last section of the questionnaire asked participants to which extent the main MOOC features supported their SRL skills. The higher average scores were obtained for MOOC Path, MOOC hints, OERs, and quizzes with average scores higher than 3.76. Forum and gamification received the lower scores, with average scores respectively of 3,51 and 3,485 (see Figure 8).    Figure 9 shows in a more detailed way how the 9 MOOC features support different SRL skills, according to the two MOOCs participants.
1) OERs supported more Task analysis, Self-motivation, Self-observation and Self reaction; 2) Forum supported more Self-observation and Self-reaction; 3) Quizzes supported more Self-judgement; 4) E-portfolio supported more Self-control; 5) Peer-assessment supported more Self-motivation and Self-judgement; 6) Open-badges supported more Self-motivation; 7) Gamification supported more Self-observation, Self-judgement and Self-reaction; 8) MOOC path and hints supported more Self-observation skills; Figure 9 A comparison of the different MOOC elements impact on 6 different SRL skills according to the participants

DISCUSSION AND CONCLUSION
The issue of engagement and completion in MOOCs has been widely discussed among educators and researchers. This article builds on research related to learning design of MOOCs and its impact on the learning experience. We described a SRL approach to designing engaging -learning experiences in miniMOOCs. Our case study from the OpenVM project investigated the experience of a sample of students who participated in the pilot assessment of the Autonomy-driven learning and SRL miniMOOCs in the OpenVM Mooc. The use of different OER formats and the productive tasks were foreseen with the aim of enriching the former design of MOOCs, that was initially criticized for being a simple list of videos with forum activities (Vázquez-Cano et al., 2018). The evaluation results indicate that the miniMOOC designed allowed learners to complete the course in a suitable amount of time and that the learning experience was enjoyable mostly due to video-based micro-content and the short learning format, as seen in previous stages of research . This current work also adds the research related to other design elements which support SRL, and our data allows us to infer some new recommendations.
Following recommendations for learning design in MOOCs based on previous research (Lee et al., 2019;Pérez-Alvarez et al., 2016), our miniMOOCs include the following MOOC hints to facilitate participants SRL processes: diagrams showing learning pathways in miniMOOCs and highlighting where the learner is located at any moment, visual information about the number of OERs in each subMOOC and if they are video or text-based to help students manage their time and plan their learning strategies, and tutorials related to the e-portfolio, peer-assessment activities, earning Open Badges and a general description of the MOOC structure.
It is very interesting to explore with some more details the results achieved of the miniMOOCs elements in terms of SRL and consider them as recommendations for further educational research and implementation. Firstly, it can be seen that the information given and the visualization of the pathways have supported students SRL, as suggested by Pérez-Alvarez et al. (2016), and not only in the planning phases for which they were mainly thought (see Table 3), but also and in particular, for monitoring learning. This means that students appreciate constant information of their allocation (Lee et al., 2019) and that it is needed when performing learning. Secondly, OERs and quizzes obtain similar results in all phases, which could involve that the objects and their assessment have been designed in alignment and that have been useful in different phases of the SRL. It could be interesting to explore if quizzes highly valued for self-reflection were also used as strategies for monitoring learning as a kind of formative assessment (Alonso-Mencía et al., 2020;Littlejohn et al., 2016). Furthermore, as for the variety of formats, it is remarkable that they were more appreciated as diverse enough in the intermediate and advanced levels than in the foundational courses. This is extremely interesting because based on this, it could be suggested that the lower the levels, the more need for diversity of objects. Thirdly, Open Badges are relevant for all SRL phases but it is worth highlighting that they are more useful for preparing learning, which is the phase for which they were initially designed for in the OpenVM MOOC (see Table 3) which is the metacognitive task that is parallel to motivational belief for which badges have been related in research (Haug et al., 2014). The lowest levels in terms of SRL are those corresponding to elements which are related to social learning, like forums, eportfolios and the tool for group formation. In this case, further work should improve students' support in the context of social learning (Lee et al., 2019), which is particularly relevant when addressing a MOOC for preparing virtual mobility in international context.
The general satisfaction results show that students ranked higher in the foundational courses than the two others, which allows generating new hypotheses for further research. On the one hand, it allows us to think that automated subMOOCs support students' autonomy. On the other hand, this is consistent with the lower rates reached by social elements in intermediate and advanced courses, like forums and eportfolios for peer-assessment and the matching tool used for group formation. It would be very interesting to explore with further detail if students who attended successfully the foundational level showed higher levels of SRL skills. Another recommendation based on these results, is that the social approach to the OpenVM MOOCs should be scaffolded and promoted not only by the elements of the design but also with the support of teachers' role.
The results presented in this article need to be integrated with the perspective of other relevant stakeholders, such as e-learning experts inside and outside of the project. However, the learners' experience is at the core of any quality assurance process and future research should try to relate the development of skills and successful rates in the context of VM training. 7. Acknowledgements: