Designing IoT applications in lower secondary schools

The paper reports on a case study where four groups of lower secondary school students participated in a workshop and undertook the demanding role of designers of Internet of Things applications. In doing that, they made use of a dedicated inventor toolkit, which facilitated students' creative solutions to problems that can appear in the context of a smart city. From a pedagogical point of view, the workshop format is in line with the experiential learning approach. The paper presents a holistic student assessment methodology for this nice domain. In particular, to analyse the impact of the workshop for the students we used four different approaches: artefacts analysis of students' design solutions, classroom observations, a post-test and a survey. The results indicate that the intervention has promoted an effective teaching methodology for the basic conceptual and design aspects of the IoT for these lower secondary school students, but it hasn't addressed equally effectively the attitude-related aspects. Nonetheless, the participant students perceived the intervention as very satisfactory in terms of the IoT concept knowledge, smart cities learning, and problem-solving skills acquired, as well as in terms of enjoyment. The paper concludes on the learning gains of the intervention and discusses the motivation aspect for the teacher as well as for the students in this highly innovative learning experience.


INTRODUCTION
The Internet of Things (IoT) is one of the most important topics in Information and Communication Technology today and IoT applications span a wide range of domains, including home, cities, and the environment [1]. Accordingly, it is associated with many viewpoints [2]. This paper takes on the viewpoint of methodologies for teaching about designing IoT applications in the contexts of education and smart cities.
The research question examined herein is: how we can use such a design methodology as a means to educate young students who are not familiarised with IoT? Designing IoT artifacts can cultivate 21st-century skills and design literacy skills [3] and, coupled with proper methodologies, can help towards making the symbolic and abstract manipulations involved in IoT design processes more concrete and manageable for young students (ibid). Typically, the design methods found in the IoT literature share the following characteristics: (a) they exploit the intersection of design thinking and user experience design, (b) they target the end users of the IoT applications (and not students) and (c) they are not used as a means to achieve some educational goals (especially for lower secondary school students). On the other hand, it seems that there is a growing need for education, which can empower digital citizens acting as designers of IoT applications [1]. In that sense, the work described herein is unique and addresses a gab in the literature of this niche domain by encouraging lower secondary school students to actually design IoT applications rather than merely learn about it.
The remainder of the paper is structured as follows: the next section provides some insights on IoT in conjunction with design methodologies, smart cities and the Tiles inventor toolkit. Following, we present the context and the procedures of the learning intervention that incorporated the IoT inventor toolkit. Next, we discuss the results with respect to the impact of the intervention for students' learning on IoT along with an interpretation of the emerging roles of the participants. Finally, we conclude on: the contribution of this case study, limitations, and future work.

A. IoT design and teaching methodologies
In the recent literature, there is a handful of methodologies and guidelines for the design of IoT applications. Examples include: a 7-step methodology (discovery, capturing, research, de-sign, prototype, evaluate and refine) for creating 'people-aware' IoT applications [4] a generic methodology for discovering and eliciting details of IoT use cases (understand, observe, synthesize, ideate, prototype, test and refine) enhanced with some special elements, like augmented representations of IoT elements, points of interest corresponding to places where information is generated, routed, read, etc [5] Typically, these methodologies are not (meant to be) used and tested as teaching/learning tools for IoT. Page 1120 they are not associated with particular educational goals and, in turn, they are not coupled with an educational assessment strategy. Even when this is the case, the papers found in the literature often involve design for IoT focusing more on the programming or technical aspects and less in the design aspects, where design is defined as the phase that follows the ideation phase; and usually this happens in the context of tertiary education. Examples include: Kortuem et al. [6] discuss an instructional intervention on IoT that took place at the Open University in the UK in 2011, when a new online course called 'My Digital life' aiming to place the IoT at the core of the first-year computing curriculum and to help students grasp the coming changes in society and technology. The course embraced experiential learning and collaborative learning theories to help students recognize their role in realizing the potential of IoT technologies. The authors concluded that there are many challenges in teaching IoT concepts to freshmen, including assessing students and keeping them engaged.
Chin and Callaghan [7] attempt to show in their paper how novice programmers can use the suggested approach, called 'Pervasive-Interactive-Programming' (PiP), to program the IoT by progressing from simple to advanced programming concepts needed to tackle university level curriculums. They evaluated PiP across six dimensions: conceptual, user control, cognitive load, information retrieval, affective experience, and future thoughts.
A notable exception involves the work of Charlton & Avramides [8] which conducted a pilot study with a number of 15-year-old students. The approach promoted knowledge construction in Computer Science and Engineering topics. It was focusing on the use of tangible toolkits for physical computing and the IoT coupled with a learning through making approach. It consisted of a mini-workshop, a brainstorming activity, a 2-day educational hackevent and a follow-up presentation. During the mini-workshop, students worked in pairs to think about smart city problems, experimented with programming and electronics engineering and presented their ideas and experiences to the whole class. A few weeks after the mini-workshop the researchers visited the school to conduct the brainstorming activity on ideas that the students would like to design and build in the context of a smart city. Following, students participated in the hackevent which had the following structure: (1) explore and design a solution, (2) present back to the whole group, (3) experiment and develop solutions, (4) demo solutions, and (5) preparation for the final presentation in a live audience.

B. Design for IoT and smart cities
The ambit of smart sustainable cities coupled with ICT is increasingly gaining the attention of diverse stakeholders due to its potential to transform the city pointing in various directions: environmental, physical, social, and economic [9]. In practice, we're talking about all sorts of worthwhile goals and potential public benefits, including more efficient traffic flow, better management of public buildings and areas, reducing the (huge) costs of public lighting, better managing waste removal and other public utilities, and more effective policing and emergency services [11]. IoT is interweaved with the smart city, powered by data collected from smart devices which can be used for the public good; for example, to increase the transparency of the local government, to enhance the public awareness of on the status of the smart city, and stimulate digital citizenship and participation [11].
Yet, there are associated technical and financial challenges, as well as political issues to overcome before the societies can safely enjoy this smart city vision. For example, governments have to deal with the privacy and security issues inherent to the massive data collection which is enabled by the sensors mounted in the smart devices (ibid). In 2013, the smart recycling bins that were located in the Cheapside area of central London logged the media access control (MAC) address of the passers-by smartphones [10]. This arose a privacy concern since in the European Union websites are legally required to inform users if a tracking cookie is placed on their computer. Tracking smartphones and other Wi-Fi devices wasn't nearly as regulated in part because the technology is new [13].

III. THE INTERNET OF THINGS INVENTOR TOOLKIT
The Tiles IoT inventor toolkit has two main components described in brief in this section: the Tiles design workshop and the Tiles inventor materials (board, playbook, cards).
In brief, the design workshop consists of two main phases, the first one being an introduction both to IoT and to the design challenge (i.e. the workshop structure). During the design challenge, students work collaboratively in small groups to create an IoT-infused design solution. At the end of this phase, they pitch their idea while presenting their design solution to their peers and the facilitator(s)/tutor(s). During the design workshop, students have the opportunity to creatively design IoT applications while being exposed to diverse issues around them, like IoT-related concepts, and design for IoT. The design workshop is supported by a set of ideation cards [14]  Page 1121 such as idea storyboarding and idea pitching. This phase generates a number of product ideas of IoT ecologies for a specific domain. The domain and the context of the problem is given to the participant students as a set of different predefined options about scenarios and personas. The cards ( Figure 1) display elements from both the physical and the digital domains, including everyday things, user interface metaphors, and online services. All the materials of Tiles inventor toolkit are freely available online at http://tilestoolkit.io/ under the Creative Commons license. Thus they can be used for educational usage without any permissions from the creators.

A. Participants profile
Seventeen lower secondary school students participated in the classroom intervention, aged 14-15 years old. Two of them were girls and the rest were boys. All of them are Norwegian and they were not familiarised with IoT before the intervention. The participant teacher has high prior knowledge in computing and a number of years of prior teaching experience. The teacher was teaching the participant students an elective course in programming that has recently been introduced in Norwegian lower secondary schools. The teacher was familiar with the Tiles toolkit having previously participated in a design workshop during a teacher training event and took the initiative to run the intervention with his class.

B. Procedures and instruments
The purpose of the intervention was to familiarise students with the basic IoT concepts and help them ideate and design IoT applications, with special focus on smart cities. Though the researchers ran the workshop, there was minimal interference. The intervention took place in the school classroom as part of the informatics course. It comprised three main phases involving: (1) an introductory presentation, (2) the Tiles inventor toolkit, and (3) a student assessment (see Figure 2). In terms of pedagogical approaches followed: an introductory presentation familiarises students with the basic IoT concepts while showing some everyday life applications, and briefly addressing the smart city aspect. The second part involved using Tiles toolkit as a card-based brainstorming tool during a 3-hour workshop in which the participant students generated ideas for IoT products. Throughout this phase, the students were working in groups of four or five persons. Their ideas were expressed both in a written form using a dedicated space in the toolkit board, but also students had the opportunity to pitch their ideas at the end of the workshop.
During the closing session students answered the quiz individually using an online freeware classroom system, which embedded some game-based elements and provided instant corrective feedback after receiving responses in a specific question (namely, kahoot! 1 ). The quiz comprises ten multiplechoice questions on IoT with four alternatives offered in each question. The students had in their disposal only twenty seconds per question in order to avoid misuse of the classroom 1 https://kahoot.com/ response system. Two researchers active in the design of IoT ecologies validated the questions included in the quiz. In particular, the questions touch upon domain-specific learning objectives like, whether students can define the concept of IoT, and recognise IoT applications and basic elements associated with the IoT concept or with the IoT equipment. These goals are suggested for the school curriculum design of the UMI (Ubiquitous computing, Mobile computing, and the Internet of Things) domain by [15]. The knowledge questionnaire is available online 2 .
To analyse the impact of the intervention for the students we used four different approaches: artefacts analysis of students' design solutions in the main phase of the workshop, classroom observations, student survey and a performance analysis of the Kahoot! quiz. To collect the data for the artefact analysis we used photographical observations (i.e. photos of the students' final solutions). They were analysed against a dedicated assessment rubric that had been prior validated by two researchers. More specifically, the predefined criteria used were the following: 1) Data exchange /online services: does the idea make use of them?
2) Use of scenario: does the idea cover all the aspects and address all the challenges proposed in the scenario?
3) Persona use: is the idea actually targeting the chosen persona? Is it too generic or too restrictive in terms of target group?
4) Implementation with [toolkit name missing for review] design elements: how is the idea using the required functionalities in terms of services, sensors, human actions, feedback etc? 5) Participation: is the idea promoting participation and collaboration between the actors of the scenario?
The assessment rubric assigns a score from zero to five in each these criteria, following the generic rubric template originally proposed by Mertler [12]. This template provides a specific description for the performance indicator at stake for each of these six scores. For example, a score of zero corresponds to the description "No response/task not attempted", a score of one corresponds to the description "demonstrates no understanding of the problem", a score of two corresponds to the description "demonstrates little understanding of the problem, many requirements of task are missing" and so on.
A researcher active in cooperation technologies conducted the observations, which were focusing on the emerging cooperation schemata and roles during the classroom intervention. Finally, the students completed the survey in a printed format individually at the end of the intervention. The survey instrument was previously used in previous research conducted by the authors. It briefly asks students (using Likert scales) about: prior programming experience, prior and current IoT knowledge, smart city learning obtained, whether the workshop was fun, whether the design task help them towards problem solving and ideation, and their willingness to proceed 2 https://play.kahoot.it/#/k/36df1589-fcff-4c5f-a03c-8d8190bfeecc Page 1122 with prototyping their ideas. The students had twenty seconds in their disposal to answer each question.
It should be noted that consent was obtained to collect data relevant to the purpose of the research, provided that the data will be anonymised and it will not be identifiable in the resulting publication. The teacher was present throughout the classroom intervention. Figure 2 shows the main phases of the intervention, indicating for each phase: (a) the participating roles, (b) whether students were working in groups or individually, and (c) the research methods used to assess students' performance. Regarding the latter, the authors during the design phase of the workshop took observations and photos of students' groupwork. The photos were used as primary data for the artefact analysis. At the end of the design workshop, students answered individually the questions included the classroom response system (i.e kahoot!) using their mobile phones, which were connected to the internet. Finally, the answered individually a workshop evaluation survey which was disseminated to them in a paper format.

C. Experiential learning for designing IoT applications
In terms of pedagogy, in this intervention we followed the collaborative learning approach in tandem with experiential learning. In particular the intervention was in line with the generic experiential learning model originally proposed by Kolb [16] which consists of four modes that effective learners need to experience: Concrete Experience (CE) abilities, that is, to be able to "involve themselves fully, openly and without bias in new experiences". In this phase, students involve themselves fully, openly and without bias in new experiences. The word "concrete" is used to denote an experience that happens in the real, physical world. In this case, CE were created for the students as they collaborated towards reaching a solution for the openended problem of designing an IoT application for a given context and persona.
Reflective Observation (RO) abilities, that is, to be able to "reflect on and observe their experiences from many perspectives". In this phase, students reflected on and observed their experiences from many perspectives. Their thoughts were triggered by the evaluation criteria (i.e. specific game cards) and questions like "what did we do well?", "what could have been better?" etc. Thus, essentially the students' thoughts were triggered by self-reflection, as well as, by feedback derived from their classmates. The latter was also facilitated via a peer-assessment process among students that took place near the end of the design workshop.
Abstract Conceptualization (AC) abilities, that is, to be able to "create concepts that integrate their observations into logically sound theories". The students thought about ideas, techniques or approaches that could helped them to improve their performance when (and if) they will engage again in the CE. For example, the students thought about the ways they could do things differently in the future to accomplish a better design solution.
Active Experimentation (AE) abilities, that is, to be able to "use these theories to make decisions and solve problems". The knowledge quiz provided to the learners the opportunity to test their ideas and knowledge by answering to questions on IoT applications in everyday life problems. Table 1 presents the main characteristics of the IoT design solutions of the students. The first idea involves a smart public bin which implements a time-saving technique towards waste reduction triggered by human actions (e.g. 'step on') combined with sensors (e.g. weight). The envisioned smart bin is meant to be used individually by the citizens of the smart city. The application is targeting towards sustainability in the sense that it is an environmentally friendly solution. The second idea involves a smart transportation sub-system which aims to enhance the driving experience for the bus driver by providing a range of services: traffic monitor, maps, public webcamera, weather forecast and so on. The idea is targeting the bus drivers and it is meant to be used individually, but the information included in the smart application can be augmented by experiences of other drivers. SeeGear (as is the name of the envisioned application) is tracking the bus position via GPS coordinates and sends the location data to remote services. For example, it can show to Finally, the fourth team designed a smart wheel chair enhancing elderlies' mobility within the smart city. The smart wheelchair has built-in sensors and services (like GPS and maps), and calculates the route based on real-time traffic data. The specific persona chosen is Per, who is a 90-year-old man who is abstained from meeting his friends to go to the theatre because of mobility problems and back pain. But this changes once he discovers that there is in the market a new wheelchair that uses GPS, maps, and calculates the route based on traffic. With this new smart wheelchair he can make it on-time to his appointments with his friends, and thus his social life is improving. With respect to the exploitation of main IoT components (i.e. the required functionalities in terms of services, sensors, human actions, feedback, etc) in a specific smart city context, the solutions of the student groups were relevant and complete. The solutions effectively exemplified the combined exploitation of two or four categories of the IoT main components, with the intended use of sensors being present in all of them. Yet, the results from the artefact analysis reveal that none of the four groups opted for a solution promoting some kind of participation or collaboration among the citizens of the smart city: in all student solutions, the ensuing application is meant to be used individually. The scores of the students' solutions with respect to the criteria mentioned in the previous paragraph are shown in Table 2.

B. Student knowledge on IoT and students' perceptions
Regarding the knowledge quiz, the majority of the students answered correctly to questions related to the lifecycle of IoT applications (Q8 on prototyping and Q7 on 3D and assembly), as well as to a question on Smart city (Q4); also, to an example of using IoT in real life (Q2). On the contrary, they scored poorly in two generic questions which involved IoT devices (Q1, Q3 and Q10), as well as in a question related to IoT data privacy (Q6). Finally, they scored acceptably when asked about IoT applications in real life (Q5 and Q9).
Regarding survey answers, Table 4 indicates that the students perceived the workshop as an intervention that: (a) promoted Smart City learning (with an average of 4.24 out of 5), (b) was joyful (with an average of 4.41 out of 5), and (c) helped them to think about different aspects of the problem and about different possible solutions (4.59 out of 5). Finally, the results indicate that, in general, students would like to advance their idea and create a physical prototype of it (3.76 out of 5). Yet, it emerges that this aspect falls short compared to the other aspects (a to c).  Twelve students assessed themselves as nonknowledgeable in IoT before the classroom intervention and knowledgeable after it. Three students assessed themselves as knowledgeable in IoT both before and after it. Two students self-assessed themselves as non-knowledgeable before and after the classroom intervention. An exact McNemar's test was run to determine if there was a significant pre-and postintervention difference in the proportion of students who assess them-selves as knowledgeable in IoT Page 1124 dichotomous dependent variable between two related groups.
Here the dichotomous variable was the perceived knowledge of the students on IoT. The authors asked the students anonymously (while using a unique code for each student for identification purposes) before and after the intervention whether they feel they know what IoT is, giving them with only two possible answers (yes or no). Using this process, we managed to set up the 2x2 matrix. The McNemar test was suitable since it is typically used to analyse pre-test/post-test study designs. The results indicate that the proportion of students who consider themselves as knowledgeable has significantly increased before (variable 'Prior IoT knowledge') and after (variable 'Current IoT knowledge') the classroom intervention.

C. Collaboration and the role of the teacher
The classroom observations involve two main emerging themes: the role of the teacher, and the collaboration schemata among participant students. Though the students have acted in a rather autonomous way in terms of help seeking behaviour, during the design workshop, we observed the critical role of the teacher in motivating students and make sure that they stay on focus. Also, the teacher used different techniques to help students stay on track. For example, when one of the group faced difficulties with the design, he suggested them to reflect: step back and think where they were in the process. With another group of students, he started challenging the idea to improve it, using examples to ground his rationale. In one case, he also acted out the scenario to help the group to visualise better their solution. Though the teacher involvement mainly intended to support the groups, we also observed teacher involvements meant to increase motivation, such as encouraging one group to reflect on the business model of their idea or to suggest a possible prototyping phase to demonstrate their idea.
Also, the teacher conveyed to the students the right message in the right format. As it is the norm with effective teaching, it seems that a good knowledge of the students and their interests helped the teacher to communicate with students in a way that it would not have been possible to external researchers.
VI. CONCLUSIONS One of the advantages of the workshop is that it provides an entry point into learning about IoT for all: it fosters students' knowledge and skills on the topic, without requiring an investment by the school in terms of resources and infrastructure. The design skills of the students are exemplified by their IoT design solutions and their characteristics as described in the section 'Student design skills on IoT'. The acquired student knowledge on the topic is evidenced by the knowledge quiz, as described in the section 'Student knowledge on IoT'. The results of the quiz can be considered as satisfactory since a) as it became evident from the survey, at the beginning of the intervention only a small number of students was familiar with the concept of IoT and b) the allocated time for each question was only twenty seconds. Also, these results are in line with the subjective perception of students, with 15 students stating that they know what IoT is after the workshop coupled with the analysis of the workshop outcomes.
More broadly speaking, the design workshop can serve as an easy entry point to promote IT education in schools in general, since it can offer a springboard that can help the teacher to address different related concepts in more details. For example, in the case described herein, it would have been possible to follow up with the prototyping phase or with lectures on (social) innovation, privacy, and possible careers connected to STEM (Science, Technology, Engineering, Mathematics) education. In Charlton and Avramides [8] the participant students explored their ideas through a simple setting where they could use sensors. This phase was led by the facilitator initially, but gradually the authors mention that students started to take ownership of their learning through experimentation with tangible IoT components. Another difference between this research work and the work in [8] is that we scaffold the ideation and design phase through a gamebased learning approach, whereas in [8] the facilitator is creating the context for students' exploration and collaboration.
In addition, the proposed methodology is not resource intensive in terms of the required teacher training. Though the workshop was run by the researchers, there was minimal intervention so that this could be easily conducted by a teacher if a clear teacher guide is provided. Based on our observations we have identified as future research the need to study more the scaffolding offered during the process, both in terms of content and modality. Also, the need to examine at what extent it is to support teachers with less experience, or teachers with no prior knowledge of the students' profile.
As a general conclusion, it can be argued that the results indicate that the workshop has promoted an effective teaching methodology in terms of students' conceptual knowledge and design skills that pertains to the ideation and design of IoT applications for the participating students, but it hasn't addressed equally effectively the attitudes-related aspects. This doesn't refer to whether the intervention motivated the students. On the contrary, students considered the workshop as a joyful learning activity. It refers to two facts: a) that the designed IoT applications did not consider the IoT citizen participation, and were meant to be used individually by the citizens of the smart city and b) that the students did not scored particularly high to the questions of the knowledge quiz that were related to the IoT privacy aspect. This should not come as a surprise since such aspects (e.g. IoT citizen participation, IoT Page 1125 data privacy) cannot be cultivated during a three-hour workshop but require a prolonged exposure of the students to diverse problematic situations. Limitations of the research include the short duration of the workshop and the small number of participant students. Additional future plans encompass intensifying our efforts to produce a robust teaching methodology for this highly innovative IT domain. Furthermore, they include investigating further whether the workshop can provide enough motivation and scaffolds to the students in order to continue with the programmingprototyping phase in an effective manner.