A Theory of Change for a Technology-Enhanced Education System in Bangladesh

An output of the EdTech Hub, https://edtechhub.org


Executive summary
In November 2020, UNICEF requested support from EdTech Hub to help the government of Bangladesh improve the reach, effectiveness, and inclusiveness of the programme of activities proposed in the government's Covid-19 Response and Recovery Plan: Education Sector (⇡Ministry of Primary and Mass Education, 2020). One year on, this plan has resulted in a wide and diverse range of programmes initiated by the government and other stakeholders. As a result, EdTech Hub worked with the government and other stakeholders to rapidly create an explicit theory of change (TOC) for the emerging technology-enhanced education system in Bangladesh.
The process to develop the TOC comprised a desktop review, interviews, and an initial stakeholder consultation workshop. The TOC draws on existing theory and practice as well as stakeholders' assumptions of how the evolving technology-enhanced education offer might achieve its goals -and what conditions need to be in place to achieve this. This working paper documents and explains the TOC (Annex 3 includes the individual TOCs that were developed for each modality used within the programmes in Bangladesh: radio, TV, mobile phone, and internet).
While we acknowledge that parents, caregivers, teachers, school, and community leaders are also key stakeholders, this first TOC focuses on students as, ultimately, the desired impacts of a technology-enhanced education system are directed towards students' educational outcomes.
In line with EdTech Hub's core focus, this initial support is focused on all students in Classes 1-12.
EdTech Hub was also asked to propose strategies for the use of the TOC, beyond this initial exercise. The recommendations are as follows.
1. Create a single, cohesive, strategy for a technology-enhanced education system in Bangladesh that incorporates and aligns the work of government and other stakeholder organisations. 2. Convene further stakeholder workshops that focus on critiquing and refining the TOC presented in this report as a multi-stakeholder dialogue to identify both additional modalities / activities that might not have been captured in this rapid exercise, as well as further refine the mid-term and short-term outcomes and impacts identified and suggested.
3. Develop a teacher-centred TOC for the technology-enhanced education system in Bangladesh to focus on the teacher's professional experiences and roles for theories of change for individual programmes. 4. Develop a parent-centred TOC for the technology-enhanced education system in Bangladesh to focus on parental experiences and roles with respect to the four main modalities (radio, TV, phone, and internet) and help build strategies for reaching parents and determining the nature of programmes to better support their children. 5. Discuss with the government and key digital providers how to ensure that relevant data is made available to teams to enhance the quality of monitoring and evaluation activities to align with the TOC. 6. Develop a research programme to investigate how the educational modalities that feature within the TOC combine to holistically support student learning.
7. Identify gaps in data that represent valid 'outputs' for the learning activities and resources that have been provided to students, which combine to provide the evidence that short-and medium-term outputs have been achieved.
8. Identify gaps in provision by using student demographic data to filter the student population and reveal the student groups that have not been able to fully access the inputs and resources provided. This will inform the design of new programmes that aim to reach these disenfranchised students.

Context of support
Following the onset of Covid-19 and the subsequent school closures, Bangladesh rapidly deployed a range of remote education services for students in Classes 1-12. These plans were presented in the Government's Covid-19 Response and Recovery Plan: Education Sector (May 2020), which outlined a range of different programmes and initiatives. Under this plan, the government collaborated with Civil Society Organisations (CSOs), development partners, and other education stakeholders to provide content to students via print, radio, TV, mobile phone, and online modalities to ensure students had access to education despite school closures.
While this approach increased the ability of many students to access educational materials, the rapid development of content meant that their design and implementation occurred in a somewhat ad hoc manner. Consequently, few programmes and initiatives had time and resources to document their theories of change, nor any associated monitoring and evaluation systems (⇡Kaye et al., 2020). This makes gaining insights into their success (or otherwise) a challenge. It also makes it difficult to concretely connect the programmes being implemented to the broader education system, particularly in relation to the use of these modalities following the re-opening of schools.
Noting the important role these programmes might play in facilitating (and indeed, expanding) education beyond the current school closures, EdTech Hub was asked to provide support to government and development partners to develop a theory of change (TOC) for technology-enhanced education in Bangladesh. Specifically, between January and June 2021, EdTech Hub was asked to: 1. Identify the implicit TOC behind Bangladesh's response across the multiple remote learning modalities. 2. Assess the implicit TOC for coherence, logic, and inclusiveness. 3. Propose strategies for the use of the TOC.
This document is the product of this support. It outlines the resulting TOC and the accompanying narrative. Annexes 3 and 4 provide detailed descriptions of the processes that led to the development of this TOC. which are summarised in Figure 1.

A TOC to inform a technology-enhanced education system in Bangladesh
A TOC is an established methodology for supporting the evaluation of programmes (or interventions). The 'theory' is the description of how a programme is expected to bring about the desired changes (or impacts).
The Bangladesh Government's Covid-19 Response and Recovery Plan was primarily concerned with mitigating the impacts of school disruptions on students' education. This plan necessarily addressed teachers, parents, and community groups. However, as it is necessary to explain in detail how the different learning modalities influence students' educational activitieswhich will ultimately result in the desired impact over time -this TOC is focused on the impacts of the different programmes on the students' learning outcomes. By deconstructing the different educational activities, it is possible to discern the evidence (in the form of metrics, measures, and data) that can inform both the design and the impact of different programmes. It also enables the impacts to be considered in an individual and cumulative way as many children will be experiencing more than one educational modality. Therefore, this TOC is student-centred. The creation of separate TOCs to support the monitoring and evaluation of programmes targeting teachers, parents, and community groups is also necessary to enable a comprehensive understanding of the wider impacts of a digitally enhanced education system in Bangladesh.
This TOC is a living tool to enable stakeholders to continually learn about and improve the various programmes that are implemented as part of an overall strategy. An effective TOC supports continuous evaluation throughout the lifespan of the programme while offering multiple stakeholders greater learning opportunities, better documentation processes, and a shared understanding of the efficacy of the programmes (Annex 5 outlines some approaches for this.) The Bangladesh government acknowledges the challenges that the pandemic context has presented for education.
"The unique nature of this challenge also calls for a more iterative planning and implementation approach with feedback loops to continuously adapt to changing circumstances expected over the next few months and possibly two to three years"

Ministry of Primary and Mass Education
(p. 1 May 2020)

The scope of the response and recovery plan
The Bangladesh Government's Covid-19 Response and Recovery Plan outlined the programmes for the school education system to cope with the adverse impacts of Covid-19 and beyond. It states a very clear set of desirable impacts for students, which include: ■ Provide access to education for: -all children aged 5-16 -hard-to-reach students -students with physical disabilities and special learning needs.
■ Reduce inequalities in learning outcomes for students.
■ Reduce school dropout rates -particularly for girls.
■ Reduce students' learning loss due to discontinuation of learning.
The delivery of the original plan, while government-initiated, was dependent on a number of civil service organisations (CSOs) and delivery partners who are working at national, regional, and local levels through numerous programmes and initiatives.

Structure of this document
The TOC for a technology-enhanced education system in Bangladesh is introduced in Section 2 along with the narrative that explains its format and resulting causal pathways.
Section 3 outlines the recommendations arising from the process of developing and evidencing the TOC.
Further details about literature reviewed, stakeholders consulted, the design and structure of both the initial stakeholder workshop (and the individual TOCs developed for radio, TV, mobile phone, and internet) and the subsequent stakeholder validation process can be found in the annexes.

The theory of change
A theory of change (TOC) is an overarching framework for understanding, systematically testing, and refining the assumed connections -the 'theory' - that underpins a programme and its anticipated impacts.
The complete TOC is shown in Figure 2 below. This figure is too small to be read in this document but can be viewed online here.
The TOC should be read from the bottom upwards. It includes the following components: • Foundational pillars.
• Educational activities for students.
• Short term outcomes • Mid-term outcomes • Impact goal In addition, the TOC has solid and dotted arrows that show the causal pathways between the educational modalities, activities, outcomes and impact. The solid lines indicate pathways for which there is emerging evidence (data) in Bangladesh whereas the dotted lines indicate pathways for which more data evidence is needed.
To the right of the TOC diagram are the legend; a yellow block of text which details the significant evidence gaps; the prerequisites for programme delivery; and the main underlying assumptions that relate to monitoring and evaluation.
The TOC is explained in detail in the narrative that follows. The TOC is built on four foundational pillars. These are: 1. Government administration and infrastructure 2. Education institutes and leadership 3. Teachers and caregivers 4. Parents and students

Government administration and infrastructure
This includes the policy frameworks, communications strategy, inclusive resources (considering diverse student needs, digital safety, and child protection) and associated professional development and support for teachers / school leaders.
Currently, there are two government plans that capture elements of what might be needed in the further development of a technology-enhanced education system in Bangladesh, which will build on the progress made during the Covid-19 Theory of Change for a Technology-Enhanced Education System in Bangladesh 13 pandemic (⇡General Economics Division, Bangladesh Planning Commission, 2020) Government and CSOs, the private sector, and other organisational stakeholders support this pillar by providing: ■ Appropriate policies, digital learning infrastructures and environments. ■ Educational content for programmes. ■ Local support for all students, teachers, and parents.
■ Oversight of quality assurance, to include support for monitoring and evaluation.

Education institutes and leadership
With support from the government, non-governmental organisations (NGOs) and other stakeholders, these can enact the policies and use the associated resources to provide access to learning opportunities for all students, to include those with disabilities and diverse learning needs.
Education institutes and leadership are supported by the government, civil service organisations (CSOs), the private sector and other organisational stakeholders to have: ■ Guidelines and an organisational culture that align with effective use of technologies. ■ Sufficient human support and infrastructure. ■ Sufficient technology resources and equitable access to them. ■ An adequate physical setup and structure for technologies.

Teachers and caregivers 1
Teachers and caregivers form a critical pillar through which systemic educational change comes about, hence the need for pre-service and in-service teacher professional development opportunities that foreground technology use and involve teachers in the ongoing development of curriculum and assessment approaches.
Teachers and caregivers are supported by the government, CSOs, the private sector, and schools to have: ■ Access to suitable devices with affordable internet. ■ Sufficient skills and proficiency with technology for inclusive learning, especially for disabled students. ■ Awareness and ability to access, utilise, and adapt local, national and international learning resources. ■ Attitudes and beliefs aligned with educational technology pedagogy. ■ Motivation and recognition for best practices of remote learning.

Parents and students
As key stakeholders in any education strategy, parents and students require special consideration and support to enable them to access the learning modalities within a technology-enhanced education system.
Parents are supported by the government, CSOs, the private sector, and other stakeholders to: ■ be aware of distance learning modalities and how their children will access / benefit from them; ■ have sufficient literacy / numeracy skills to provide adequate associated support.
Students are supported by the government, NGOs, and other organisational stakeholders through: 1 Caregivers refers specifically to adults who support young children's education in kindergarten and nursery settings.
■ offering accessible technologies that are close to any prior technology experiences, with which the students are comfortable and students are supported to develop attitudes, beliefs, and engagement patterns that align with the learning modalities.

Educational modalities
The Educational modalities included on the TOC are the 12 modalities outlined in the UNICEF Guidance on Distance Learning Modalities (⇡UNICEF, 2020). For Bangladesh, these are grouped (and colour coded) according to the four modalities initially outlined in the Bangladesh Covid-19 Education Response Plan: Radio (dark blue), TV (pink), Mobile phone (light blue), and Internet (green). On the TOC, each modality includes letter codes that refer to their specific underlying assumptions and dependencies. These letter codes are explained in Section 2.7. This is the most critical component of the TOC as it is only by considering what students actually do when they are engaging in learning that we can begin to appreciate how these activities contribute to students' achievements of the relevant outcome measures over time. This section of the TOC is deliberately detailed to highlight the sense of progression within a high-quality learning experience. On the TOC, this progression is indicated from left to right through the following educational activities:

Educational activities for students
■ Students listen to teacher(s) (audio only). ■ Students view synchronous or asynchronous lessons. ■ Students participate in synchronous lessons. ■ Students respond to teacher(s) in synchronous sessions.

Theory of Change for a Technology-Enhanced Education System in Bangladesh
■ Students complete tasks and assignments. ■ Students submit tasks and assignments. ■ Students and parents engage with teacher(s)' feedback from submitted tasks and assignments. ■ Students and parents engage with formative / summative assessment and progress data.
A traditional classroom lesson can be conceived as a synchronous ("live") session. Asynchronous learning activities would normally take place away from the classroom or school environment (such as in homework or during periods of school closure, remote learning). All activities that involve students can be considered both at the individual student level and as student-to-student collaborative work. For example, peer assessment activities through which students review and comment on each other's work are known to be particularly effective (⇡Black & Wiliam, 2010).

Short-term outcomes
The short-term outcomes might be evidenced over days and weeks. They are: ■ Increased numbers of students accessing and engaging with learning resources.

Theory of Change for a Technology-Enhanced Education System in Bangladesh
■ Increased synchronous and quality engagement between teachers and students (includes assessment / remediation of material). ■ Increased interactions within and between students, teachers, and parents for learning and vital social connection.
These three outcomes capture the key elements of effective education systems, with or without educational technologies. Each is vital to provide a learning environment in which all children learn and thrive within and beyond classrooms. The addition of digital technology increases the complexities as students, teachers, and parents learn new ways of working and interacting with each other. It cannot be assumed that the addition of technology will automatically lead to improvements in these three outcomes.

Mid-term outcomes
The stated impact goal relies on a combination of mid-term outcomes that must be achieved over time periods of months and years. These align with the Bangladesh government's longer-term education strategy. They are:

Theory of Change for a Technology-Enhanced Education System in Bangladesh
■ Improved students' access to and engagement with learning resource(s). ■ Improved students' wellbeing and motivation to learn, which requires some consideration of how these two constructs are widely understood and evidenced and some common methods for doing so. ■ Improvements in all students' achievements of benchmark learning outcomes, where benchmark outcomes are widely understood and collected frequently enough to support agile monitoring and evaluation. ■ Increased positive reinforcements from teachers and parents.

Impacts
The TOC has one widely-understood impact goal, namely:

All children (Class 1-12) are fully engaged in high-quality education enhanced by technology.
This resonates with both the Bangladesh Government's Education Sector Plan (2020-21 to 2024-25) and its Vision 2041 plans, which rely on a highly educated population that is digitally literate. Hence, the children who are in school right now are a significant group and their progress through the school system over the coming years will need to be carefully monitored and evaluated to support the system to be improved in time.
The TOC focuses on the processes through which all students (Classes 1-12) might be enabled to engage fully in a high-quality education when engaging with the digital resources being provided in Bangladesh. This relies on a wide and ongoing effort to decrease the technological and other barriers for students that prevent many from learning from the current materials that are provided. Hence the words fully engaged in high-quality education set a major goal as these barriers must be addressed if a fully equitable education system is to be achieved.

Underlying assumptions and dependencies
The TOC is underpinned by a wide range of assumptions and dependencies. These relate to either the particular educational modality or the wider context for its use.
For programme delivery students' access is dependent on the following conditions being met.
a. Household having access to electricity and quality network coverage.

Significant evidence gaps (the yellow box)
Finally, the TOC includes a section, which is shown in yellow, that highlights some significant evidence gaps relating to a lack of accurate student-level data that is hampering opportunities to monitor and evaluate the impact of programmes on specific groups of students (see Figure 8).
For interventions targeted to disadvantaged students, it will only be possible to gauge if students have achieved the stated medium-term outcomes on the TOC if the student data is disaggregated and the programme design is sensitive to the students' needs and context.
Hence measures of students' wellbeing, motivation and learning outcomes will be needed, which are disaggregated by the student demographics (socio-economic status, geography, ethnicity, gender, special educational needs, physical disabilities etc.) The following outcomes might then become evident: ■ Increased access to education for hard-to-reach students and those with disabilities and special learning needs. ■ Reduction in inequalities in learning outcomes for students. ■ Reduction in school dropout rates -particularly for girls.

Theory of Change for a Technology-Enhanced Education System in Bangladesh 22
EdTech Hub The TOC components are connected by solid and dotted arrows that indicate evidenced (solid arrow) and hypothesised (dotted arrow) pathways that explain how multiple students' engagements in the educational activities lead to the outcomes over time, to result in the desired impact.
The causal pathways are inferred by tracing an educational modality upwards to indicate how the learning activities it has facilitated should enable the student to achieve the mid-and long-term outcomes providing the prerequisites have been satisfied and the associated assumptions have been validated. For example, a student's viewing of a series of TV broadcasts, while relatively easy to evidence, would only result in a high-quality educational impact if the student was Theory of Change for a Technology-Enhanced Education System in Bangladesh also completing and submitting associated tasks and engaging in associated formative and summative activities involving rich dialogue with teachers and parents.

Theory of Change for a Technology-Enhanced Education System in Bangladesh 24
EdTech Hub

Recommendations
The goal of a theory of change (TOC) is to provide a diagram that communicates how a programme design is intended to achieve its goals; and to create a tool that can enable multiple stakeholders to rally their efforts. In particular, it communicates the critical links that need to be foregrounded in monitoring and evaluation activities. The overarching TOC supports this process at the level of the education system.
In addition, the initial stakeholder workshop and follow-up activities resulted in a set of individual TOCs for the four educational modalities initiated by the Response and Recovery Plan. These TOCs are provided in Annex 3 and they include more detail with respect to the particular programmes that have been developed.
The TOC offered in this paper captures a theory for how a technology-enhanced education system might work in Bangladesh to support student learning. It helps to identify gaps where more investigation / thinking is required to better understand the way in which interventions can support student learning in Bangladesh. Furthermore, the TOCs allow the government and key stakeholders to see what data is needed on a national scale to support the individual programme teams to monitor and evaluate the outcomes and impacts of their own work.
Based on the rigorous data collection and analysis conducted to support this paper, we offer the following recommendations: 1. Create a single, cohesive, strategy for a technology-enhanced education system in Bangladesh. While this working paper has attempted to review the response to remote education in Bangladesh as a whole, it appears that the approaches are a set of discrete activities being deployed by stakeholders to fill gaps in service provision. While this is a common occurrence at the current time -governments and development partners have foregone overarching cohesion to rapidly provide services for students during school closures -it would be good to now regroup, reanalyse the solutions that are available, and design a single, cohesive strategy for a technology-enhanced education system that builds on the advances made in the last year and align the work of government and other stakeholder organisations. 2. Convene further stakeholder workshops that focus on critiquing and refining the TOC presented in this report as a multi-stakeholder dialogue. The act of bringing multiple stakeholders together to share information about the programmes that they have designed can be facilitated by framing the activity through the lens of refining the TOC. This should include identifying both additional inputs / activities that might not have been captured in this rapid exercise, as well as further refining the short-and mid-term outcomes and impacts identified and suggested. For example, the internet platforms for primary and secondary education revealed many different programmes that had some overlapping and some very distinct inputs and goals. The respective TOCs might be used to discuss these different programmes and, in doing so, widen knowledge of existing monitoring and evaluation approaches and associated findings. Be mindful that newcomers will need time and support to make sense of the key terms and concepts of the TOC. Annex 5 offers some guidance on how to facilitate activities that enable such ways of working. 3. Develop a teacher-centred TOC for a technology-enhanced education system in Bangladesh. This TOC should focus on the teachers' professional experiences and roles that are detailed in the foundational pillars. This will help identify gaps in the current plans and inform the further development of resources to support the longer-term development of teachers' digital capacities towards the broader goals of the country's education plans. 4. Develop a parent-centred TOC for a technology-enhanced education system in Bangladesh. This should focus on the parental experiences and roles as detailed in the foundational pillar and on aligning with related assumptions. This will help to build strategies for reaching parents and determining the nature of programmes that will enable them to better support their children. 5. Hold discussions with the government and key digital providers about how to ensure that relevant data is made available to teams. This approach could enhance the quality of monitoring and evaluation activities for the TOC. For example, the back-end data from digital platforms might be used to triangulate data from surveys and interviews. 6. Develop a research programme. While the TOCs presented within this document require further and ongoing refinement, they are nuanced enough to begin informing the development of a research programme that can support the refinement and enhancement of Bangladesh's education plans. As a starting point, the process of developing the overarching TOC has revealed that there is much scope to investigate how different programmes and initiatives can combine to holistically support student learning. As mentioned above, currently each of the strands seems to operate independently, rather than being part of a strategy where each of the elements combine to generate outcomes that are greater than the sum of the individual parts. Additionally, the process of developing the TOC revealed that to date there has also been very little assessment of the impact that the various programmes are having on student learning outcomes, and how the inputs and activities are leading to short-and mid-term outcomes and impacts. This is another area that would benefit from further focused research in the short term. Beyond monitoring, the overarching and platform-specific theories of change can be used in the future as a conceptual framework for larger-scale evaluation. Both the linkages and, in some parts, the lack thereof, can be used to identify areas for large-scale evaluation. ■ Student motivation data.
■ Benchmarks for agreed tasks / assessments that can be administered remotely, when needed.

Identify gaps in provision.
The overarching TOC provides a framework to track the particular inputs and resources that have been developed for all students. However, once the demographic data is used to filter the student population, it is likely to reveal that certain student groups have not been able to fully access the inputs and resources provided. In this case, the TOC will need to be reviewed and additional inputs and resources provided to address the gaps. For example, a gap might be that particular student cohorts (e.g., those with physical disabilities and special learning needs, or those in remote and rural areas) may not be reached by any platform. In this case, the following actions could be taken: ■ Further research initiated to understand the most impactful ways to reach such students - to include exploration of (digital) devices and resources to support students to access learning content.

Annex 1. Stakeholder engagement
Annex 1 includes the list of stakeholders that were represented at the workshops according to their role designation and institutions.
Those that Organisations that were involved in the initial workshop are indicated with a cedilla (~) and those that were interviewed are indicated with an asterisk (*) .

Organisation Designation
Access to Information (A2i)

Annex 2. The initial stakeholder workshops
As the Government's Covid-19 Response and Recovery Plan has led to numerous programmes that have been devised by multiple stakeholders, we began by adopting a particular structure (a logic model), to enable stakeholders at the initial workshop to begin to analyse their own programmes. We used the Kellogg Foundation's theory of change (TOC) approach, which has been used extensively in numerous programme evaluations worldwide (⇡Kellogg Foundation, 2004). Although we did not eventually adopt this structure, we describe the process below to highlight the iterative nature of TOC creation.
A TOC needs to be focused on the main recipients of the programme -which in our case is the students -and the resources and activities through which the desired impacts on students will be achieved.
In our initial approach, we created individual TOCs for each educational modality that can be read from left to right as a causal chain. The desired impacts of each modality form the goals for the various remote education programmes that have been developed. As these impacts have driven the design of the individual programmes, the TOC terms are explained in reverse order (i.e., from right to left) in the sub-sections that follow.
It should be noted that the Bangladesh Government's Covid-19 Response and Recovery Plan adopts some different meanings for the words 'outputs', 'outcomes', and 'impacts'. This section also includes the individual TOCs for the four modalities specified within the Response and Recovery Plan: TV, radio, mobile phone, and internet. Where relevant, the TOCs for both primary and secondary education are included.

Impacts
Impacts are organisational, community, and / or system-level changes expected to result from the programmes provided. More generally, these might include improved conditions, increased capacity, and changes in the policy arena. In other words, impacts are the overarching goals that implemented programmes intend to achieve. These can be expressed as short-term impacts (weeks), mid-term impacts (months) and long-term impacts (years).

The Bangladesh Government's Covid-19 Response and Recovery Plan:
Education Sector was initially framed by the following timelines, which provide a context for the interpretation of the TOC: • Short Term (Immediate to 6 months): Children's safety and learning continuity. • Medium Term (Immediate to 12 months): Recovery and re-opening.
• Long Term (Immediate to 24 months): Sustaining and integrating gains.

Outcomes
Outcomes are specific changes in attitudes, behaviours, knowledge, skills, status, or level of functioning expected to result from programme activities and which are most often expressed at an individual or cohort level. These outcomes should be measurable through a change in the value of an output measure. Outcomes can be expressed as an increase, a decrease, or a constant value. For example, in the case of remote learning through the modality of radio in Bangladesh, a desirable outcome would be an increase in learning outcomes for a population of students who had accessed the radio platform in a way that aligns with the programme designers' theory of change.

Outputs
Outputs are the metrics or measures that are collected as participants engage with a programme or intervention. The programme monitoring and evaluation plans should detail each measure, what it represents, the methods for its collection, and subsequent analysis. The information gathered will indicate whether the programme has been delivered to the intended audiences at the intended levels.
In the Bangladesh context, the outputs need to relate to the students who have engaged with relevant programmes. For example, the number of activities a student has engaged in, the length of time, the frequency of engagement, the number of related assignments completed etc.

Activities
Activities are the processes, techniques, tools, events, and actions that take place within the planned programmes. In the case of remote learning programmes, there are multiple activities needed by programme implementers that contribute to a successful programme. Our student-focused approach has placed the emphasis on the description of the learning activities for students, as created by the programme designers / implementers as these drive the desired outputs, outcomes, and ultimately impacts. For example, students learning through the radio platform might be expected to listen to the programmes and then complete assignments. They might also submit their assignments somewhere and / or engage in feedback activities with a teacher, parent, or sibling.
It is useful to use 'action verbs' to describe students' learning activities that form part of the theory for how the programme works to achieve the desired impacts over time. It is the cumulative impacts of many occurrences over time, which may take place across different modalities, that dictate the scale of the impacts in the longer term.

Inputs or resources
Inputs are the resources such as the human, financial, organisational, and community resources a programme directs towards the work. A 'student-centred' focus means that these inputs should describe everything that needs to be put in place to enable the students to carry out the desired learning activities. The four educational modalities - TV, radio, mobile phone, and internet -each have very different sets of inputs. Prior to students being able to access the resources, many preconditions need to be in place, and these are detailed in the TOCs.

Assumptions
Assumptions are the list of conditions that are highly desirable for a successful programme. For example, although students can access the radio platform independently from their schools, it might be desirable that the school promotes the existence of the radio platform to parents and links the radio lessons with the school textbooks and relevant curriculum. Many programmes are based on assumptions, which need to be documented as part of the TOC. Some of these assumptions relate to preconditions that must be implemented for programmes to have the potential to achieve their outputs / outcomes / impacts. For example, we already know that programmes require: ■ Connectivity, for example, using TV, radio, the internet, mobile data.
■ Access to devices (e.g., a TV, a radio, a smartphone, a tablet, a computer).
■ Access to support from a parent / carer, teacher, community leader etc.
In many cases, early programme monitoring gathers data to support the evaluation of whether preconditions have been met and, indeed, to check if assumptions are correct.

How the individual theories of change were created
At the initial stakeholder workshop, we gave participants two templates to guide discussions. The first template was created using a digital collaboration tool that multiple stakeholders could contribute to during the synchronous online workshop. As an alternative, if they wanted, the groups had the option of using a second template, a Google document, which was organised as a What are the overarching goals?
Workshop participants were grouped according to the type of educational modality their programme or initiative addressed. They then listed the desired impacts from the Bangladesh Government's Covid-19 Response and Recovery Plan: Education Sector in the far right-hand column, grouping these by short-(weeks), medium-(months) or long-term (years) impacts.
Next, the programme teams noted the inputs or resources that had already been designed (or were planned) - and the underlying preconditions and assumptions that related to these designs.
The groups then either worked backwards from the desired impacts -or forward from the designed activities, to decide on the most appropriate data that might need to be gathered. This data is needed to: ■ validate or refute assumptions that have been made in constructing the theory; ■ support the development of the theory by highlighting gaps where more or better data is needed.

Gathering information
We recommended that the groups work with sticky notes (or virtual sticky notes, such as Miro) to list the ideas for each element of the TOC. By writing each contribution on a single sticky note, it was possible to move the ideas around. It is very common to 'over assume' the impacts of programmes without realising that we might lack evidence to support this aspiration. In all cases, it is necessary to think deeply about the learning activities that students undertake -and the data that is generated through these activities, which is often not very visible remotely. Hence, the need for additional Covid-safe data gathering using surveys, questionnaires, field observations, and interviews (⇡Kaye et al., 2020).
For example, there is a subtle difference between the data that evidences the programme outputs alongside data that evidences the programme outcomes. For example, TV programme viewing figures at any point in time is an output measure, whereas an increase (or decrease) in these figures would be an outcome measure.
Deciding what data is a good proxy for the impact that is being sought will normally involve a number of conversations, especially in relation to learning outcomes. Using the example of TV, more data would be needed to be confident that only watching the programmes resulted in improved learning.
Other activities, such as completing accompanying textbook tasks or receiving and acting on feedback are often desirable to ensure high-quality learning.
In a multiple stakeholder programme, discussions (in-person conversations, phone calls, surveys, workshops) are necessary to find the extent of available data and determine its usefulness in relation to a particular intervention.
In addition, an accompanying narrative is necessary to explain the components in depth so that multiple stakeholders can easily access and understand the information that the TOC is trying to provide.
The best results will be achieved if there is an intentional and critical approach in which participants question each other supportively about their 'theory' and the evidence, as well as assumptions underlying those theories.
These TOCs were developed in two stages.
1. Stakeholder groups added information (shown in black text in Figures  10-15 below) about their programmes during the online workshop. Some theories of change included multiple programmes that have been developed by different stakeholders. 2. The EdTech Hub team added suggestions for additional activities, outputs, and outcomes for a more comprehensive TOC (in blue text in Figures 10-15 below).
The TOCs that were initiated during the stakeholder workshop and then modified by the EdTech hub team are given below, organised by their corresponding educational modality.

Television modality for primary and secondary education
The 'Ghore Boshe Shikhi' (Learn at Home) lessons, broadcast by Sangshad TV were initially offered for primary-age students, and a series of lessons for secondary-age students were developed subsequently. Hence, we have developed a separate TOC for each to reflect the different stages of development.
Synchronous televised video lessons for primary-age students were conducted in 20-minute segments with three classes per day. The main subjects taught were Bangla, English, mathematics, science and Bangladesh global studies. An additional hour of programming was also provided each day for remedial learning and focused on repeat broadcasts of previous lessons.
Synchronous televised lessons for secondary-age students were also conducted in 20-minute segments but with eight classes per day. The main subjects taught were Bangla, English, mathematics, Bangladesh global studies, ICT, science (physics, chemistry, biology), history, and accounting. An extra hour of programming was provided for Technical and Vocational Education and Training (TVET) subjects as well as for Madrasa students.
A number of sources of evidence were available that reported on the early impacts of the TV modality (⇡World Bank, 2020). These reports mainly focused on gathering data to monitor the availability and access of the TV programmes for students in various locations. This is important data, needed to support the validation of key assumptions within the TOC. However, these reports did not provide evidence of programme impact on students' learning experiences and outcomes for the TV modality as a whole.
The individual TOCs for primary and secondary education are presented in Figures 10 and 11 below. Figure 11. Theory of change for television for primary education.
The black text indicates information provided by the stakeholders and the blue text indicates suggestions for additional information made by the EdTech Hub team. Figure 12. Theory of change for television for secondary education.

Theory of Change for a Technology-Enhanced Education System in Bangladesh
The black text indicates information provided by the stakeholders and the blue text indicates suggestions for additional information made by the EdTech Hub team.

Theory of Change for a Technology-Enhanced Education System in Bangladesh
Both TOCs share common approaches with respect to student inputs and activities. However, existing data reveals little about the nature of students' participation in learning activities that accompany televised lessons and how these students represent the wider population of school-age students. Thus, we have identified existing gaps in evidence and proposed areas for exploration when designing and monitoring programmes.
For further exploration: ■ Is the learning content and format appropriate for the full range of students? If not, how might it be adapted to this goal?
■ What supplementary resources might be developed to improve student engagement? These might be resources for parents and teachers.
■ Are assessments providing sufficient data of acceptable quality? How is data collected and collated? How are teachers and parents involved?
■ How can the TV modality complement the formal education system in the longer term?
■ What new content areas (for example, co-curricular classes or career awareness) might be explored through the TV modality?

Radio modality
Bangladesh's radio platform was created in collaboration with UNESCO in August 2020. It broadcast condensed, 8-minute lessons based on the 'Ghore Boshe Shikhi' ('Learn at Home') TV programmes and NCTB curriculum content. Lessons in the subject areas of English and Bangla were broadcast through radio networks and could also be accessed on mobile phones with radio capacity.
The TOC for the radio modality is presented in Figure 13. The black text indicates information provided by the stakeholders and the blue text indicates suggestions for additional information made by the EdTech Hub team.
Unsurprisingly, there are fewer assumptions that need to be satisfied for radio than those required for the digital platforms' TOC. However, several elements require clarification. For example, the student tasks and assignments are a critical factor to ensure learning and can be more clearly articulated. Clarification of how adults (teachers and parents) will provide support and feedback is also required. Existing data reveals little about the nature of students' participation in learning activities that accompany televised lessons and how these students represent the wider population of school-age students. Thus, we have identified existing gaps in evidence and proposed areas for exploration when designing and monitoring programmes.
Recommendation and questions for further exploration: ■ The design principles for the radio programme are not evident. These need to indicate how the activities will initiate student learning and outline the design of student tasks and assessment activities that accompany each programme.
■ What are the learning activities associated with the lessons? Is the learning content and format appropriate for the full range of students? If not, how might it be adapted to this goal?
■ Are teachers involved in this platform? What are pathways for feedback?

Mobile phone modality
Distance education by mobile phones is being delivered by various civil society organisations (CSOs) such as BRAC. Through its Home School programme, BRAC conducts audio lessons over the telephone in 20-minute segments. Students are divided into groups, which consist of four students supported by a teacher. Support is also provided for students with disabilities by training a guardian who is available during the audio lesson. In addition, Friendship, another CSO, has also established a mobile school programme in order to reach students. The TOC is given in Figure 8 below.
The mobile initiative, although relatively small-scale, is promising. It facilitates planned and regular communications between teachers and students. It is also personalised and therefore more likely to motivate and engage students. It offers a conduit for teachers to direct students to other learning resources within the other platforms, should the student have access to the necessary device and associated resources.
In addition, we have identified existing gaps in evidence and proposed areas for exploration when designing and monitoring programmes.
Questions for further exploration: ■ How are learning tasks associated with the programme designed, monitored, and assessed?
■ To what extent can students engage with their learning independently?
■ How can students seek additional support if needed?
■ What are the other effective ways that mobile platforms can be used? Figure 14. Theory of change for mobile phone.
The black text indicates information provided by the stakeholders and the blue text indicates suggestions for additional information made by the EdTech Hub team.

Internet modality for primary and secondary education
Multiple initiatives using internet-based resources for students have been developed, many of which have wide geographical reach. These range from the government-funded platforms (e.g., Konnect) to smaller-scale, school-and teacher-led initiatives (e.g., Facebook groups) for primary and secondary school education, which all aim to build digital capacities in the country. Student learning is supported through live synchronous instruction as well as asynchronous methods in the various internet-based platforms designed for web and mobile devices. In addition, we have identified existing gaps in evidence and proposed areas for exploration when designing and monitoring programmes.
The TOCs are presented in Figures 9 and 10 below.
Questions for further exploration ■ How can the back-end data that is available be accessed and leveraged to support monitoring and evaluation activities? (For example Facebook and YouTube analytics data.) ■ How might the diverse initiatives be mapped and/or aligned to enable key lessons to be learned in relation to the most effective content design, delivery and impacts?
■ How will a complimentary teacher-centred TOC support the monitoring and evaluation of the impacts of the Internet Platforms and their future developments? Figure 15. Theory of change for internet for primary education.
The black text indicates information provided by the stakeholders and the blue text indicates suggestions for additional information made by the EdTech Hub team. Figure 16. Theory of change for internet for secondary education.

Theory of Change for a Technology-Enhanced Education System in Bangladesh 51
The black text indicates information provided by the stakeholders and the blue text indicates suggestions for additional information made by the EdTech Hub team. Figure 18. Mid-term outcomes (non-government participants). Responses to poll question "Which of the Mid-term outcomes do your programmes aim to achieve?" (n= 29).

Breadth of educational modalities on offer in Bangladesh
The non-government stakeholders are offering educational programmes that collectively include all 12 modalities.

Data sources to support monitoring and evaluation of programmes
Feedback from both goverment and non-government providers implied that their approaches to monitoring and evaluation of programmes are still in development. Across all respondents, the following data sources are emerging that might support validation of programme-level theories of change.
■ Student attendance data (according to the different learning modalities). ■ Student task completion data. ■ Student test score data. ■ Student work (i.e., submitted tasks and assignments). ■ Student collaboration data (i.e., evidence of if, and how students have worked productively on group tasks). ■ Teacher feedback data (i.e., specific evidence of if, and how teachers have provided feedback to students and parents). ■ Student feedback data (i.e., specific evidence of if, and how, students have interacted with teachers about their wellbeing, motivation, and learning). ■ Parent / caregiver feedback data (i.e., specific evidence of if, and how, parents / caregivers have interacted with teachers about their child's wellbeing, motivation, and learning).

Summary of changes made to the TOC as a result of the stakeholder validation process
The process of stakeholder validation led to a number of common questions being asked about the TOC and its relationship to educational programmes that had been designed for particular content areas or contexts. Hence we offer a set of Frequently Asked Questions (FAQs) in Annex 4.
A number of stakeholders provided feedback that a student's psychosocial context (indicating their 'readiness to learn') alongside the students' views of their learning experiences were important aspects to capture and monitor within programme-specific TOCs. Hence, each programme would need to devise suitable and appropriate outcome data to support the monitoring and evaluation of programme outcomes.

Annex 4. Frequently asked questions Learning content
Are we missing some key aspects of education such as soft skills, ethics, careers education, emotional intelligence, empathy?
The TOC is 'content agnostic' -it does not suggest any particular learning content. Consequently, it is for each programme to decide the content-specific learning objectives, from which the suitable outcome measures would be determined.

Student engagement
How do we evidence 'fully engaged'? This seems a very ambitious impact goal!
We will never be able to test or evidence this as humans are too complex and the world around us is too dynamic. It is an aspirational goal that we should aim towards as a collective mission. We should also understand that moving towards such an aim requires us all to collaborate and iterate when designing an education system.

How do we classify different forms of EdTech by their modalities?
This is always very difficult -particularly as many technologies work and share data together -they are 'interoperable'. The main purpose for classifying educational technologies is so that we can have some common understandings of how a student will interact with each modality as a learner, and what data is automatically captured as a result of their interactions. It is vital that educationalists work alongside technologists to decide which data is valid from an educational perspective. Many digital systems collect vast amounts of data that is of no real value when trying to understand if, when, and how learning has happened!