Teacher digital self-efficacy interventions

Preregistration File:

1. Project Title: How to build teacher self-efficacy as a part of digital competence: Experimenting with teacher training in two countries
2. Authors and Affiliations:
• Ley, Tobias, professor, Head of the Center for Digitisation in Lifelong Learning, University for Continuing Education Krems, Austria
• Tammets, Kairit, professor, Head of the Centre for Educational Technology,   Tallinn University, Estonia
• Põder, Kaire, professor, Head of Labour market and Education Research Group, Estonian Business School, Estonia
• Wagner, Marlene, Research Fellow, University for Continuing Education Krems, Austria
3. Date of Preregistration: 14^th of May 2025
4. Research Question(s) or Hypotheses:

·         H1 (design of training hypothesis): The effect of collaborative design is smaller than the effect of situated learning on the teacher's self-efficacy.

·         H2 (complexity hypotheses): The complexity of teacher training (more training methods) is increasing the self-efficacy beliefs of teachers.

·         H3 (cognitive activation hypotheses): Tealess effective if it does not bring along cognitive activation in building teachers' self-efficacy beliefs?

1. Study Design:
• Type: Experimental. 
• Units of analysis: teachers

Our data originates from a quasi-experimental design study from two countries: Austria and Estonia.  Data are collected using a pre-post design of pre- and in-service teachers, where teachers could self-select for the training.  We apply reliable instruments for data collection and collect data about various background characteristics, which allow us to control many heterogeneous effects. 

1. Planned Sample

o   Sample size: 188 observations from 7 treatments.

o   Inclusion/exclusion criteria: in and pre-service teachers

1. Variables
• Treatment: Teacher training method: Knowledge Instruction (KI), Collaborative Design (CD), Situated Learning (SL)
• Independent: Cognitive Activation Scale (CAS)

CAS (6 items) from Richter & Richter (2024) was used to assess teachers' assessment of the course in level 1.   The items are:

All 6 items, rated on a 5-point Likert scale (1 = strongly disagree, 5 = strongly agree). A reliability analysis indicated good internal consistency (in  Cronbach’s Alpha = 0.91 in case of Austria and 0.87 in case of Estonia).

• Dependent: Self-efficacy of digital teaching

Self-Efficacy Scale (SES), we used van Acker et al.'s (2013) instrument with 3 items. 

The internal consistency of the teacher SES (3 items) was used to assess teachers' beliefs of self-efficacy.  All 3 items were rated on a 6-point Likert scale (1 = fully agree, 6 = fully disagree). A reliability analysis indicated good internal consistency (in pre-test, Cronbach’s Alpha = 0.92 and post-test 0.93 for Estonia and respectively 0.87 and 0.93 for Austria ), suggesting that the scale items measured a cohesive construct. To measure SES as a composite measure, a three-item scale was used (see descriptive statistics Table 2).  The SES was computed by averaging the responses across the three items, ensuring that each component contributed equally to the overall self-efficacy measurement. Higher scores indicate greater self-efficacy, while lower scores suggest reduced confidence in one’s abilities.

o   Controls: age, gender, education.

1. Methods of Data Collection
• Pre-post survey, collected in all 7 groups
2. Planned Analyses
• Country and group fixed effect regressions

The dependent variable is self-efficacy in digital teaching measured after the intervention (selfeff_post). All models include group (training case) and country fixed effects. In all cases, we control for participants' background characteristics such as age, gender and education. The treatment variables are binary indicators for instructional methods: KI+CD (knowledge instruction and case discussion), KI+SL (knowledge instruction and simulation learning), and KI+SL+CD (a combination of all three), with KI only serving as the reference category. Model 2 estimates the effect on the subset (excluding 2 Cases which have no measure for cognitive activation).  Cognitive activation (C) captures the extent to which learners experienced cognitively engaging instruction in Model 3. Model 4 includes an interaction term between treatment method and cognitive activation (T×C), allowing for the examination of differential effects of activation by instructional method.  All models control for baseline self-efficacy (selfeff_pre), age, gender and education of the teachers.

1. Deviations and Limitations

While the study followed a quasi-experimental pre-post design to examine the effects of various instructional methods on teachers’ self-efficacy in digital competence.  Limitations from the data and method:

o   Random assignment was not feasible, as participants self-selected into training formats. This may have introduced selection bias, although demographic controls (age, gender, education) were used to mitigate it.

o   Due to the use of group and country fixed effects in the regression models, it was not possible to isolate the effect of delivery mode (online vs. face-to-face), even though these modes varied across treatment groups and may have influenced outcomes.

o   Measurement of cognitive activation, which was only available for five of the seven groups. The remaining groups (1 and 2) lacked data on this mediating variable, reducing statistical power for tests of the third hypothesis.

1. Ethical Considerations

The subjects of the study are teachers (age 21-63) who gave informed consent for voluntary participation.  The participants were adults enrolled in professional development programs, either as pre-service or in-service teachers. Prior to data collection, participants were informed of the study’s purpose, their right to withdraw at any time, and the anonymous treatment of their responses. Informed consent was obtained either in written or digital form, depending on the delivery format.

Data were anonymised at the point of collection: no names, personal identifiers, or school affiliations were recorded. Each participant was assigned a unique case ID, and all statistical analyses were conducted on de-identified datasets. Data storage and handling conform to EU GDPR standards, and the final dataset is stored securely with access limited to the research team.

The training content posed no psychological or physical risk, and no deception was used. Participation in the study had no bearing on teachers’ evaluations or institutional standing. The research design and tools (e.g., self-efficacy scales) were based on established instruments and approved under the broader ethics guidelines of the EffecTive (https://project-effective.eu/) project consortium.