BY-COVID - WP5 - Baseline Use Case: COVID-19 vaccine effectiveness assessment

Survival analysis

Vaccination schedules dropped from the subgroup analyses

Vaccination schedules with less than 500 were dropped from the subgroup analyses.

Vaccination schedule N
BP-MD 288
MD-BP 228
JJ-JJ 92
BP-JJ 52
MD-AZ 52
AZ-MD 36
AZ-JJ 16
JJ-AZ 2
JJ-NV 2
MD-JJ 2

Survival plot

We estimate the survival function using the Kaplan-Meier estimator and represent this function visually using a Kaplan-Meier curve, showing the probability of not getting infected by SARS-CoV-2 at a certain time after onset of follow-up. The survival function is estimated for the control and intervention group.


The cumulative incidence of the event (SARS-CoV-2 infection) was additionally plotted.


Survival (time-to-event)

The probability of not getting infected by SARS-CoV-2 beyond a certain time after onset of follow-up (survival function, estimated using the Kaplan-Meier estimator) is reported for different periods.

Strata Time Number at risk Cumulative sum of number of events Cumulative sum of number censored Survival Std. error Cumulative hazard Std. error cumulative hazard
Not fully vaccinated 0 755055 0 94 1.0000 0.00000 0.0000 0.00000
Not fully vaccinated 100 118538 5340 631888 0.9771 0.00035 0.0232 0.00035
Not fully vaccinated 200 23739 25915 705747 0.6894 0.00185 0.3712 0.00268
Not fully vaccinated 300 9975 29046 716114 0.5583 0.00264 0.5819 0.00472
Not fully vaccinated 400 3241 30428 721516 0.4680 0.00319 0.7581 0.00681
Not fully vaccinated 500 210 30607 724239 0.3885 0.00765 0.9438 0.01964
Fully vaccinated 0 754136 102 1013 0.9999 0.00001 0.0001 0.00001
Fully vaccinated 100 122427 2863 630447 0.9891 0.00023 0.0109 0.00023
Fully vaccinated 200 29524 16051 709808 0.8106 0.00151 0.2097 0.00186
Fully vaccinated 300 16511 17682 720934 0.7500 0.00202 0.2874 0.00269
Fully vaccinated 400 6282 18661 730346 0.7000 0.00246 0.3564 0.00352
Fully vaccinated 500 399 18838 735819 0.6177 0.00811 0.4813 0.01310


Median survival time

The median survival time is the time corresponding to a probability of not obtaining a SARS-CoV-2 infection probability of 0.5. (if NA, the probability of not obtaining a SARS-CoV-2 infection did not drop below 50%)

Characteristic Median survival (95% CI)
fully_vaccinated_bl
    FALSE 349 (342, 358)
    TRUE — (—, —)
Cox regression and estimation of the average treatment effect

A Cox regression model was built to examine the relationship between the distribution of the probability of not obtaining a SARS-CoV-2 infection (survival distribution) and completing a primary vaccination schedule (covariate). The Cox proportional hazards regression model was fitted with ‘fully_vaccinated_bl’ as a covariate and accounts for clustering within individuals (as one individual can be re-sampled as control).

A hazard ratio (HR) is computed for the covariate ‘fully_vaccinated_bl’. A hazard can be interpreted as the instantaneous rate of SARS-CoV-2 infections in individuals that are at risk for obtaining an infection (Cox proportional hazards regression assumes stable proportional hazards over time). A HR < 1 indicates reduced hazard of SARS-CoV-2 infection when having completed a primary vaccination schedule whereas a HR > 1 indicates an increased hazard of SARS-CoV-2 infection.

Parameter estimate SE coefficient Robust SE coefficient P-value Hazard Ratio (HR) (95% CI for HR)
fully_vaccinated_blTRUE -0.631 0.009 0.012 0 0.532 (0.52, 0.544)

The overall significance of the model is tested.

Test statistic Df P-value
Likelihood ratio test 4786.588 1 0
Wald test 2897.440 1 0
Score (logrank) test 4779.628 1 0
Robust score test 2322.623 1 0

Proportional hazards during the study period might be unlikely. As such, the RMST and RMTL ratios are additionally calculated, providing an alternative estimate for the the Average Treatment Effect (ATE), without requiring the proportional hazards assumption to be met.

Arm Measure Estimate SE CI.lower CI.upper
fully_vaccinated_bl==FALSE RMST 280.758 0.416 279.943 281.573
fully_vaccinated_bl==TRUE RMST 316.696 0.337 316.035 317.357
fully_vaccinated_bl==FALSE RMTL 84.242 0.416 83.427 85.057
fully_vaccinated_bl==TRUE RMTL 48.304 0.337 47.643 48.965
Measure Estimate CI.lower CI.upper p_value
RMST (fully_vaccinated_bl==TRUE)-(fully_vaccinated_bl==FALSE) 35.938 34.889 36.988 0
RMST (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 1.128 1.124 1.132 0
RMTL (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 0.573 0.564 0.583 0
Survival plot

We estimate the survival function using the Kaplan-Meier estimator and represent this function visually using a Kaplan-Meier curve, showing the probability of not getting infected by SARS-CoV-2 at a certain time after onset of follow-up. The survival function is estimated for the control and intervention group, within subsets defined by the vaccination schedule.


The cumulative incidence of the event (SARS-CoV-2 infection) was additionally plotted within subsets defined by the vaccination schedule.


Cox regression and estimation of the average treatment effect

A Cox regression model was built to examine the relationship between the distribution of the probability of not obtaining a SARS-CoV-2 infection (survival distribution) and completing a primary vaccination schedule (covariate), and whether this differs according to the administered vaccination schedule. A stratified Cox proportional hazards regression model was fitted with ‘fully_vaccinated_bl’ as a covariate, ‘vaccination_schedule_cd’ as a stratification factor, and accounting for clustering within individuals (as one individual can be re-sampled as control).

Parameter estimate SE coefficient Robust SE coefficient P-value Hazard Ratio (HR) (95% CI for HR)
fully_vaccinated_blTRUE -0.474 0.030 0.034 0.000 0.623 (0.581, 0.665)
fully_vaccinated_blTRUE:strata(vaccination_schedule_cd)AZ-BP -0.173 0.137 0.136 0.204 0.841 (0.616, 1.066)
fully_vaccinated_blTRUE:strata(vaccination_schedule_cd)BP-AZ -0.084 0.452 0.451 0.852 0.92 (0.107, 1.732)
fully_vaccinated_blTRUE:strata(vaccination_schedule_cd)BP-BP -0.154 0.032 0.036 0.000 0.857 (0.797, 0.918)
fully_vaccinated_blTRUE:strata(vaccination_schedule_cd)JJ- -0.255 0.062 0.065 0.000 0.775 (0.677, 0.873)
fully_vaccinated_blTRUE:strata(vaccination_schedule_cd)JJ-BP -0.328 0.071 0.074 0.000 0.72 (0.616, 0.825)
fully_vaccinated_blTRUE:strata(vaccination_schedule_cd)JJ-MD -0.225 0.071 0.074 0.002 0.798 (0.682, 0.915)
fully_vaccinated_blTRUE:strata(vaccination_schedule_cd)MD-MD -0.227 0.043 0.046 0.000 0.797 (0.725, 0.868)


The overall significance of the model is tested.

Test statistic Df P-value
Likelihood ratio test 4795.951 8 0
Wald test 2914.020 8 0
Score (logrank) test 4789.637 8 0
Robust score test 2342.611 8 0

Proportional hazards during the study period might be unlikely. As such, the RMST and RMTL ratios are additionally calculated, providing an alternative estimate for the the Average Treatment Effect (ATE), without requiring the proportional hazards assumption to be met.

Vaccination_schedule Arm Measure Estimate SE CI.lower CI.upper
BP-BP fully_vaccinated_bl==FALSE RMST 283.142 0.474 282.212 284.072
BP-BP fully_vaccinated_bl==TRUE RMST 317.235 0.387 316.478 317.993
BP-BP fully_vaccinated_bl==FALSE RMTL 81.858 0.474 80.928 82.788
BP-BP fully_vaccinated_bl==TRUE RMTL 47.765 0.387 47.007 48.522
BP-AZ fully_vaccinated_bl==FALSE RMST 267.002 23.201 221.528 312.476
BP-AZ fully_vaccinated_bl==TRUE RMST 317.534 16.707 284.789 350.278
BP-AZ fully_vaccinated_bl==FALSE RMTL 97.998 23.201 52.524 143.472
BP-AZ fully_vaccinated_bl==TRUE RMTL 47.466 16.707 14.722 80.211
MD-MD fully_vaccinated_bl==FALSE RMST 275.375 1.372 272.687 278.064
MD-MD fully_vaccinated_bl==TRUE RMST 314.208 1.138 311.977 316.439
MD-MD fully_vaccinated_bl==FALSE RMTL 89.625 1.372 86.936 92.313
MD-MD fully_vaccinated_bl==TRUE RMTL 50.792 1.138 48.561 53.023
AZ-AZ fully_vaccinated_bl==FALSE RMST 271.276 1.712 267.921 274.631
AZ-AZ fully_vaccinated_bl==TRUE RMST 307.061 1.532 304.059 310.064
AZ-AZ fully_vaccinated_bl==FALSE RMTL 93.724 1.712 90.369 97.079
AZ-AZ fully_vaccinated_bl==TRUE RMTL 57.939 1.532 54.936 60.941
AZ-BP fully_vaccinated_bl==FALSE RMST 256.153 7.025 242.384 269.921
AZ-BP fully_vaccinated_bl==TRUE RMST 292.088 6.626 279.100 305.075
AZ-BP fully_vaccinated_bl==FALSE RMTL 108.847 7.025 95.079 122.616
AZ-BP fully_vaccinated_bl==TRUE RMTL 72.912 6.626 59.925 85.900
JJ-MD fully_vaccinated_bl==FALSE RMST 280.234 2.818 274.711 285.758
JJ-MD fully_vaccinated_bl==TRUE RMST 329.165 1.970 325.304 333.026
JJ-MD fully_vaccinated_bl==FALSE RMTL 84.766 2.818 79.242 90.289
JJ-MD fully_vaccinated_bl==TRUE RMTL 35.835 1.970 31.974 39.696
JJ- fully_vaccinated_bl==FALSE RMST 273.604 2.379 268.941 278.267
JJ- fully_vaccinated_bl==TRUE RMST 317.537 1.879 313.854 321.220
JJ- fully_vaccinated_bl==FALSE RMTL 91.396 2.379 86.733 96.059
JJ- fully_vaccinated_bl==TRUE RMTL 47.463 1.879 43.780 51.146
JJ-BP fully_vaccinated_bl==FALSE RMST 275.907 2.774 270.470 281.344
JJ-BP fully_vaccinated_bl==TRUE RMST 328.134 1.956 324.300 331.969
JJ-BP fully_vaccinated_bl==FALSE RMTL 89.093 2.774 83.656 94.530
JJ-BP fully_vaccinated_bl==TRUE RMTL 36.866 1.956 33.031 40.700
Vaccination_schedule Measure Estimate CI.lower CI.upper p_value
BP-BP RMST (fully_vaccinated_bl==TRUE)-(fully_vaccinated_bl==FALSE) 34.093 32.894 35.293 0.000
BP-BP RMST (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 1.120 1.116 1.125 0.000
BP-BP RMTL (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 0.584 0.572 0.595 0.000
BP-AZ RMST (fully_vaccinated_bl==TRUE)-(fully_vaccinated_bl==FALSE) 50.532 -5.505 106.568 0.077
BP-AZ RMST (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 1.189 0.975 1.451 0.088
BP-AZ RMTL (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 0.484 0.211 1.112 0.087
MD-MD RMST (fully_vaccinated_bl==TRUE)-(fully_vaccinated_bl==FALSE) 38.833 35.339 42.326 0.000
MD-MD RMST (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 1.141 1.127 1.155 0.000
MD-MD RMTL (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 0.567 0.537 0.598 0.000
AZ-AZ RMST (fully_vaccinated_bl==TRUE)-(fully_vaccinated_bl==FALSE) 35.785 31.283 40.288 0.000
AZ-AZ RMST (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 1.132 1.114 1.150 0.000
AZ-AZ RMTL (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 0.618 0.580 0.658 0.000
AZ-BP RMST (fully_vaccinated_bl==TRUE)-(fully_vaccinated_bl==FALSE) 35.935 17.008 54.862 0.000
AZ-BP RMST (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 1.140 1.063 1.223 0.000
AZ-BP RMTL (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 0.670 0.538 0.833 0.000
JJ-MD RMST (fully_vaccinated_bl==TRUE)-(fully_vaccinated_bl==FALSE) 48.931 42.191 55.670 0.000
JJ-MD RMST (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 1.175 1.148 1.202 0.000
JJ-MD RMTL (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 0.423 0.373 0.479 0.000
JJ- RMST (fully_vaccinated_bl==TRUE)-(fully_vaccinated_bl==FALSE) 43.933 37.991 49.875 0.000
JJ- RMST (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 1.161 1.137 1.185 0.000
JJ- RMTL (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 0.519 0.473 0.570 0.000
JJ-BP RMST (fully_vaccinated_bl==TRUE)-(fully_vaccinated_bl==FALSE) 52.228 45.574 58.881 0.000
JJ-BP RMST (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 1.189 1.162 1.217 0.000
JJ-BP RMTL (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 0.414 0.367 0.467 0.000
Survival plot

We estimate the survival function using the Kaplan-Meier estimator and represent this function visually using a Kaplan-Meier curve, showing the probability of not getting infected by SARS-CoV-2 at a certain time after onset of follow-up. The survival function is estimated for the control and intervention group, within subsets defined by the NUTS3 residence area.


The cumulative incidence of the event (SARS-CoV-2 infection) was additionally plotted within subsets defined by the NUTS3 residence area.


Cox regression and estimation of the average treatment effect

A Cox regression model was built to examine the relationship between the distribution of the probability of not obtaining a SARS-CoV-2 infection (survival distribution) and completing a primary vaccination schedule (covariate), and whether this differs according to the area of residence (NUTS3 level). A stratified Cox proportional hazards regression model was fitted with ‘fully_vaccinated_bl’ as a covariate, ‘residence_area_cd’ as a stratification factor, and accounting for clustering within individuals (as one individual can be re-sampled as control).

Parameter estimate SE coefficient Robust SE coefficient P-value Hazard Ratio (HR) (95% CI for HR)
fully_vaccinated_blTRUE -0.681 0.021 0.027 0.000 0.506 (0.48, 0.533)
fully_vaccinated_blTRUE:strata(residence_area_cd)ES242 0.080 0.039 0.050 0.109 1.083 (0.978, 1.188)
fully_vaccinated_blTRUE:strata(residence_area_cd)ES243 0.058 0.024 0.030 0.051 1.06 (0.998, 1.122)


The overall significance of the model is tested.

Test statistic Df P-value
Likelihood ratio test 4799.421 3 0
Wald test 2909.140 3 0
Score (logrank) test 4792.951 3 0
Robust score test 2332.446 3 0

Proportional hazards during the study period might be unlikely. As such, the RMST and RMTL ratios are additionally calculated, providing an alternative estimate for the the Average Treatment Effect (ATE), without requiring the proportional hazards assumption to be met.

Residence_area Arm Measure Estimate SE CI.lower CI.upper
ES243 fully_vaccinated_bl==FALSE RMST 280.648 0.488 279.692 281.604
ES243 fully_vaccinated_bl==TRUE RMST 315.893 0.399 315.111 316.674
ES243 fully_vaccinated_bl==FALSE RMTL 84.352 0.488 83.396 85.308
ES243 fully_vaccinated_bl==TRUE RMTL 49.107 0.399 48.326 49.889
ES241 fully_vaccinated_bl==FALSE RMST 275.355 0.981 273.433 277.277
ES241 fully_vaccinated_bl==TRUE RMST 315.780 0.791 314.230 317.329
ES241 fully_vaccinated_bl==FALSE RMTL 89.645 0.981 87.723 91.567
ES241 fully_vaccinated_bl==TRUE RMTL 49.220 0.791 47.671 50.770
ES242 fully_vaccinated_bl==FALSE RMST 291.877 1.353 289.226 294.528
ES242 fully_vaccinated_bl==TRUE RMST 324.385 1.054 322.319 326.450
ES242 fully_vaccinated_bl==FALSE RMTL 73.123 1.353 70.472 75.774
ES242 fully_vaccinated_bl==TRUE RMTL 40.615 1.054 38.550 42.681
Residence_area Measure Estimate CI.lower CI.upper p_value
ES243 RMST (fully_vaccinated_bl==TRUE)-(fully_vaccinated_bl==FALSE) 35.245 34.010 36.479 0
ES243 RMST (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 1.126 1.121 1.130 0
ES243 RMTL (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 0.582 0.571 0.594 0
ES241 RMST (fully_vaccinated_bl==TRUE)-(fully_vaccinated_bl==FALSE) 40.425 37.956 42.894 0
ES241 RMST (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 1.147 1.137 1.157 0
ES241 RMTL (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 0.549 0.529 0.570 0
ES242 RMST (fully_vaccinated_bl==TRUE)-(fully_vaccinated_bl==FALSE) 32.507 29.146 35.868 0
ES242 RMST (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 1.111 1.099 1.124 0
ES242 RMTL (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 0.555 0.522 0.591 0