Methodological Considerations on the External Validity of the Kim HJ et al. COVID-19 Vaccination Study (Biomark Res 13:114, 2025): A Quantitative Analysis

Despite the fundamental importance of retrospective studies in assessing the real-world impact of COVID-19 vaccination, many of these works employ cohort construction methodologies that do not adhere to the most basic rules of biostatistics, thus compromising the validity of their results. The objective of this study is to formally evaluate the methodology and test the external validity of a recent large-scale cohort study that reported a surprising and significantly higher 1-year cancer incidence risk in the COVID-19 vaccinated group. Aggregated raw data (n=2,975,035 individuals) from the cohort were used to calculate the overall Crude Incidence Rate CR of cancer. The resulting cohort CR was then compared against the established official national average CR for the reference period (2020–2022) to assess external validity. A secondary analysis employed the Chi-Squared Goodness-of-Fit Test to quantify the impact of the 1:4 Propensity Score Matching (PSM) on the age structure of the final cohort against the national demographic benchmark. The cohort's overall CR was 40.78 per 10,000, a substantial 22.26% downward deviation from the national average 52.46 per 10,000 (SD 2.97). This discrepancy establishes a pronounced epidemiological paradox, strongly indicating a lack of external validity. Furthermore, the Chi-Squared test revealed a profound structural alteration, with a value of 69,370 (p < 0.00001), confirming that the PSM procedure underrepresented the high-risk demographic group >= 65 years) compared to the national average (12.15% observed vs 18.00% expected). In conclusion, the reliability of the statistical associations reported by the scrutinized study are significantly challenged by the lack of external validity and the methodological ambiguities concerning the composition of the cohort. Independent validation is mandatory, necessitating the immediate public access to the underlying administrative health data sources