Mapping of Target Trial Emulations and Randomized Controlled Trials

PROTOCOL

TITLE: Mapping of Target Trial Emulations and Randomized Controlled Trials

AUTHORS: Karl Abou Chacra, Kenny Lim, Philippe Ravaud, Isabelle Boutron, Sally Yaacoub

AUTHOR AFFILIATIONS:

Université Paris Cité and Université Sorbonne Paris Nord, Inserm, INRAe, Centre for Research in Epidemiology and Statistics (CRESS), F-75004 Paris, France

Centre d'Épidémiologie Clinique, Hôpital Hôtel Dieu, AP-HP, 1 Parvis de Notre Dame, Université Paris Cité, 75004 Paris, France.

Cochrane France

CORRESPONDING AUTHOR:

Sally Yaacoub, BS Pharm, MPH, PhD

Université Paris Cité and Université Sorbonne Paris Nord, Inserm, INRAe

Center for Research in Epidemiology and Statistics (CRESS)

Cochrane France

Hôpital Hôtel Dieu, 1 place du Parvis Notre-Dame, 75004 Paris, France

Email: sally.yaacoub@u-paris.fr  

INTRODUCTION

Randomized controlled trials (RCTs) have been considered by researchers and clinicians to be the gold standard for establishing a causal relationship between a specific intervention and a respective outcome. Decision makers favor RCTs not just because of their inherent randomization, but also due to their ability to precisely control patient characteristics and health outcomes.1 However, not all clinical questions are suited to be answered by RCTs, either because of problems regarding funding, feasibility, or time related reasons. In addition, RCTs often have the problem of being applicable to only a very narrow patient population.

Thus, real world data (RWD) are especially relevant as they provide the evidence to answer clinical questions for which there is no existing or suitable RCT, as well as allow for a larger, more representative population to be studied. 2 Unfortunately, nonrandomized studies bring with them their own set of analytical problems: such as confounding bias, and the various associated assumptions. These assumptions include: exchangeability, where participants in the study may be exchanged between treatment and control groups without altering the average treatment effect; consistency, where only one version of the treatment is being tested; positivity, where every participant has a non-zero probability of receiving any treatment option under investigation given their measured covariates; and non- interference, where the treatment given to any one participant does not affect the outcomes of other participants.13

However, analyzing RWD in a framework designed to emulate RCTs as close as possible may mitigate some of the biases that would be present in the standard observational study. This design, the target trial emulation (TTE), involves specifying the protocol of the target trial and then emulating it as using observational data. It requires the eligibility criteria, treatment strategies, follow-up, and outcomes of interest to be prespecified in the protocol.3

There is an increasing amount TTEs currently being published due to their ability to provide evidence on larger patient populations, more diverse treatments, and longer outcomes (such as rare events and adverse events).2 Several studies, such as the RCT-DUPLICATE project, have been undertaken to assess the alignment of RWD specifically aiming to emulate a target trial with RCT findings, whereby the FDA selected 32 published RCTs of the treatment of various diseases, for which the corresponding RWD studies were conducted.12

TTEs are an important approach to validate and complement RCT findings, as they allow for the assessment of the external validity of the proposed causal pathway, especially over longer periods of time. Benchmarking TTEs against RCTs with similar causal questions may provide insight on their relevance in answering clinical questions, especially in the context where TTEs do not have any corresponding RCTs. This may help clinicians and readers to understand the scope of a TTE, and its role in addressing clinical questions.

To our knowledge, no study has mapped the evidence of existing observational studies specifically aiming to emulate a target trial with the evidence provided by randomized controlled trials.

Objectives

This study aims to map target trial emulations to randomized controlled trials and compare their characteristics and effect estimates.

METHODS

Study Design

This will be a meta-epidemiological study.

Identifying target trial emulations

Eligibility and search

We will use the same eligibility criteria for indication of a TTE as Hansford et. al in their systematic review.3 We will include studies that explicitly aim to emulate a target trial of a medical intervention, using the terms specified in the Appendix. We will exclude studies that do not investigate a medical intervention, that were not conducted in humans, or were not written in English. We will also exclude protocols and pre-prints.

We will search three electronic databases: Medline (Ovid), Embase, and Web of Science, using the search strategy reported by Hansford et al. in their systematic review (Appendix 3), however we will limit the search from database inception to the day of the search.3 Similarly, we will use CitationChaser for forward citation tracking of five main articles on target trial emulation framework.5, 6, 7, 8, 9 We will restrict the forward citation to Jan. 1, 2022 – Jan. 15, 2025, to complement the search of Hansford et al. We will also screen the references of relevant systematic reviews, including those of Hansford et al.

Screening and Classification

We will first do a preliminary screening on solely the title and abstract. This will be done by two independent reviewers in duplicate on Covidence. We will resolve disagreements through discussion, or the input of a third reviewer. With the identification of potentially eligible TTEs, we will then classify them into different medical fields and type of intervention. These tags are specified in the Appendix.

Following this, we will retrieve the full-texts of potentially eligible studies and two reviewers will independently screen them against the eligibility criteria. We will resolve disagreements through discussion, or the input of a third reviewer.

Identifying the clinical questions and eligibility criteria of target trial emulations

After the classification of each eligible study, we will extract the characteristics of the studies (the title, authors, and date of publication). We will also extract the components of the clinical question in the PICO format (participants, intervention, comparators, outcomes), and the eligibility criteria and setting.

Identifying matching randomized controlled trials

Using the clinical question of the each TTE, we will develop a search strategy to conduct a literature search using relevant terms and keywords. We aim to identify individual randomized controlled trials (RCTs) through relevant databases.  Following this, we will use the P, I and C components of the TTEs to retrieve potential matches with RCTs. Then, we will establish these matches using eligibility criteria of each TTE to screen the search hits and identify the relevant RCTs.

Data Extraction

For each TTE and RCT, we will extract:

-              The general characteristics e.g. authors, title, publication date, aim, region, setting, number of centers, the data used, the source of the data, recruitment duration, and overall follow up.

-              The participant characteristics:

o             target populations, condition, participants eligible, participants analyzed, number of centers, condition severity, demographic characteristics (e.g., age and gender distribution) and the eligibility criteria (inclusion and exclusion)

-              The intervention characteristics:

o             definition, description, drug, initiation/continuation status, number of participants, type of intervention, and assignment procedure

-              comparator characteristics

o             definition, drug, number of participants, and type of comparator

-              The outcome characteristics:

o             Primary and secondary outcomes of interest, duration of follow up for each outcome, statistical measure for each outcome, effect size, and confidence interval.

Additionally, the type of statistical analysis used to emulate the target trial (propensity score matching, inverse probability weighting methods…etc.) will be documented.  We will also document the confounding factors considered and methods for adjustment.  We will also note the causal contrast of interest for the RCTs.

Analysis

We will report the proportion of TTEs that do not have a match to any RCT, those that have only one matched RCT, and those that have several matched RCTs. We will conduct descriptive statistics to the characteristics among those included in the matched TTE and RCT pairs and highlight the differences between them. The characteristics to be compared are:

-              Participant characteristics:

o             disease severity (if applicable)

o             age and gender distribution

o             exclusion criteria

-              Sample size

-              Setting

-              Timeline of the study

-              Intervention characteristics:

o             Definition, type of intervention

-              Comparator characteristics:

o             Definition, type of comparator (e.g., active comparator, or standard care)

-              Outcome characteristics:

o             outcome of interest, the effect estimates

-              Duration of follow up.

Additionally, with respect to their effect estimates, we will report the proportion of matched TTEs and RCTs results that are concordant or discordant. If the results are binary, concordance is determined through the assessment of the relative risk ratio, calculated as the relative risk (RR) of the RCT over the RR of the TTE. If the resulting value is lower than 0.7 or greater than 1.43 (the reciprocal), this would signify extreme discordance between the results of the study.  if the difference in findings of the studies was due to more than chance.11 Additionally, if the outcomes of interest are continuous, the standardized mean differences will be compared. We will conduct hypothesis tests to evaluate whether there is a difference in findings by calculating the standardized difference between the RCT and RWE effect estimates

If the studies are discordant, we will attempt to identify the characteristics that make it so.10 Individual RCTs that are deemed similar enough to each other and matched to the same TTE will be meta-analytically combined using a random effects model.

We will use R software version 21 for the statistical analysis.

Implications

We will be able to identify the most common characteristics in TTEs, and that may lend to understanding which features of specific clinical questions can be suited to be answered by performing TTEs. In addition, through the comparison with RCTs and noting their concordance, we can understand the role TTEs play with respect to complimenting existing RCTs.

Data availability statement

All datasets will be publicly available on Zenodo.

Ethical considerations

Not applicable

Conflict of interest

There are no conflicts of interest.

Publication plan

We plan to publish the work as a peer-reviewed article.

Bibliography:

1.            Franklin JM, Schneeweiss S. When and How Can Real World Data Analyses Substitute for Randomized Controlled Trials? Clin Pharma and Therapeutics. 2017;102(6):924-933. doi:10.1002/cpt.857

2.            Yaacoub S, Porcher R, Pellat A, et al. Characteristics of non-randomised studies of drug treatments: cross sectional study. bmjmed. 2024;3(1):e000932. doi:10.1136/bmjmed-2024-000932

3.            Hansford HJ, Cashin AG, Jones MD, et al. Reporting of Observational Studies Explicitly Aiming to Emulate Randomized Trials: A Systematic Review. JAMA Netw Open. 2023;6(9):e2336023. doi:10.1001/jamanetworkopen.2023.36023

4.            Zuo H, Yu L, Campbell SM, Yamamoto SS, Yuan Y. The implementation of target trial emulation for causal inference: a scoping review. Journal of Clinical Epidemiology. 2023;162:29-37. doi:10.1016/j.jclinepi.2023.08.003

5.            Robins J. A new approach to causal inference in mortality studies with a sustained exposure period—application to control of the healthy worker survivor effect. Mathematical Modelling. 1986;7(9-12):1393-1512. doi:10.1016/0270-0255(86)90088-6

6.            Hernán MA, Sauer BC, Hernández-Díaz S, Platt R, Shrier I. Specifying a target trial prevents immortal time bias and other self-inflicted injuries in observational analyses. Journal of Clinical Epidemiology. 2016;79:70-75. doi:10.1016/j.jclinepi.2016.04.014

7.            Hernán MA, Robins JM. Using Big Data to Emulate a Target Trial When a Randomized Trial Is Not Available: Table 1. Am J Epidemiol. 2016;183(8):758-764. doi:10.1093/aje/kwv254

8.            Hernán MA. How to estimate the effect of treatment duration on survival outcomes using observational data. BMJ. Published online February 1, 2018:k182. doi:10.1136/bmj.k182

9.            Robins J, Hernan M. Estimation of the causal effects of time-varying exposures. In: Verbeke G, Davidian M, Fitzmaurice G, Molenberghs G, eds. Longitudinal Data Analysis. Vol 20085746. Chapman & Hall/CRC Handbooks of Modern Statistical Methods. Chapman and Hall/CRC; 2008:553-599. doi:10.1201/9781420011579.ch23

10.          Moneer O, Daly G, Skydel JJ, et al. Agreement of treatment effects from observational studies and randomized controlled trials evaluating hydroxychloroquine, lopinavir-ritonavir, or dexamethasone for covid-19: meta-epidemiological study. BMJ. Published online May 10, 2022:e069400. doi:10.1136/bmj-2021-069400

11.          Wang SV, Schneeweiss S, Franklin JM, et al. Emulation of Randomized Clinical Trials With Nonrandomized Database Analyses. JAMA. 2023;329(16):1376-1376. doi:https://doi.org/10.1001/jama.2023.4221

12.          Wang SV, Schneeweiss S, RCT-DUPLICATE Initiative, et al. Emulation of Randomized Clinical Trials With Nonrandomized Database Analyses: Results of 32 Clinical Trials. JAMA. 2023;329(16):1376. doi:10.1001/jama.2023.4221

13.          Bigirumurame T, Hiu SKW, Teare MD, Wason JMS, Bryant A, Breckons M. Current practices in studies applying the target trial emulation framework: a protocol for a systematic review. BMJ Open. 2023;13(6):e070963. doi:10.1136/bmjopen-2022-070963