This archive contains all code needed to reproduce the main and supplementary analysis for Chekroud et al., 2024 “Illusory Generalizability of Clinical Prediction Models”. 

The first 3 scripts (`01_preprocess_functions.R` through  `03_revised_ML_processing.R`) contain code to harmonize and process raw data from the formats stored on YODA to a format suitable for machine learning models. This includes harmonizing and rescoring items, imputing missing values, and creating other data scoring transformations such as the four outcome definitions and drug dose standardization.

The next 6 scripts (`04_CrossTrial_pipeline.R` through `09_RunBootstraps_and_SaveResults.R`) contain code to conduct modeling and summarize metrics and other statistics. Some of this code, in particular the leave-one-trial-out validations, can be quite time consuming because—at least at the time of analysis—only a single processor was available on YODA. We recommend that users adapting the code consider parallelization. 

The last script, `10_Plots_and_tables.Rmd`, reads the results and compiles them into the final tables and figures.

Code should be run in numerical order, as later scripts rely on functions included in earlier scripts. Note also that several interim datasets are created and you may need to change directories where data are stored and code is referenced to match your own directory structure.