DeSide: A unified deep learning approach for cellular decomposition of bulk tumors based on limited scRNA-seq data

DeSide_workflow

This is the whole workflow and results of the DeSide project, which includes the following parts:

•  main_workflow.py: the main workflow of the DeSide project
• datasets/*: the datasets used in the DeSide project, including the TCGA dataset, the simulated bulk cell dataset, and the single cell dataset
• model_evaluation/*: the predicted cell proportions of different methods for performance evaluation
• results/*: the results of the DeSide project, including the predicted cell proportions by DeSide with different datasets
• plot_fig/*: the scripts for plotting the figures in the DeSide project
• single_cell_dataset_integration/*: the scripts for integrating the single cell datasets

The file structure of the project

DeSide_workflow
├── LICENSE
├── README.md
├── main_workflow.py
├── TCGA_dataset_preprocessing.py
├── datasets
│   ├── Independent_Test_Set
│   ├── TCGA
│   │   └── tpm
│   │       ├── merged_tpm.csv
│   │       └── tcga_sample_id2cancer_type.csv
│   ├── cancer_purity
│   │   └── cancer_purity.csv
│   ├── generated_sc_dataset_7ds_n_base100
│   │   ├── generated_frac_SCT_POS_N100.csv
│   │   ├── generated_frac_SCT_POS_N10K.csv
│   │   ├── simu_bulk_exp_SCT_POS_N100_log2cpm1p.h5ad
│   │   └── simu_bulk_exp_SCT_POS_N10K_log2cpm1p.h5ad
│   ├── simulated_bulk_cell_dataset
│   │   ├── README.md
│   │   ├── random_7ds_n_base100
│   │   │   ├── generated_frac_Mixed_N100K_random.csv
│   │   │   ├── simu_bulk_exp_Mixed_N100K_random_log2cpm1p.h5ad
│   │   │   └── simu_bulk_exp_Test_set0_log2cpm1p.h5ad
│   │   └── segment_7ds_0.95_n_base100_median_gep
│   │       ├── D2
│   │       │   ├── corr_cell_frac_with_gene_exp_D2.csv
│   │       │   ├── corr_cell_frac_with_gene_exp_D2.xlsx
│   │       │   └── gene_list_filtered_by_high_corr_gene.csv
│   │       ├── Mixed_N100K_segment
│   │       │   ├── Mixed_N100K_segment_PCA_with_TCGA_high_corr_gene_and_quantile_range.csv
│   │       │   ├── Mixed_N100K_segment_PCA_with_TCGA_high_corr_gene_and_quantile_range_PC0_PC1.png
│   │       │   ├── Mixed_N100K_segment_PCA_with_TCGA_high_corr_gene_and_quantile_range_PC0_PC2.png
│   │       │   ├── Mixed_N100K_segment_PCA_with_TCGA_high_corr_gene_and_quantile_range_PC1_PC2.png
│   │       │   └── gene_list_filtered_by_high_corr_gene_and_quantile_range_q_5.0_q_95.0.csv
│   │       ├── simu_bulk_exp_Mixed_N100K_segment_log2cpm1p.h5ad
│   │       ├── simu_bulk_exp_Mixed_N100K_segment_log2cpm1p_filtered_by_high_corr_gene_and_quantile_range_q_5.0_q_95.0.h5ad
│   │       ├── simu_bulk_exp_Mixed_N100K_segment_without_filtering_log2cpm1p.h5ad
│   │       ├── simu_bulk_exp_Test_set1_log2cpm1p.csv
│   │       ├── simu_bulk_exp_Test_set1_log2cpm1p.h5ad
│   │       ├── simu_bulk_exp_Test_set1_sampled_sc_cell_id.csv
│   │       ├── simu_bulk_exp_Test_set2_log2cpm1p.csv
│   │       ├── simu_bulk_exp_Test_set2_log2cpm1p.h5ad
│   │       └── simu_bulk_exp_Test_set2_sampled_sc_cell_id.csv
│   └── single_cell
│       ├── Add_subgroup_name.ipynb
│       ├── Pan_cancer_cell_lines
│       │   ├── Metadata.txt
│       │   ├── Metadata.xlsx
│       │   └── pan_cancer_07_GSE157220_log2cpm1p_filtered.h5ad
│       ├── gene_list_in_merged_7_sc_datasets.csv
│       ├── merged_7_sc_datasets_log2cpm1p.h5ad
│       ├── merged_7_sc_datasets_sample_info.csv
│       ├── merged_7_sc_datasets_sample_info.xlsx
│       ├── merged_after_batch_correction
│       │   ├── 01_all_HNSCC_TIL_Cillo_Immunity_2020_merged_after_batch_correction.h5ad
│       │   ├── 02_PDAC_Peng_J_Cell_Res_2019_merged_after_batch_correction.h5ad
│       │   ├── 03_all_HNSCC_Puram_Cell_2017_merged_after_batch_correction.h5ad
│       │   ├── 04_PDAC_Steele_NCancer_2020_merged_after_batch_correction.h5ad
│       │   ├── 05_LUAD_primary_site_cancer_Kim_NCommun_2020_merged_after_batch_correction.h5ad
│       │   ├── 06_NSCLC_tumor_tissue_Guo_NMed_2018_merged_after_batch_correction.h5ad
│       │   └── 07_pan_cancer_07_GSE157220_log2cpm1p_filtered.h5ad
│       └── selected_marker_genes.csv
├── model_evaluation
│   ├── Fig.3a_compare_different_algos.py
│   ├── Fig.3c_predict_cell_prop_other_test_set.py
│   ├── cibersortx_preprocessing.py
│   └── predicted_cell_fractions
│       ├── CIBERSORT_result/*
│       ├── DeSide_result/*
│       ├── EPIC_1125/*
│       ├── Kassandra_result/*
│       ├── Scaden_result/*
│       └── merged_result
│           ├── CIBERSORT_hnscc_pred_cell_prop.csv
│           ├── CIBERSORT_luad_pred_cell_prop.csv
│           ├── DeSide_pred_cell_prop.csv
│           ├── DeSide_softmax_pred_cell_prop.csv
│           ├── EPIC_self_pred_cell_prop.csv
│           ├── Kassandra_self_pred_cell_prop.csv
│           ├── Scaden_ascites_pred_cell_prop.csv
│           └── Scaden_simu_bulk_pred_cell_prop.csv
├── plot_fig
│   ├── Fig.1 - PCA Plot.ipynb
│   ├── Fig.1.ipynb
│   ├── Fig.2.ipynb
│   ├── Fig.3.ipynb
│   ├── sup_figures/*
│   └── sup_figures.ipynb
├── results
│   └── whole_workflow_20230216_7ds_new_hp_without_caner_cell
│       ├── DeSide_009
│       │   ├── Mixed_N100K_random/*
│       │   ├── Mixed_N100K_random_1/*
│       │   ├── Mixed_N100K_random_2/*
│       │   ├── Mixed_N100K_random_3/*
│       │   ├── Mixed_N100K_random_4/*
│       │   ├── Mixed_N100K_segment_filtered/*
│       │   ├── Mixed_N100K_segment_filtered_1/*
│       │   ├── Mixed_N100K_segment_filtered_2/*
│       │   ├── Mixed_N100K_segment_filtered_3/*
│       │   ├── Mixed_N100K_segment_filtered_4/*
│       │   ├── Mixed_N100K_segment_filtered-Mixed_N100K_segment_without_filtering/*
│       │   ├── Mixed_N100K_segment_filtered_1-Mixed_N100K_segment_without_filtering/*
│       │   ├── Mixed_N100K_segment_filtered_2-Mixed_N100K_segment_without_filtering/*
│       │   ├── Mixed_N100K_segment_filtered_3-Mixed_N100K_segment_without_filtering/*
│       │   ├── Mixed_N100K_segment_filtered_4-Mixed_N100K_segment_without_filtering/*
│       │   ├── Mixed_N100K_segment_without_filtering/*
│       │   ├── Mixed_N100K_segment_without_filtering_1/*
│       │   ├── Mixed_N100K_segment_without_filtering_2/*
│       │   ├── Mixed_N100K_segment_without_filtering_3/*
│       │   ├── Mixed_N100K_segment_without_filtering_4/*
│       │   ├── Mixed_N200K_random/*
│       │   ├── Mixed_N200K_segment_filtered/*
│       │   └── Mixed_N200K_segment_without_filtering/*
│       ├── new_workflow_2_009.log
│       ├── new_workflow_2_009_2.log
│       ├── new_workflow_2_009_3.log
│       ├── new_workflow_2_009_3_random.log
│       ├── new_workflow_2_random_sampling1.log
│       └── new_workflow_2_random_sampling2.log
└── single_cell_dataset_integration
    ├── 01deal_with_Cillo et al Immunity.ipynb
    ├── 02deal_with_PengJ et al Cell Res.ipynb
    ├── 03deal_with_Puram et al Cell.ipynb
    ├── 04deal_with_Steele et al Nat Cancer.ipynb
    ├── 05deal_with_Kim et al Nat Commun.ipynb
    ├── 06deal_with_Guo et al Nat Med.ipynb
    ├── 07deal_with_pan_cancer_cell_lines.ipynb
    ├── 08filter_and_merge_01_06.py
    ├── 09add_pan_cancer_dataset.ipynb
    └── UMAP_of_D1.ipynb