Skip to contents

# Fit a classifier to feature matrix to extract top performers

Source: `R/fit_single_feature_classifier.R`
`fit_single_feature_classifier.Rd`

Fit a classifier to feature matrix to extract top performers

## Usage

    fit_single_feature_classifier(
      data,
      id_var = "id",
      group_var = "group",
      test_method = "gaussprRadial",
      use_balanced_accuracy = FALSE,
      use_k_fold = FALSE,
      num_folds = 10,
      use_empirical_null = FALSE,
      null_testing_method = c("ModelFreeShuffles", "NullModelFits"),
      p_value_method = c("empirical", "gaussian"),
      num_permutations = 50,
      pool_empirical_null = FALSE,
      seed = 123
    )

## Arguments

data

  the dataframe containing the raw feature matrix

id_var

  a string specifying the ID variable to group data on (if one exists). Defaults to `"id"`

group_var

  a string specifying the grouping variable that the data aggregates to. Defaults to `"group"`

test_method

  the algorithm to use for quantifying class separation. Defaults to `"gaussprRadial"`. Should be either `"t-test"`, `"wilcox"`, or `"binomial logistic"` for two-class problems to obtain exact statistics, or a valid `caret` classification model for everything else

use_balanced_accuracy

  a Boolean specifying whether to use balanced accuracy as the summary metric for caret model training. Defaults to `FALSE`

use_k_fold

  a Boolean specifying whether to use k-fold procedures for generating a distribution of classification accuracy estimates if a `caret` model is specified for `test_method`. Defaults to `FALSE`

num_folds

  an integer specifying the number of k-folds to perform if `use_k_fold` is set to `TRUE`. Defaults to `10`

use_empirical_null

  a Boolean specifying whether to use empirical null procedures to compute p-values if a `caret` model is specified for `test_method`. Defaults to `FALSE`

null_testing_method

  a string specifying the type of statistical method to use to calculate p-values. Defaults to `model free shuffles`

p_value_method

  a string specifying the method of calculating p-values. Defaults to `"empirical"`

num_permutations

  an integer specifying the number of class label shuffles to perform if `use_empirical_null` is `TRUE`. Defaults to `50`

pool_empirical_null

  a Boolean specifying whether to use the pooled empirical null distribution of all features or each features' individual empirical null distribution if a `caret` model is specified for `test_method` use_empirical_null is `TRUE`. Defaults to `FALSE`

seed

  fixed number for R's random number generator to ensure reproducibility

## Value

an object of class dataframe containing results

## Author

Trent Henderson

## Examples

    # \donttest{
    featMat <- calculate_features(data = simData,
      id_var = "id",
      time_var = "timepoint",
      values_var = "values",
      group_var = "process",
      feature_set = "catch22",
      seed = 123)
    #> No IDs removed. All value vectors good for feature extraction.
    #> Running computations for catch22...
    #>
    #> Calculations completed for catch22.

    # Mimic machinery of theft::compute_top_features
    # which calls fit_single_feature_classifier and
    # does these operations prior

    featMat$group <- make.names(featMat$group)
    featMat$group <- as.factor(featMat$group)
    featMat$values <- as.numeric(featMat$values)

    fit_single_feature_classifier(featMat,
      id_var = "id",
      group_var = "group",
      test_method = "gaussprRadial",
      use_balanced_accuracy = FALSE,
      use_k_fold = TRUE,
      num_folds = 10,
      use_empirical_null = TRUE,
      null_testing_method = "ModelFreeShuffles",
      p_value_method = "gaussian",
      num_permutations = 50,
      pool_empirical_null = FALSE,
      seed = 123)
    #> This will take a while. Great reason to go grab a coffee and relax ^_^
    #>                                                 feature  accuracy
    #> 1                           catch22_dn_histogram_mode_5 0.3611111
    #> 2                          catch22_dn_histogram_mode_10 0.3111111
    #> 3                                     catch22_co_f1ecac 0.7888889
    #> 4                               catch22_co_first_min_ac 0.6666667
    #> 5                     catch22_co_histogram_ami_even_2_5 0.6722222
    #> 6                                 catch22_co_trev_1_num 0.4388889
    #> 7                          catch22_md_hrv_classic_pnn40 0.4333333
    #> 8             catch22_sb_binary_stats_mean_longstretch1 0.6388889
    #> 9           catch22_sb_transition_matrix_3ac_sumdiagcov 0.5333333
    #> 10                   catch22_pd_periodicity_wang_th0_01 0.5111111
    #> 11         catch22_co_embed2_dist_tau_d_expfit_meandiff 0.6944444
    #> 12   catch22_in_auto_mutual_info_stats_40_gaussian_fmmi 0.5333333
    #> 13              catch22_fc_local_simple_mean1_tauresrat 0.6222222
    #> 14               catch22_dn_outlier_include_p_001_mdrmd 0.3222222
    #> 15               catch22_dn_outlier_include_n_001_mdrmd 0.3555556
    #> 16             catch22_sp_summaries_welch_rect_area_5_1 0.9500000
    #> 17            catch22_sb_binary_stats_diff_longstretch0 0.4333333
    #> 18                   catch22_sb_motif_three_quantile_hh 0.7777778
    #> 19 catch22_sc_fluct_anal_2_rsrangefit_50_1_logi_prop_r1 0.2722222
    #> 20      catch22_sc_fluct_anal_2_dfa_50_1_2_logi_prop_r1 0.2500000
    #> 21             catch22_sp_summaries_welch_rect_centroid 0.8722222
    #> 22                 catch22_fc_local_simple_mean3_stderr 0.8222222
    #>    p_value_accuracy classifier_name               statistic_name
    #> 1      2.646837e-10   gaussprRadial Mean classification accuracy
    #> 2      2.436996e-06   gaussprRadial Mean classification accuracy
    #> 3      2.120380e-91   gaussprRadial Mean classification accuracy
    #> 4      1.488492e-59   gaussprRadial Mean classification accuracy
    #> 5      7.518342e-61   gaussprRadial Mean classification accuracy
    #> 6      9.654280e-19   gaussprRadial Mean classification accuracy
    #> 7      4.783652e-18   gaussprRadial Mean classification accuracy
    #> 8      2.756663e-53   gaussprRadial Mean classification accuracy
    #> 9      9.659005e-33   gaussprRadial Mean classification accuracy
    #> 10     4.474297e-29   gaussprRadial Mean classification accuracy
    #> 11     3.515038e-66   gaussprRadial Mean classification accuracy
    #> 12     9.659005e-33   gaussprRadial Mean classification accuracy
    #> 13     1.068134e-49   gaussprRadial Mean classification accuracy
    #> 14     4.016075e-07   gaussprRadial Mean classification accuracy
    #> 15     8.304186e-10   gaussprRadial Mean classification accuracy
    #> 16    5.107928e-144   gaussprRadial Mean classification accuracy
    #> 17     4.783652e-18   gaussprRadial Mean classification accuracy
    #> 18     3.229540e-88   gaussprRadial Mean classification accuracy
    #> 19     4.928403e-04   gaussprRadial Mean classification accuracy
    #> 20     5.148504e-03   gaussprRadial Mean classification accuracy
    #> 21    4.182436e-117   gaussprRadial Mean classification accuracy
    #> 22    2.709871e-101   gaussprRadial Mean classification accuracy
    # }