ROC and AUC in plsda / splsda/ sgccda/ gccda

Issue #16 resolved
Kim-Anh Le Cao repo owner created an issue

For now:
* ROC and AUC implementation as in shiny.

  • comment code

  • add pvalue

Now that we are moving towards diagnostic values with the supervised analyses, it would be worthwhile including ROC and AUC.
We should consider a loo ROC or a bootstrap ROC.
For multiclasses, we would need to discuss if we want to put HUM or not (another dependency ...)

This is not urgent but we should consider this soon.

Comments (3)

  1. Kim-Anh Le Cao reporter

    I am just adding to that issue some literature to choose an appropriate methodology (based on a review I received when handling a manuscript). We can discuss later.

    BMC Bioinformatics. 2007 Sep 2;8:326.
    Stratification bias in low signal microarray studies.
    Parker BJ, Günter S, Bedo J.

    and

    Computational Statistics & Data Analysis
    Volume 55, Issue 4, 1 April 2011, Pages 1828–1844
    An experimental comparison of cross-validation techniques for estimating the area under the ROC curve
    Antti Airola, Tapio Pahikkalaa, Willem Waegeman, Bernard De Baets, Tapio Salakoskia

    Pattern Recognition Letters
    Volume 26, Issue 16, December 2005, Pages 2600–2610
    Estimating the uncertainty in the estimated mean area under the ROC curve of a classifier
    Waleed A. Yousefa, Robert F. Wagnerb, Murray H. Loewa

    addresses these issues, proposing the leave-pair-out bootstrap for AUC estimation via bootstrapping, that it would seem should be preferred over the bootstrapped leave-one-out.

    discussing limitations of leave-on-out for AUC estimation, that result from the fact that AUC is a multivariate measure defined in terms of positive-negative pairs.

  2. Kim-Anh Le Cao reporter

    Done for PLS and sPLS-DA (although we do not necessarily recommend using those measures, as PLS and PLS-DA use their own prediction threshold rules from the latent components. Results from the PLS-DA prediction and the AUC may differ).

  3. Log in to comment