ROC and AUC in plsda / splsda/ sgccda/ gccda

[Kim-Anh Le Cao]

For now: * ROC and AUC implementation as in shiny.

• comment code

• add pvalue

Now that we are moving towards diagnostic values with the supervised analyses, it would be worthwhile including ROC and AUC. We should consider a loo ROC or a bootstrap ROC. For multiclasses, we would need to discuss if we want to put HUM or not (another dependency ...)

This is not urgent but we should consider this soon.

[Kim-Anh Le Cao]

I am just adding to that issue some literature to choose an appropriate methodology (based on a review I received when handling a manuscript). We can discuss later.

BMC Bioinformatics. 2007 Sep 2;8:326. Stratification bias in low signal microarray studies. Parker BJ, Günter S, Bedo J.

and

Computational Statistics & Data Analysis Volume 55, Issue 4, 1 April 2011, Pages 1828–1844 An experimental comparison of cross-validation techniques for estimating the area under the ROC curve Antti Airola, Tapio Pahikkalaa, Willem Waegeman, Bernard De Baets, Tapio Salakoskia

Pattern Recognition Letters Volume 26, Issue 16, December 2005, Pages 2600–2610 Estimating the uncertainty in the estimated mean area under the ROC curve of a classifier Waleed A. Yousefa, Robert F. Wagnerb, Murray H. Loewa

addresses these issues, proposing the leave-pair-out bootstrap for AUC estimation via bootstrapping, that it would seem should be preferred over the bootstrapped leave-one-out.

discussing limitations of leave-on-out for AUC estimation, that result from the fact that AUC is a multivariate measure defined in terms of positive-negative pairs.

[Kim-Anh Le Cao]

Done for PLS and sPLS-DA (although we do not necessarily recommend using those measures, as PLS and PLS-DA use their own prediction threshold rules from the latent components. Results from the PLS-DA prediction and the AUC may differ).