Commits

Zoltan Szabo committed e50e170

Kernel on distributions (K) object type, expected and Bhattacharyya kernel estimation: added; see K_initialization.m', 'K_estimation.m', 'Kexpected_initialization.m', 'Kexpected_estimation.m', 'KBhattacharyya_kNN_k_initialization.m', 'KBhattacharyya_kNN_k_estimation.m'.

Comments (0)

Files changed (445)

+v0.40 (June 23, 2013):
+-Bhattacharyya kernel estimation based on k-nearest neighbors: added; see 'KBhattacharyya_kNN_k_initialization.m' and 'KBhattacharyya_kNN_k_estimation.m'.
+-Expected kernel estimation: added; see 'Kexpected_initialization.m', 'Kexpected_estimation.m'.
+-Directory 'H_I_D_A_C' renamed to 'H_I_D_A_C_K'; 'ITE_install.m' modified accordingly.
+-Kernel on distributions (K) object type: added; see 'K_initialization.m', 'K_estimation.m'. 
+
 v0.39 (June 12, 2013):
 -Symmetric Bregman distance estimation based on nonsymmetric Bregman distance: added; see 'DsymBregman_DBregman_initialization.m', 'DsymBregman_DBregman_estimation.m'.
 -Symmetric Bregman distance estimation based on k-nearest neighbors: added; see 'DsymBregman_kNN_k_initialization.m', 'DsymBregman_kNN_k_estimation.m'. 
 #ITE (Information Theoretical Estimators)
 
-ITE is capable of estimating many different variants of entropy, mutual information, divergence, association measures and cross quantities. Thanks to its highly modular design, ITE supports additionally 
+ITE is capable of estimating many different variants of entropy, mutual information, divergence, association measures, cross quantities, and kernels on distributions. Thanks to its highly modular design, ITE supports additionally 
 
 - the combinations of the estimation techniques, 
 - the easy construction and embedding of novel information theoretical estimators, and 
 - `mutual information (I)`: generalized variance, kernel canonical correlation analysis, kernel generalized variance, Hilbert-Schmidt independence criterion, Shannon mutual information, L2 mutual information, R�nyi mutual information, Tsallis mutual information, copula-based kernel dependency, multivariate version of Hoeffding's Phi, Schweizer-Wolff's sigma and kappa, complex mutual information, Cauchy-Schwartz quadratic mutual information, Euclidean distance based quadratic mutual information, distance covariance, distance correlation, approximate correntropy independence measure,
 - `divergence (D)`: Kullback-Leibler divergence (relative entropy, I directed divergence), L2 divergence, R�nyi divergence, Tsallis divergence, Hellinger distance, Bhattacharyya distance, maximum mean discrepancy (kernel distance), J-distance (symmetrised Kullback-Leibler divergence, J divergence), Cauchy-Schwartz divergence, Euclidean distance based divergence, energy distance (specially the Cramer-Von Mises distance), Jensen-Shannon divergence, Jensen-R�nyi divergence, K divergence, L divergence, certain f-divergences (Csisz�r-Morimoto divergence, Ali-Silvey distance), non-symmetric Bregman distance (Bregman divergence), Jensen-Tsallis divergence, symmetric Bregman distance,
 - `association measures (A)`, including `measures of concordance`: multivariate extensions of Spearman's rho (Spearman's rank correlation coefficient, grade correlation coefficient), correntropy, centered correntropy, correntropy coefficient, correntropy induced metric, centered correntropy induced metric, multivariate extension of Blomqvist's beta (medial correlation coefficient), multivariate conditional version of Spearman's rho, lower/upper tail dependence via conditional Spearman's rho,
-- `cross quantities (C)`: cross-entropy.
+- `cross quantities (C)`: cross-entropy,
+- `kernels on distributions (K)`: expected kernel, Bhattacharyya kernel.
 
 ITE offers solution methods for 
 
 
 - the evolution of the ITE code is briefly summarized in CHANGELOG.txt.
 - become a [Follower](https://bitbucket.org/szzoli/ite/follow) to be always up-to-date with ITE.
-- if you have an H/I/D/A/C estimator/subtask solver with a GPLv3(>=)-compatible license that you would like to be embedded into ITE, feel free to contact me.
+- if you have an H/I/D/A/C/K estimator/subtask solver with a GPLv3(>=)-compatible license that you would like to be embedded into ITE, feel free to contact me.
 
 **Download** the latest release: 
 
-- code: [zip](https://bitbucket.org/szzoli/ite/downloads/ITE-0.39_code.zip), [tar.bz2](https://bitbucket.org/szzoli/ite/downloads/ITE-0.39_code.tar.bz2), 
-- [documentation (pdf)](https://bitbucket.org/szzoli/ite/downloads/ITE-0.39_documentation.pdf).
+- code: [zip](https://bitbucket.org/szzoli/ite/downloads/ITE-0.40_code.zip), [tar.bz2](https://bitbucket.org/szzoli/ite/downloads/ITE-0.40_code.tar.bz2), 
+- [documentation (pdf)](https://bitbucket.org/szzoli/ite/downloads/ITE-0.40_documentation.pdf).
 
 

code/H_I_D_A_C/A_estimation.m

-function [A] = A_estimation(Y,ds,co)
-%Estimates association measure A(y^1,...,y^M), where the m^th subspace is ds(m)-dimensional; using the method/cost object co.
-%
-%INPUT:
-%   Y: Y(:,t) is the t^th sample.
-%  ds: subspace dimensions.
-%  co: association measure estimator object (structure).
-%
-%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
-%
-%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
-%
-%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
-%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
-%
-%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
-%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
-%
-%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
-
-%Here, we make use of the naming convention 'A<name>_estimation':
-    eval(['A=A',co.name,'_estimation(Y,ds,co);']);%example: A = ASpearman1_estimation(Y,ds,co);
-    

code/H_I_D_A_C/A_initialization.m

-function [co] = A_initialization(cost_name,mult)
-%Initialization of an A (association measure) estimator. The estimator is treated as a cost object (co). 
-%
-%INPUT:
-%   mult: is a multiplicative constant relevant (needed) in the estimation; '=1' means yes, '=0' no.
-%OUTPUT:
-%   co: cost object (structure).
-%
-%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
-%
-%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
-%
-%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
-%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
-%
-%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
-%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
-%
-%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
-
-%Here, we make use of the naming convention: 'A<name>_initialization':
-   eval(['co=A',cost_name,'_initialization(mult)']), %example: co = ASpearman1_initialization(mult);
-    
-disp('A initialization: ready.');

code/H_I_D_A_C/C_estimation.m

-function [C] = C_estimation(Y1,Y2,co)
-%Cross quantity estimation (C) of Y1 and Y2 using the specified estimator.
-%
-%INPUT:
-%  Y1: Y1(:,t) is the t^th sample from the first distribution.
-%  Y2: Y2(:,t) is the t^th sample from the second distribution. 
-%  co: cross quantity estimator object.
-%
-%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
-%
-%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
-%
-%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
-%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
-%
-%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
-%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
-%
-%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
-
-%Here, we make use of the naming convention 'C<name>_estimation':
-    eval(['C=C',co.name,'_estimation(Y1,Y2,co);']); %example: CE = CCE_kNN_k_estimation(Y1,Y2,co);

code/H_I_D_A_C/C_initialization.m

-function [co] = C_initialization(cost_name,mult)
-%Initialization of a cross quantity estimator. The estimator is treated as a cost object (co). 
-%   
-%INPUT:
-%   mult: is a multiplicative constant relevant (needed) in the estimation; '=1' means yes, '=0' no.
-%OUTPUT:
-%   co: cost object (structure).
-%
-%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
-%
-%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
-%
-%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
-%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
-%
-%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
-%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
-%
-%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
-
-%Here, we make use of the naming convention 'C<name>_initialization':
-   eval(['co=C',cost_name,'_initialization(mult)']); %example: co = CCE_kNN_k_initialization(mult);
-    
-disp('C initialization: ready.');

code/H_I_D_A_C/D_estimation.m

-function [D] = D_estimation(Y1,Y2,co)
-%Estimates divergence between Y1 and Y2.
-%
-%INPUT:
-%   Y1: Y1(:,t) is the t^th sample from the first distribution. 
-%   Y2: Y2(:,t) is the t^th sample from the second distribution.
-%   co: divergence estimator object (structure).
-%
-%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
-%
-%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
-%
-%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
-%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
-%
-%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
-%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
-%
-%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
-
-%Here, we make use of the naming convention 'D<name>_estimation':
-    eval(['D=D',co.name,'_estimation(Y1,Y2,co);']);%example: D = DRenyi_kNN_k_estimation(Y1,Y2,co);

code/H_I_D_A_C/D_initialization.m

-function [co] = D_initialization(cost_name,mult)
-%Initialization of a divergence estimator. The estimator is treated as a cost object (co). 
-%
-%INPUT:
-%   mult: is a multiplicative constant relevant (needed) in the estimation; '=1' means yes, '=0' no.
-%OUTPUT:
-%   co: cost object (structure).
-%
-%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
-%
-%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
-%
-%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
-%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
-%
-%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
-%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
-%
-%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
-
-%Here, we make use of the naming convention 'D<name>_initialization':
-   eval(['co=D',cost_name,'_initialization(mult)']); %example: co = DRenyi_kNN_k_initialization(mult);
-    
-disp('D initialization: ready.');

code/H_I_D_A_C/H_estimation.m

-function [H] = H_estimation(Y,co)
-%Entropy estimation (H) of Y using the specified entropy estimator.
-%
-%INPUT:
-%   Y: Y(:,t) is the t^th sample.
-%  co: entropy estimator object.
-%
-%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
-%
-%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
-%
-%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
-%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
-%
-%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
-%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
-%
-%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
-
-%Here, we make use of the naming convention 'H<name>_estimation':
-	eval(['H=H',co.name,'_estimation(Y,co);']);%example: H = HRenyi_kNN_1tok_estimation(Y,co);

code/H_I_D_A_C/H_initialization.m

-function [co] = H_initialization(cost_name,mult)
-%Initialization of an entropy estimator. The estimator is treated as a cost object (co). 
-%
-%INPUT:
-%   mult: is a multiplicative constant relevant (needed) in the estimation; '=1' means yes, '=0' no.
-%OUTPUT:
-%   co: cost object (structure).
-%
-%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
-%
-%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
-%
-%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
-%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
-%
-%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
-%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
-%
-%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
-
-%Here, we make use of the naming convention 'H<name>_initialization':
-   eval(['co=H',cost_name,'_initialization(mult)']); %example: co = HRenyi_kNN_k_initialization(mult);
-    
-disp('H initialization: ready.');

code/H_I_D_A_C/I_estimation.m

-function [I] = I_estimation(Y,ds,co)
-%Estimates mutual information I(y^1,...,y^M), where the m^th subspace is ds(m)-dimensional (I), using the specified estimator.
-%
-%INPUT:
-%   Y: Y(:,t) is the t^th sample.
-%  ds: subspace dimensions.
-%  co: mutual information estimator object (structure).
-%
-%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
-%
-%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
-%
-%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
-%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
-%
-%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
-%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
-%
-%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
-
-%Here, we make use of the naming convention 'I<name>_estimation':
-    eval(['I=I',co.name,'_estimation(Y,ds,co);']);%example: I = IShannon_HShannon_estimation(Y,ds,co);
-    

code/H_I_D_A_C/I_initialization.m

-function [co] = I_initialization(cost_name,mult)
-%Initialization of a I (mutual information) estimator. The estimator is treated as a cost object (co). 
-%
-%INPUT:
-%   mult: is a multiplicative constant relevant (needed) in the estimation; '=1' means yes, '=0' no.
-%OUTPUT:
-%   co: cost object (structure).
-%
-%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
-%
-%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
-%
-%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
-%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
-%
-%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
-%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
-%
-%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
-
-%Here, we make use of the naming convention 'I<name>_initialization':
-   eval(['co=I',cost_name,'_initialization(mult)']), %example: co = IGV_initialization(mult);
-    
-disp('I initialization: ready.');

code/H_I_D_A_C/base_estimators/ABlomqvist_estimation.m

-function [A] = ABlomqvist_estimation(Y,ds,co)
-%Estimates the multivariate extension of Blomqvist's beta (medial correlation coefficient). 
-%
-%We use the naming convention 'A<name>_estimation' to ease embedding new association estimator methods.
-%
-%INPUT:
-%   Y: Y(:,t) is the t^th sample.
-%  ds: subspace dimensions.
-%  co: association estimator object.
-%
-%REFERENCE: 
-%   Friedrich Schmid, Rafael Schmidt, Thomas Blumentritt, Sandra Gaiser, and Martin Ruppert. Copula Theory and Its Applications, Chapter Copula based Measures of Multivariate Association. Lecture Notes in Statistics. Springer, 2010. (multidimensional case, length(ds)>=2)
-%   Manuel Ubeda-Flores. Multivariate versions of Blomqvist's beta and Spearman's footrule. Annals of the Institute of Statistical Mathematics, 57:781-788, 2005.
-%   Nils Blomqvist. On a measure of dependence between two random variables. The Annals of Mathematical Statistics, 21:593-600, 1950. (2D case, statistical properties)
-%   Frederick Mosteller. On some useful ''inefficient'' statistics. Annals of Mathematical Statistics, 17:377--408, 1946. (2D case, def)
-%
-%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
-%
-%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
-%
-%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
-%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
-%
-%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
-%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
-%
-%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
-
-%co.mult:OK.
-
-%verification:
-    if sum(ds) ~= size(Y,1);
-        error('The subspace dimensions are not compatible with Y.');
-    end
-    if ~one_dimensional_problem(ds)
-        error('The subspaces must be one-dimensional for this estimator.');
-    end
-
-[d,num_of_samples] = size(Y);
-U = copula_transformation(Y);
-
-h = 2^(d-1) / (2^(d-1)-1); %h(d)
-C1 = mean(all(U<=1/2,1)); %C(1/2)
-C2 = mean(all(U>1/2,1)); %\bar{C}(1/2)
-A = h * ( C1 + C2 - 2^(1-d) );

code/H_I_D_A_C/base_estimators/ABlomqvist_initialization.m

-function [co] = ABlomqvist_initialization(mult)
-%Initialization of the estimator of the multivariate extension of Blomqvist's beta (medial correlation coefficient).
-%
-%Note:
-%   1)The estimator is treated as a cost object (co).
-%   2)We use the naming convention 'A<name>_initialization' to ease embedding new association estimator methods.
-%
-%INPUT:
-%   mult: is a multiplicative constant relevant (needed) in the estimation; '=1' means yes, '=0' no.
-%OUTPUT:
-%   co: cost object (structure).
-%
-%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
-%
-%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
-%
-%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
-%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
-%
-%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
-%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
-%
-%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
-
-%mandatory fields:
-    co.name = 'Blomqvist';
-    co.mult = mult;
- 

code/H_I_D_A_C/base_estimators/ACCorrEntr_KDE_Lapl_estimation.m

-function [A] = ACCorrEntr_KDE_Lapl_estimation(Y,ds,co)
-%Estimates the centered correntropy (A) using Laplacian KDE (kernel density estimation) + a sorting based trick.
-%
-%We use the naming convention 'A<name>_estimation' to ease embedding new association measure estimator methods.
-%
-%INPUT:
-%   Y: Y(:,t) is the t^th sample.
-%  ds: subspace dimensions.
-%  co: association measure estimator object.
-%
-%REFERENCE:
-%   Murali Rao, Sohan Seth, Jianwu Xu, Yunmei Chen, Hemant Tagare, and Jose C. Principe. A test of independence based on a generalized correlation function. Signal Processing, 91:15-27, 2011.
-%   Aiyou Chen. Fast kernel density independent component analysis. In Independent Component Analysis and Blind Signal Separation (ICA), pages 24-31, 2006. (sorting based trick)
-%
-%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
-%
-%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
-%
-%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
-%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
-%
-%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
-%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
-%
-%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
-
-%co.mult:OK.
-
-%verification:
-    if sum(ds) ~= size(Y,1);
-        error('The subspace dimensions are not compatible with Y.');
-    end
-    if ~one_dimensional_problem(ds) || length(ds)~=2
-        error('There must be 2 pieces of one-dimensional subspaces (coordinates) for this estimator.');
-    end
-    
-A = centcorrenexp(Y(1,:).',Y(2,:).',co.sigma);

code/H_I_D_A_C/base_estimators/ACCorrEntr_KDE_Lapl_initialization.m

-function [co] = ACCorrEntr_KDE_Lapl_initialization(mult)
-%Initialization of the Laplacian KDE (kernel density estimation) + a sorting trick based centered correntropy estimator.
-%
-%Note:
-%   1)The estimator is treated as a cost object (co).
-%   2)We use the naming convention 'A<name>_initialization' to ease embedding new association measure estimator methods.
-%
-%INPUT:
-%   mult: is a multiplicative constant relevant (needed) in the estimation; '=1' means yes, '=0' no.
-%OUTPUT:
-%   co: cost object (structure).
-%
-%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
-%
-%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
-%
-%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
-%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
-%
-%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
-%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
-%
-%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
-
-%mandatory fields:
-    co.name = 'CCorrEntr_KDE_Lapl';
-    co.mult = mult;
-    
-%other fields:
-   co.sigma = 1; %scaling parameter of the Laplacian kernel

code/H_I_D_A_C/base_estimators/ACCorrEntr_KDE_iChol_estimation.m

-function [A] = ACCorrEntr_KDE_iChol_estimation(Y,ds,co)
-%Estimates the centered correntropy (A) using Gaussian KDE (kernel density estimation) + incomplete Cholesky decomposition.
-%
-%We use the naming convention 'A<name>_estimation' to ease embedding new association measure estimator methods.
-%
-%INPUT:
-%   Y: Y(:,t) is the t^th sample.
-%  ds: subspace dimensions.
-%  co: association measure estimator object.
-%
-%REFERENCE:
-%   Murali Rao, Sohan Seth, Jianwu Xu, Yunmei Chen, Hemant Tagare, and Jose C. Principe. A test of independence based on a generalized correlation function. Signal Processing, 91:15-27, 2011.
-%   Sohan Seth and Jose C. Principe. On speeding up computation in information theoretic learning. In International Joint Conference on Neural Networks (IJCNN), pages 2883-2887, 2009.
-%
-%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
-%
-%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
-%
-%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
-%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
-%
-%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
-%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
-%
-%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
-
-%co.mult:OK.
-
-%verification:
-    if sum(ds) ~= size(Y,1);
-        error('The subspace dimensions are not compatible with Y.');
-    end
-    if ~one_dimensional_problem(ds) || length(ds)~=2
-        error('There must be 2 pieces of one-dimensional subspaces (coordinates) for this estimator.');
-    end
-
-A = centcorren(Y(1,:).',Y(2,:).',co.sigma);

code/H_I_D_A_C/base_estimators/ACCorrEntr_KDE_iChol_initialization.m

-function [co] = ACCorrEntr_KDE_iChol_initialization(mult)
-%Initialization of the Gaussian KDE (kernel density estimation) + incomplete Cholesky decomposition based centered correntropy estimator.
-%
-%Note:
-%   1)The estimator is treated as a cost object (co).
-%   2)We use the naming convention 'A<name>_initialization' to ease embedding new association measure estimator methods.
-%
-%INPUT:
-%   mult: is a multiplicative constant relevant (needed) in the estimation; '=1' means yes, '=0' no.
-%OUTPUT:
-%   co: cost object (structure).
-%
-%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
-%
-%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
-%
-%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
-%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
-%
-%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
-%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
-%
-%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
-
-%mandatory fields:
-    co.name = 'CCorrEntr_KDE_iChol';
-    co.mult = mult;
-    
-%other fields:    
-    co.sigma = 1;%std in the RBF (Gaussian) kernel

code/H_I_D_A_C/base_estimators/ACorrEntrCoeff_KDE_direct_estimation.m

-function [A] = ACorrEntrCoeff_KDE_direct_estimation(Y,ds,co)
-%Estimates the correntropy coefficient (A) directly using Gaussian KDE (kernel density estimation).
-%
-%We use the naming convention 'A<name>_estimation' to ease embedding new association measure estimator methods.
-%
-%INPUT:
-%   Y: Y(:,t) is the t^th sample.
-%  ds: subspace dimensions.
-%  co: association measure estimator object.
-%
-%REFERENCE:
-%   Murali Rao, Sohan Seth, Jianwu Xu, Yunmei Chen, Hemant Tagare, and Jose C. Principe. A test of independence based on a generalized correlation function. Signal Processing, 91:15-27, 2011.
-%
-%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
-%
-%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
-%
-%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
-%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
-%
-%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
-%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
-%
-%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
-
-%co.mult:OK.
-
-%verification:
-    if sum(ds) ~= size(Y,1);
-        error('The subspace dimensions are not compatible with Y.');
-    end
-    if ~one_dimensional_problem(ds) || length(ds)~=2
-        error('There must be 2 pieces of one-dimensional subspaces (coordinates) for this estimator.');
-    end
-
-A = correncoef_2(Y(1,:).',Y(2,:).',co.sigma);

code/H_I_D_A_C/base_estimators/ACorrEntrCoeff_KDE_direct_initialization.m

-function [co] = ACorrEntrCoeff_KDE_direct_initialization(mult)
-%Initialization of the Gaussian KDE (kernel density estimation) based direct correntropy coefficient estimator.
-%
-%Note:
-%   1)The estimator is treated as a cost object (co).
-%   2)We use the naming convention 'A<name>_initialization' to ease embedding new association measure estimator methods.
-%
-%INPUT:
-%   mult: is a multiplicative constant relevant (needed) in the estimation; '=1' means yes, '=0' no.
-%OUTPUT:
-%   co: cost object (structure).
-%
-%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
-%
-%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
-%
-%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
-%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
-%
-%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
-%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
-%
-%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
-
-%mandatory fields:
-    co.name = 'CorrEntrCoeff_KDE_direct';
-    co.mult = mult;
-    
-%other fields:    
-    co.sigma = 1;%std in the RBF (Gaussian) kernel
-   

code/H_I_D_A_C/base_estimators/ACorrEntrCoeff_KDE_iChol_estimation.m

-function [A] = ACorrEntrCoeff_KDE_iChol_estimation(Y,ds,co)
-%Estimates the correntropy coefficient (A) using Gaussian KDE (kernel density estimation) + incomplete Cholesky decomposition.
-%
-%We use the naming convention 'A<name>_estimation' to ease embedding new association measure estimator methods.
-%
-%INPUT:
-%   Y: Y(:,t) is the t^th sample.
-%  ds: subspace dimensions.
-%  co: association measure estimator object.
-%
-%REFERENCE:
-%   Murali Rao, Sohan Seth, Jianwu Xu, Yunmei Chen, Hemant Tagare, and Jose C. Principe. A test of independence based on a generalized correlation function. Signal Processing, 91:15-27, 2011.
-%   Sohan Seth and Jose C. Principe. On speeding up computation in information theoretic learning. In International Joint Conference on Neural Networks (IJCNN), pages 2883-2887, 2009.
-%
-%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
-%
-%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
-%
-%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
-%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
-%
-%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
-%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
-%
-%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
-
-%co.mult:OK.
-
-%verification:
-    if sum(ds) ~= size(Y,1);
-        error('The subspace dimensions are not compatible with Y.');
-    end
-    if ~one_dimensional_problem(ds) || length(ds)~=2
-        error('There must be 2 pieces of one-dimensional subspaces (coordinates) for this estimator.');
-    end
-
-A = correncoef(Y(1,:).',Y(2,:).',co.sigma);

code/H_I_D_A_C/base_estimators/ACorrEntrCoeff_KDE_iChol_initialization.m

-function [co] = ACorrEntrCoeff_KDE_iChol_initialization(mult)
-%Initialization of the Gaussian KDE (kernel density estimation) + incomplete Cholesky decomposition based correntropy coefficient estimator.
-%
-%Note:
-%   1)The estimator is treated as a cost object (co).
-%   2)We use the naming convention 'A<name>_initialization' to ease embedding new association measure estimator methods.
-%
-%INPUT:
-%   mult: is a multiplicative constant relevant (needed) in the estimation; '=1' means yes, '=0' no.
-%OUTPUT:
-%   co: cost object (structure).
-%
-%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
-%
-%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
-%
-%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
-%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
-%
-%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
-%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
-%
-%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
-
-%mandatory fields:
-    co.name = 'CorrEntrCoeff_KDE_iChol';
-    co.mult = mult;
-    
-%other fields:    
-    co.sigma = 1;%std in the RBF (Gaussian) kernel
-   

code/H_I_D_A_C/base_estimators/ACorrEntr_KDE_direct_estimation.m

-function [A] = ACorrEntr_KDE_direct_estimation(Y,ds,co)
-%Estimates the correntropy (A) directly using Gaussian KDE (kernel density estimation).
-%
-%We use the naming convention 'A<name>_estimation' to ease embedding new association measure estimator methods.
-%
-%INPUT:
-%   Y: Y(:,t) is the t^th sample.
-%  ds: subspace dimensions.
-%  co: association measure estimator object.
-%
-%REFERENCE:
-%   Murali Rao, Sohan Seth, Jianwu Xu, Yunmei Chen, Hemant Tagare, and Jose C. Principe. A test of independence based on a generalized correlation function. Signal Processing, 91:15-27, 2011.
-%
-%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
-%
-%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
-%
-%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
-%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
-%
-%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
-%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
-%
-%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
-
-%co.mult:OK.
-
-%verification:
-    if sum(ds) ~= size(Y,1);
-        error('The subspace dimensions are not compatible with Y.');
-    end
-    if ~one_dimensional_problem(ds) || length(ds)~=2
-        error('There must be 2 pieces of one-dimensional subspaces (coordinates) for this estimator.');
-    end
-    
-A = corren(Y(1,:).',Y(2,:).',co.sigma);    

code/H_I_D_A_C/base_estimators/ACorrEntr_KDE_direct_initialization.m

-function [co] = ACorrEntr_KDE_direct_initialization(mult)
-%Initialization of the Gaussian KDE (kernel density estimation) based direct correntropy estimator.
-%
-%Note:
-%   1)The estimator is treated as a cost object (co).
-%   2)We use the naming convention 'A<name>_initialization' to ease embedding new association measure estimator methods.
-%
-%INPUT:
-%   mult: is a multiplicative constant relevant (needed) in the estimation; '=1' means yes, '=0' no.
-%OUTPUT:
-%   co: cost object (structure).
-%
-%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
-%
-%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
-%
-%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
-%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
-%
-%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
-%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
-%
-%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
-
-%mandatory fields:
-    co.name = 'CorrEntr_KDE_direct';
-    co.mult = mult;
-    
-%other fields:    
-    co.sigma = 1; %std in the RBF (Gaussian) kernel

code/H_I_D_A_C/base_estimators/ASpearman1_estimation.m

-function [A] = ASpearman1_estimation(Y,ds,co)
-%Estimates the first multivariate extension of Spearman's rho using empirical copulas.
-%
-%We use the naming convention 'A<name>_estimation' to ease embedding new association measure estimator methods.
-%
-%INPUT:
-%   Y: Y(:,t) is the t^th sample.
-%  ds: subspace dimensions.
-%  co: association measure estimator object.
-%
-%REFERENCE:
-%   Friedrich Shmid, Rafael Schmidt, Thomas Blumentritt, Sandra Gaiser, and Martin Ruppert. Copula Theory and Its Applications, Chapter Copula based Measures of Multivariate Association. Lecture Notes in Statistics. Springer, 2010.
-%   Friedrich Schmid and Rafael Schmidt. Multivariate extensions of Spearman's rho and related statistics. Statistics & Probability Letters, 77:407-416, 2007.
-%   Roger B. Nelsen. Nonparametric measures of multivariate association. Lecture Notes-Monograph Series, Distributions with Fixed Marginals and Related Topics, 28:223-232, 1996.
-%   Edward F. Wolff. N-dimensional measures of dependence. Stochastica, 4:175-188, 1980.
-%   C. Spearman. The proof and measurement of association between two things. The American Journal of Psychology, 15:72-101, 1904.
-%
-%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
-%
-%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
-%
-%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
-%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
-%
-%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
-%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
-%
-%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
-
-%co.mult:OK.
-
-%verification:
-    if sum(ds) ~= size(Y,1);
-        error('The subspace dimensions are not compatible with Y.');
-    end
-    if ~one_dimensional_problem(ds)
-        error('The subspaces must be one-dimensional for this estimator.');
-    end
-    
-[d,num_of_samples] = size(Y); %dimension, number of samples
-U = copula_transformation(Y);
-
-h = (d+1) / (2^d - (d+1)); %h_rho(d)
-
-A = h * (2^d * mean(prod(1-U)) -1);
-

code/H_I_D_A_C/base_estimators/ASpearman1_initialization.m

-function [co] = ASpearman1_initialization(mult)
-%Initialization of the first multivariate extension of Spearman's rho estimator.
-%
-%Note:
-%   1)The estimator is treated as a cost object (co).
-%   2)We use the naming convention 'A<name>_initialization' to ease embedding new association measure estimator methods.
-%
-%INPUT:
-%   mult: is a multiplicative constant relevant (needed) in the estimation; '=1' means yes, '=0' no.
-%OUTPUT:
-%   co: cost object (structure).
-%
-%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
-%
-%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
-%
-%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
-%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
-%
-%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
-%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
-%
-%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
-
-%mandatory fields:
-    co.name = 'Spearman1';
-    co.mult = mult;
-    
-   

code/H_I_D_A_C/base_estimators/ASpearman2_estimation.m

-function [A] = ASpearman2_estimation(Y,ds,co)
-%Estimates the second multivariate extension of Spearman's rho using empirical copulas.
-%
-%We use the naming convention 'A<name>_estimation' to ease embedding new association measure estimator methods.
-%
-%INPUT:
-%   Y: Y(:,t) is the t^th sample.
-%  ds: subspace dimensions.
-%  co: association measure estimator object.
-%
-%REFERENCE:
-%   Friedrich Shmid, Rafael Schmidt, Thomas Blumentritt, Sandra Gaiser, and Martin Ruppert. Copula Theory and Its Applications, Chapter Copula based Measures of Multivariate Association. Lecture Notes in Statistics. Springer, 2010.
-%   Friedrich Schmid and Rafael Schmidt. Multivariate extensions of Spearman's rho and related statistics. Statistics & Probability Letters, 77:407-416, 2007.
-%   Roger B. Nelsen. Nonparametric measures of multivariate association. Lecture Notes-Monograph Series, Distributions with Fixed Marginals and Related Topics, 28:223-232, 1996.
-%   Harry Joe. Multivariate concordance. Journal of Multivariate Analysis, 35:12-30, 1990.
-%   C. Spearman. The proof and measurement of association between two things. The American Journal of Psychology, 15:72-101, 1904.
-%
-%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
-%
-%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
-%
-%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
-%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
-%
-%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
-%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
-%
-%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
-
-%co.mult:OK.
-
-%verification:
-    if sum(ds) ~= size(Y,1);
-        error('The subspace dimensions are not compatible with Y.');
-    end
-    if ~one_dimensional_problem(ds)
-        error('The subspaces must be one-dimensional for this estimator.');
-    end
-    
-[d,num_of_samples] = size(Y); %dimension, number of samples
-U = copula_transformation(Y);
-
-h = (d+1) / (2^d - (d+1)); %h_rho(d)
-
-A = h * (2^d * mean(prod(U)) -1);
-

code/H_I_D_A_C/base_estimators/ASpearman2_initialization.m

-function [co] = ASpearman2_initialization(mult)
-%Initialization of the second multivariate extension of Spearman's rho estimator.
-%
-%Note:
-%   1)The estimator is treated as a cost object (co).
-%   2)We use the naming convention 'A<name>_initialization' to ease embedding new association measure estimator methods.
-%
-%INPUT:
-%   mult: is a multiplicative constant relevant (needed) in the estimation; '=1' means yes, '=0' no.
-%OUTPUT:
-%   co: cost object (structure).
-%
-%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
-%
-%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
-%
-%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
-%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
-%
-%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
-%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
-%
-%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
-
-%mandatory fields:
-    co.name = 'Spearman2';
-    co.mult = mult;
-    
-   

code/H_I_D_A_C/base_estimators/ASpearman3_estimation.m

-function [A] = ASpearman3_estimation(Y,ds,co)
-%Estimates the second multivariate extension of Spearman's rho using empirical copulas.
-%
-%We use the naming convention 'A<name>_estimation' to ease embedding new association measure estimator methods.
-%
-%INPUT:
-%   Y: Y(:,t) is the t^th sample.
-%  ds: subspace dimensions.
-%  co: association measure estimator object.
-%
-%REFERENCE:
-%   Friedrich Shmid, Rafael Schmidt, Thomas Blumentritt, Sandra Gaiser, and Martin Ruppert. Copula Theory and Its Applications, Chapter Copula based Measures of Multivariate Association. Lecture Notes in Statistics. Springer, 2010.
-%   Roger B. Nelsen. An Introduction to Copulas (Springer Series in Statistics). Springer, 2006.
-%   Roger B. Nelsen. Distributions with Given Marginals and Statistical Modelling, chapter Concordance and copulas: A survey, pages 169-178. Kluwer Academic Publishers, Dordrecht, 2002.
-%   C. Spearman. The proof and measurement of association between two things. The American Journal of Psychology, 15:72-101, 1904.
-%
-%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
-%
-%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
-%
-%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
-%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
-%
-%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
-%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
-%
-%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
-
-%co.mult:OK.
-
-%verification:
-    if sum(ds) ~= size(Y,1);
-        error('The subspace dimensions are not compatible with Y.');
-    end
-    if ~one_dimensional_problem(ds)
-        error('The subspaces must be one-dimensional for this estimator.');
-    end
-    
-[d,num_of_samples] = size(Y); %dimension, number of samples
-U = copula_transformation(Y);
-
-h = (d+1) / (2^d - (d+1)); %h_rho(d)
-
-A1 = h * (2^d * mean(prod(1-U)) -1);
-A2 = h * (2^d * mean(prod(U)) -1);
-A = (A1+A2)/2;
-
-

code/H_I_D_A_C/base_estimators/ASpearman3_initialization.m

-function [co] = ASpearman3_initialization(mult)
-%Initialization of the third multivariate extension of Spearman's rho estimator.
-%
-%Note:
-%   1)The estimator is treated as a cost object (co).
-%   2)We use the naming convention 'A<name>_initialization' to ease embedding new association measure estimator methods.
-%
-%INPUT:
-%   mult: is a multiplicative constant relevant (needed) in the estimation; '=1' means yes, '=0' no.
-%OUTPUT:
-%   co: cost object (structure).
-%
-%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
-%
-%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
-%
-%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
-%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
-%
-%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
-%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
-%
-%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
-
-%mandatory fields:
-    co.name = 'Spearman3';
-    co.mult = mult;
-    
-   

code/H_I_D_A_C/base_estimators/ASpearman4_estimation.m

-function [A] = ASpearman4_estimation(Y,ds,co)
-%Estimates the fourth multivariate extension of Spearman's rho, the average of pairwise Spearman's rho using empirical copulas.
-%
-%We use the naming convention 'A<name>_estimation' to ease embedding new association measure estimator methods.
-%
-%INPUT:
-%   Y: Y(:,t) is the t^th sample.
-%  ds: subspace dimensions.
-%  co: association measure estimator object.
-%
-%REFERENCE:
-%   Friedrich Schmid, Rafael Schmidt, Thomas Blumentritt, Sandra Gaiser, and Martin Ruppert. Copula Theory and Its Applications, Chapter Copula based Measures of Multivariate Association. Lecture Notes in Statistics. Springer, 2010.
-%   Friedrich Schmid and Rafael Schmidt. Multivariate extensions of Spearman's rho and related statistics. Statistics & Probability Letters, 77:407-416, 2007.
-%   Maurice G. Kendall. Rank correlation methods. London, Griffin, 1970.
-%   C. Spearman. The proof and measurement of association between two things. The American Journal of Psychology, 15:72-101, 1904.
-%
-%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
-%
-%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
-%
-%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
-%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
-%
-%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
-%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
-%
-%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
-
-%co.mult:OK.
-
-%verification:
-    if sum(ds) ~= size(Y,1);
-        error('The subspace dimensions are not compatible with Y.');
-    end
-    if ~one_dimensional_problem(ds)
-        error('The subspaces must be one-dimensional for this estimator.');
-    end
-    
-[d,num_of_samples] = size(Y); %dimension, number of samples
-U = copula_transformation(Y);
-
-M_triu = triu(ones(d),1);%upper triangular mask
-b = nchoosek(d,2);
-A = 12 * sum(sum(((1-U)*(1-U).').*M_triu)) / (b*num_of_samples) -3;
-

code/H_I_D_A_C/base_estimators/ASpearman4_initialization.m

-function [co] = ASpearman4_initialization(mult)
-%Initialization of the fourth multivariate extension of Spearman's rho estimator, the average of pairwise Spearman's rho.
-%
-%Note:
-%   1)The estimator is treated as a cost object (co).
-%   2)We use the naming convention 'A<name>_initialization' to ease embedding new association measure estimator methods.
-%
-%INPUT:
-%   mult: is a multiplicative constant relevant (needed) in the estimation; '=1' means yes, '=0' no.
-%OUTPUT:
-%   co: cost object (structure).
-%
-%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
-%
-%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
-%
-%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
-%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
-%
-%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
-%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
-%
-%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
-
-%mandatory fields:
-    co.name = 'Spearman4';
-    co.mult = mult;
-    
-   

code/H_I_D_A_C/base_estimators/ASpearman_lt_estimation.m

-function [A] = ASpearman_lt_estimation(Y,ds,co)
-%Estimates the multivariate conditional version of Spearman's rho; the measure weights the lower tail of the copula. The estimation is based on the empirical copula.
-%
-%We use the naming convention 'A<name>_estimation' to ease embedding new association measure estimator methods.
-%
-%INPUT:
-%   Y: Y(:,t) is the t^th sample.
-%  ds: subspace dimensions.
-%  co: association measure estimator object.
-%
-%REFERENCE:
-%   Friedrich Schmid and Rafael Schmidt. Multivariate conditional versions of Spearman's rho and related measures of tail dependence. Journal of Multivariate Analysis, 98:1123-1140, 2007.
-%   C. Spearman. The proof and measurement of association between two things. The American Journal of Psychology, 15:72-101, 1904.
-%
-%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
-%
-%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
-%
-%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
-%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
-%
-%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
-%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
-%
-%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
-
-%co.mult:OK.
-
-%verification:
-    if sum(ds) ~= size(Y,1);
-        error('The subspace dimensions are not compatible with Y.');
-    end
-    if ~one_dimensional_problem(ds)
-        error('The subspaces must be one-dimensional for this estimator.');
-    end
-    
-[d,num_of_samples] = size(Y); %dimension, number of samples
-U = copula_transformation(Y);
-
-%constants:
-    p = co.p;
-    c1 = (p^2/2)^d;
-    c2 = p^(d+1)/(d+1);
-
-A = (mean(prod(max(p-U,0))) - c1) / (c2 - c1);
-

code/H_I_D_A_C/base_estimators/ASpearman_lt_initialization.m

-function [co] = ASpearman_lt_initialization(mult)
-%Initialization of the multivariate conditional version of Spearman's rho estimator; the measure weights the lower tail of the copula.
-%
-%Note:
-%   1)The estimator is treated as a cost object (co).
-%   2)We use the naming convention 'A<name>_initialization' to ease embedding new association measure estimator methods.
-%
-%INPUT:
-%   mult: is a multiplicative constant relevant (needed) in the estimation; '=1' means yes, '=0' no.
-%OUTPUT:
-%   co: cost object (structure).
-%
-%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
-%
-%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
-%
-%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
-%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
-%
-%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
-%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
-%
-%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
-
-%mandatory fields:
-    co.name = 'Spearman_lt';
-    co.mult = mult;
-    
-%other fields:
-    co.p = 1/2; %0<p<=1
-   

code/H_I_D_A_C/base_estimators/ASpearman_ut_estimation.m

-function [A] = ASpearman_ut_estimation(Y,ds,co)
-%Estimates the multivariate conditional version of Spearman's rho; the measure weights the upper tail of the copula. The estimation is based on the empirical copula.
-%
-%We use the naming convention 'A<name>_estimation' to ease embedding new association measure estimator methods.
-%
-%INPUT:
-%   Y: Y(:,t) is the t^th sample.
-%  ds: subspace dimensions.
-%  co: association measure estimator object.
-%
-%REFERENCE:
-%   Friedrich Schmid and Rafael Schmidt. Multivariate conditional versions of Spearman's rho and related measures of tail dependence. Journal of Multivariate Analysis, 98:1123-1140, 2007.
-%   C. Spearman. The proof and measurement of association between two things. The American Journal of Psychology, 15:72-101, 1904.
-%
-%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
-%
-%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
-%
-%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
-%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
-%
-%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
-%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
-%
-%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
-
-%co.mult:OK.
-
-%verification:
-    if sum(ds) ~= size(Y,1);
-        error('The subspace dimensions are not compatible with Y.');
-    end
-    if ~one_dimensional_problem(ds)
-        error('The subspaces must be one-dimensional for this estimator.');
-    end
-    
-[d,num_of_samples] = size(Y); %dimension, number of samples
-U = copula_transformation(Y);
-
-p = co.p;
-
-c = mean(prod(1 - max(U,1-p)));
-c1 = (p*(2-p)/2)^d;
-c2 = p^d * (d+1-p*d) / (d+1);
-
-A = (c-c1) / (c2-c1);
-

code/H_I_D_A_C/base_estimators/ASpearman_ut_initialization.m

-function [co] = ASpearman_ut_initialization(mult)
-%Initialization of the multivariate conditional version of Spearman's rho estimator; the measure weights the upper tail of the copula.
-%
-%Note:
-%   1)The estimator is treated as a cost object (co).
-%   2)We use the naming convention 'A<name>_initialization' to ease embedding new association measure estimator methods.
-%
-%INPUT:
-%   mult: is a multiplicative constant relevant (needed) in the estimation; '=1' means yes, '=0' no.
-%OUTPUT:
-%   co: cost object (structure).
-%
-%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
-%
-%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
-%
-%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
-%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
-%
-%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
-%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
-%
-%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
-
-%mandatory fields:
-    co.name = 'Spearman_ut';
-    co.mult = mult;
-    
-%other fields:
-    co.p = 1/2; %0<p<=1
-    
-   

code/H_I_D_A_C/base_estimators/CCE_kNN_k_estimation.m

-function [CE] = CCE_kNN_k_estimation(Y1,Y2,co)
-%Estimates the cross-entropy (CE) of Y1 and Y2 using the kNN method (S={k}).
-%
-%We use the naming convention 'C<name>_estimation' to ease embedding new cross quantity estimation methods.
-%
-%INPUT:
-%  Y1: Y1(:,t) is the t^th sample from the first distribution.
-%  Y2: Y2(:,t) is the t^th sample from the second distribution. Note: the number of samples in Y1 [=size(Y1,2)] and Y2 [=size(Y2,2)] can be different.
-%  co: cross quantity estimator object.
-%
-%REFERENCE: 
-%   Nikolai Leonenko, Luc Pronzato, and Vippal Savani. A class of Renyi information estimators for multidimensional densities. Annals of Statistics, 36(5):2153-2182, 2008.
-%
-%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
-%
-%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
-%
-%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
-%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
-%
-%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
-%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
-%
-%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
-
-%co.mult:OK.
-
-d1 = size(Y1,1);
-[d2,num_of_samples2] = size(Y2);
-
-%verification:
-    if d1~=d2
-        error('The dimension of the samples in Y1 and Y2 must be equal.');
-    end
-
-d = d1;
-V = volume_of_the_unit_ball(d); 
-squared_distancesY1Y2 = kNN_squared_distances(Y2,Y1,co,0);
-dist_k_Y1Y2 = sqrt(squared_distancesY1Y2(end,:));
-CE = log(V) + log(num_of_samples2) - psi(co.k) + d * mean(log(dist_k_Y1Y2));
-

code/H_I_D_A_C/base_estimators/CCE_kNN_k_initialization.m

-function [co] = CCE_kNN_k_initialization(mult)
-%Initialization of the kNN (k-nearest neighbor, S={k}) based cross-entropy estimator.
-%
-%Note:
-%   1)The estimator is treated as a cost object (co).
-%   2)We use the naming convention 'C<name>_initialization' to ease embedding new cross quantity estimation methods.
-%
-%INPUT: