Commits

Zoltan Szabo  committed fa6edf9

Correntropy, centered correntropy, correntropy coefficient, correntropy induced metric, centered correntropy induced metric, approximate correntropy independence measure estimators: added; see 'ACorrEntr_KDE_direct_initialization.m', 'ACorrEntr_KDE_direct_estimation.m', 'ACCorrEntr_KDE_iChol_initialization.m', 'ACCorrEntr_KDE_iChol_estimation.m', 'ACCorrEntr_KDE_Lapl_initialization.m', 'ACCorrEntr_KDE_Lapl_estimation.m', 'ACorrEntrCoeff_KDE_direct_initialization.m', 'ACorrEntrCoeff_KDE_direct_estimation.m', 'ACorrEntrCoeff_KDE_iChol_initialization.m', 'ACorrEntrCoeff_KDE_iChol_estimation.m', 'ACIM_initialization.m', 'ACIM_estimation.m', 'ACCIM_initialization.m', 'ACCIM_estimation.m', 'IApprCorrEntr_initialization.m', 'IApprCorrEntr_estimation.m'. Handling of identically constant random variables in distance correlation computation: included; see 'IdCor_estimation.m'.

  • Participants
  • Parent commits 98e0522
  • Tags release-0.27

Comments (0)

Files changed (39)

File CHANGELOG.txt

+v0.27 (Dec 28, 2012):
+-Approximate correntropy independence measure estimator: added; see 'IApprCorrEntr_initialization.m', 'IApprCorrEntr_estimation.m'.
+-Correntropy induced metric, centered correntropy induced metric estimators: added; see 'ACIM_initialization.m', 'ACIM_estimation.m', 'ACCIM_initialization.m', 'ACCIM_estimation.m'.
+-Correntropy, centered correntropy, correntropy coefficient estimators: added; see 'ACorrEntr_KDE_direct_initialization.m', 'ACorrEntr_KDE_direct_estimation.m', 'ACCorrEntr_KDE_iChol_initialization.m', 'ACCorrEntr_KDE_iChol_estimation.m', 'ACCorrEntr_KDE_Lapl_initialization.m', 'ACCorrEntr_KDE_Lapl_estimation.m', 'ACorrEntrCoeff_KDE_direct_initialization.m', 'ACorrEntrCoeff_KDE_direct_estimation.m', 'ACorrEntrCoeff_KDE_iChol_initialization.m', 'ACorrEntrCoeff_KDE_iChol_estimation.m'.
+-Some references fine-tuned/added (code+doc).
 -Handling of identically constant random variables in distance correlation computation: included; see 'IdCor_estimation.m'.
 
 v0.26 (Dec 22, 2012):
 ITE can estimate 
 
 - `entropy (H)`: Shannon entropy, R�nyi entropy, Tsallis entropy (Havrda and Charv�t entropy), complex entropy,
-- `mutual information (I)`: generalized variance, kernel canonical correlation analysis, kernel generalized variance, Hilbert-Schmidt independence criterion, Shannon mutual information, L2 mutual information, R�nyi mutual information, Tsallis mutual information, copula-based kernel dependency, multivariate version of Hoeffding's Phi, Schweizer-Wolff's sigma and kappa, complex mutual information, Cauchy-Schwartz quadratic mutual information, Euclidean distance based quadratic mutual information, distance covariance, distance correlation,
+- `mutual information (I)`: generalized variance, kernel canonical correlation analysis, kernel generalized variance, Hilbert-Schmidt independence criterion, Shannon mutual information, L2 mutual information, R�nyi mutual information, Tsallis mutual information, copula-based kernel dependency, multivariate version of Hoeffding's Phi, Schweizer-Wolff's sigma and kappa, complex mutual information, Cauchy-Schwartz quadratic mutual information, Euclidean distance based quadratic mutual information, distance covariance, distance correlation, approximate correntropy independence measure,
 - `divergence (D)`: Kullback-Leibler divergence (relative entropy), L2 divergence, R�nyi divergence, Tsallis divergence, Hellinger distance, Bhattacharyya distance, maximum mean discrepancy (kernel distance, an integral probability metric), J-distance (symmetrised Kullback-Leibler divergence), Cauchy-Schwartz divergence, Euclidean distance based divergence, energy distance (specially the Cramer-Von Mises distance),
-- `association measures (A)`, including `measures of concordance`: multivariate extensions of Spearman's rho (Spearman's rank correlation coefficient, grade correlation coefficient),
+- `association measures (A)`, including `measures of concordance`: multivariate extensions of Spearman's rho (Spearman's rank correlation coefficient, grade correlation coefficient), correntropy, centered correntropy, correntropy coefficient, correntropy induced metric, centered correntropy induced metric,
 - `cross quantities (C)`: cross-entropy.
 
 ITE offers solution methods for 
 
 **Download** the latest release: 
 
-- code: [zip](https://bitbucket.org/szzoli/ite/downloads/ITE-0.26_code.zip), [tar.bz2](https://bitbucket.org/szzoli/ite/downloads/ITE-0.26_code.tar.bz2), 
-- [documentation (pdf)](https://bitbucket.org/szzoli/ite/downloads/ITE-0.26_documentation.pdf).
+- code: [zip](https://bitbucket.org/szzoli/ite/downloads/ITE-0.27_code.zip), [tar.bz2](https://bitbucket.org/szzoli/ite/downloads/ITE-0.27_code.tar.bz2), 
+- [documentation (pdf)](https://bitbucket.org/szzoli/ite/downloads/ITE-0.27_documentation.pdf).
 
 

File code/H_I_D_A_C/A_estimation.m

 function [A] = A_estimation(Y,ds,co)
-%Estimates association A(y^1,...,y^M), where the m^th subspace is ds(m)-dimensional; using the method/cost object co.
+%Estimates association measure A(y^1,...,y^M), where the m^th subspace is ds(m)-dimensional; using the method/cost object co.
 %
 %INPUT:
 %   Y: Y(:,t) is the t^th sample.
 %  ds: subspace dimensions.
-%  co: association estimator object (structure).
+%  co: association measure estimator object (structure).
 %
 %Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
 %
 
 %Here, we make use of the naming convention 'A<name>_estimation':
     eval(['A=A',co.name,'_estimation(Y,ds,co);']);%example: A = ASpearman1_estimation(Y,ds,co);
-    
+    

File code/H_I_D_A_C/A_initialization.m

 function [co] = A_initialization(cost_name,mult)
-%Initialization of an A (association) estimator. The estimator is treated as a cost object (co). 
+%Initialization of an A (association measure) estimator. The estimator is treated as a cost object (co). 
 %
 %INPUT:
 %   mult: is a multiplicative constant relevant (needed) in the estimation; '=1' means yes, '=0' no.
 %Here, we make use of the naming convention: 'A<name>_initialization':
    eval(['co=A',cost_name,'_initialization(mult)']), %example: co = ASpearman1_initialization(mult);
     
-disp('A initialization: ready.');
+disp('A initialization: ready.');

File code/H_I_D_A_C/C_estimation.m

 function [C] = C_estimation(Y1,Y2,co)
-%Cross estimation (C) of Y1 and Y2 using the specified cross estimator.
+%Cross quantity estimation (C) of Y1 and Y2 using the specified estimator.
 %
 %INPUT:
 %  Y1: Y1(:,t) is the t^th sample from the first distribution.
 %  Y2: Y2(:,t) is the t^th sample from the second distribution. 
-%  co: cross estimator object.
+%  co: cross quantity estimator object.
 %
 %Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
 %

File code/H_I_D_A_C/C_initialization.m

 function [co] = C_initialization(cost_name,mult)
-%Initialization of a cross estimator. The estimator is treated as a cost object (co). 
+%Initialization of a cross quantity estimator. The estimator is treated as a cost object (co). 
 %   
 %INPUT:
 %   mult: is a multiplicative constant relevant (needed) in the estimation; '=1' means yes, '=0' no.

File code/H_I_D_A_C/base_estimators/ACCorrEntr_KDE_Lapl_estimation.m

+function [A] = ACCorrEntr_KDE_Lapl_estimation(Y,ds,co)
+%Estimates the centered correntropy (A) using Laplacian KDE (kernel density estimation) + a sorting based trick.
+%
+%We use the naming convention 'A<name>_estimation' to ease embedding new association measure estimator methods.
+%
+%INPUT:
+%   Y: Y(:,t) is the t^th sample.
+%  ds: subspace dimensions.
+%  co: association measure estimator object.
+%
+%REFERENCE:
+%   Murali Rao, Sohan Seth, Jianwu Xu, Yunmei Chen, Hemant Tagare, and Jose C. Principe. A test of independence based on a generalized correlation function. Signal Processing, 91:15-27, 2011.
+%   Aiyou Chen. Fast kernel density independent component analysis. In Independent Component Analysis and Blind Signal Separation (ICA), pages 24-31, 2006. (sorting based trick)
+%
+%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
+%
+%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
+%
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
+%
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+
+%co.mult:OK.
+
+%verification:
+    if sum(ds) ~= size(Y,1);
+        error('The subspace dimensions are not compatible with Y.');
+    end
+    if ~one_dimensional_problem(ds) || length(ds)~=2
+        error('There must be 2 pieces of one-dimensional subspaces (coordinates) for this estimator.');
+    end
+    
+A = centcorrenexp(Y(1,:).',Y(2,:).',co.sigma);

File code/H_I_D_A_C/base_estimators/ACCorrEntr_KDE_Lapl_initialization.m

+function [co] = ACCorrEntr_KDE_Lapl_initialization(mult)
+%Initialization of the Laplacian KDE (kernel density estimation) + a sorting trick based centered correntropy estimator.
+%
+%Note:
+%   1)The estimator is treated as a cost object (co).
+%   2)We use the naming convention 'A<name>_initialization' to ease embedding new association measure estimator methods.
+%
+%INPUT:
+%   mult: is a multiplicative constant relevant (needed) in the estimation; '=1' means yes, '=0' no.
+%OUTPUT:
+%   co: cost object (structure).
+%
+%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
+%
+%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
+%
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
+%
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+
+%mandatory fields:
+    co.name = 'CCorrEntr_KDE_Lapl';
+    co.mult = mult;
+    
+%other fields:
+   co.sigma = 1; %scaling parameter of the Laplacian kernel

File code/H_I_D_A_C/base_estimators/ACCorrEntr_KDE_iChol_estimation.m

+function [A] = ACCorrEntr_KDE_iChol_estimation(Y,ds,co)
+%Estimates the centered correntropy (A) using Gaussian KDE (kernel density estimation) + incomplete Cholesky decomposition.
+%
+%We use the naming convention 'A<name>_estimation' to ease embedding new association measure estimator methods.
+%
+%INPUT:
+%   Y: Y(:,t) is the t^th sample.
+%  ds: subspace dimensions.
+%  co: association measure estimator object.
+%
+%REFERENCE:
+%   Murali Rao, Sohan Seth, Jianwu Xu, Yunmei Chen, Hemant Tagare, and Jose C. Principe. A test of independence based on a generalized correlation function. Signal Processing, 91:15-27, 2011.
+%   Sohan Seth and Jose C. Principe. On speeding up computation in information theoretic learning. In International Joint Conference on Neural Networks (IJCNN), pages 2883-2887, 2009.
+%
+%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
+%
+%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
+%
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
+%
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+
+%co.mult:OK.
+
+%verification:
+    if sum(ds) ~= size(Y,1);
+        error('The subspace dimensions are not compatible with Y.');
+    end
+    if ~one_dimensional_problem(ds) || length(ds)~=2
+        error('There must be 2 pieces of one-dimensional subspaces (coordinates) for this estimator.');
+    end
+
+A = centcorren(Y(1,:).',Y(2,:).',co.sigma);

File code/H_I_D_A_C/base_estimators/ACCorrEntr_KDE_iChol_initialization.m

+function [co] = ACCorrEntr_KDE_iChol_initialization(mult)
+%Initialization of the Gaussian KDE (kernel density estimation) + incomplete Cholesky decomposition based centered correntropy estimator.
+%
+%Note:
+%   1)The estimator is treated as a cost object (co).
+%   2)We use the naming convention 'A<name>_initialization' to ease embedding new association measure estimator methods.
+%
+%INPUT:
+%   mult: is a multiplicative constant relevant (needed) in the estimation; '=1' means yes, '=0' no.
+%OUTPUT:
+%   co: cost object (structure).
+%
+%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
+%
+%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
+%
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
+%
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+
+%mandatory fields:
+    co.name = 'CCorrEntr_KDE_iChol';
+    co.mult = mult;
+    
+%other fields:    
+    co.sigma = 1;%std in the RBF (Gaussian) kernel

File code/H_I_D_A_C/base_estimators/ACorrEntrCoeff_KDE_direct_estimation.m

+function [A] = ACorrEntrCoeff_KDE_direct_estimation(Y,ds,co)
+%Estimates the correntropy coefficient (A) directly using Gaussian KDE (kernel density estimation).
+%
+%We use the naming convention 'A<name>_estimation' to ease embedding new association measure estimator methods.
+%
+%INPUT:
+%   Y: Y(:,t) is the t^th sample.
+%  ds: subspace dimensions.
+%  co: association measure estimator object.
+%
+%REFERENCE:
+%   Murali Rao, Sohan Seth, Jianwu Xu, Yunmei Chen, Hemant Tagare, and Jose C. Principe. A test of independence based on a generalized correlation function. Signal Processing, 91:15-27, 2011.
+%
+%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
+%
+%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
+%
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
+%
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+
+%co.mult:OK.
+
+%verification:
+    if sum(ds) ~= size(Y,1);
+        error('The subspace dimensions are not compatible with Y.');
+    end
+    if ~one_dimensional_problem(ds) || length(ds)~=2
+        error('There must be 2 pieces of one-dimensional subspaces (coordinates) for this estimator.');
+    end
+
+A = correncoef_2(Y(1,:).',Y(2,:).',co.sigma);

File code/H_I_D_A_C/base_estimators/ACorrEntrCoeff_KDE_direct_initialization.m

+function [co] = ACorrEntrCoeff_KDE_direct_initialization(mult)
+%Initialization of the Gaussian KDE (kernel density estimation) based direct correntropy coefficient estimator.
+%
+%Note:
+%   1)The estimator is treated as a cost object (co).
+%   2)We use the naming convention 'A<name>_initialization' to ease embedding new association measure estimator methods.
+%
+%INPUT:
+%   mult: is a multiplicative constant relevant (needed) in the estimation; '=1' means yes, '=0' no.
+%OUTPUT:
+%   co: cost object (structure).
+%
+%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
+%
+%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
+%
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
+%
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+
+%mandatory fields:
+    co.name = 'CorrEntrCoeff_KDE_direct';
+    co.mult = mult;
+    
+%other fields:    
+    co.sigma = 1;%std in the RBF (Gaussian) kernel
+   

File code/H_I_D_A_C/base_estimators/ACorrEntrCoeff_KDE_iChol_estimation.m

+function [A] = ACorrEntrCoeff_KDE_iChol_estimation(Y,ds,co)
+%Estimates the correntropy coefficient (A) using Gaussian KDE (kernel density estimation) + incomplete Cholesky decomposition.
+%
+%We use the naming convention 'A<name>_estimation' to ease embedding new association measure estimator methods.
+%
+%INPUT:
+%   Y: Y(:,t) is the t^th sample.
+%  ds: subspace dimensions.
+%  co: association measure estimator object.
+%
+%REFERENCE:
+%   Murali Rao, Sohan Seth, Jianwu Xu, Yunmei Chen, Hemant Tagare, and Jose C. Principe. A test of independence based on a generalized correlation function. Signal Processing, 91:15-27, 2011.
+%   Sohan Seth and Jose C. Principe. On speeding up computation in information theoretic learning. In International Joint Conference on Neural Networks (IJCNN), pages 2883-2887, 2009.
+%
+%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
+%
+%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
+%
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
+%
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+
+%co.mult:OK.
+
+%verification:
+    if sum(ds) ~= size(Y,1);
+        error('The subspace dimensions are not compatible with Y.');
+    end
+    if ~one_dimensional_problem(ds) || length(ds)~=2
+        error('There must be 2 pieces of one-dimensional subspaces (coordinates) for this estimator.');
+    end
+
+A = correncoef(Y(1,:).',Y(2,:).',co.sigma);

File code/H_I_D_A_C/base_estimators/ACorrEntrCoeff_KDE_iChol_initialization.m

+function [co] = ACorrEntrCoeff_KDE_iChol_initialization(mult)
+%Initialization of the Gaussian KDE (kernel density estimation) + incomplete Cholesky decomposition based correntropy coefficient estimator.
+%
+%Note:
+%   1)The estimator is treated as a cost object (co).
+%   2)We use the naming convention 'A<name>_initialization' to ease embedding new association measure estimator methods.
+%
+%INPUT:
+%   mult: is a multiplicative constant relevant (needed) in the estimation; '=1' means yes, '=0' no.
+%OUTPUT:
+%   co: cost object (structure).
+%
+%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
+%
+%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
+%
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
+%
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+
+%mandatory fields:
+    co.name = 'CorrEntrCoeff_KDE_iChol';
+    co.mult = mult;
+    
+%other fields:    
+    co.sigma = 1;%std in the RBF (Gaussian) kernel
+   

File code/H_I_D_A_C/base_estimators/ACorrEntr_KDE_direct_estimation.m

+function [A] = ACorrEntr_KDE_direct_estimation(Y,ds,co)
+%Estimates the correntropy (A) directly using Gaussian KDE (kernel density estimation).
+%
+%We use the naming convention 'A<name>_estimation' to ease embedding new association measure estimator methods.
+%
+%INPUT:
+%   Y: Y(:,t) is the t^th sample.
+%  ds: subspace dimensions.
+%  co: association measure estimator object.
+%
+%REFERENCE:
+%   Murali Rao, Sohan Seth, Jianwu Xu, Yunmei Chen, Hemant Tagare, and Jose C. Principe. A test of independence based on a generalized correlation function. Signal Processing, 91:15-27, 2011.
+%
+%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
+%
+%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
+%
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
+%
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+
+%co.mult:OK.
+
+%verification:
+    if sum(ds) ~= size(Y,1);
+        error('The subspace dimensions are not compatible with Y.');
+    end
+    if ~one_dimensional_problem(ds) || length(ds)~=2
+        error('There must be 2 pieces of one-dimensional subspaces (coordinates) for this estimator.');
+    end
+    
+A = corren(Y(1,:).',Y(2,:).',co.sigma);    

File code/H_I_D_A_C/base_estimators/ACorrEntr_KDE_direct_initialization.m

+function [co] = ACorrEntr_KDE_direct_initialization(mult)
+%Initialization of the Gaussian KDE (kernel density estimation) based direct correntropy estimator.
+%
+%Note:
+%   1)The estimator is treated as a cost object (co).
+%   2)We use the naming convention 'A<name>_initialization' to ease embedding new association measure estimator methods.
+%
+%INPUT:
+%   mult: is a multiplicative constant relevant (needed) in the estimation; '=1' means yes, '=0' no.
+%OUTPUT:
+%   co: cost object (structure).
+%
+%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
+%
+%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
+%
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
+%
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+
+%mandatory fields:
+    co.name = 'CorrEntr_KDE_direct';
+    co.mult = mult;
+    
+%other fields:    
+    co.sigma = 1; %std in the RBF (Gaussian) kernel

File code/H_I_D_A_C/base_estimators/ASpearman1_estimation.m

 function [A] = ASpearman1_estimation(Y,ds,co)
 %Estimates the first multivariate extension of Spearman's rho using empirical copulas.
 %
-%We use the naming convention 'A<name>_estimation' to ease embedding new association estimator methods.
+%We use the naming convention 'A<name>_estimation' to ease embedding new association measure estimator methods.
 %
 %INPUT:
 %   Y: Y(:,t) is the t^th sample.
 %  ds: subspace dimensions.
-%  co: association estimator object.
+%  co: association measure estimator object.
 %
 %REFERENCE:
 %   Friedrich Shmid, Rafael Schmidt, Thomas Blumentritt, Sandra Gaiser, and Martin Ruppert. Copula Theory and Its Applications, Chapter Copula based Measures of Multivariate Association. Lecture Notes in Statistics. Springer, 2010.

File code/H_I_D_A_C/base_estimators/ASpearman1_initialization.m

 %
 %Note:
 %   1)The estimator is treated as a cost object (co).
-%   2)We use the naming convention 'A<name>_initialization' to ease embedding new association estimator methods.
+%   2)We use the naming convention 'A<name>_initialization' to ease embedding new association measure estimator methods.
 %
 %INPUT:
 %   mult: is a multiplicative constant relevant (needed) in the estimation; '=1' means yes, '=0' no.

File code/H_I_D_A_C/base_estimators/ASpearman2_estimation.m

 function [A] = ASpearman2_estimation(Y,ds,co)
 %Estimates the second multivariate extension of Spearman's rho using empirical copulas.
 %
-%We use the naming convention 'A<name>_estimation' to ease embedding new association estimator methods.
+%We use the naming convention 'A<name>_estimation' to ease embedding new association measure estimator methods.
 %
 %INPUT:
 %   Y: Y(:,t) is the t^th sample.
 %  ds: subspace dimensions.
-%  co: association estimator object.
+%  co: association measure estimator object.
 %
 %REFERENCE:
 %   Friedrich Shmid, Rafael Schmidt, Thomas Blumentritt, Sandra Gaiser, and Martin Ruppert. Copula Theory and Its Applications, Chapter Copula based Measures of Multivariate Association. Lecture Notes in Statistics. Springer, 2010.

File code/H_I_D_A_C/base_estimators/ASpearman2_initialization.m

 %
 %Note:
 %   1)The estimator is treated as a cost object (co).
-%   2)We use the naming convention 'A<name>_initialization' to ease embedding new association estimator methods.
+%   2)We use the naming convention 'A<name>_initialization' to ease embedding new association measure estimator methods.
 %
 %INPUT:
 %   mult: is a multiplicative constant relevant (needed) in the estimation; '=1' means yes, '=0' no.

File code/H_I_D_A_C/base_estimators/ASpearman3_estimation.m

 function [A] = ASpearman3_estimation(Y,ds,co)
 %Estimates the second multivariate extension of Spearman's rho using empirical copulas.
 %
-%We use the naming convention 'A<name>_estimation' to ease embedding new association estimator methods.
+%We use the naming convention 'A<name>_estimation' to ease embedding new association measure estimator methods.
 %
 %INPUT:
 %   Y: Y(:,t) is the t^th sample.
 %  ds: subspace dimensions.
-%  co: association estimator object.
+%  co: association measure estimator object.
 %
 %REFERENCE:
 %   Friedrich Shmid, Rafael Schmidt, Thomas Blumentritt, Sandra Gaiser, and Martin Ruppert. Copula Theory and Its Applications, Chapter Copula based Measures of Multivariate Association. Lecture Notes in Statistics. Springer, 2010.

File code/H_I_D_A_C/base_estimators/ASpearman3_initialization.m

 %
 %Note:
 %   1)The estimator is treated as a cost object (co).
-%   2)We use the naming convention 'A<name>_initialization' to ease embedding new association estimator methods.
+%   2)We use the naming convention 'A<name>_initialization' to ease embedding new association measure estimator methods.
 %
 %INPUT:
 %   mult: is a multiplicative constant relevant (needed) in the estimation; '=1' means yes, '=0' no.

File code/H_I_D_A_C/base_estimators/CCE_kNN_k_estimation.m

 function [CE] = CCE_kNN_k_estimation(Y1,Y2,co)
 %Estimates the cross-entropy (CE) of Y1 and Y2 using the kNN method (S={k}).
 %
-%We use the naming convention 'C<name>_estimation' to ease embedding new cross estimation methods.
+%We use the naming convention 'C<name>_estimation' to ease embedding new cross quantity estimation methods.
 %
 %INPUT:
 %  Y1: Y1(:,t) is the t^th sample from the first distribution.
 %  Y2: Y2(:,t) is the t^th sample from the second distribution. Note: the number of samples in Y1 [=size(Y1,2)] and Y2 [=size(Y2,2)] can be different.
-%  co: cross estimator object.
+%  co: cross quantity estimator object.
 %
 %REFERENCE: 
 %   Nikolai Leonenko, Luc Pronzato, and Vippal Savani. A class of Renyi information estimators for multidimensional densities. Annals of Statistics, 36(5):2153-2182, 2008.

File code/H_I_D_A_C/base_estimators/CCE_kNN_k_initialization.m

 %
 %Note:
 %   1)The estimator is treated as a cost object (co).
-%   2)We use the naming convention 'C<name>_initialization' to ease embedding new cross estimation methods.
+%   2)We use the naming convention 'C<name>_initialization' to ease embedding new cross quantity estimation methods.
 %
 %INPUT:
 %   mult: is a multiplicative constant relevant (needed) in the estimation; '=1' means yes, '=0' no.

File code/H_I_D_A_C/meta_estimators/ACCIM_estimation.m

+function [A] = ACCIM_estimation(Y,ds,co)
+%Estimates the centered correntropy induced metric of Y1 and Y2 (A) using the relation CCIM(y^1,y^_2) = [CCE(y^1,y^1)+CCE(y^2,y^2)-2CCE(y^1,y^2)]^{1/2}, where CCE denotes centered correntropy.
+%
+%INPUT:
+%   Y: Y(:,t) is the t^th sample.
+%  ds: subspace dimensions.
+%  co: association measure estimator object.
+%
+%Note:
+%   1)We use the naming convention 'A<name>_estimation' to ease embedding new association measure estimator methods.
+%   2)This is a meta method: the centered correntropy estimator can be arbitrary.
+%
+%REFERENCE: 
+%   Murali Rao, Sohan Seth, Jianwu Xu, Yunmei Chen, Hemant Tagare, and Jose C. Principe. A test of independence based on a generalized correlation function. Signal Processing, 91:15-27, 2011.
+%
+%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
+%
+%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
+%
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
+%
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+
+%co.mult:OK.
+
+%verification:
+    if sum(ds) ~= size(Y,1);
+        error('The subspace dimensions are not compatible with Y.');
+    end
+    if ~one_dimensional_problem(ds) || length(ds)~=2
+        error('There must be 2 pieces of one-dimensional subspaces (coordinates) for this estimator.');
+    end
+
+A_Y1Y1 = A_estimation([Y(1,:);Y(1,:)],ds,co.member_co);
+A_Y2Y2 = A_estimation([Y(2,:);Y(2,:)],ds,co.member_co);
+A_Y1Y2 = A_estimation([Y(1,:);Y(2,:)],ds,co.member_co);
+
+%A =  sqrt(A_Y1Y1 + A_Y2Y2 - 2*A_Y1Y2);%theoretically
+A =  sqrt(abs(A_Y1Y1 + A_Y2Y2 - 2*A_Y1Y2));%abs(): to guarantee that the argument of sqrt is non-negative (due to the finite number of samples)
+

File code/H_I_D_A_C/meta_estimators/ACCIM_initialization.m

+function [co] = ACCIM_initialization(mult)
+%Initialization of the centered correntropy induced metric estimator, defined according to the relation CCIM(y^1,y^_2) = [CCE(y^1,y^1)+CCE(y^2,y^2)-2CCE(y^1,y^2)]^{1/2}, where CCE denotes centered correntropy.
+%
+%Note:
+%   1)The estimator is treated as a cost object (co).
+%   2)We use the naming convention 'A<name>_initialization' to ease embedding new association measure estimator methods.
+%
+%INPUT:
+%   mult: is a multiplicative constant relevant (needed) in the estimation; '=1' means yes, '=0' no.
+%OUTPUT:
+%   co: cost object (structure).
+%
+%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
+%
+%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
+%
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
+%
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+
+%mandatory fields:
+    co.name = 'CCIM';
+    co.mult = mult;
+    
+%other fields:
+    co.member_name = 'CCorrEntr_KDE_iChol'; %you can change it to any centered correntropy estimator
+    co.member_co = A_initialization(co.member_name,mult);

File code/H_I_D_A_C/meta_estimators/ACIM_estimation.m

+function [A] = ACIM_estimation(Y,ds,co)
+%Estimates the correntropy induced metric of Y1 and Y2 using the relation CIM(y^1,y^_2) = [k(0,0)-correntropy(y^1,y^2)]^{1/2}, where k is the applied (Gaussian) kernel. 
+%
+%We use the naming convention 'A<name>_estimation' to ease embedding new association measure estimator methods.
+%
+%INPUT:
+%   Y: Y(:,t) is the t^th sample.
+%  ds: subspace dimensions.
+%  co: association measure estimator object.
+%
+%Note:
+%   1)We use the naming convention 'A<name>_estimation' to ease embedding new association estimator methods.
+%   2)This is a meta method: the (Gaussian) correntropy estimator can be arbitrary.
+%
+%REFERENCE: 
+%   Sohan Seth and Jose C. Principe. Compressed signal reconstruction using the correntropy induced metric. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 3845-3848, 2008.
+%   Weifeng Liu, P.P. Pokharel, and Jose C. Principe. Correntropy: Properties and applications in non-Gaussian signal processing. IEEE Transactions on Signal Processing, 55:5286-5298, 2007.
+%
+%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
+%
+%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
+%
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
+%
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+
+%co.mult:OK.
+
+%verification:
+    if sum(ds) ~= size(Y,1);
+        error('The subspace dimensions are not compatible with Y.');
+    end
+    if ~one_dimensional_problem(ds) || length(ds)~=2
+        error('There must be 2 pieces of one-dimensional subspaces (coordinates) for this estimator.');
+    end
+
+A0 = A_estimation(Y,ds,co.member_co); %correntropy
+k0 = 1; %Assumption: k(u,u)=1. Example (Gaussian kernel): k(u,v)=e^{-(u-v)^2}, k(u,u)=1
+A =  sqrt(k0-A0);
+

File code/H_I_D_A_C/meta_estimators/ACIM_initialization.m

+function [co] = ACIM_initialization(mult)
+%Initialization of the correntropy induced metric estimator, defined according to the relation CIM(y^1,y^_2) = [k(0,0)-correntropy(y^1,y^2)]^{1/2}, where k is the applied (Gaussian) kernel.
+%
+%Note:
+%   1)The estimator is treated as a cost object (co).
+%   2)We use the naming convention 'A<name>_initialization' to ease embedding new association measure estimator methods.
+%
+%INPUT:
+%   mult: is a multiplicative constant relevant (needed) in the estimation; '=1' means yes, '=0' no.
+%OUTPUT:
+%   co: cost object (structure).
+%
+%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
+%
+%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
+%
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
+%
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+
+%mandatory fields:
+    co.name = 'CIM';
+    co.mult = mult;
+    
+%other fields:
+    co.member_name = 'CorrEntr_KDE_direct'; %you can change it to any (Gaussian kernel based) correntropy estimator
+    co.member_co = A_initialization(co.member_name,mult);

File code/H_I_D_A_C/meta_estimators/DEnergyDist_DMMD_estimation.m

 %
 %REFERENCE:
 %   Dino Sejdinovic, Arthur Gretton, Bharath Sriperumbudur, and Kenji Fukumizu. Hypothesis testing using pairwise distances and associated kernels. International Conference on Machine Learning (ICML), pages 1111-1118, 2012. (semimetric space; energy distance <=> MMD, with a suitable kernel)
-%   Russell Lyons. Distance Covariance in metric spaces. Technical report, Indiana University, 2011. http://arxiv.org/abs/1106.5758. (energy distance, metric space of negative type; pre-equivalence to MMD)
+%   Russell Lyons. Distance covariance in metric spaces. Annals of Probability, 2012. (To appear. http://php.indiana.edu/~rdlyons/pdf/dcov.pdf; http://arxiv.org/abs/1106.5758; energy distance, metric space of negative type; pre-equivalence to MMD).
 %   Gabor J. Szekely and Maria L. Rizzo. A new test for multivariate normality. Journal of Multivariate Analysis, 93:58-80, 2005. (energy distance; metric space of negative type)
 %   Gabor J. Szekely and Maria L. Rizzo. Testing for equal distributions in high dimension. InterStat, 5, 2004. (energy distance; R^d)
 %

File code/H_I_D_A_C/meta_estimators/IApprCorrEntr_estimation.m

+function [I] = IApprCorrEntr_estimation(Y,ds,co)
+%Estimates the approximate correntropy independence measure (I) based on centered correntropy, using the relation: I(y^1,y^2) = max(|CCorrEntr(y^1,y^2)|,|CCorrEntr(-y^1,y^2)|), where CCorrEntr denotes centered correntropy.
+%
+%Note:
+%   1)We use the naming convention 'I<name>_estimation' to ease embedding new mutual information estimation methods.
+%   2)This a meta method: the centered correntropy estimator can be arbitrary.
+%
+%INPUT:
+%   Y: Y(:,t) is the t^th sample.
+%  ds: subspace dimensions.
+%  co: mutual information estimator object.
+%
+%REFERENCE:
+%   Murali Rao, Sohan Seth, Jianwu Xu, Yunmei Chen, Hemant Tagare, and Jose C. Principe. A test of independence based on a generalized correlation function. Signal Processing, 91:15-27, 2011.
+%
+%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
+%
+%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
+%
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
+%
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+
+%co.mult:OK.
+
+%verification:
+    if sum(ds) ~= size(Y,1);
+        error('The subspace dimensions are not compatible with Y.');
+    end
+    if ~one_dimensional_problem(ds) || length(ds)~=2 
+        error('There must be 2 pieces of one-dimensional subspaces (coordinates) for this estimator.');
+    end
+
+A1 = A_estimation(Y,ds,co.member_co);%[Y1;Y2]
+A2 = A_estimation([-Y(1,:);Y(2,:)],ds,co.member_co);%[-Y1;Y2]
+I = max(abs(A1),abs(A2));
+

File code/H_I_D_A_C/meta_estimators/IApprCorrEntr_initialization.m

+function [co] = IApprCorrEntr_initialization(mult)
+%Initialization of the approximate correntropy independence measure estimator. The estimated quantity is computed according to the relation: I(y^1,y^2) = max(|CCorrEntr(y^1,y^2)|,|CCorrEntr(-y^1,y^2)|), where CCorrEntr denotes centered correntropy.
+%
+%Note:
+%   1)The estimator is treated as a cost object (co). 
+%   2)We use the naming convention 'I<name>_initialization' to ease embedding new mutual information estimation methods.
+%   3)This is a meta method: the centered correntropy estimator can be arbitrary.
+%
+%INPUT:
+%   mult: is a multiplicative constant relevant (needed) in the estimation; '=1' means yes, '=0' no.
+%OUTPUT:
+%   co: object (structure).
+%
+%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
+%
+%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
+%
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
+%
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+
+%mandatory fields:
+    co.name = 'ApprCorrEntr';
+    co.mult = mult;
+	
+%other fields:    
+    co.member_name = 'CCorrEntr_KDE_iChol'; %you can change it to any centered correntropy estimator
+    co.member_co = A_initialization(co.member_name,mult);
+ 

File code/H_I_D_A_C/meta_estimators/IdCov_IHSIC_estimation.m

 %
 %REFERENCE:
 %   Dino Sejdinovic, Arthur Gretton, Bharath Sriperumbudur, and Kenji Fukumizu. Hypothesis testing using pairwise distances and associated kernels. International Conference on Machine Learning (ICML), pages 1111-1118, 2012. (equivalence to HSIC)
-%   Russell Lyons. Distance Covariance in metric spaces. Technical report, Indiana University, 2011. http://arxiv.org/abs/1106.5758. (generalized distance covariance, rho_i; equivalence to HSIC)
+%   Russell Lyons. Distance covariance in metric spaces. Annals of Probability, 2012. (To appear. http://php.indiana.edu/~rdlyons/pdf/dcov.pdf; http://arxiv.org/abs/1106.5758; generalized distance covariance, rho_i; equivalence to HSIC).
 %   Gabor J. Szekely and Maria L. Rizzo and. Brownian distance covariance. The Annals of Applied Statistics, 3:1236-1265, 2009. (distance covariance)
 %   Gabor J. Szekely, Maria L. Rizzo, and Nail K. Bakirov. Measuring and testing dependence by correlation of distances. The Annals of Statistics, 35:2769-2794, 2007. (distance covariance)
 %

File code/IPA/demos/estimate_ISA.m

 %   de_hat: in case of known subspace dimensions ('unknown_dimensions = 0') de_hat = de; else it contains the estimated subspace dimensions; ordered increasingly.
 %
 %REFERENCE:
+%   Jason A. Palmer and Scott Makeig. Contrast functions for independent subspace analysis. In International conference on Latent Variable Analysis and Signal Separation (LVA/ICA), pages 115-122, 2012. (exciting, alternative proof idea for deflation methods)
 %   Zoltan Szabo, Barnabas Poczos, Andras Lorincz: Undercomplete Blind Subspace Deconvolution. Journal of Machine Learning Research 8(May):1063-1095, 2007. (proof; sufficient conditions for the ISA separation theorem)
 %   Jean-Francois Cardoso. Multidimensional independent component analysis. International Conference on Acoustics, Speech, and Signal Processing (ICASSP), pages 1941-1944, 1998. (conjecture)
 %   Lieven De Lathauwer, Bart De Moor, and Joos Vandewalle. Fetal electrocardiogram extraction by source subspace separation. In IEEE SP/Athos Workshop on Higher-Order Statistics, pages 134-138, 1995. ('conjecture')

File code/shared/embedded/ITL/centcorren.m

+% The following function computes the parametric centered correntropy between two vectors X and Y using Gaussian kernel.
+%
+% Input:   Both X and Y should be COLUMN vectors of SAME length (nx1).
+%				'kSize' is a scalar for the kernel size.
+%				'param' is a 2x1 vector containing the parameter coeffues 'a' and 'b' in ORDER.
+%
+% Output: 'corren' contains the parametric correntropy coefficient
+%
+% Default:  param = [a,b] = [1,0] and kSize = 1.
+%
+% Comments: The code uses Incomplete Cholesky Decomposition.
+%
+% Author: Sohan Seth (sohan@cnel.ufl.edu)	Date: 11.03.2008
+
+function corren = centcorren(X,Y,kSize,param)
+
+if nargin == 2
+	kSize = 1;
+	param = [1,0];
+end
+
+if nargin == 3
+	param = [1,0];
+end
+
+n = size(X,1);
+a = param(1); b = param(2);
+twokSizeSquare = 2*kSize^2;
+
+X = a*X+b;
+
+G =  incompleteCholeskyMulti([X;Y],kSize);
+OZ = [ones(n,1);zeros(n,1)]/n;
+ZO = [zeros(n,1);ones(n,1)]/n;
+corren = (1/n)*sum(exp(-(X - Y).^2/twokSizeSquare)) - (ZO'*G)*(G'*OZ);
+
+% ~ Done

File code/shared/embedded/ITL/centcorrenexp.m

+% The following function computes the parametric centered correntropy
+%               between two vectors X and Y using double exponential kernel
+%
+% Input:   X and Y are real valued COLUMN vectors of SAME length
+%				kSize is a scalar for the kernel size, i.e. K(x,y) = exp(-|x-y|/kSize),
+%				param is a [2x1] vector containing the coefficients a and b in ORDER.
+%
+% Output: corren contains the parametric centered correntropy
+%
+% Default:  param = [a,b] = [1,0] and kSize = 1.
+%
+% Comments: The code uses cipexp
+%
+% Author: Sohan Seth (sohan@cnel.ufl.edu)	Date: 15.07.2009
+
+function corren = centcorrenexp(X,Y,kSize,param)
+
+if nargin == 2
+	kSize = 1;
+	param = [1,0];
+end
+
+if nargin == 3
+	param = [1,0];
+end
+
+n = size(X,1);
+a = param(1); b = param(2);
+
+X = a*X+b;
+
+corren = (1/n)*sum(exp(-abs(X - Y)/kSize)) - cipexp(X,Y,kSize);

File code/shared/embedded/ITL/cipexp.m

+% The following function computes the cross information potential between
+%               two vectors X and Y using a double exponential kernel
+%
+% Input:   X and Y real valued COLUMN vectors of same length
+%				kSize is a scalar for the kernel size, i.e. K(x,y) =  exp(-|x-y|/kSize)
+%
+% Output: cip contains the cross information potential
+%
+% Default:  kSize = 1.
+%
+% Author: IL "Memming" Park (memming@cnel.ufl.edu)	Date: 15.07.2009
+
+function [cip] = cipexp(X,Y,kSize)
+
+if nargin == 2
+	kSize = 1;
+end
+
+X = X ./ kSize; Y = Y ./ kSize;
+
+X_sort = sort(X); Y_sort = sort(Y);
+
+X_pos_exp = exp(X_sort); X_neg_exp = exp(-X_sort);
+Y_pos_exp = exp(Y_sort); Y_neg_exp = exp(-Y_sort);
+
+Y_pos_cum_sum_exp  = cumsum(Y_pos_exp);
+Y_neg_cum_sum_exp  = flipud(cumsum(flipud(Y_neg_exp)));
+
+Y_sort = [Y_sort; Inf];
+cip = 0;
+yidx = 0; % no y is smaller than x yet
+for xidx = 1:length(X)
+    x = X_sort(xidx);
+
+    % at the end of the next Y_sort(yidx) >= X_sort(xidx)
+    while Y_sort(yidx+1) <= x
+        yidx = yidx + 1;
+    end
+
+    if yidx == 0
+        cip = cip + Y_neg_cum_sum_exp(1) * X_pos_exp(xidx);
+    elseif yidx == length(Y)
+        cip = cip + Y_pos_cum_sum_exp(end) * X_neg_exp(xidx);
+    else
+        cip = cip + Y_pos_cum_sum_exp(yidx) * X_neg_exp(xidx) + Y_neg_cum_sum_exp(yidx+1) * X_pos_exp(xidx);
+    end
+end
+cip = cip / (length(X) * length(Y));

File code/shared/embedded/ITL/corren.m

+% The following function computes the parametric correntropy between two vectors X and Y.
+%
+% Input:   Both X and Y should be COLUMN vectors of SAME length (nx1).
+%				'kSize' is a scalar for the kernel size.
+%				'param' is a 2x1 vector containing the parameter coeffues 'a' and 'b' in ORDER.
+%
+% Output: 'corren' contains the parametric correntropy coefficient
+%
+% Default:  param = [a,b] = [1,0] and kSize = 1.
+%
+% Comments: The code uses Incomplete Cholesky Decomposition.
+%
+% Author: Sohan Seth (sohan@cnel.ufl.edu)	Date: 11.03.2008
+
+function corren = corren(X,Y,kSize,param)
+
+if nargin == 2
+	kSize = 1;
+	param = [1,0];
+end
+
+if nargin == 3
+	param = [1,0];
+end
+
+n = size(X,1);
+a = param(1); b = param(2);
+twokSizeSquare = 2*kSize^2;
+
+X = a*X+b;
+corren = (1/n)*sum(exp(-(X - Y).^2/twokSizeSquare));
+
+% ~ Done

File code/shared/embedded/ITL/correncoef.m

+% The following function computes the parametric correntropy coefficient between two vectors X and Y.
+%
+% Input:   Both X and Y should be COLUMN vectors of SAME length (nx1).
+%				'kSize' is a scalar for the kernel size.
+%				'param' is a 2x1 vector containing the parameter coeffues 'a' and 'b' in ORDER.
+%
+% Output: 'coeff' contains the parametric correntropy coefficient
+%
+% Default:  param = [a,b] = [1,0] and kSize = 1.
+%
+% Caution: a ~= 0.
+%
+% Comments: The code uses Incomplete Cholesky Decomposition.
+%
+% Author: Sohan Seth (sohan@cnel.ufl.edu)	Date: 11.03.2008
+
+function coeff = correncoef(X,Y,kSize,param)
+
+if nargin == 2
+	kSize = 1;
+	param = [1,0];
+end
+
+if nargin == 3
+	param = [1,0];
+end
+
+n = size(X,1);
+a = param(1); b = param(2);
+twokSizeSquare = 2*kSize^2;
+
+if a == 0;
+	error('a must NOT be 0');
+end
+
+X = a*X+b;
+
+G =  incompleteCholeskyMulti([X;Y],kSize);
+OZ = [ones(n,1);zeros(n,1)]/n;
+ZO = [zeros(n,1);ones(n,1)]/n;
+corren = (1/n)*sum(exp(-(X - Y).^2/twokSizeSquare));
+coeff = (corren - (ZO'*G)*(G'*OZ)) /sqrt((1 - (OZ'*G)*(G'*OZ))*(1 - (ZO'*G)*(G'*ZO)));
+
+% ~ Done

File code/shared/embedded/ITL/correncoef_2.m

+% Direct computation of Correntropy coefficient
+%
+% Author : Sohan Seth (sohan@cnel.ufl.edu) Date : Date: 01.06.2009
+
+function val = correncoef_2(X,Y,kernelSize)
+
+n = length(X);
+Kxx = ones(n,n); Kyy = ones(n,n);
+for count1 = 1:n
+    for count2 = count1+1:n
+        Kxx(count1,count2) = exp(-(norm(X(count1,:) - X(count2,:)))^2/(2*kernelSize^2));
+        Kxx(count2,count1) = Kxx(count1,count2);
+        Kyy(count1,count2) = exp(-(norm(Y(count1,:) - Y(count2,:)))^2/(2*kernelSize^2));
+        Kyy(count2,count1) = Kyy(count1,count2);
+    end
+end
+
+Kxy = ones(n,n);
+for count1 = 1:n
+    for count2 =1:n
+        Kxy(count1,count2) = exp(-(norm(X(count1,:) - Y(count2,:)))^2/(2*kernelSize^2));
+    end
+end
+
+A  = sum(exp(-((X - Y)).^2/(2*kernelSize^2)))/n;
+
+val = (A - sum(Kxy(:))/n^2)/sqrt((1 - sum(Kxx(:))/n^2)*(1 - sum(Kyy(:))/n^2));