Commits

Zoltan Szabo committed fa6edf9

Correntropy, centered correntropy, correntropy coefficient, correntropy induced metric, centered correntropy induced metric, approximate correntropy independence measure estimators: added; see 'ACorrEntr_KDE_direct_initialization.m', 'ACorrEntr_KDE_direct_estimation.m', 'ACCorrEntr_KDE_iChol_initialization.m', 'ACCorrEntr_KDE_iChol_estimation.m', 'ACCorrEntr_KDE_Lapl_initialization.m', 'ACCorrEntr_KDE_Lapl_estimation.m', 'ACorrEntrCoeff_KDE_direct_initialization.m', 'ACorrEntrCoeff_KDE_direct_estimation.m', 'ACorrEntrCoeff_KDE_iChol_initialization.m', 'ACorrEntrCoeff_KDE_iChol_estimation.m', 'ACIM_initialization.m', 'ACIM_estimation.m', 'ACCIM_initialization.m', 'ACCIM_estimation.m', 'IApprCorrEntr_initialization.m', 'IApprCorrEntr_estimation.m'. Handling of identically constant random variables in distance correlation computation: included; see 'IdCor_estimation.m'.

Comments (0)

Files changed (39)

+v0.27 (Dec 28, 2012):
+-Approximate correntropy independence measure estimator: added; see 'IApprCorrEntr_initialization.m', 'IApprCorrEntr_estimation.m'.
+-Correntropy induced metric, centered correntropy induced metric estimators: added; see 'ACIM_initialization.m', 'ACIM_estimation.m', 'ACCIM_initialization.m', 'ACCIM_estimation.m'.
+-Correntropy, centered correntropy, correntropy coefficient estimators: added; see 'ACorrEntr_KDE_direct_initialization.m', 'ACorrEntr_KDE_direct_estimation.m', 'ACCorrEntr_KDE_iChol_initialization.m', 'ACCorrEntr_KDE_iChol_estimation.m', 'ACCorrEntr_KDE_Lapl_initialization.m', 'ACCorrEntr_KDE_Lapl_estimation.m', 'ACorrEntrCoeff_KDE_direct_initialization.m', 'ACorrEntrCoeff_KDE_direct_estimation.m', 'ACorrEntrCoeff_KDE_iChol_initialization.m', 'ACorrEntrCoeff_KDE_iChol_estimation.m'.
+-Some references fine-tuned/added (code+doc).
 -Handling of identically constant random variables in distance correlation computation: included; see 'IdCor_estimation.m'.
 
 v0.26 (Dec 22, 2012):
 ITE can estimate 
 
 - `entropy (H)`: Shannon entropy, R�nyi entropy, Tsallis entropy (Havrda and Charv�t entropy), complex entropy,
-- `mutual information (I)`: generalized variance, kernel canonical correlation analysis, kernel generalized variance, Hilbert-Schmidt independence criterion, Shannon mutual information, L2 mutual information, R�nyi mutual information, Tsallis mutual information, copula-based kernel dependency, multivariate version of Hoeffding's Phi, Schweizer-Wolff's sigma and kappa, complex mutual information, Cauchy-Schwartz quadratic mutual information, Euclidean distance based quadratic mutual information, distance covariance, distance correlation,
+- `mutual information (I)`: generalized variance, kernel canonical correlation analysis, kernel generalized variance, Hilbert-Schmidt independence criterion, Shannon mutual information, L2 mutual information, R�nyi mutual information, Tsallis mutual information, copula-based kernel dependency, multivariate version of Hoeffding's Phi, Schweizer-Wolff's sigma and kappa, complex mutual information, Cauchy-Schwartz quadratic mutual information, Euclidean distance based quadratic mutual information, distance covariance, distance correlation, approximate correntropy independence measure,
 - `divergence (D)`: Kullback-Leibler divergence (relative entropy), L2 divergence, R�nyi divergence, Tsallis divergence, Hellinger distance, Bhattacharyya distance, maximum mean discrepancy (kernel distance, an integral probability metric), J-distance (symmetrised Kullback-Leibler divergence), Cauchy-Schwartz divergence, Euclidean distance based divergence, energy distance (specially the Cramer-Von Mises distance),
-- `association measures (A)`, including `measures of concordance`: multivariate extensions of Spearman's rho (Spearman's rank correlation coefficient, grade correlation coefficient),
+- `association measures (A)`, including `measures of concordance`: multivariate extensions of Spearman's rho (Spearman's rank correlation coefficient, grade correlation coefficient), correntropy, centered correntropy, correntropy coefficient, correntropy induced metric, centered correntropy induced metric,
 - `cross quantities (C)`: cross-entropy.
 
 ITE offers solution methods for 
 
 **Download** the latest release: 
 
-- code: [zip](https://bitbucket.org/szzoli/ite/downloads/ITE-0.26_code.zip), [tar.bz2](https://bitbucket.org/szzoli/ite/downloads/ITE-0.26_code.tar.bz2), 
-- [documentation (pdf)](https://bitbucket.org/szzoli/ite/downloads/ITE-0.26_documentation.pdf).
+- code: [zip](https://bitbucket.org/szzoli/ite/downloads/ITE-0.27_code.zip), [tar.bz2](https://bitbucket.org/szzoli/ite/downloads/ITE-0.27_code.tar.bz2), 
+- [documentation (pdf)](https://bitbucket.org/szzoli/ite/downloads/ITE-0.27_documentation.pdf).
 
 

code/H_I_D_A_C/A_estimation.m

 function [A] = A_estimation(Y,ds,co)
-%Estimates association A(y^1,...,y^M), where the m^th subspace is ds(m)-dimensional; using the method/cost object co.
+%Estimates association measure A(y^1,...,y^M), where the m^th subspace is ds(m)-dimensional; using the method/cost object co.
 %
 %INPUT:
 %   Y: Y(:,t) is the t^th sample.
 %  ds: subspace dimensions.
-%  co: association estimator object (structure).
+%  co: association measure estimator object (structure).
 %
 %Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
 %
 
 %Here, we make use of the naming convention 'A<name>_estimation':
     eval(['A=A',co.name,'_estimation(Y,ds,co);']);%example: A = ASpearman1_estimation(Y,ds,co);
-    
+    

code/H_I_D_A_C/A_initialization.m

 function [co] = A_initialization(cost_name,mult)
-%Initialization of an A (association) estimator. The estimator is treated as a cost object (co). 
+%Initialization of an A (association measure) estimator. The estimator is treated as a cost object (co). 
 %
 %INPUT:
 %   mult: is a multiplicative constant relevant (needed) in the estimation; '=1' means yes, '=0' no.
 %Here, we make use of the naming convention: 'A<name>_initialization':
    eval(['co=A',cost_name,'_initialization(mult)']), %example: co = ASpearman1_initialization(mult);
     
-disp('A initialization: ready.');
+disp('A initialization: ready.');

code/H_I_D_A_C/C_estimation.m

 function [C] = C_estimation(Y1,Y2,co)
-%Cross estimation (C) of Y1 and Y2 using the specified cross estimator.
+%Cross quantity estimation (C) of Y1 and Y2 using the specified estimator.
 %
 %INPUT:
 %  Y1: Y1(:,t) is the t^th sample from the first distribution.
 %  Y2: Y2(:,t) is the t^th sample from the second distribution. 
-%  co: cross estimator object.
+%  co: cross quantity estimator object.
 %
 %Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
 %

code/H_I_D_A_C/C_initialization.m

 function [co] = C_initialization(cost_name,mult)
-%Initialization of a cross estimator. The estimator is treated as a cost object (co). 
+%Initialization of a cross quantity estimator. The estimator is treated as a cost object (co). 
 %   
 %INPUT:
 %   mult: is a multiplicative constant relevant (needed) in the estimation; '=1' means yes, '=0' no.

code/H_I_D_A_C/base_estimators/ACCorrEntr_KDE_Lapl_estimation.m

+function [A] = ACCorrEntr_KDE_Lapl_estimation(Y,ds,co)
+%Estimates the centered correntropy (A) using Laplacian KDE (kernel density estimation) + a sorting based trick.
+%
+%We use the naming convention 'A<name>_estimation' to ease embedding new association measure estimator methods.
+%
+%INPUT:
+%   Y: Y(:,t) is the t^th sample.
+%  ds: subspace dimensions.
+%  co: association measure estimator object.
+%
+%REFERENCE:
+%   Murali Rao, Sohan Seth, Jianwu Xu, Yunmei Chen, Hemant Tagare, and Jose C. Principe. A test of independence based on a generalized correlation function. Signal Processing, 91:15-27, 2011.
+%   Aiyou Chen. Fast kernel density independent component analysis. In Independent Component Analysis and Blind Signal Separation (ICA), pages 24-31, 2006. (sorting based trick)
+%
+%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
+%
+%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
+%
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
+%
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+
+%co.mult:OK.
+
+%verification:
+    if sum(ds) ~= size(Y,1);
+        error('The subspace dimensions are not compatible with Y.');
+    end
+    if ~one_dimensional_problem(ds) || length(ds)~=2
+        error('There must be 2 pieces of one-dimensional subspaces (coordinates) for this estimator.');
+    end
+    
+A = centcorrenexp(Y(1,:).',Y(2,:).',co.sigma);

code/H_I_D_A_C/base_estimators/ACCorrEntr_KDE_Lapl_initialization.m

+function [co] = ACCorrEntr_KDE_Lapl_initialization(mult)
+%Initialization of the Laplacian KDE (kernel density estimation) + a sorting trick based centered correntropy estimator.
+%
+%Note:
+%   1)The estimator is treated as a cost object (co).
+%   2)We use the naming convention 'A<name>_initialization' to ease embedding new association measure estimator methods.
+%
+%INPUT:
+%   mult: is a multiplicative constant relevant (needed) in the estimation; '=1' means yes, '=0' no.
+%OUTPUT:
+%   co: cost object (structure).
+%
+%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
+%
+%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
+%
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
+%
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+
+%mandatory fields:
+    co.name = 'CCorrEntr_KDE_Lapl';
+    co.mult = mult;
+    
+%other fields:
+   co.sigma = 1; %scaling parameter of the Laplacian kernel

code/H_I_D_A_C/base_estimators/ACCorrEntr_KDE_iChol_estimation.m

+function [A] = ACCorrEntr_KDE_iChol_estimation(Y,ds,co)
+%Estimates the centered correntropy (A) using Gaussian KDE (kernel density estimation) + incomplete Cholesky decomposition.
+%
+%We use the naming convention 'A<name>_estimation' to ease embedding new association measure estimator methods.
+%
+%INPUT:
+%   Y: Y(:,t) is the t^th sample.
+%  ds: subspace dimensions.
+%  co: association measure estimator object.
+%
+%REFERENCE:
+%   Murali Rao, Sohan Seth, Jianwu Xu, Yunmei Chen, Hemant Tagare, and Jose C. Principe. A test of independence based on a generalized correlation function. Signal Processing, 91:15-27, 2011.
+%   Sohan Seth and Jose C. Principe. On speeding up computation in information theoretic learning. In International Joint Conference on Neural Networks (IJCNN), pages 2883-2887, 2009.
+%
+%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
+%
+%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
+%
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
+%
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+
+%co.mult:OK.
+
+%verification:
+    if sum(ds) ~= size(Y,1);
+        error('The subspace dimensions are not compatible with Y.');
+    end
+    if ~one_dimensional_problem(ds) || length(ds)~=2
+        error('There must be 2 pieces of one-dimensional subspaces (coordinates) for this estimator.');
+    end
+
+A = centcorren(Y(1,:).',Y(2,:).',co.sigma);

code/H_I_D_A_C/base_estimators/ACCorrEntr_KDE_iChol_initialization.m

+function [co] = ACCorrEntr_KDE_iChol_initialization(mult)
+%Initialization of the Gaussian KDE (kernel density estimation) + incomplete Cholesky decomposition based centered correntropy estimator.
+%
+%Note:
+%   1)The estimator is treated as a cost object (co).
+%   2)We use the naming convention 'A<name>_initialization' to ease embedding new association measure estimator methods.
+%
+%INPUT:
+%   mult: is a multiplicative constant relevant (needed) in the estimation; '=1' means yes, '=0' no.
+%OUTPUT:
+%   co: cost object (structure).
+%
+%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
+%
+%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
+%
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
+%
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+
+%mandatory fields:
+    co.name = 'CCorrEntr_KDE_iChol';
+    co.mult = mult;
+    
+%other fields:    
+    co.sigma = 1;%std in the RBF (Gaussian) kernel

code/H_I_D_A_C/base_estimators/ACorrEntrCoeff_KDE_direct_estimation.m

+function [A] = ACorrEntrCoeff_KDE_direct_estimation(Y,ds,co)
+%Estimates the correntropy coefficient (A) directly using Gaussian KDE (kernel density estimation).
+%
+%We use the naming convention 'A<name>_estimation' to ease embedding new association measure estimator methods.
+%
+%INPUT:
+%   Y: Y(:,t) is the t^th sample.
+%  ds: subspace dimensions.
+%  co: association measure estimator object.
+%
+%REFERENCE:
+%   Murali Rao, Sohan Seth, Jianwu Xu, Yunmei Chen, Hemant Tagare, and Jose C. Principe. A test of independence based on a generalized correlation function. Signal Processing, 91:15-27, 2011.
+%
+%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
+%
+%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
+%
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
+%
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+
+%co.mult:OK.
+
+%verification:
+    if sum(ds) ~= size(Y,1);
+        error('The subspace dimensions are not compatible with Y.');
+    end
+    if ~one_dimensional_problem(ds) || length(ds)~=2
+        error('There must be 2 pieces of one-dimensional subspaces (coordinates) for this estimator.');
+    end
+
+A = correncoef_2(Y(1,:).',Y(2,:).',co.sigma);

code/H_I_D_A_C/base_estimators/ACorrEntrCoeff_KDE_direct_initialization.m

+function [co] = ACorrEntrCoeff_KDE_direct_initialization(mult)
+%Initialization of the Gaussian KDE (kernel density estimation) based direct correntropy coefficient estimator.
+%
+%Note:
+%   1)The estimator is treated as a cost object (co).
+%   2)We use the naming convention 'A<name>_initialization' to ease embedding new association measure estimator methods.
+%
+%INPUT:
+%   mult: is a multiplicative constant relevant (needed) in the estimation; '=1' means yes, '=0' no.
+%OUTPUT:
+%   co: cost object (structure).
+%
+%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
+%
+%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
+%
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
+%
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+
+%mandatory fields:
+    co.name = 'CorrEntrCoeff_KDE_direct';
+    co.mult = mult;
+    
+%other fields:    
+    co.sigma = 1;%std in the RBF (Gaussian) kernel
+   

code/H_I_D_A_C/base_estimators/ACorrEntrCoeff_KDE_iChol_estimation.m

+function [A] = ACorrEntrCoeff_KDE_iChol_estimation(Y,ds,co)
+%Estimates the correntropy coefficient (A) using Gaussian KDE (kernel density estimation) + incomplete Cholesky decomposition.
+%
+%We use the naming convention 'A<name>_estimation' to ease embedding new association measure estimator methods.
+%
+%INPUT:
+%   Y: Y(:,t) is the t^th sample.
+%  ds: subspace dimensions.
+%  co: association measure estimator object.
+%
+%REFERENCE:
+%   Murali Rao, Sohan Seth, Jianwu Xu, Yunmei Chen, Hemant Tagare, and Jose C. Principe. A test of independence based on a generalized correlation function. Signal Processing, 91:15-27, 2011.
+%   Sohan Seth and Jose C. Principe. On speeding up computation in information theoretic learning. In International Joint Conference on Neural Networks (IJCNN), pages 2883-2887, 2009.
+%
+%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
+%
+%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
+%
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
+%
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+
+%co.mult:OK.
+
+%verification:
+    if sum(ds) ~= size(Y,1);
+        error('The subspace dimensions are not compatible with Y.');
+    end
+    if ~one_dimensional_problem(ds) || length(ds)~=2
+        error('There must be 2 pieces of one-dimensional subspaces (coordinates) for this estimator.');
+    end
+
+A = correncoef(Y(1,:).',Y(2,:).',co.sigma);

code/H_I_D_A_C/base_estimators/ACorrEntrCoeff_KDE_iChol_initialization.m

+function [co] = ACorrEntrCoeff_KDE_iChol_initialization(mult)
+%Initialization of the Gaussian KDE (kernel density estimation) + incomplete Cholesky decomposition based correntropy coefficient estimator.
+%
+%Note:
+%   1)The estimator is treated as a cost object (co).
+%   2)We use the naming convention 'A<name>_initialization' to ease embedding new association measure estimator methods.
+%
+%INPUT:
+%   mult: is a multiplicative constant relevant (needed) in the estimation; '=1' means yes, '=0' no.
+%OUTPUT:
+%   co: cost object (structure).
+%
+%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
+%
+%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
+%
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
+%
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+
+%mandatory fields:
+    co.name = 'CorrEntrCoeff_KDE_iChol';
+    co.mult = mult;
+    
+%other fields:    
+    co.sigma = 1;%std in the RBF (Gaussian) kernel
+   

code/H_I_D_A_C/base_estimators/ACorrEntr_KDE_direct_estimation.m

+function [A] = ACorrEntr_KDE_direct_estimation(Y,ds,co)
+%Estimates the correntropy (A) directly using Gaussian KDE (kernel density estimation).
+%
+%We use the naming convention 'A<name>_estimation' to ease embedding new association measure estimator methods.
+%
+%INPUT:
+%   Y: Y(:,t) is the t^th sample.
+%  ds: subspace dimensions.
+%  co: association measure estimator object.
+%
+%REFERENCE:
+%   Murali Rao, Sohan Seth, Jianwu Xu, Yunmei Chen, Hemant Tagare, and Jose C. Principe. A test of independence based on a generalized correlation function. Signal Processing, 91:15-27, 2011.
+%
+%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
+%
+%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
+%
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
+%
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+
+%co.mult:OK.
+
+%verification:
+    if sum(ds) ~= size(Y,1);
+        error('The subspace dimensions are not compatible with Y.');
+    end
+    if ~one_dimensional_problem(ds) || length(ds)~=2
+        error('There must be 2 pieces of one-dimensional subspaces (coordinates) for this estimator.');
+    end
+    
+A = corren(Y(1,:).',Y(2,:).',co.sigma);    

code/H_I_D_A_C/base_estimators/ACorrEntr_KDE_direct_initialization.m

+function [co] = ACorrEntr_KDE_direct_initialization(mult)
+%Initialization of the Gaussian KDE (kernel density estimation) based direct correntropy estimator.
+%
+%Note:
+%   1)The estimator is treated as a cost object (co).
+%   2)We use the naming convention 'A<name>_initialization' to ease embedding new association measure estimator methods.
+%
+%INPUT:
+%   mult: is a multiplicative constant relevant (needed) in the estimation; '=1' means yes, '=0' no.
+%OUTPUT:
+%   co: cost object (structure).
+%
+%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
+%
+%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
+%
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
+%
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+
+%mandatory fields:
+    co.name = 'CorrEntr_KDE_direct';
+    co.mult = mult;
+    
+%other fields:    
+    co.sigma = 1; %std in the RBF (Gaussian) kernel

code/H_I_D_A_C/base_estimators/ASpearman1_estimation.m

 function [A] = ASpearman1_estimation(Y,ds,co)
 %Estimates the first multivariate extension of Spearman's rho using empirical copulas.
 %
-%We use the naming convention 'A<name>_estimation' to ease embedding new association estimator methods.
+%We use the naming convention 'A<name>_estimation' to ease embedding new association measure estimator methods.
 %
 %INPUT:
 %   Y: Y(:,t) is the t^th sample.
 %  ds: subspace dimensions.
-%  co: association estimator object.
+%  co: association measure estimator object.
 %
 %REFERENCE:
 %   Friedrich Shmid, Rafael Schmidt, Thomas Blumentritt, Sandra Gaiser, and Martin Ruppert. Copula Theory and Its Applications, Chapter Copula based Measures of Multivariate Association. Lecture Notes in Statistics. Springer, 2010.

code/H_I_D_A_C/base_estimators/ASpearman1_initialization.m

 %
 %Note:
 %   1)The estimator is treated as a cost object (co).
-%   2)We use the naming convention 'A<name>_initialization' to ease embedding new association estimator methods.
+%   2)We use the naming convention 'A<name>_initialization' to ease embedding new association measure estimator methods.
 %
 %INPUT:
 %   mult: is a multiplicative constant relevant (needed) in the estimation; '=1' means yes, '=0' no.

code/H_I_D_A_C/base_estimators/ASpearman2_estimation.m

 function [A] = ASpearman2_estimation(Y,ds,co)
 %Estimates the second multivariate extension of Spearman's rho using empirical copulas.
 %
-%We use the naming convention 'A<name>_estimation' to ease embedding new association estimator methods.
+%We use the naming convention 'A<name>_estimation' to ease embedding new association measure estimator methods.
 %
 %INPUT:
 %   Y: Y(:,t) is the t^th sample.
 %  ds: subspace dimensions.
-%  co: association estimator object.
+%  co: association measure estimator object.
 %
 %REFERENCE:
 %   Friedrich Shmid, Rafael Schmidt, Thomas Blumentritt, Sandra Gaiser, and Martin Ruppert. Copula Theory and Its Applications, Chapter Copula based Measures of Multivariate Association. Lecture Notes in Statistics. Springer, 2010.

code/H_I_D_A_C/base_estimators/ASpearman2_initialization.m

 %
 %Note:
 %   1)The estimator is treated as a cost object (co).
-%   2)We use the naming convention 'A<name>_initialization' to ease embedding new association estimator methods.
+%   2)We use the naming convention 'A<name>_initialization' to ease embedding new association measure estimator methods.
 %
 %INPUT:
 %   mult: is a multiplicative constant relevant (needed) in the estimation; '=1' means yes, '=0' no.

code/H_I_D_A_C/base_estimators/ASpearman3_estimation.m

 function [A] = ASpearman3_estimation(Y,ds,co)
 %Estimates the second multivariate extension of Spearman's rho using empirical copulas.
 %
-%We use the naming convention 'A<name>_estimation' to ease embedding new association estimator methods.
+%We use the naming convention 'A<name>_estimation' to ease embedding new association measure estimator methods.
 %
 %INPUT:
 %   Y: Y(:,t) is the t^th sample.
 %  ds: subspace dimensions.
-%  co: association estimator object.
+%  co: association measure estimator object.
 %
 %REFERENCE:
 %   Friedrich Shmid, Rafael Schmidt, Thomas Blumentritt, Sandra Gaiser, and Martin Ruppert. Copula Theory and Its Applications, Chapter Copula based Measures of Multivariate Association. Lecture Notes in Statistics. Springer, 2010.

code/H_I_D_A_C/base_estimators/ASpearman3_initialization.m

 %
 %Note:
 %   1)The estimator is treated as a cost object (co).
-%   2)We use the naming convention 'A<name>_initialization' to ease embedding new association estimator methods.
+%   2)We use the naming convention 'A<name>_initialization' to ease embedding new association measure estimator methods.
 %
 %INPUT:
 %   mult: is a multiplicative constant relevant (needed) in the estimation; '=1' means yes, '=0' no.

code/H_I_D_A_C/base_estimators/CCE_kNN_k_estimation.m

 function [CE] = CCE_kNN_k_estimation(Y1,Y2,co)
 %Estimates the cross-entropy (CE) of Y1 and Y2 using the kNN method (S={k}).
 %
-%We use the naming convention 'C<name>_estimation' to ease embedding new cross estimation methods.
+%We use the naming convention 'C<name>_estimation' to ease embedding new cross quantity estimation methods.
 %
 %INPUT:
 %  Y1: Y1(:,t) is the t^th sample from the first distribution.
 %  Y2: Y2(:,t) is the t^th sample from the second distribution. Note: the number of samples in Y1 [=size(Y1,2)] and Y2 [=size(Y2,2)] can be different.
-%  co: cross estimator object.
+%  co: cross quantity estimator object.
 %
 %REFERENCE: 
 %   Nikolai Leonenko, Luc Pronzato, and Vippal Savani. A class of Renyi information estimators for multidimensional densities. Annals of Statistics, 36(5):2153-2182, 2008.

code/H_I_D_A_C/base_estimators/CCE_kNN_k_initialization.m

 %
 %Note:
 %   1)The estimator is treated as a cost object (co).
-%   2)We use the naming convention 'C<name>_initialization' to ease embedding new cross estimation methods.
+%   2)We use the naming convention 'C<name>_initialization' to ease embedding new cross quantity estimation methods.
 %
 %INPUT:
 %   mult: is a multiplicative constant relevant (needed) in the estimation; '=1' means yes, '=0' no.

code/H_I_D_A_C/meta_estimators/ACCIM_estimation.m

+function [A] = ACCIM_estimation(Y,ds,co)
+%Estimates the centered correntropy induced metric of Y1 and Y2 (A) using the relation CCIM(y^1,y^_2) = [CCE(y^1,y^1)+CCE(y^2,y^2)-2CCE(y^1,y^2)]^{1/2}, where CCE denotes centered correntropy.
+%
+%INPUT:
+%   Y: Y(:,t) is the t^th sample.
+%  ds: subspace dimensions.
+%  co: association measure estimator object.
+%
+%Note:
+%   1)We use the naming convention 'A<name>_estimation' to ease embedding new association measure estimator methods.
+%   2)This is a meta method: the centered correntropy estimator can be arbitrary.
+%
+%REFERENCE: 
+%   Murali Rao, Sohan Seth, Jianwu Xu, Yunmei Chen, Hemant Tagare, and Jose C. Principe. A test of independence based on a generalized correlation function. Signal Processing, 91:15-27, 2011.
+%
+%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
+%
+%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
+%
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
+%
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+
+%co.mult:OK.
+
+%verification:
+    if sum(ds) ~= size(Y,1);
+        error('The subspace dimensions are not compatible with Y.');
+    end
+    if ~one_dimensional_problem(ds) || length(ds)~=2
+        error('There must be 2 pieces of one-dimensional subspaces (coordinates) for this estimator.');
+    end
+
+A_Y1Y1 = A_estimation([Y(1,:);Y(1,:)],ds,co.member_co);
+A_Y2Y2 = A_estimation([Y(2,:);Y(2,:)],ds,co.member_co);
+A_Y1Y2 = A_estimation([Y(1,:);Y(2,:)],ds,co.member_co);
+
+%A =  sqrt(A_Y1Y1 + A_Y2Y2 - 2*A_Y1Y2);%theoretically
+A =  sqrt(abs(A_Y1Y1 + A_Y2Y2 - 2*A_Y1Y2));%abs(): to guarantee that the argument of sqrt is non-negative (due to the finite number of samples)
+

code/H_I_D_A_C/meta_estimators/ACCIM_initialization.m

+function [co] = ACCIM_initialization(mult)
+%Initialization of the centered correntropy induced metric estimator, defined according to the relation CCIM(y^1,y^_2) = [CCE(y^1,y^1)+CCE(y^2,y^2)-2CCE(y^1,y^2)]^{1/2}, where CCE denotes centered correntropy.
+%
+%Note:
+%   1)The estimator is treated as a cost object (co).
+%   2)We use the naming convention 'A<name>_initialization' to ease embedding new association measure estimator methods.
+%
+%INPUT:
+%   mult: is a multiplicative constant relevant (needed) in the estimation; '=1' means yes, '=0' no.
+%OUTPUT:
+%   co: cost object (structure).
+%
+%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
+%
+%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
+%
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
+%
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+
+%mandatory fields:
+    co.name = 'CCIM';
+    co.mult = mult;
+    
+%other fields:
+    co.member_name = 'CCorrEntr_KDE_iChol'; %you can change it to any centered correntropy estimator
+    co.member_co = A_initialization(co.member_name,mult);

code/H_I_D_A_C/meta_estimators/ACIM_estimation.m

+function [A] = ACIM_estimation(Y,ds,co)
+%Estimates the correntropy induced metric of Y1 and Y2 using the relation CIM(y^1,y^_2) = [k(0,0)-correntropy(y^1,y^2)]^{1/2}, where k is the applied (Gaussian) kernel. 
+%
+%We use the naming convention 'A<name>_estimation' to ease embedding new association measure estimator methods.
+%
+%INPUT:
+%   Y: Y(:,t) is the t^th sample.
+%  ds: subspace dimensions.
+%  co: association measure estimator object.
+%
+%Note:
+%   1)We use the naming convention 'A<name>_estimation' to ease embedding new association estimator methods.
+%   2)This is a meta method: the (Gaussian) correntropy estimator can be arbitrary.
+%
+%REFERENCE: 
+%   Sohan Seth and Jose C. Principe. Compressed signal reconstruction using the correntropy induced metric. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 3845-3848, 2008.
+%   Weifeng Liu, P.P. Pokharel, and Jose C. Principe. Correntropy: Properties and applications in non-Gaussian signal processing. IEEE Transactions on Signal Processing, 55:5286-5298, 2007.
+%
+%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
+%
+%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
+%
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
+%
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+
+%co.mult:OK.
+
+%verification:
+    if sum(ds) ~= size(Y,1);
+        error('The subspace dimensions are not compatible with Y.');
+    end
+    if ~one_dimensional_problem(ds) || length(ds)~=2
+        error('There must be 2 pieces of one-dimensional subspaces (coordinates) for this estimator.');
+    end
+
+A0 = A_estimation(Y,ds,co.member_co); %correntropy
+k0 = 1; %Assumption: k(u,u)=1. Example (Gaussian kernel): k(u,v)=e^{-(u-v)^2}, k(u,u)=1
+A =  sqrt(k0-A0);
+

code/H_I_D_A_C/meta_estimators/ACIM_initialization.m

+function [co] = ACIM_initialization(mult)
+%Initialization of the correntropy induced metric estimator, defined according to the relation CIM(y^1,y^_2) = [k(0,0)-correntropy(y^1,y^2)]^{1/2}, where k is the applied (Gaussian) kernel.
+%
+%Note:
+%   1)The estimator is treated as a cost object (co).
+%   2)We use the naming convention 'A<name>_initialization' to ease embedding new association measure estimator methods.
+%
+%INPUT:
+%   mult: is a multiplicative constant relevant (needed) in the estimation; '=1' means yes, '=0' no.
+%OUTPUT:
+%   co: cost object (structure).
+%
+%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
+%
+%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
+%
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
+%
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+
+%mandatory fields:
+    co.name = 'CIM';
+    co.mult = mult;
+    
+%other fields:
+    co.member_name = 'CorrEntr_KDE_direct'; %you can change it to any (Gaussian kernel based) correntropy estimator
+    co.member_co = A_initialization(co.member_name,mult);

code/H_I_D_A_C/meta_estimators/DEnergyDist_DMMD_estimation.m

 %
 %REFERENCE:
 %   Dino Sejdinovic, Arthur Gretton, Bharath Sriperumbudur, and Kenji Fukumizu. Hypothesis testing using pairwise distances and associated kernels. International Conference on Machine Learning (ICML), pages 1111-1118, 2012. (semimetric space; energy distance <=> MMD, with a suitable kernel)
-%   Russell Lyons. Distance Covariance in metric spaces. Technical report, Indiana University, 2011. http://arxiv.org/abs/1106.5758. (energy distance, metric space of negative type; pre-equivalence to MMD)
+%   Russell Lyons. Distance covariance in metric spaces. Annals of Probability, 2012. (To appear. http://php.indiana.edu/~rdlyons/pdf/dcov.pdf; http://arxiv.org/abs/1106.5758; energy distance, metric space of negative type; pre-equivalence to MMD).
 %   Gabor J. Szekely and Maria L. Rizzo. A new test for multivariate normality. Journal of Multivariate Analysis, 93:58-80, 2005. (energy distance; metric space of negative type)
 %   Gabor J. Szekely and Maria L. Rizzo. Testing for equal distributions in high dimension. InterStat, 5, 2004. (energy distance; R^d)
 %

code/H_I_D_A_C/meta_estimators/IApprCorrEntr_estimation.m

+function [I] = IApprCorrEntr_estimation(Y,ds,co)
+%Estimates the approximate correntropy independence measure (I) based on centered correntropy, using the relation: I(y^1,y^2) = max(|CCorrEntr(y^1,y^2)|,|CCorrEntr(-y^1,y^2)|), where CCorrEntr denotes centered correntropy.
+%
+%Note:
+%   1)We use the naming convention 'I<name>_estimation' to ease embedding new mutual information estimation methods.
+%   2)This a meta method: the centered correntropy estimator can be arbitrary.
+%
+%INPUT:
+%   Y: Y(:,t) is the t^th sample.
+%  ds: subspace dimensions.
+%  co: mutual information estimator object.
+%
+%REFERENCE:
+%   Murali Rao, Sohan Seth, Jianwu Xu, Yunmei Chen, Hemant Tagare, and Jose C. Principe. A test of independence based on a generalized correlation function. Signal Processing, 91:15-27, 2011.
+%
+%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
+%
+%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
+%
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
+%
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+
+%co.mult:OK.
+
+%verification:
+    if sum(ds) ~= size(Y,1);
+        error('The subspace dimensions are not compatible with Y.');
+    end
+    if ~one_dimensional_problem(ds) || length(ds)~=2 
+        error('There must be 2 pieces of one-dimensional subspaces (coordinates) for this estimator.');
+    end
+
+A1 = A_estimation(Y,ds,co.member_co);%[Y1;Y2]
+A2 = A_estimation([-Y(1,:);Y(2,:)],ds,co.member_co);%[-Y1;Y2]
+I = max(abs(A1),abs(A2));
+

code/H_I_D_A_C/meta_estimators/IApprCorrEntr_initialization.m

+function [co] = IApprCorrEntr_initialization(mult)
+%Initialization of the approximate correntropy independence measure estimator. The estimated quantity is computed according to the relation: I(y^1,y^2) = max(|CCorrEntr(y^1,y^2)|,|CCorrEntr(-y^1,y^2)|), where CCorrEntr denotes centered correntropy.
+%
+%Note:
+%   1)The estimator is treated as a cost object (co). 
+%   2)We use the naming convention 'I<name>_initialization' to ease embedding new mutual information estimation methods.
+%   3)This is a meta method: the centered correntropy estimator can be arbitrary.
+%
+%INPUT:
+%   mult: is a multiplicative constant relevant (needed) in the estimation; '=1' means yes, '=0' no.
+%OUTPUT:
+%   co: object (structure).
+%
+%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
+%
+%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
+%
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
+%
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+
+%mandatory fields:
+    co.name = 'ApprCorrEntr';
+    co.mult = mult;
+	
+%other fields:    
+    co.member_name = 'CCorrEntr_KDE_iChol'; %you can change it to any centered correntropy estimator
+    co.member_co = A_initialization(co.member_name,mult);
+ 

code/H_I_D_A_C/meta_estimators/IdCov_IHSIC_estimation.m

 %
 %REFERENCE:
 %   Dino Sejdinovic, Arthur Gretton, Bharath Sriperumbudur, and Kenji Fukumizu. Hypothesis testing using pairwise distances and associated kernels. International Conference on Machine Learning (ICML), pages 1111-1118, 2012. (equivalence to HSIC)
-%   Russell Lyons. Distance Covariance in metric spaces. Technical report, Indiana University, 2011. http://arxiv.org/abs/1106.5758. (generalized distance covariance, rho_i; equivalence to HSIC)
+%   Russell Lyons. Distance covariance in metric spaces. Annals of Probability, 2012. (To appear. http://php.indiana.edu/~rdlyons/pdf/dcov.pdf; http://arxiv.org/abs/1106.5758; generalized distance covariance, rho_i; equivalence to HSIC).
 %   Gabor J. Szekely and Maria L. Rizzo and. Brownian distance covariance. The Annals of Applied Statistics, 3:1236-1265, 2009. (distance covariance)
 %   Gabor J. Szekely, Maria L. Rizzo, and Nail K. Bakirov. Measuring and testing dependence by correlation of distances. The Annals of Statistics, 35:2769-2794, 2007. (distance covariance)
 %

code/IPA/demos/estimate_ISA.m

 %   de_hat: in case of known subspace dimensions ('unknown_dimensions = 0') de_hat = de; else it contains the estimated subspace dimensions; ordered increasingly.
 %
 %REFERENCE:
+%   Jason A. Palmer and Scott Makeig. Contrast functions for independent subspace analysis. In International conference on Latent Variable Analysis and Signal Separation (LVA/ICA), pages 115-122, 2012. (exciting, alternative proof idea for deflation methods)
 %   Zoltan Szabo, Barnabas Poczos, Andras Lorincz: Undercomplete Blind Subspace Deconvolution. Journal of Machine Learning Research 8(May):1063-1095, 2007. (proof; sufficient conditions for the ISA separation theorem)
 %   Jean-Francois Cardoso. Multidimensional independent component analysis. International Conference on Acoustics, Speech, and Signal Processing (ICASSP), pages 1941-1944, 1998. (conjecture)
 %   Lieven De Lathauwer, Bart De Moor, and Joos Vandewalle. Fetal electrocardiogram extraction by source subspace separation. In IEEE SP/Athos Workshop on Higher-Order Statistics, pages 134-138, 1995. ('conjecture')

code/shared/embedded/ITL/centcorren.m

+% The following function computes the parametric centered correntropy between two vectors X and Y using Gaussian kernel.
+%
+% Input:   Both X and Y should be COLUMN vectors of SAME length (nx1).
+%				'kSize' is a scalar for the kernel size.
+%				'param' is a 2x1 vector containing the parameter coeffues 'a' and 'b' in ORDER.
+%
+% Output: 'corren' contains the parametric correntropy coefficient
+%
+% Default:  param = [a,b] = [1,0] and kSize = 1.
+%
+% Comments: The code uses Incomplete Cholesky Decomposition.
+%
+% Author: Sohan Seth (sohan@cnel.ufl.edu)	Date: 11.03.2008
+
+function corren = centcorren(X,Y,kSize,param)
+
+if nargin == 2
+	kSize = 1;
+	param = [1,0];
+end
+
+if nargin == 3
+	param = [1,0];
+end
+
+n = size(X,1);
+a = param(1); b = param(2);
+twokSizeSquare = 2*kSize^2;
+
+X = a*X+b;
+
+G =  incompleteCholeskyMulti([X;Y],kSize);
+OZ = [ones(n,1);zeros(n,1)]/n;
+ZO = [zeros(n,1);ones(n,1)]/n;
+corren = (1/n)*sum(exp(-(X - Y).^2/twokSizeSquare)) - (ZO'*G)*(G'*OZ);
+
+% ~ Done

code/shared/embedded/ITL/centcorrenexp.m

+% The following function computes the parametric centered correntropy
+%               between two vectors X and Y using double exponential kernel
+%
+% Input:   X and Y are real valued COLUMN vectors of SAME length
+%				kSize is a scalar for the kernel size, i.e. K(x,y) = exp(-|x-y|/kSize),
+%				param is a [2x1] vector containing the coefficients a and b in ORDER.
+%
+% Output: corren contains the parametric centered correntropy
+%
+% Default:  param = [a,b] = [1,0] and kSize = 1.
+%
+% Comments: The code uses cipexp
+%
+% Author: Sohan Seth (sohan@cnel.ufl.edu)	Date: 15.07.2009
+
+function corren = centcorrenexp(X,Y,kSize,param)
+
+if nargin == 2
+	kSize = 1;
+	param = [1,0];
+end
+
+if nargin == 3
+	param = [1,0];
+end
+
+n = size(X,1);
+a = param(1); b = param(2);
+
+X = a*X+b;
+
+corren = (1/n)*sum(exp(-abs(X - Y)/kSize)) - cipexp(X,Y,kSize);

code/shared/embedded/ITL/cipexp.m

+% The following function computes the cross information potential between
+%               two vectors X and Y using a double exponential kernel
+%
+% Input:   X and Y real valued COLUMN vectors of same length
+%				kSize is a scalar for the kernel size, i.e. K(x,y) =  exp(-|x-y|/kSize)
+%
+% Output: cip contains the cross information potential
+%
+% Default:  kSize = 1.
+%
+% Author: IL "Memming" Park (memming@cnel.ufl.edu)	Date: 15.07.2009
+
+function [cip] = cipexp(X,Y,kSize)
+
+if nargin == 2
+	kSize = 1;
+end
+
+X = X ./ kSize; Y = Y ./ kSize;
+
+X_sort = sort(X); Y_sort = sort(Y);
+
+X_pos_exp = exp(X_sort); X_neg_exp = exp(-X_sort);
+Y_pos_exp = exp(Y_sort); Y_neg_exp = exp(-Y_sort);
+
+Y_pos_cum_sum_exp  = cumsum(Y_pos_exp);
+Y_neg_cum_sum_exp  = flipud(cumsum(flipud(Y_neg_exp)));
+
+Y_sort = [Y_sort; Inf];
+cip = 0;
+yidx = 0; % no y is smaller than x yet
+for xidx = 1:length(X)
+    x = X_sort(xidx);
+
+    % at the end of the next Y_sort(yidx) >= X_sort(xidx)
+    while Y_sort(yidx+1) <= x
+        yidx = yidx + 1;
+    end
+
+    if yidx == 0
+        cip = cip + Y_neg_cum_sum_exp(1) * X_pos_exp(xidx);
+    elseif yidx == length(Y)
+        cip = cip + Y_pos_cum_sum_exp(end) * X_neg_exp(xidx);
+    else
+        cip = cip + Y_pos_cum_sum_exp(yidx) * X_neg_exp(xidx) + Y_neg_cum_sum_exp(yidx+1) * X_pos_exp(xidx);
+    end
+end
+cip = cip / (length(X) * length(Y));

code/shared/embedded/ITL/corren.m

+% The following function computes the parametric correntropy between two vectors X and Y.
+%
+% Input:   Both X and Y should be COLUMN vectors of SAME length (nx1).
+%				'kSize' is a scalar for the kernel size.
+%				'param' is a 2x1 vector containing the parameter coeffues 'a' and 'b' in ORDER.
+%
+% Output: 'corren' contains the parametric correntropy coefficient
+%
+% Default:  param = [a,b] = [1,0] and kSize = 1.
+%
+% Comments: The code uses Incomplete Cholesky Decomposition.
+%
+% Author: Sohan Seth (sohan@cnel.ufl.edu)	Date: 11.03.2008
+
+function corren = corren(X,Y,kSize,param)
+
+if nargin == 2
+	kSize = 1;
+	param = [1,0];
+end
+
+if nargin == 3
+	param = [1,0];
+end
+
+n = size(X,1);
+a = param(1); b = param(2);
+twokSizeSquare = 2*kSize^2;
+
+X = a*X+b;
+corren = (1/n)*sum(exp(-(X - Y).^2/twokSizeSquare));
+
+% ~ Done

code/shared/embedded/ITL/correncoef.m

+% The following function computes the parametric correntropy coefficient between two vectors X and Y.
+%
+% Input:   Both X and Y should be COLUMN vectors of SAME length (nx1).
+%				'kSize' is a scalar for the kernel size.
+%				'param' is a 2x1 vector containing the parameter coeffues 'a' and 'b' in ORDER.
+%
+% Output: 'coeff' contains the parametric correntropy coefficient
+%
+% Default:  param = [a,b] = [1,0] and kSize = 1.
+%
+% Caution: a ~= 0.
+%
+% Comments: The code uses Incomplete Cholesky Decomposition.
+%
+% Author: Sohan Seth (sohan@cnel.ufl.edu)	Date: 11.03.2008
+
+function coeff = correncoef(X,Y,kSize,param)
+
+if nargin == 2
+	kSize = 1;
+	param = [1,0];
+end
+
+if nargin == 3
+	param = [1,0];
+end
+
+n = size(X,1);
+a = param(1); b = param(2);
+twokSizeSquare = 2*kSize^2;
+
+if a == 0;
+	error('a must NOT be 0');
+end
+
+X = a*X+b;
+
+G =  incompleteCholeskyMulti([X;Y],kSize);
+OZ = [ones(n,1);zeros(n,1)]/n;
+ZO = [zeros(n,1);ones(n,1)]/n;
+corren = (1/n)*sum(exp(-(X - Y).^2/twokSizeSquare));
+coeff = (corren - (ZO'*G)*(G'*OZ)) /sqrt((1 - (OZ'*G)*(G'*OZ))*(1 - (ZO'*G)*(G'*ZO)));
+
+% ~ Done

code/shared/embedded/ITL/correncoef_2.m

+% Direct computation of Correntropy coefficient
+%
+% Author : Sohan Seth (sohan@cnel.ufl.edu) Date : Date: 01.06.2009
+
+function val = correncoef_2(X,Y,kernelSize)
+
+n = length(X);
+Kxx = ones(n,n); Kyy = ones(n,n);
+for count1 = 1:n
+    for count2 = count1+1:n
+        Kxx(count1,count2) = exp(-(norm(X(count1,:) - X(count2,:)))^2/(2*kernelSize^2));
+        Kxx(count2,count1) = Kxx(count1,count2);
+        Kyy(count1,count2) = exp(-(norm(Y(count1,:) - Y(count2,:)))^2/(2*kernelSize^2));
+        Kyy(count2,count1) = Kyy(count1,count2);
+    end
+end
+
+Kxy = ones(n,n);
+for count1 = 1:n
+    for count2 =1:n
+        Kxy(count1,count2) = exp(-(norm(X(count1,:) - Y(count2,:)))^2/(2*kernelSize^2));
+    end
+end
+
+A  = sum(exp(-((X - Y)).^2/(2*kernelSize^2)))/n;
+
+val = (A - sum(Kxy(:))/n^2)/sqrt((1 - sum(Kxx(:))/n^2)*(1 - sum(Kyy(:))/n^2));
Tip: Filter by directory path e.g. /media app.js to search for public/media/app.js.
Tip: Use camelCasing e.g. ProjME to search for ProjectModifiedEvent.java.
Tip: Filter by extension type e.g. /repo .js to search for all .js files in the /repo directory.
Tip: Separate your search with spaces e.g. /ssh pom.xml to search for src/ssh/pom.xml.
Tip: Use ↑ and ↓ arrow keys to navigate and return to view the file.
Tip: You can also navigate files with Ctrl+j (next) and Ctrl+k (previous) and view the file with Ctrl+o.
Tip: You can also navigate files with Alt+j (next) and Alt+k (previous) and view the file with Alt+o.