Comments (0)
Files changed (13)

+5 0CHANGELOG.txt

+1 1README.md

+38 0code/H_I_D/base_estimators/DBhattacharyya_kNN_k_estimation.m

+60 0code/H_I_D/base_estimators/DBhattacharyya_kNN_k_initialization.m

+36 0code/H_I_D/base_estimators/DHellinger_kNN_k_estimation.m

+60 0code/H_I_D/base_estimators/DHellinger_kNN_k_initialization.m

+6 6code/H_I_D/base_estimators/DL2_kNN_k_estimation.m

+1 1code/H_I_D/base_estimators/HRenyi_weightedkNN_estimation.m

+3 3code/H_I_D/base_estimators/HShannon_kNN_k_estimation.m

+36 0code/H_I_D/utilities/estimate_Dab.m

+1 1code/H_I_D/utilities/estimate_Ialpha.m

+16 0code/H_I_D/utilities/volume_of_the_unit_ball.m

+0 0doc/ITE_documentation.pdf
CHANGELOG.txt
+The Hellinger and Bhattacharyya distances are now available in ITE. They can be estimated via knearest neighbor methods; see 'DHellinger_kNN_k_initialization.m', 'DHellinger_kNN_k_estimation.m', 'DBhattacharyya_kNN_k_initialization.m', and 'DBhattacharyya_kNN_k_estimation.m'.
MonteCarlo simulation to compute the additive constants in Renyi entropy estimation: added; see 'estimate_HRenyi_constant.m'.
README.md
ITE can estimate Shannon, R�nyi, Tsallis entropy; generalized variance, kernel canonical correlation analysis, kernel generalized variance, HilbertSchmidt independence criterion, Shannon, L2, R�nyi, Tsallis mutual information, copulabased kernel dependency, multivariate version of Hoeffding's Phi, SchweizerWolff's sigma and kappa; complex variants of entropy and mutual information; L2, R�nyi, Tsallis divergence, maximum mean discrepancy, and Jdistance.
+ITE can estimate Shannon, R�nyi, Tsallis entropy; generalized variance, kernel canonical correlation analysis, kernel generalized variance, HilbertSchmidt independence criterion, Shannon, L2, R�nyi, Tsallis mutual information, copulabased kernel dependency, multivariate version of Hoeffding's Phi, SchweizerWolff's sigma and kappa; complex variants of entropy and mutual information; L2, R�nyi, Tsallis divergence; Hellinger, Bhattacharyya distance; maximum mean discrepancy, and Jdistance.
code/H_I_D/base_estimators/DBhattacharyya_kNN_k_estimation.m
+%using the kNN method (S={k}). The number of samples in X [=size(X,2)] and Y [=size(Y,2)] can be different. Cost parameters are provided in the cost object co.
+%We make use of the naming convention 'D<name>_estimation', to ease embedding new divergence estimation methods.
+% Barnabas Poczos and Liang Xiong and Dougal Sutherland and Jeff Schneider. Support Distribution Machines. Technical Report, 2012. "http://arxiv.org/abs/1202.0302"
+%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+D = log(abs(D_ab));%abs() to avoid possible 'log(negative)' values due to the finite number of samples
code/H_I_D/base_estimators/DBhattacharyya_kNN_k_initialization.m
+% 2)We make use of the naming convention 'D<name>_initialization', to ease embedding new divergence estimation methods.
+%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+ %III: 'knnsearch' (Matlab Statistics Toolbox): parameters: co.k, co.NSmethod ('kdtree' or 'exhaustive').
+ %co.epsi = 0; %=0: exact kNN; >0: approximate kNN, the true (not squared) distances can not exceed the real distance more than a factor of (1+epsi).
code/H_I_D/base_estimators/DHellinger_kNN_k_estimation.m
+%using the kNN method (S={k}). The number of samples in X [=size(X,2)] and Y [=size(Y,2)] can be different. Cost parameters are provided in the cost object co.
+%We make use of the naming convention 'D<name>_estimation', to ease embedding new divergence estimation methods.
+% Barnabas Poczos and Liang Xiong and Dougal Sutherland and Jeff Schneider. Support Distribution Machines. Technical Report, 2012. "http://arxiv.org/abs/1202.0302"
+%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
code/H_I_D/base_estimators/DHellinger_kNN_k_initialization.m
+% 2)We make use of the naming convention 'D<name>_initialization', to ease embedding new divergence estimation methods.
+%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+ %III: 'knnsearch' (Matlab Statistics Toolbox): parameters: co.k, co.NSmethod ('kdtree' or 'exhaustive').
+ %co.epsi = 0; %=0: exact kNN; >0: approximate kNN, the true (not squared) distances can not exceed the real distance more than a factor of (1+epsi).
code/H_I_D/base_estimators/DL2_kNN_k_estimation.m
term3 = (num_of_samplesX  1) * c * (co.k2) * (co.k1) / (co.k * (num_of_samplesY*c)^2) * (dist_k_XX.^d ./ dist_k_YX.^(2*d));
+term3 = mean((dist_k_XX.^d) ./ (dist_k_YX.^(2*d))) * (num_of_samplesX  1) * (co.k2) * (co.k1) / (num_of_samplesY^2*c*co.k);
code/H_I_D/base_estimators/HShannon_kNN_k_estimation.m
%We make use of the naming convention 'H<name>_estimation', to ease embedding new entropy estimation methods.
% M. N. Goria, Nikolai N. Leonenko, V. V. Mergel, and P. L. Novi Inverardi. A new class of random vector entropy estimators and its applications in testing statistical hypotheses. Journal of Nonparametric Statistics, 17: 277�297, 2005. (S={k})
+% M. N. Goria, Nikolai N. Leonenko, V. V. Mergel, and P. L. Novi Inverardi. A new class of random vector entropy estimators and its applications in testing statistical hypotheses. Journal of Nonparametric Statistics, 17: 277�297, 2005. (S={k})
% Harshinder Singh, Neeraj Misra, Vladimir Hnizdo, Adam Fedorowicz and Eugene Demchuk. Nearest neighbor estimates of entropy. American Journal of Mathematical and Management Sciences, 23, 301321, 2003. (S={k})
% L. F. Kozachenko and Nikolai N. Leonenko. A statistical estimate for the entropy of a random vector. Problems of Information Transmission, 23:9�16, 1987. (S={1})
+% L. F. Kozachenko and Nikolai N. Leonenko. A statistical estimate for the entropy of a random vector. Problems of Information Transmission, 23:9�16, 1987. (S={1})
%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
H = log(num_of_samples1)  psi(co.k) + log(V) + d / num_of_samples * sum(log(sqrt(squared_distances(co.k,:)))); %sqrt <= squared_distances,
code/H_I_D/utilities/estimate_Dab.m
+%Estimates D_ab = \int p^a(x)q^b(x)dx; the Hellinger distance and the Bhattacharyya distance are simple functions of this quantity.
+%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+D_ab = (num_of_samplesX1)^(a) * num_of_samplesY^(b) * B * mean(dist_k_XX.^(d*a).*dist_k_YX.^(d*b));
code/H_I_D/utilities/estimate_Ialpha.m
I_alpha = (num_of_samples1) / num_of_samples * V^(1co.alpha) * C^(1co.alpha) * s / (num_of_samples1)^(co.alpha);
code/H_I_D/utilities/volume_of_the_unit_ball.m
+%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.