e640efa
committed
Commits
Comments (0)
Files changed (9)

+6 0CHANGELOG.txt

+4 4README.md

+43 0code/estimators/base_estimators/DChiSquare_kNN_k_estimation.m

+73 0code/estimators/base_estimators/DChiSquare_kNN_k_initialization.m

+41 0code/estimators/base_estimators/HPhi_spacing_estimation.m

+41 0code/estimators/base_estimators/HPhi_spacing_initialization.m

+58 0code/estimators/quick_tests/quick_test_DChiSquare.m

+1 0code/estimators/quick_tests/quick_test_Dequality.m

+62 0code/estimators/quick_tests/quick_test_HPhi.m
CHANGELOG.txt
+Phientropy (fentropy) estimation based on the spacing method: added; see 'HPhi_spacing_initialization.m', 'HPhi_spacing_estimation.m'.
+Pearson chi square divergence (chi square distance) estimation based on knearest neighbors: added; see 'DChiSquare_kNN_k_initialization.m', 'DChiSquare_kNN_k_estimation.m'.
+Quick test for Phientropy and Pearson chi square divergence: introduced; see 'quick_test_HPhi.m', 'quick_test_DChiSquare.m'.
Exponentiated JensenTsallis kernel estimators based on Tsallis entropy and JensenTsallis divergence: added; see 'KEJT1_HT_initialization.m', 'KEJT1_HT_estimation.m', 'KEJT2_DJT_initialization.m', 'KEJT2_DJT_estimation.m'.
Quick tests for the exponentiated JensenTsallis kernel estimators: added, see 'quick_test_EJT1.m', 'quick_test_KEJT2.m'. 'quick_test_Kpos_semidef.m': changed to cover the 2 new distribution kernel estimators.
README.md
 `entropy (H)`: Shannon entropy, R�nyi entropy, Tsallis entropy (Havrda and Charv�t entropy), complex entropy,
+ `entropy (H)`: Shannon entropy, R�nyi entropy, Tsallis entropy (Havrda and Charv�t entropy), complex entropy, Phientropy (fentropy),
 `mutual information (I)`: generalized variance, kernel canonical correlation analysis, kernel generalized variance, HilbertSchmidt independence criterion, Shannon mutual information (total correlation, multiinformation), L2 mutual information, R�nyi mutual information, Tsallis mutual information, copulabased kernel dependency, multivariate version of Hoeffding's Phi, SchweizerWolff's sigma and kappa, complex mutual information, CauchySchwartz quadratic mutual information, Euclidean distance based quadratic mutual information, distance covariance, distance correlation, approximate correntropy independence measure,
 `divergence (D)`: KullbackLeibler divergence (relative entropy, I directed divergence), L2 divergence, R�nyi divergence, Tsallis divergence, Hellinger distance, Bhattacharyya distance, maximum mean discrepancy (kernel distance), Jdistance (symmetrised KullbackLeibler divergence, J divergence), CauchySchwartz divergence, Euclidean distance based divergence, energy distance (specially the CramerVon Mises distance), JensenShannon divergence, JensenR�nyi divergence, K divergence, L divergence, certain fdivergences (Csisz�rMorimoto divergence, AliSilvey distance), nonsymmetric Bregman distance (Bregman divergence), JensenTsallis divergence, symmetric Bregman distance,
+ `divergence (D)`: KullbackLeibler divergence (relative entropy, I directed divergence), L2 divergence, R�nyi divergence, Tsallis divergence, Hellinger distance, Bhattacharyya distance, maximum mean discrepancy (kernel distance), Jdistance (symmetrised KullbackLeibler divergence, J divergence), CauchySchwartz divergence, Euclidean distance based divergence, energy distance (specially the CramerVon Mises distance), JensenShannon divergence, JensenR�nyi divergence, K divergence, L divergence, certain fdivergences (Csisz�rMorimoto divergence, AliSilvey distance), nonsymmetric Bregman distance (Bregman divergence), JensenTsallis divergence, symmetric Bregman distance, Pearson chi square divergence (chi square distance),
 `association measures (A)`, including `measures of concordance`: multivariate extensions of Spearman's rho (Spearman's rank correlation coefficient, grade correlation coefficient), correntropy, centered correntropy, correntropy coefficient, correntropy induced metric, centered correntropy induced metric, multivariate extension of Blomqvist's beta (medial correlation coefficient), multivariate conditional version of Spearman's rho, lower/upper tail dependence via conditional Spearman's rho,
 `kernels on distributions (K)`: expected kernel, Bhattacharyya kernel, probability product kernel, JensenShannon kernel, exponentiated JensenShannon kernel, JensenTsallis kernel, exponentiated JensenRenyi kernel(s), exponentiated JensenTsallis kernel(s).
 code: [zip](https://bitbucket.org/szzoli/ite/downloads/ITE0.45_code.zip), [tar.bz2](https://bitbucket.org/szzoli/ite/downloads/ITE0.45_code.tar.bz2),
+ code: [zip](https://bitbucket.org/szzoli/ite/downloads/ITE0.46_code.zip), [tar.bz2](https://bitbucket.org/szzoli/ite/downloads/ITE0.46_code.tar.bz2),
code/estimators/base_estimators/DChiSquare_kNN_k_estimation.m
+%We use the naming convention 'D<name>_estimation' to ease embedding new divergence estimation methods.
+% Y2: Y2(:,t) is the t^th sample from the second distribution. Note: the number of samples in Y1 [=size(Y1,2)] and Y2 [=size(Y2,2)] can be different.
+% Barnabas Poczos, Liang Xiong, Dougal Sutherland, and Jeff Schneider. Support distribution machines. Technical Report, Carnegie Mellon University, 2012. http://arxiv.org/abs/1202.0302. (estimation: Dtemp2 below)
+% Karl Pearson. On the criterion that a given system of deviations from the probable in the case of correlated system of variables is such that it can be reasonable supposed to have arisen from random sampling. Philosophical Magazine Series, 50:157172, 1900.
+%Copyright (C) 2013 Zoltan Szabo ("http://www.gatsby.ucl.ac.uk/~szabo/", "zoltan (dot) szabo (at) gatsby (dot) ucl (dot) ac (dot) uk")
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+%co.mult:OK. The information theoretical quantity of interest can be (and is!) estimated exactly [co.mult=1]; the computational complexity of the estimation is essentially the same as that of the 'up to multiplicative constant' case [co.mult=0].
code/estimators/base_estimators/DChiSquare_kNN_k_initialization.m
+%Initialization of the kNN (knearest neighbor, S={k}) based Pearson chi square divergence estimator.
+% 2)We use the naming convention 'D<name>_initialization' to ease embedding new divergence estimation methods.
+% post_init: {field_name1,field_value1,field_name2,field_value2,...}; cell array containing the names and the values of the cost object fields that are to be used
+%Copyright (C) 2013 Zoltan Szabo ("http://www.gatsby.ucl.ac.uk/~szabo/", "zoltan (dot) szabo (at) gatsby (dot) ucl (dot) ac (dot) uk")
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+%mandatory fields (following the template structure of the estimators to make uniform usage of the estimators possible):
+ %III: 'knnsearch' (Matlab Statistics Toolbox): parameters: co.k, co.NSmethod ('kdtree' or 'exhaustive').
+ %co.epsi = 0; %=0: exact kNN; >0: approximate kNN, the true (not squared) distances can not exceed the real distance more than a factor of (1+epsi).
code/estimators/base_estimators/HPhi_spacing_estimation.m
+%We use the naming convention 'H<name>_estimation' to ease embedding new entropy estimation methods.
+% Bert van Es. Estimating Functionals Related to a Density by a Class of Statistics Based on Spacings. Scandinavian Journal of Statistics, 19:6172, 1992.
+%Copyright (C) 2013 Zoltan Szabo ("http://www.gatsby.ucl.ac.uk/~szabo/", "zoltan (dot) szabo (at) gatsby (dot) ucl (dot) ac (dot) uk")
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+%co.mult:OK. The information theoretical quantity of interest can be (and is!) estimated exactly [co.mult=1]; the computational complexity of the estimation is essentially the same as that of the 'up to multiplicative constant' case [co.mult=0].
+m = floor(sqrt(num_of_samples));%m/num_of_samples>0; m/log(num_of_samples)>infty; m,num_of_samples>infty
code/estimators/base_estimators/HPhi_spacing_initialization.m
+% 2)We use the naming convention 'H<name>_initialization' to ease embedding new entropy estimation methods.
+% post_init: {field_name1,field_value1,field_name2,field_value2,...}; cell array containing the names and the values of the cost object fields that are to be used
+%Copyright (C) 2013 Zoltan Szabo ("http://www.gatsby.ucl.ac.uk/~szabo/", "zoltan (dot) szabo (at) gatsby (dot) ucl (dot) ac (dot) uk")
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+%mandatory fields (following the template structure of the estimators to make uniform usage of the estimators possible):
code/estimators/quick_tests/quick_test_DChiSquare.m
+%Quick test for (Pearson) chi^2 divergence estimators: analytical expression vs estimated value as a function of the sample number. In the test, uniform variables are considered.
+%Copyright (C) 2013 Zoltan Szabo ("http://www.gatsby.ucl.ac.uk/~szabo/", "zoltan (dot) szabo (at) gatsby (dot) ucl (dot) ac (dot) uk")
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+ Y2 = rand(d,num_of_samples_max) .* repmat(b,1,num_of_samples_max); %U[0,b], a<=b (coordinatewise) => Y1<<Y2
code/estimators/quick_tests/quick_test_HPhi.m
+%Quick test for Phientropy estimators: analytical expression vs estimated value as a function of the sample number. In the test, uniform variables are considered.
+%Copyright (C) 2013 Zoltan Szabo ("http://www.gatsby.ucl.ac.uk/~szabo/", "zoltan (dot) szabo (at) gatsby (dot) ucl (dot) ac (dot) uk")
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.