Commits
Comments (0)
Files changed (11)

+5 0CHANGELOG.txt

+2 2README.md

+38 0code/estimators/base_estimators/DKL_expF_estimation.m

+39 0code/estimators/base_estimators/DKL_expF_initialization.m

+49 0code/estimators/base_estimators/HShannon_spacing_VKDE_estimation.m

+29 0code/estimators/base_estimators/HShannon_spacing_VKDE_initialization.m

+2 0code/estimators/quick_tests/tests_analytical_vs_estimation/quick_test_DKL.m

+2 1code/estimators/quick_tests/tests_analytical_vs_estimation/quick_test_HShannon.m

+2 0code/estimators/quick_tests/tests_other_consistency/quick_test_Dequality.m

+20 0code/estimators/utilities/exp_family/expF_Bregman_distance.m

+22 0code/estimators/utilities/exp_family/expF_np1_np2_subtract.m
README.md
 code: [zip](https://bitbucket.org/szzoli/ite/downloads/ITE0.56_code.zip), [tar.bz2](https://bitbucket.org/szzoli/ite/downloads/ITE0.56_code.tar.bz2),
+ code: [zip](https://bitbucket.org/szzoli/ite/downloads/ITE0.57_code.zip), [tar.bz2](https://bitbucket.org/szzoli/ite/downloads/ITE0.57_code.tar.bz2),
code/estimators/base_estimators/DKL_expF_estimation.m
+%Estimates the KullbackLeibler divergence (D) of Y1 and Y2 using maximum likelihood estimation (MLE) + analytical formula associated to the chosen exponential family.
+%We use the naming convention 'D<name>_estimation' to ease embedding new divergence estimation methods.
+% Y2: Y2(:,t) is the t^th sample from the second distribution. Note: the number of samples in Y1 [=size(Y1,2)] and Y2 [=size(Y2,2)] can be different.
+% Frank Nielsen and Sylvain Boltz. The BurbeaRao and Bhattacharyya centroids. IEEE Transaction on Information Theory, 57:5455–5466, 2011.
+%Copyright (C) 20122014 Zoltan Szabo ("http://www.gatsby.ucl.ac.uk/~szabo/", "zoltan (dot) szabo (at) gatsby (dot) ucl (dot) ac (dot) uk")
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+%co.mult:OK. The information theoretical quantity of interest can be (and is!) estimated exactly [co.mult=1]; the computational complexity of the estimation is essentially the same as that of the 'up to multiplicative constant' case [co.mult=0]. In other words, the estimation is carried out 'exactly' (instead of up to 'proportionality').
code/estimators/base_estimators/DKL_expF_initialization.m
+%Initialization of the exponential family based KullbackLeibler divergence estimator (maximum likelihood + analytical formula associated to the chosen exponential family).
+% 2)We use the naming convention 'D<name>_initialization' to ease embedding new divergence estimation methods.
+% mult: is a multiplicative constant relevant (needed) in the estimation; '=1' means yes (='exact' estimation), '=0' no (=estimation up to 'proportionality').
+% post_init: {field_name1,field_value1,field_name2,field_value2,...}; cell array containing the names and the values of the cost object fields that are to be used
+%Copyright (C) 20122014 Zoltan Szabo ("http://www.gatsby.ucl.ac.uk/~szabo/", "zoltan (dot) szabo (at) gatsby (dot) ucl (dot) ac (dot) uk")
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+%mandatory fields (following the template structure of the estimators to make uniform usage of the estimators possible):
code/estimators/base_estimators/HShannon_spacing_VKDE_estimation.m
+%Estimates the Shannon entropy (H) of Y using the Vasicek's spacing method corrected with KDE (kernel density estimation) at the left and right sides.
+%We use the naming convention 'H<name>_estimation' to ease embedding new entropy estimation methods.
+%REFERENCE: Havva Alizadeh Noughabi and Reza Alizadeh Noughabi. On the entropy estimators. Journal of Statistical Computatiion and Simulation, 83:784792, 2013.
+%Copyright (C) 2013 Zoltan Szabo ("http://www.gatsby.ucl.ac.uk/~szabo/", "zoltan (dot) szabo (at) gatsby (dot) ucl (dot) ac (dot) uk")
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+%co.mult:OK. The information theoretical quantity of interest can be (and is!) estimated exactly [co.mult=1]; the computational complexity of the estimation is essentially the same as that of the 'up to multiplicative constant' case [co.mult=0].
code/estimators/base_estimators/HShannon_spacing_VKDE_initialization.m
+%Initialization of the Shannon differential entropy (H) estimator based on Vasicek's spacing method corrected with KDE (kernel density estimation) at the left and right sides.
+% 2)We use the naming convention 'H<name>_initialization' to ease embedding new entropy estimation methods.
+%Copyright (C) 2013 Zoltan Szabo ("http://www.gatsby.ucl.ac.uk/~szabo/", "zoltan (dot) szabo (at) gatsby (dot) ucl (dot) ac (dot) uk")
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+%mandatory fields (following the template structure of the estimators to make uniform usage of the estimators possible):
code/estimators/quick_tests/tests_analytical_vs_estimation/quick_test_HShannon.m
code/estimators/utilities/exp_family/expF_Bregman_distance.m
+%Computes the Bregman distance (B) of the natural parameters np1 and np2. The Bregman distance is defined by the lognormalizer of the given exponential family (distr).
+%Copyright (C) 20122014 Zoltan Szabo ("http://www.gatsby.ucl.ac.uk/~szabo/", "zoltan (dot) szabo (at) gatsby (dot) ucl (dot) ac (dot) uk")
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+B = expF_F(distr,np1)  expF_F(distr,np2)  expF_np1_np2_mult(diff_nps,expF_gradF(distr,np2)); %F(theta1)  F(theta2)  <theta1  theta2, nabla F(theta2)>
code/estimators/utilities/exp_family/expF_np1_np2_subtract.m
+%Subtracts the natural parameter np2 from np1, the result is np. (np1, np2: structures with the same fields; example: np1.t1, np1.t2, np2.t1, np2.t2).
+%Copyright (C) 20122014 Zoltan Szabo ("http://www.gatsby.ucl.ac.uk/~szabo/", "zoltan (dot) szabo (at) gatsby (dot) ucl (dot) ac (dot) uk")
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
CHANGELOG.txt