5452501
committed
Commits
Comments (0)
Files changed (13)

+7 1CHANGELOG.txt

+2 2README.md

+44 0code/estimators/base_estimators/CCE_expF_estimation.m

+39 0code/estimators/base_estimators/CCE_expF_initialization.m

+34 0code/estimators/base_estimators/HShannon_expF_estimation.m

+39 0code/estimators/base_estimators/HShannon_expF_initialization.m

+1 0code/estimators/quick_tests/quick_test_CCE.m

+2 0code/estimators/quick_tests/quick_test_HShannon.m

+1 0code/estimators/quick_tests/quick_test_Himreg.m

+30 0code/estimators/utilities/exp_family/expF_gradF.m

+22 0code/estimators/utilities/exp_family/expF_np1_np2_mult.m

+17 0code/estimators/utilities/ip.m

+1 0doc/ITE_documentation.txt
CHANGELOG.txt
+Shannon entropy and crossentropy estimation based on maximum likelihood estimation + analytical formula in the chosen exponential family: added; see 'HShannon_expF_initialization.m', 'HShannon_expF_estimation.m', 'CCE_expF_initialization', 'CCE_expF_estimation.m'.
+Quick tests: updated with the new estimators, see 'quick_test_HShannon.m', 'quick_test_Himreg.m' and 'quick_test_CCE.m'.
Renyi entropy estimation based on maximum likelihood estimation (MLE) + analytical formula in the exponential family: added; see 'HRenyi_expF_initialization.m', 'HRenyi_expF_estimation.m'.
+Renyi entropy estimation based on maximum likelihood estimation + analytical formula in the exponential family: added; see 'HRenyi_expF_initialization.m', 'HRenyi_expF_estimation.m'.
Tsallis entropy estimation based on MLE + analytical formula in the exponential family: added; see 'HTsallis_expF_initialization.m', 'HRenyi_Tsallis_estimation.m'.
Quick tests: updated according to the new estimators; see 'quick_test_HRenyi.m', 'quick_test_HTsallis.m', 'quick_test_Himreg.m'.
README.md
 code: [zip](https://bitbucket.org/szzoli/ite/downloads/ITE0.54_code.zip), [tar.bz2](https://bitbucket.org/szzoli/ite/downloads/ITE0.54_code.tar.bz2),
+ code: [zip](https://bitbucket.org/szzoli/ite/downloads/ITE0.55_code.zip), [tar.bz2](https://bitbucket.org/szzoli/ite/downloads/ITE0.55_code.tar.bz2),
code/estimators/base_estimators/CCE_expF_estimation.m
+%Estimates the crossentropy (CE) of Y1 and Y2 using maximum likelihood estimation (MLE) + analytical formula associated to the chosen exponential family.
+%We use the naming convention 'C<name>_estimation' to ease embedding new cross quantity estimation methods.
+% Y2: Y2(:,t) is the t^th sample from the second distribution. Note: the number of samples in Y1 [=size(Y1,2)] and Y2 [=size(Y2,2)] can be different.
+% Frank Nielsen and Richard Nock. Entropies and crossentropies of exponential families. In IEEE International Conference on Image Processing (ICIP), pages 3621–3624, 2010.
+%Copyright (C) 20122014 Zoltan Szabo ("http://www.gatsby.ucl.ac.uk/~szabo/", "zoltan (dot) szabo (at) gatsby (dot) ucl (dot) ac (dot) uk")
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+%co.mult:OK. The information theoretical quantity of interest can be (and is!) estimated exactly [co.mult=1]; the computational complexity of the estimation is essentially the same as that of the 'up to multiplicative constant' case [co.mult=0]. In other words, the estimation is carried out 'exactly' (instead of up to 'proportionality').
code/estimators/base_estimators/CCE_expF_initialization.m
+%Initialization of the exponential family based crossentropy estimator (maximum likelihood + analytical formula associated to the chosen exponential family).
+% 2)We use the naming convention 'C<name>_initialization' to ease embedding new cross quantity estimation methods.
+% mult: is a multiplicative constant relevant (needed) in the estimation; '=1' means yes (='exact' estimation), '=0' no (=estimation up to 'proportionality').
+% post_init: {field_name1,field_value1,field_name2,field_value2,...}; cell array containing the names and the values of the cost object fields that are to be used
+%Copyright (C) 20122014 Zoltan Szabo ("http://www.gatsby.ucl.ac.uk/~szabo/", "zoltan (dot) szabo (at) gatsby (dot) ucl (dot) ac (dot) uk")
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+%mandatory fields (following the template structure of the estimators to make uniform usage of the estimators possible):
code/estimators/base_estimators/HShannon_expF_estimation.m
+%Estimates the Shannon (H) of Y using maximum likelihood estimation (MLE) + analytical formula corresponding in the chosen exponential family.
+%We use the naming convention 'H<name>_estimation' to ease embedding new entropy estimation methods.
+% Frank Nielsen and Richard Nock. A closedform expression for the SharmaMittal entropy of exponential families. Journal of Physics A: Mathematical and Theoretical, 45:032003, 2012. (analytical formula)
+%Copyright (C) 20122014 Zoltan Szabo ("http://www.gatsby.ucl.ac.uk/~szabo/", "zoltan (dot) szabo (at) gatsby (dot) ucl (dot) ac (dot) uk")
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+%co.mult:OK. The information theoretical quantity of interest can be (and is!) estimated exactly [co.mult=1]; the computational complexity of the estimation is essentially the same as that of the 'up to multiplicative constant' case [co.mult=0]. In other words, the estimation is carried out 'exactly' (instead of up to 'proportionality').
code/estimators/base_estimators/HShannon_expF_initialization.m
+%Initialization of the exponential family based Shannon entropy estimator (maximum likelihood + analytical formula in the chosen exponential family).
+% 2)We use the naming convention 'H<name>_initialization' to ease embedding new entropy estimation methods.
+% mult: is a multiplicative constant relevant (needed) in the estimation; '=1' means yes (='exact' estimation), '=0' no (=estimation up to 'proportionality').
+% post_init: {field_name1,field_value1,field_name2,field_value2,...}; cell array containing the names and the values of the cost object fields that are to be used
+%Copyright (C) 20122014 Zoltan Szabo ("http://www.gatsby.ucl.ac.uk/~szabo/", "zoltan (dot) szabo (at) gatsby (dot) ucl (dot) ac (dot) uk")
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+%mandatory fields (following the template structure of the estimators to make uniform usage of the estimators possible):
code/estimators/quick_tests/quick_test_HShannon.m
code/estimators/utilities/exp_family/expF_gradF.m
+%Computes the gradient of the lognormalizer (gradF) at a given natural parameter value (np) for the input exponential family.
+% distr = 'normal': np.t1 = C^{1}*m, np.t2 = 1/2*C^{1}, where m is the mean, C is the covariance matrix.
+%Copyright (C) 20122014 Zoltan Szabo ("http://www.gatsby.ucl.ac.uk/~szabo/", "zoltan (dot) szabo (at) gatsby (dot) ucl (dot) ac (dot) uk")
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+ case 'normal' %Ref: Frank Nielsen, Vincent Garcia. Statistical exponential families: A digest with flash cards. "http://arxiv.org/abs/0911.4863"
code/estimators/utilities/exp_family/expF_np1_np2_mult.m
+%Inner product of the 'natural parameter' variables, np1 and np2. (np1,np2: structures with the same fields; example: np1.t1, np1.t2, np2.t1, np2.t2).
+%Copyright (C) 20122014 Zoltan Szabo ("http://www.gatsby.ucl.ac.uk/~szabo/", "zoltan (dot) szabo (at) gatsby (dot) ucl (dot) ac (dot) uk")
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
code/estimators/utilities/ip.m
+%Copyright (C) 20122014 Zoltan Szabo ("http://www.gatsby.ucl.ac.uk/~szabo/", "zoltan (dot) szabo (at) gatsby (dot) ucl (dot) ac (dot) uk")
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.