Zoltan Szabo avatar Zoltan Szabo committed 5e2af9c

Probability product kernel estimation based on k-nearest neighbors, Jensen-Shannon kernel estimation: added; see 'KPP_kNN_k_initialization.m', 'KPP_kNN_k_estimation.m', 'KJS_DJS_initialization.m' and 'KJS_DJS_estimation.m'.

Comments (0)

Files changed (7)

+v0.41 (July 12, 2013):
+-Probability product kernel estimation based on k-nearest neighbors: added; see 'KPP_kNN_k_initialization.m' and 'KPP_kNN_k_estimation.m'.
+-Jensen-Shannon kernel estimation: added; see 'KJS_DJS_initialization.m' and 'KJS_DJS_estimation.m'.
+
 v0.40 (June 23, 2013):
 -Bhattacharyya kernel estimation based on k-nearest neighbors: added; see 'KBhattacharyya_kNN_k_initialization.m' and 'KBhattacharyya_kNN_k_estimation.m'.
 -Expected kernel estimation: added; see 'Kexpected_initialization.m', 'Kexpected_estimation.m'.
 - `divergence (D)`: Kullback-Leibler divergence (relative entropy, I directed divergence), L2 divergence, Rényi divergence, Tsallis divergence, Hellinger distance, Bhattacharyya distance, maximum mean discrepancy (kernel distance), J-distance (symmetrised Kullback-Leibler divergence, J divergence), Cauchy-Schwartz divergence, Euclidean distance based divergence, energy distance (specially the Cramer-Von Mises distance), Jensen-Shannon divergence, Jensen-Rényi divergence, K divergence, L divergence, certain f-divergences (Csiszár-Morimoto divergence, Ali-Silvey distance), non-symmetric Bregman distance (Bregman divergence), Jensen-Tsallis divergence, symmetric Bregman distance,
 - `association measures (A)`, including `measures of concordance`: multivariate extensions of Spearman's rho (Spearman's rank correlation coefficient, grade correlation coefficient), correntropy, centered correntropy, correntropy coefficient, correntropy induced metric, centered correntropy induced metric, multivariate extension of Blomqvist's beta (medial correlation coefficient), multivariate conditional version of Spearman's rho, lower/upper tail dependence via conditional Spearman's rho,
 - `cross quantities (C)`: cross-entropy,
-- `kernels on distributions (K)`: expected kernel, Bhattacharyya kernel.
+- `kernels on distributions (K)`: expected kernel, Bhattacharyya kernel, probability product kernel, Jensen-Shannon kernel.
 
 ITE offers solution methods for 
 
 
 **Download** the latest release: 
 
-- code: [zip](https://bitbucket.org/szzoli/ite/downloads/ITE-0.40_code.zip), [tar.bz2](https://bitbucket.org/szzoli/ite/downloads/ITE-0.40_code.tar.bz2), 
-- [documentation (pdf)](https://bitbucket.org/szzoli/ite/downloads/ITE-0.40_documentation.pdf).
+- code: [zip](https://bitbucket.org/szzoli/ite/downloads/ITE-0.41_code.zip), [tar.bz2](https://bitbucket.org/szzoli/ite/downloads/ITE-0.410_code.tar.bz2), 
+- [documentation (pdf)](https://bitbucket.org/szzoli/ite/downloads/ITE-0.41_documentation.pdf).
 
 

code/H_I_D_A_C_K/base_estimators/KPP_kNN_k_estimation.m

+function [K] = KPP_kNN_k_estimation(Y1,Y2,co)
+%Estimates the probability product kernel of two distributions from which we have samples, Y1 and Y2. The estimation is based on k-nearest neighbors (S={k}).
+%
+%We use the naming convention 'K<name>_estimation' to ease embedding new kernels on distributions.
+%
+%INPUT:
+%  Y1: Y1(:,t) is the t^th sample from the first distribution.
+%  Y2: Y2(:,t) is the t^th sample from the second distribution. Note: the number of samples in Y1 [=size(Y1,2)] and Y2 [=size(Y2,2)] can be different.
+%  co: estimator object of a kernel on distributions.
+%
+%REFERENCE: 
+%   Barnabas Poczos and Liang Xiong and Dougal Sutherland and Jeff Schneider. Support Distribution Machines. Technical Report, 2012. "http://arxiv.org/abs/1202.0302" (k-nearest neighbor based estimation)
+%   Tony Jebara, Risi Kondor, and Andrew Howard. Probability product kernels. Journal of Machine Learning Research, 5:819-844, 2004. (probability product kernels --spec--> Bhattacharyya kernel)
+%
+%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
+%
+%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
+%
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
+%
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+
+%co.mult:OK.
+
+%verification:
+    if size(Y1,1)~=size(Y2,1)
+        error('The dimension of the samples in Y1 and Y2 must be equal.');
+    end
+
+if co.p %[p(x)dx]
+    K = estimate_Dtemp2(Y1,Y2,co);
+else %[q(x)dx]
+    K = estimate_Dtemp2(Y2,Y1,co);
+end

code/H_I_D_A_C_K/base_estimators/KPP_kNN_k_initialization.m

+function [co] = KPP_kNN_k_initialization(mult)
+%Initialization of the probability product kernel estimator. The estimation is based on k-nearest neighbors (S={k}).
+%
+%Note:
+%   1)The estimator is treated as a cost object (co).
+%   2)We use the naming convention 'K<name>_initialization' to ease embedding new estimators for kernels on distributions.
+%
+%INPUT:
+%   mult: is a multiplicative constant relevant (needed) in the estimation; '=1' means yes, '=0' no.
+%OUTPUT:
+%   co: cost object (structure).
+%
+%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
+%
+%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
+%
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
+%
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+
+%mandatory fields:
+    co.name = 'PP_kNN_k';
+    co.mult = mult;
+    
+%other fields:
+    %Possibilities for 'co.kNNmethod' (see 'kNN_squared_distances.m'): 
+        %I: 'knnFP1': fast pairwise distance computation and C++ partial sort; parameter: co.k.        
+        %II: 'knnFP2': fast pairwise distance computation; parameter: co.k.
+        %III: 'knnsearch' (Matlab Statistics Toolbox): parameters: co.k, co.NSmethod ('kdtree' or 'exhaustive').
+        %IV: 'ANN' (approximate nearest neighbor); parameters: co.k, co.epsi. 
+		%I:
+            co.kNNmethod = 'knnFP1';
+            co.k = 3;%k-nearest neighbors
+  	    %II:
+            %co.kNNmethod = 'knnFP2';
+            %co.k = 3;%k-nearest neighbors
+        %III:
+            %co.kNNmethod = 'knnsearch';
+            %co.k = 3;%k-nearest neighbors
+            %co.NSmethod = 'kdtree';
+        %IV:
+            %co.kNNmethod = 'ANN';
+            %co.k = 3;%k-nearest neighbors
+            %co.epsi = 0; %=0: exact kNN; >0: approximate kNN, the true (not squared) distances can not exceed the real distance more than a factor of (1+epsi).
+    %Kernel parameter:
+        co.rho  = 2; %assumption: >0          
+	%Possibilities for rewriting the probability product kernel:
+		%I [\int p^{rho}(x)q^{rho}(x)dx = \int p^{rho-1}(x)q^{rho}(x) p(x)dx]:
+			co.p = 1; %use p [p(x)dx]
+		%II [\int p^{rho}(x)q^{rho}(x)dx = \int q^{rho-1}(x)p^{rho}(x) q(x)dx]:
+			%co.p = 0; %use q instead [q(x)dx]
+	%Fixed, do not change it:
+		co.a = co.rho-1; 
+		co.b = co.rho;
+				
+%initialize the ann wrapper in Octave, if needed:
+    initialize_Octave_ann_wrapper_if_needed(co.kNNmethod);
+
+

code/H_I_D_A_C_K/meta_estimators/KJS_DJS_estimation.m

+function [K] = KJS_DJS_estimation(Y1,Y2,co)
+%Estimates the Jensen-Shannon kernel of two distributions from which we have samples (Y1 and Y2) using the relation: K_JS(f_1,f_2) = log(2) - D_JS(f_1,f_2), where D_JS is the Jensen-Shannon divergence.
+%
+%%Note:
+%   1)We use the naming convention 'K<name>_estimation' to ease embedding new kernels on distributions.
+%   2)This is a meta method: the Jensen-Shannon divergence estimator can be arbitrary.
+%
+%INPUT:
+%  Y1: Y1(:,t) is the t^th sample from the first distribution.
+%  Y2: Y2(:,t) is the t^th sample from the second distribution. Note: the number of samples in Y1 [=size(Y1,2)] and Y2 [=size(Y2,2)] can be different.
+%  co: estimator object of a kernel on distributions.
+%
+%REFERENCE: 
+%   Andre F. T. Martins, Noah A. Smith, Eric P. Xing, Pedro M. Q. Aguiar, and Mario A. T. Figueiredo. Nonextensive information theoretical kernels on measures. Journal of Machine Learning Research, 10:935-975, 2009.
+%   Andre F. T. Martins, Pedro M. Q. Aguiar, and Mario A. T. Figueiredo. Tsallis kernels on measures. In Information Theory Workshop (ITW), pages 298-302, 2008.
+%
+%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
+%
+%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
+%
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
+%
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+
+%co.mult:OK.
+
+%verification:
+    if size(Y1,1)~=size(Y2,1)
+        error('The dimension of the samples in Y1 and Y2 must be equal.');
+    end
+
+K = log(2) - D_estimation(Y1,Y2,co.member_co);

code/H_I_D_A_C_K/meta_estimators/KJS_DJS_initialization.m

+function [co] = KJS_DJS_initialization(mult)
+%Initialization of the Jensen-Shannon kernel estimator defined according to the relation: K_JS(f_1,f_2) = log(2) - D_JS(f_1,f_2), where D_JS is the Jensen-Shannon divergence.
+%
+%Note:
+%   1)The estimator is treated as a cost object (co).
+%   2)We use the naming convention 'K<name>_initialization' to ease embedding new estimators for kernels on distributions.
+%   3)This is a meta method: the Jensen-Shannon divergence estimator can arbitrary.
+%
+%INPUT:
+%   mult: is a multiplicative constant relevant (needed) in the estimation; '=1' means yes, '=0' no.
+%OUTPUT:
+%   co: cost object (structure).
+%
+%Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu")
+%
+%This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox.
+%
+%ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
+%the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+%
+%This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
+%
+%You should have received a copy of the GNU General Public License along with ITE. If not, see <http://www.gnu.org/licenses/>.
+
+%mandatory fields:
+    co.name = 'JS_DJS';
+    co.mult = mult;
+    
+%other fields:
+    co.member_name = 'JensenShannon_HShannon'; %you can change it to any Jensen-Shannon divergence estimator
+    co.member_co = D_initialization(co.member_name,mult);

code/shared/embedded/KDP/mat_oct/mexme.m

 else
 	if isunix
 		mex CFLAGS='-std=gnu99 -D_GNU_SOURCE' -outdir private   -I../kdpee  private/kdpeemex.c ../src/kdpee.c
+		%mex CFLAGS='-D_GNU_SOURCE' -outdir private   -I../kdpee  private/kdpeemex.c ../src/kdpee.c
 	else ispc
 		mex -outdir private   -I../kdpee  private/kdpeemex.c ../src/kdpee.c
 	end
Tip: Filter by directory path e.g. /media app.js to search for public/media/app.js.
Tip: Use camelCasing e.g. ProjME to search for ProjectModifiedEvent.java.
Tip: Filter by extension type e.g. /repo .js to search for all .js files in the /repo directory.
Tip: Separate your search with spaces e.g. /ssh pom.xml to search for src/ssh/pom.xml.
Tip: Use ↑ and ↓ arrow keys to navigate and return to view the file.
Tip: You can also navigate files with Ctrl+j (next) and Ctrl+k (previous) and view the file with Ctrl+o.
Tip: You can also navigate files with Alt+j (next) and Alt+k (previous) and view the file with Alt+o.