1. Zoltán Szabó
  2. ITE



Zoltán Szabó 3dfc21d 

Zoltan Szabo 5e2af9c 

Zoltan Szabo e50e170 

Zoltan Szabo 63da916 

Zoltan Szabo 2c2556b 

Zoltan Szabo d0b19a8 

Zoltan Szabo d708ee0 

Zoltan Szabo 8744654 

Zoltan Szabo 5da18f8 

Zoltan Szabo e448f3f 
Zoltan Szabo f4fc2a3 
Zoltan Szabo e448f3f 
Zoltan Szabo 2e8afa0 

Zoltan Szabo 5810f95 

Zoltan Szabo d50e55c 

Zoltan Szabo 5810f95 
Zoltan Szabo 2e7c93c 

Zoltan Szabo 5810f95 
Zoltan Szabo 1e31d39 

Zoltan Szabo fa6edf9 

Zoltan Szabo 98e0522 

Zoltan Szabo 750bcf2 

Zoltan Szabo 933410b 

Zoltan Szabo ec09b46 

Zoltan Szabo c2c43aa 
Zoltan Szabo 0f83211 

Zoltan Szabo f90e2f2 
Zoltan Szabo 0b610c4 

Zoltan Szabo 184c85e 

Zoltan Szabo 0da37f9 

Zoltan Szabo 30943a7 

Zoltan Szabo cb07f5b 

Zoltan Szabo 0a3d2e6 
Zoltan Szabo 0fca81f 

Zoltan Szabo 2069255 
Zoltan Szabo 0fca81f 
Zoltan Szabo e9e40a2 
Zoltan Szabo 09db668 

Zoltan Szabo 87856c5 
Zoltan Szabo 8b71da6 
Zoltan Szabo 87856c5 
Zoltan Szabo b5181ca 

Zoltan Szabo 210e7ee 

Zoltan Szabo db2c2e4 

Zoltan Szabo 3b72fd2 

Zoltan Szabo 210e7ee 
Zoltan Szabo 3b72fd2 
Zoltan Szabo 1fb133f 

Zoltan Szabo 3e697ee 
Zoltan Szabo ca6860b 

Zoltan Szabo 210e7ee 
Zoltan Szabo ca6860b 
Zoltan Szabo d64cd16 

Zoltan Szabo 87856c5 
Zoltan Szabo d64cd16 

Zoltan Szabo b5181ca 
Zoltan Szabo d64cd16 

Zoltan Szabo 14141a1 

v0.42 (September 7, 2013):

-Initialization and estimation of high-level information theoretical estimators (H/I/D/A/C/K): 'eval' changed to 'function handles' to speed up computations; see '<X>_initialization.m' and '<X>_estimation.m', where <X>='H'/'I'/'D'/'A'/'C'/'K'.
-Cost object initialization now allows setting field values (alpha, number of kNN-s, ...) through its argument. This makes ITE more user-friendly. It also enables overriding default values and is useful in meta estimators for parameter inheritence. See 'post_initialization.m'; '<X>_initialization.m' and '<X><cost_name>_initialization.m', where <X>='H'/'I'/'D'/'A'/'C'/'K'. For the usage in meta estimators, see
'HTsallis_HRenyi_initialization.m', 'IRenyi_DRenyi_initialization.m', 'ITsallis_DTsallis_initialization.m', 'IRenyi_HRenyi_initialization.m', 'DJensenRenyi_HRenyi_initialization.m', 'DJensenTsallis_HTsallis_initialization.m', 'DsymBregman_DBregman_initialization.m', 'KJS_DJS_initialization.m'. For further details, see doc: Section 3.3.2.
-Quick tests introduced: consistency of the estimators, positive semi-definiteness of Gram matrices determined by distribution kernels, image registration; see directory 'quick_tests', doc: Section 5, Section F.

Refactorization; documentation improved:
-Function names added to the first line of the code descriptions to improve help.
-Copyright information: detached from help (one empty line inserted before license information).
-Homepage, email address: updated.
-Importance of co.mult: explained in more detail (doc + code (see 'co.mult:OK...')).
-co.alpha: changed to be the same in all Renyi/Tsallis estimators, co.alpha = 0.99 -- uniformity. 
-Typo discovered and corrected: 'HShannon_DKL_N_estimation.m': 'Y_normal = ... + m' -> 'Y_normal = ... + repmat(m,1,num_of_samples)' (typo for d>1).
-Directory 'H_I_D_A_C_K' renamed to 'estimators'. 'ITE_install.m': updated accordingly.
-Comment on the relation between mutual information and total correlation (or multi-information): added [doc + code (see 'IShannon_HShannon_initialization.m')].
-To ease the usage of ITE 'ITE_add_to_path.m' and 'ITE_remove_from_path.m': added. These functions make it possible to add/remove the ITE code directory to/from the Matlab/Octave PATH.
-Explicit instructions for the installation of the toolbox added to 'ITE_install.m' when no compilation is carried out.
-'HRPensemble_estimation.m': verification that 'RP dimension <= dimension of the samples' added.
-'mexme.m' (=compilation of KDP): an alternative compilation option added.
-In pathological cases (quite small dimension and large number of samples) the estimated Renyi additive constant (see 'estimate_HRenyi_constant.m') can be Inf. This case is now handled by introducing 'replace_Infs_with_max.m', see 'compute_length_HRenyi_kNN_1tok.m', 'compute_length_HRenyi_kNN_S.m'.
- 'MatlabBGL' and the associated GSF based Renyi entropy estimator ('HRenyi_GSF_initialization.m', HRenyi_GSF_estimation.m', 'compute_length_HRenyi_GSF.m'): deleted (self-containedness improved + space considerations); 'compute_MST.m','HRenyi_MST_initialization.m', 'HRenyi_MST_estimation.m': modified accordingly.
-The embedded 'sqdistance.m' (squared Euclidean distance computation) function could give some small, but negative values in the diagonal of 'sqdistance(Y)'; corrected. 
-'Guideance' on the choice of the estimators: included, see doc: Section 3.4.
-Figure listing all the estimators and their relations: added, see doc: Section 3.3.2: Fig. 1.
-Note on compiler requirements: added (see doc).
-ITE_install.m: made to be more user-friendly; detection of the already (i) deleted 'ann_wrapperO'/'ann_wrapperM' directory, (ii) downloaded ARfit package: added.

v0.41 (July 12, 2013):
-Probability product kernel estimation based on k-nearest neighbors: added; see 'KPP_kNN_k_initialization.m' and 'KPP_kNN_k_estimation.m'.
-Jensen-Shannon kernel estimation: added; see 'KJS_DJS_initialization.m' and 'KJS_DJS_estimation.m'.

v0.40 (June 23, 2013):
-Bhattacharyya kernel estimation based on k-nearest neighbors: added; see 'KBhattacharyya_kNN_k_initialization.m' and 'KBhattacharyya_kNN_k_estimation.m'.
-Expected kernel estimation: added; see 'Kexpected_initialization.m', 'Kexpected_estimation.m'.
-Directory 'H_I_D_A_C' renamed to 'H_I_D_A_C_K'; 'ITE_install.m' modified accordingly.
-Kernel on distributions (K) object type: added; see 'K_initialization.m', 'K_estimation.m'. 

v0.39 (June 12, 2013):
-Symmetric Bregman distance estimation based on nonsymmetric Bregman distance: added; see 'DsymBregman_DBregman_initialization.m', 'DsymBregman_DBregman_estimation.m'.
-Symmetric Bregman distance estimation based on k-nearest neighbors: added; see 'DsymBregman_kNN_k_initialization.m', 'DsymBregman_kNN_k_estimation.m'. 

v0.38 (June 1, 2013):
-Jensen-Tsallis divergence estimation: added; see 'DJensenTsallis_HTsallis_initialization.m' and 'DJensenTsallis_HTsallis_estimation.m'.
-Bregman distance estimation: added; see 'DBregman_kNN_k_initialization.m' and 'DBregman_kNN_k_estimation.m'.

v0.37 (May 12, 2013):
-K divergence estimation: added; see 'DK_DKL_initialization.m' and 'DK_DKL_estimation.m'.
-L divergence estimation: added; see 'DL_DKL_initialization.m' and 'DL_DKL_estimation.m'.
-Handling 'Y==Q' in case of co.kNNmethod = 'knnsearch': included; see 'kNN_squared_distances.m'.
-Dimension verification: added to 
(i) meta estimators, see 'ITsallis_DTsallis_estimation.m', 'DEnergyDist_DMMD_estimation.m', 'DJdistance_estimation.m', 'DJensenRenyi_HRenyi_estimation.m', 'DJensenShannon_HShannon_estimation.m', 'DKL_CCE_HShannon_estimation.m'.
(ii) utilities 'estimate_Dtemp1.m' and 'estimate_Dtemp2.m'.

v0.36 (Apr 26, 2013):
-Jensen-Renyi divergence estimation: added; see 'DJensenRenyi_HRenyi_initialization.m' and 'DJensenRenyi_HRenyi_estimation.m'.
-Jensen-Shannon divergence estimation: added; see 'DJensenShannon_HShannon_initialization.m' and 'DJensenShannon_HShannon_estimation.m'.

v0.35 (Apr 1, 2013):
-An alternative Jacobi optimization based ICA solution with general entropy/mutual information estimators: added; see 'estimate_ICA.m', 'estimate_ICA_Jacobi2.m'. The method is using a different fining scheme in the rotation angle search compared to 'estimate_ICA_Jacobi1.m'.

v0.34 (Mar 22, 2013):
-Jacobi optimization based ICA solution with general entropy/mutual information estimators: added; see 'estimate_ICA.m', 'estimate_ICA_Jacobi1.m'.

v0.33 (Mar 6, 2013):
-Two one-dimensional Shannon entropy estimators based on the maximum entropy method: added; see 'HShannon_MaxEnt1_initialization.m', 'HShannon_MaxEnt1_estimation.m', 'HShannon_MaxEnt2_initialization.m', 'HShannon_MaxEnt2_estimation.m'.

v0.32 (Feb 25, 2013):
-ICA and ISA structures: introduced for unified treatment of the estimators. It will also enable embedding of general ICA optimization algorithms such as the Jacobi method. 
-Some variables: renamed; see ARmethod_parameters -> AR, ARXmethod_parameters -> ARX, fARmethod_parameters -> fAR, MAparameters -> MA, gaussianizationmethod_parameters -> gaussianization.
-'estimate_mAR.m': 'stepwiseLS' method deleted.
-Example 20 (ISA-3): cost_type = 'KGV' typo corrected (-> cost_type = 'Ipairwise1d'; doc).
-'kdpee.c': MSVC does not provide log2. A more elegant solution: added.

v0.31 (Feb 9, 2013):
-EASI (equivariant adaptive separation via independence) real/complex ICA method: added; see 'estimate_ICA_EASI.m', 'estimate_ICA.m', 'estimate_complex_ICA.m'.
-Adaptive (k-d) partitioning based Shannon entropy estimation: added; see 'HShannon_KDP_initialization.m', 'HShannon_KDP_estimation.m'.

v0.30 (Jan 25, 2013):
-Upper tail dependence via conditional Spearman's rho: added; see 'ASpearman_U_initialization.m', 'ASpearman_U_estimation.m'.
-Multivariate conditional version of Spearman's rho weighting the lower tail: added; see 'ASpearman_ut_initialization.m', 'ASpearman_ut_estimation.m'.

v0.29 (Jan 13, 2013):
-Lower tail dependence via conditional Spearman's rho: added; see 'ASpearman_L_initialization.m', 'ASpearman_L_estimation.m'.
-Multivariate conditional version of Spearman's rho weighting the lower tail: added; see 'ASpearman_lt_initialization.m', 'ASpearman_lt_estimation.m'.
-Notes on lower- and upper orthant dependence: added (doc: see 'Spearman1' and 'Spearman2').

v0.28 (Jan 2, 2013):
-Multivariate extension of Blomqvist's beta (medial correlation coefficient): added; see 'ABlomqvist_initialization.m', 'ABlomqvist_estimation.m'.
-Average pairwise Spearman's rho: added; see 'ASpearman4_initialization.m', 'ASpearman4_estimation.m'.
-Definition of the multivariate measure of concordance and -independence measure: added (doc).
-Further references: included; see 'ISW1_estimation.m', 'ISWinf_estimation.m', 'ASpearman1_estimation.m', 'ASpearman2_estimation.m', 'ASpearman3_estimation.m', +doc.

v0.27 (Dec 28, 2012):
-Approximate correntropy independence measure estimator: added; see 'IApprCorrEntr_initialization.m', 'IApprCorrEntr_estimation.m'.
-Correntropy induced metric, centered correntropy induced metric estimators: added; see 'ACIM_initialization.m', 'ACIM_estimation.m', 'ACCIM_initialization.m', 'ACCIM_estimation.m'.
-Correntropy, centered correntropy, correntropy coefficient estimators: added; see 'ACorrEntr_KDE_direct_initialization.m', 'ACorrEntr_KDE_direct_estimation.m', 'ACCorrEntr_KDE_iChol_initialization.m', 'ACCorrEntr_KDE_iChol_estimation.m', 'ACCorrEntr_KDE_Lapl_initialization.m', 'ACCorrEntr_KDE_Lapl_estimation.m', 'ACorrEntrCoeff_KDE_direct_initialization.m', 'ACorrEntrCoeff_KDE_direct_estimation.m', 'ACorrEntrCoeff_KDE_iChol_initialization.m', 'ACorrEntrCoeff_KDE_iChol_estimation.m'.
-Some references fine-tuned/added (code+doc).
-Handling of identically constant random variables in distance correlation computation: included; see 'IdCor_estimation.m'.

v0.26 (Dec 22, 2012):
-Distance covariance estimation via HSIC: added; see IdCov_IHSIC_initialization.m', 'IdCov_IHSIC_estimation.m'.
-Energy distance estimation via MMD: added; see 'DEnergyDist_DMMD_initialization.m', 'DEnergyDist_DMMD_estimation'.
-Energy distance estimation: added; see 'DEnergyDist_initialization.m', 'DEnergyDist_estimation.m'.
-We computed the square of distance correlation: sqrt added, see 'IdCor_estimation.m'.
-The ARfit website is again available: 'ITE_install.m' changed to its original form.

v0.25 (Dec 15, 2012):
-Distance covariance, distance correlation estimation: added; see 'IdCov_initialization.m', 'IdCov_estimation.m', 'IdCor_initialization.m', 'IdCor_estimation.m'.
-Temporarily the homepage of the downloaded ARfit website seems to be unavailable. Download link changed to 'http://www.mathworks.com/matlabcentral/fileexchange/174-arfit?download=true'; see 'ITE_install.m'.

v0.24 (Dec 12, 2012):
-MMD estimation based on U- and V-statistics: added; see 'DMMD_Ustat_initialization.m', 'DMMD_Ustat_estimation.m', 'DMMD_Vstat_initialization.m', 'DMMD_Vstat_estimation.m'.
-Notes on HSIC, MMD and measures of concordance: added (doc).
-Online MMD estimation: now RBF and linear kernels are both available; see 'DMMD_online_initialization.m', 'DMMD_online_estimation.m'.
-'MMDonline' renamed to 'MMD_online'; see 'DMMD_online_initialization.m', 'DMMD_online_estimation.m'; 'IMMD_DMMD_initialization.m': modified accordingly.

v0.23 (Dec 07, 2012):
-Three multivariate extensions of Spearman's rho: added; see 'ASpearman1_initialization.m', 'ASpearman1_estimation.m', 'ASpearman2_initialization.m', 'ASpearman2_estimation.m', 'ASpearman3_initialization.m', 'ASpearman3_estimation.m'.
-Association (A) cost object type: added; see 'A_initialization.m', 'A_estimation.m'. 
-Directory 'H_I_D_C' renamed to 'H_I_D_A_C'; 'ITE_install.m' modified accordingly.
-Verification of the H/I/D/C function arguments: unified across different estimators. Further verification (compatibility of ds and Y): added to I-estimators (the estimations have not changed). Some comment unification: carried out. 

v0.22 (Dec 01, 2012):
-Cauchy-Schwartz and Euclidean distance based divergence estimators: added; see 'DCS_KDE_iChol_initialization.m', 'DCS_KDE_iChol_estimation.m', 'DED_KDE_iChol_initialization.m', 'DED_KDE_iChol_estimation.m'.
-Cauchy-Schwartz and Euclidean distance based quadratic mutual information estimators: added; see 'IQMI_CS_KDE_direct_initialization.m', 'IQMI_CS_KDE_direct_estimation.m', 'IQMI_CS_KDE_iChol_initialization.m', 'IQMI_CS_KDE_iChol_estimation.m', 'IQMI_ED_KDE_iChol_initialization.m', 'IQMI_ED_KDE_iChol_estimation.m'.

v0.21 (Nov 25, 2012):
-Kullback-Leibler divergence meta estimator based on cross-entropy and entropy: added; see 'DKL_CCE_HShannon_initialization.m', 'DKL_CCE_HShannon_estimation.m'.
-Cross-entropy estimation based on k-nearest neighbors: added; see 'CCE_kNN_k_initialization.m', 'CCE_kNN_k_estimation.m'.
-Cross (C) cost object type: added; see 'C_initialization.m', 'C_estimation.m'. 
-Directory 'H_I_D' renamed to 'H_I_D_C'; 'ITE_install.m' modified accordingly.
-IGV_estimation: added (#2); documentation: modified accordingly.

v0.20 (Nov 19, 2012):
-Two Shannon entropy estimators based on the distance (KL divergence) from the uniform/Gaussian distributions: added; see 'HShannon_DShannon_U_initialization.m', 'HShannon_DShannon_U_estimation.m', 'HShannon_DShannon_N_initialization.m', 'HShannon_DShannon_N_estimation.m'.
-Shannon entropy estimator based on Voronoi regions: added; see 'HShannon_Voronoi_initialization.m', 'HShannon_Voronoi_estimation.m'.
-The documentation of the ITE package has been banished to 'https://bitbucket.org/szzoli/ite/downloads': Downloads. (Its history unnecessarily increased the size of the Mercurial repository.) For further details, see 'doc:ITE_documentation.txt'.
-some code refactoring: (i) divergence estimators: notation (X,Y) -> (Y1,Y2); (ii) disp('Error:...')/disp('Warning:...') -> error('...')/warning('...'); (iii) 'Renyi_CDSS' renamed to 'qRenyi_CDSS'.

v0.19 (Nov 16, 2012):
-2 k-nearest neighbor based Kullback-Leibler divergence estimators: added. See 'DKL_kNN_k_initialization.m', 'DKL_kNN_k_estimation.m', 'DKL_kNN_kiTi_initialization.m', 'DKL_kNN_kiTi_estimation.m'.
-compute_CDSS.cpp: 'sqrt(T)' -> 'sqrt(double(T))', to increase compatibility with compilers.
-Note on Jensen-Shannon divergence: deleted (doc).

v0.18 (Nov 10, 2012):
-8 sample spacing based 1d Shannon/Rényi entropy estimators: added; see 'HShannon_spacing_V_initialization.m', 'HShannon_spacing_V_estimation.m', 'HShannon_spacing_Vb_initialization.m', 'HShannon_spacing_Vb_estimation.m', 'HShannon_spacing_Vpconst_initialization.m', 'HShannon_spacing_Vpconst_estimation.m', 'HShannon_spacing_Vplin_initialization.m', 'HShannon_spacing_Vplin_estimation.m', 'HShannon_spacing_LL_initialization.m', 'HShannon_spacing_LL_estimation.m', 'HRenyi_spacing_V_initialization.m', 'HRenyi_spacing_V_estimation.m', 'HRenyi_spacing_E_initialization.m', 'HRenyi_spacing_E_estimation.m', 'HRenyi_CDSS_initialization.m', 'HRenyi_CDSS_estimation.m'.
-minimum spanning tree, geodesic spanning forest: reference updated (doc+comments); For the latter, see 'HRenyi_MST_estimation', 'HRenyi_GSF_estimation.m'.
-A note on a built-in helping automatism of meta estimators: added (comments); see 'HTsallis_HRenyi_initialization.m', 'IRenyi_DRenyi_initialization.m', 'IRenyi_HRenyi_initialization.m', 'ITsallis_DTsallis_initialization.m'.
-some 'co.mul=mult' -> 'co.mult=mult' typo corrected (to be precise, the computations were OK); see 'ITsallis_DTsallis_initialization.m', 'IShannon_HShannon_initialization.m', 'IRenyi_HRenyi_initialization.m', 'IRenyi_DRenyi_initialization.m', 'IL2_DL2_initialization.m', 'Icomplex_initialization.m', 'HRPensemble_initialization.m', 'Hensemble_initialization.m', 'Hcomplex_initialization.m'.

v0.17 (Nov 6, 2012):
-Edgeworth expansion based Shannon entropy estimator: accelerated (C++ alternative); see 'Edgeworth_t1_t2_t3.cpp'.
-'Tsallis entropy <- Renyi entropy' meta estimator: added; see 'HTsallis_HRenyi_initialization.m', 'HTsallis_HRenyi_estimation.m'. 

v0.16 (Nov 2, 2012):
-Edgeworth expansion based Shannon entropy estimator: added; see 'HShannon_Edgeworth_initialization.m', 'HShannon_Edgeworth_estimation.m'.
-Lookup table for the underlying H/I/D estimation formulas: added (see ITE_documentation.pdf: Section C).
-estimate_HRenyi_constant.m: simplified; gam: deleted.

v0.15 (Oct 29, 2012):
-The Hellinger and Bhattacharyya distances are now available in ITE. They can be estimated via k-nearest neighbor methods; see 'DHellinger_kNN_k_initialization.m', 'DHellinger_kNN_k_estimation.m', 'DBhattacharyya_kNN_k_initialization.m', and 'DBhattacharyya_kNN_k_estimation.m'.
-volume_of_the_unit_ball.m: added. 
-DL2_kNN_k_estimation.m: a '/'->'*' typo corrected (c: volume of the unit ball).

v0.14 (Oct 29, 2012):
-Monte-Carlo simulation to compute the additive constants in Renyi entropy estimation: added; see 'estimate_HRenyi_constant.m'.
-compute_length_HRenyi_MST.m: pdist -> sqdistance (acceleration).
-The embedded 'knnFP1' method can produce an '1e-15' rounding error in squared_distances => W is not _perfectly_ sym. (in the verification of compute_MST.m:kruskal_mst.m), correction made; see 'compute_length_HRenyi_GSF.m'.
-kNN_squared_distances.m, co.kNNmethod='knnFP2': for compatibility reasons variable 'indices' has been converted to int32.

v0.13 (Oct 27, 2012):
-Tsallis entropy is now available in ITE; it can be estimated via k-nearest neighbors, see 'HTsallis_kNN_k_initialization.m', 'HTsallis_kNN_k_estimation.m'.
-A '/'->'*' typo corrected in 'HRenyi_kNN_k_estimation.m'; see 'estimate_Ialpha.m' (V: volume of the unit ball).

v0.12 (Oct 27, 2012):
-Schweizer-Wolff's sigma and kappa: added; see 'ISW1_initialization.m', 'ISW1_estimation.m', 'ISWinf_initialization.m', 'ISWinf_estimation.m'.
-Hoeffding's Phi computation: scaled-up (C++ alternative); see 'Hoeffding_term1.cpp'.
-HRenyi_weightedkNN_initialization.m: contained a superfluous co.k parameter, deleted.
-chol_gauss.c: a superfluous 'double *y' variable deleted.
-dY==dX (dimension of Y and X) verification added to 'DRenyi_kNN_k_estimation.m', 'DTsallis_kNN_k_estimation.m', 'DL2_kNN_k_estimation.m'.
-some comments improved.

v0.11 (Oct 20, 2012):
-multivariate version of the Hoeffding's Phi estimator: added (see 'IHoeffding_initialization.m', 'IHoeffding_estimation.m').
-ITE_install.m: updated to perform only the 'compiled' quick tests. I am a greedy user;) See #1.

v0.1 (Oct 10, 2012): initial release.