Source

ITE / README.md

Full commit
Zoltan Szabo fb79e28 

Zoltan Szabo 0f83211 
Zoltan Szabo fb79e28 










Zoltan Szabo 0b610c4 



Zoltan Szabo c2c43aa 
Zoltan Szabo fa6edf9 
Zoltan Szabo d0b19a8 
Zoltan Szabo d50e55c 
Zoltan Szabo c2c43aa 
Zoltan Szabo fb79e28 



Zoltan Szabo 30943a7 
Zoltan Szabo ca6860b 
Zoltan Szabo 0b610c4 

Zoltan Szabo 3e697ee 

Zoltan Szabo e9e40a2 
Zoltan Szabo 3e697ee 
Zoltan Szabo c2c43aa 
Zoltan Szabo 2331b9a 
Zoltan Szabo 30943a7 

Zoltan Szabo d0b19a8 

Zoltan Szabo 30943a7 
#ITE (Information Theoretical Estimators)

ITE is capable of estimating many different variants of entropy, mutual information, divergence, association measures and cross quantities. Thanks to its highly modular design, ITE supports additionally 

- the combinations of the estimation techniques, 
- the easy construction and embedding of novel information theoretical estimators, and 
- their immediate application in information theoretical optimization problems. 

ITE is 

- written in Matlab/Octave,
- multi-platform (tested extensively on Windows and Linux),
- free and open source (released under the GNU GPLv3(>=) license).

* * *

ITE can estimate 

- `entropy (H)`: Shannon entropy, Rényi entropy, Tsallis entropy (Havrda and Charvát entropy), complex entropy,
- `mutual information (I)`: generalized variance, kernel canonical correlation analysis, kernel generalized variance, Hilbert-Schmidt independence criterion, Shannon mutual information, L2 mutual information, Rényi mutual information, Tsallis mutual information, copula-based kernel dependency, multivariate version of Hoeffding's Phi, Schweizer-Wolff's sigma and kappa, complex mutual information, Cauchy-Schwartz quadratic mutual information, Euclidean distance based quadratic mutual information, distance covariance, distance correlation, approximate correntropy independence measure,
- `divergence (D)`: Kullback-Leibler divergence (relative entropy, I directed divergence), L2 divergence, Rényi divergence, Tsallis divergence, Hellinger distance, Bhattacharyya distance, maximum mean discrepancy (kernel distance, an integral probability metric), J-distance (symmetrised Kullback-Leibler divergence, J divergence), Cauchy-Schwartz divergence, Euclidean distance based divergence, energy distance (specially the Cramer-Von Mises distance), Jensen-Shannon divergence, Jensen-Rényi divergence, K divergence, L divergence,
- `association measures (A)`, including `measures of concordance`: multivariate extensions of Spearman's rho (Spearman's rank correlation coefficient, grade correlation coefficient), correntropy, centered correntropy, correntropy coefficient, correntropy induced metric, centered correntropy induced metric, multivariate extension of Blomqvist's beta (medial correlation coefficient), multivariate conditional version of Spearman's rho, lower/upper tail dependence via conditional Spearman's rho,
- `cross quantities (C)`: cross-entropy.

ITE offers solution methods for 

- Independent Subspace Analysis (ISA) and 
- its extensions to different linear-, controlled-, post nonlinear-, complex valued-, partially observed models, as well as to systems with nonparametric source dynamics. 

* * *

Note: 

- the evolution of the ITE code is briefly summarized in CHANGELOG.txt.
- become a [Follower](https://bitbucket.org/szzoli/ite/follow) to be always up-to-date with ITE.
- if you have an H/I/D/A/C estimator/subtask solver with a GPLv3(>=)-compatible license that you would like to be embedded into ITE, feel free to contact me.

**Download** the latest release: 

- code: [zip](https://bitbucket.org/szzoli/ite/downloads/ITE-0.37_code.zip), [tar.bz2](https://bitbucket.org/szzoli/ite/downloads/ITE-0.37_code.tar.bz2), 
- [documentation (pdf)](https://bitbucket.org/szzoli/ite/downloads/ITE-0.37_documentation.pdf).