# ITE / code / IPA / optimization / clustering_UD0.m

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 function [perm_ICA] = clustering_UD0(s_ICA,ds,opt_type,cost_type,cost_name) %Clusters the ICA (s_ICA) elements to subspaces of given dimensions (ds). % %INPUT: % s_ICA: s_ICA(:,t) is the t^th sample. % opt_type: optimization type; 'greedy','CE'(cross-entropy),'exhaustive' % cost_type: % i)Values: 'I','sumH', 'sum-I','Irecursive', 'Ipairwise' or 'Ipairwise1d'. % ii)Special cost types/schemes allow for more efficient optimization procedures. % cost_name: depending on entropy/mutual information based ISA formulation [see below A)-F)], it can take the values: % i)entropy: % a)base methods: 'Shannon_kNN_k', 'Renyi_kNN_k', 'Renyi_kNN_1tok', 'Renyi_kNN_S', 'Renyi_weightedkNN', 'Renyi_MST', 'Renyi_GSF', 'Tsallis_kNN_k', 'Shannon_Edgeworth'. % b)meta methods: 'complex', 'ensemble', 'RPensemble'. % ii)mutual information: % a)base methods: 'GV', 'KCCA', 'KGV', 'HSIC', 'Hoeffding', 'SW1', 'SWinf'. % b)meta methods: % -Hilbert transformation based method: 'complex', % -divergence based techniques: 'L2_DL2', 'Renyi_DRenyi', 'Tsallis_DTsallis', 'MMD_DMMD', % -entropy based techniques (copula): 'Renyi_HRenyi', % -entropy based techniques (de Morgan's law): 'Shannon_HShannon'. %-------- % Introduction: The ISA problem consists of the minimization: J(W) = I(y^1,...,y^M) -> % min_W, where W is orthogonal (the latter can be assumed w.l.o.g.=whitening). Provided that the ISA separation theorem holds % the ISA solution is a permutation of the ICA elements (W=W_ICA(perm_ICA,:), where W_ICA is the % ICA demixing matrix, p:=perm_ICA is a permutation). The ISA optimization % is carried out making use of this principle. %-------- % Let y=[y^1,...,y^M]=s_ICA(p,:) denote the estimated ISA source with d_m-dimensional components y^m; H and I denotes Shannon differential entropy and mutual information, respectively. % Possible combinations for opt_type, cost_type and cost_name: % A)cost_type = 'I': % Example: J(p) = I(y^1,...,y^M) -> min_p. % B)cost_type = 'sumH' % Example: J(p) = \sum_{m=1}^M H(y^m) -> min_p, % C)cost_type = 'sum-I' (sum minus I) % J(p) = -\sum_{m=1}^M I(y_1^m,...,y_{d_m}^M) -> min_p. % D)cost_type = 'Irecursive': % Example: J(p) = \sum_{m=1}^{M-1} I(y^m,[y^{m+1},...,y^M]) -> min_p. % E)cost_type = 'Ipairwise' % Example: J(p) = \sum_{m1,m2: different subspace indices} I(y^{m1},y^{m2}) -> min_p. % F)cost_type = 'Ipairwise1d': % Example: J(p) = \sum_{m1,m2,i1,i2; m1,m2:different subspace indices; i1,i2: coordinates of different (m1\ne m2) subspaces} I(y_{i1}^{m1},y_{i2}^{m2}) -> min_p. % Note: % A),B),C),D): ISA is *equivalent* to A),B),C),D) in case of Shannon H and I. % E): The goal of E) is to make the estimated ISA subspaces pairwise independent. % F): The goal of F) is to make the estimated ISA subspaces pairwise independent; pairwise independence of the subspaces is estimated by the pairwise independende of their coordinates. % E) and F) are *necessary* conditions of ISA demixing, which also work often efficiently in practice. %OUTPUT: % perm_ICA: permutation of the ICA elements. % %Copyright (C) 2012 Zoltan Szabo ("http://nipg.inf.elte.hu/szzoli", "szzoli (at) cs (dot) elte (dot) hu") % %This file is part of the ITE (Information Theoretical Estimators) Matlab/Octave toolbox. % %ITE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by %the Free Software Foundation, either version 3 of the License, or (at your option) any later version. % %This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of %MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. % %You should have received a copy of the GNU General Public License along with ITE. If not, see . %cost object initialization: %cost_type,ds => mult: mult = set_mult(cost_type,ds); co = co_initialization(cost_type,cost_name,mult); switch opt_type case 'greedy' switch cost_type case {'I','Irecursive'} perm_ICA = clustering_UD0_greedy_general(s_ICA,ds,cost_type,co); case {'sumH','sum-I'} perm_ICA = clustering_UD0_greedy_additive_wrt_subspaces(s_ICA,ds,cost_type,co); case 'Ipairwise' perm_ICA = clustering_UD0_greedy_pairadditive_wrt_subspaces(s_ICA,ds,co); case 'Ipairwise1d' S = I_similarity_matrix(s_ICA,co); %similarity matrix perm_ICA = clustering_UD0_greedy_pairadditive_wrt_coordinates(S,ds); otherwise disp('Error: cost type=?'); end case 'CE' switch cost_type case {'I','sumH','sum-I','Irecursive','Ipairwise'} perm_ICA = clustering_UD0_CE_general(s_ICA,ds,cost_type,co); case 'Ipairwise1d'%'global JBD' S = I_similarity_matrix(s_ICA,co); %similarity matrix perm_ICA = clustering_UD0_CE_pairadditive_wrt_coordinates(S,ds); otherwise disp('Error: optimization type=?'); end case 'exhaustive' %can be useful for small dimensions (small sum(ds)), or for cost verification switch cost_type case {'I','Irecursive','sumH','sum-I','Ipairwise'} perm_ICA = clustering_UD0_exhaustive_general(s_ICA,ds,cost_type,co); case 'Ipairwise1d'%'exhaustive JBD (joint block diagonalization)' S = I_similarity_matrix(s_ICA,co); %similarity matrix perm_ICA = clustering_UD0_exhaustive_pairadditive_wrt_coordinates(S,ds); otherwise disp('Error: cost type=?'); end otherwise disp('Error: optimization type=?'); end 
Tip: Filter by directory path e.g. /media app.js to search for public/media/app.js.
Tip: Use camelCasing e.g. ProjME to search for ProjectModifiedEvent.java.
Tip: Filter by extension type e.g. /repo .js to search for all .js files in the /repo directory.
Tip: Separate your search with spaces e.g. /ssh pom.xml to search for src/ssh/pom.xml.
Tip: Use ↑ and ↓ arrow keys to navigate and return to view the file.
Tip: You can also navigate files with Ctrl+j (next) and Ctrl+k (previous) and view the file with Ctrl+o.
Tip: You can also navigate files with Alt+j (next) and Alt+k (previous) and view the file with Alt+o.