Wiki

bnpy comes with a profiling tool to create easy-to-read HTML analysis of how much time is spend on each line of code when executing a specific experiment. All relevant code lives in the profiler/ directory of this repository (many thanks to Jincheng Li for help with this feature).

Suppose we want to study a simple DPMixtureModel on the AsteriskK8 toy dataset. We would normally run an experiment from the command line like so:

python -m bnpy.Run AsteriskK8 DPMixtureModel Gauss VB --nLap 3 --K 10

Instead, we can run the same experiment with the profiler, as follows:

cd $BNPYROOT/profiler/
python RunWithProfiler.py AsteriskK8 DPMixtureModel Gauss VB --nLap 3 --K 10

Expected output

Decorating ... [DONE]
Asterisk Toy Data. 8 true clusters.
  size: 25000 units (single observations)
  dimension: 2
Allocation Model:  DP mixture with K=10. Concentration gamma0= 1.00
Obs. Data  Model:  Gaussian with full covariance.
Obs. Data  Prior:  Gauss-Wishart on each mean/prec matrix pair: mu, Lam
  E[ mu[k] ]     = [ 0.  0.]
  E[ CovMat[k] ] = 
  [[ 1.  0.]
   [ 0.  1.]]
Learn Alg: VB
Trial  1/1 | alg. seed: 4226944 | data order seed: 8541952
savepath: /results/AsteriskK8/defaultjob/1
        1/3 after      0 sec. | K   10 | ev -9.551913351e-01 |  
        2/3 after      0 sec. | K   10 | ev -9.030340484e-01 | Ndiff  138.572 
        3/3 after      0 sec. | K   10 | ev -8.602065852e-01 | Ndiff  171.317 
... done. not converged. max laps thru data exceeded.
Building HTML ... [DONE]
Undecorating ... [DONE]
Wrote HTML to /path/to/bnpy-dev/profiler/reports/20150921-1533-455628//index.html

Clicking on the HTML link at the bottom of the output yields a nice browse-able display of all functions the experiment executed and the total time spent at each line of Python code.

Wiki

bnpy-dev / Profiler

Expected output

HTML output: main view

HTML output: function hotspots view