Commits

Ruben Martinez-Cantin  committed 98269d8

Gaussian Process with normal prior working. Improved documentation.

  • Participants
  • Parent commits da7755d

Comments (0)

Files changed (9)

File CMakeLists.txt

   ./src/hierarchical_gaussian_process.cpp
   ./src/gaussian_process.cpp
   ./src/gaussian_process_ml.cpp
+  ./src/gaussian_process_normal.cpp
   ./src/student_t_process_jef.cpp
   ./src/student_t_process_nig.cpp
 #  ./src/gaussian_process_ign.cpp
 # The EXTRA_PACKAGES tag can be to specify one or more names of LaTeX
 # packages that should be included in the LaTeX output.
 
-EXTRA_PACKAGES         =
+EXTRA_PACKAGES         = amssymb
 
 # The LATEX_HEADER tag can be used to specify a personal LaTeX header for
 # the generated latex document. The header should contain everything until
 BayesOpt: A Bayesian optimization toolbox            {#mainpage}
 =========================================
 
-This is an efficient, C++ implementation of the Bayesian
+BayesOpt is an free, efficient, implementation of the Bayesian
 optimization methodology for nonlinear-optimization, experimental
 design and stochastic bandits. In the literature it is also called
 Sequential Kriging Optimization (SKO) or Efficient Global
 The online HTML version of these docs:
 <http://rmcantin.bitbucket.org/html/>
 
-Basically, it uses a distribution over functions to build a
+Bayesian optimization uses a distribution over functions to build a
 metamodel of the unknown function for we are looking the extrema,
 and then apply some active learning strategy to select the query
 points that provides most potential interest for the seek. For that
 research. At the same time, it does everything the "right way". For
 example:
 
-- latin hypercube sampling is use for the preliminary sampling step
-- kernel parameters are trained with the preliminary samples and
-  fixed afterwards to avoid bias and divergence
-- matrix algebra tricks are used to guarantee that any covariance
-  matrix remains SPD and reduce computational cost.
+- latin hypercube sampling is used for the preliminary design step,
+- extensive use of Cholesky decomposition and related tecniques to 
+  improve numeral stability and reduce computational cost,
+- kernels, criteria and parametric functions can be combined to 
+  produce more advanced functions,
 - etc.
 
-Originally, it was developed for as part of a robotics research
-project \cite MartinezCantin09AR \cite MartinezCantin07RSS, where a
-Gaussian process with hyperpriors on the mean and signal covariance
-parameters. Then, the metamodel was constructed using the Maximum a
-Posteriory (MAP) of the parameters.
-
-However, the library now has grown to support many more surrogate
-models, with different distributions (Gaussian processes,
-Student's-t processes, etc.), with many kernels and mean
-functions. It also provides different criteria (even some combined
-criteria) so the library can be used to any problem involving some
-bounded optimization, stochastic bandits, active learning for
-regression, etc.
-
-Start by reading the \ref install and the \ref reference
+Start by reading the \ref install and the \ref reference. You
+can also check about \ref bopttheory.
 
 You can also find more details at:
 <http://bitbucket.org/rmcantin/bayesopt/wiki/Home>
 
-**Important:** This code is free to use. However, if you are using,
-or plan to use, the library, specially if it is for research or
-academic purposes, please send me an email at <rmcantin@unizar.es>
-with your name, institution and a brief description of your
-interest for this code (one or two lines).
+**Important:** This code is free to use. However, if you are using the
+library, specially if it is for research or academic purposes, please
+send me an email at <rmcantin@unizar.es> with your name, institution
+and a brief description of your interest for this code (one or two
+lines).
 
-If you use BayesOpt in work that leads to a publication, we would
-appreciate it if you would kindly cite BayesOpt in your
+If you use BayesOpt in a work that leads to a scientific publication,
+we would appreciate it if you would kindly cite BayesOpt in your
 manuscript. Cite BayesOpt as something like:
 
 ----------------------------------------------------------------------
 
 ----------------------------------------------------------------------
 
-Copyright (C) 2011-2012 Ruben Martinez-Cantin <rmcantin@unizar.es>
+Copyright (C) 2011-2013 Ruben Martinez-Cantin <rmcantin@unizar.es>
 
 BayesOpt is free software: you can redistribute it and/or modify it
 under the terms of the GNU General Public License as published by the

File app/bo_display.cpp

 
 class MP :public MatPlot{ 
 void DISPLAY(){
-    if ((is_run) && (state_ii < 150))
+  size_t nruns = GLOBAL_MODEL->getParameters()->n_iterations;
+    if ((is_run) && (state_ii < nruns))
     {
       ++state_ii;
       GLOBAL_MODEL->stepOptimization(state_ii); 
   bopt_params parameters = initialize_parameters_to_default();
   parameters.n_init_samples = 10;
   parameters.n_iter_relearn = 0;
-  parameters.n_iterations = 150;
-  parameters.surr_name = "STUDENT_T_PROCESS_NORMAL_INV_GAMMA";
+  parameters.n_iterations = 300;
+  //parameters.surr_name = "STUDENT_T_PROCESS_NORMAL_INV_GAMMA";
+  parameters.surr_name = "GAUSSIAN_PROCESS_NORMAL";
+
+  parameters.crit_name = "cLCB";
+  parameters.crit_params[0] = 5;
+  parameters.n_crit_params = 1;
+
   parameters.kernel.hp_mean[0] = 1;
   parameters.kernel.hp_std[0] = 5;
   parameters.kernel.n_hp = 1;
-  //  parameters.noise = 0.01;
-  // parameters.l_type = L_ML;
-  // parameters.mean.name = "mZero";
-  //  parameters.crit_name = "cHedge(cEI,cLCB,cExpReturn,cOptimisticSampling)";
-  // parameters.epsilon = 0.0;
+
   parameters.verbose_level = 2;
 
   state_ii = 0;

File doxygen/reference.dox

 /*! \page reference Reference Manual
 \tableofcontents
 
-After installing the library, several test program will also be
-generated and can be found in the \em bin folder. For the external
-interfaces, there are test programs respectively under the \em python
-and \em matlab folders.  They provide examples of the different
-interfaces that Bayesian-optimization provide.
+Originally, it was developed for as part of a robotics research
+project \cite MartinezCantin09AR \cite MartinezCantin07RSS, where a
+Gaussian process with hyperpriors on the mean and signal covariance
+parameters. Then, the metamodel was constructed using the Maximum a
+Posteriory (MAP) of the parameters.
+
+However, the library now has grown to support many more surrogate
+models, with different distributions (Gaussian processes,
+Student's-t processes, etc.), with many kernels and mean
+functions. It also provides different criteria (even some combined
+criteria) so the library can be used to any problem involving some
+bounded optimization, stochastic bandits, active learning for
+regression, etc.
+
+\section testing Testing the installation.
+
+The library includes several test program that can be found in the \em
+bin folder. Furthermore, there are test programs respectively under
+the \em python and \em matlab folders.  They provide examples of the
+different interfaces that BayesOpt provide.
 
 First of all, make sure your system finds the correct libraries. In
 Windows, you can copy the dlls to your working folder or include the
 
 In some *nix systems, including Ubuntu, Debian and Mac OS, the library
 is installed by default in /usr/local/lib/. However, this folder is
-not included in the linker, Python or Matlab paths by default. The
-script \em exportlocalpaths.sh makes sure that the folder is included
-in all the necessary paths.
+not included by default in the linker, Python or Matlab paths by
+default. This is specially critical when building shared libraries
+(mandatory for Python usage). The script \em exportlocalpaths.sh makes
+sure that the folder is included in all the necessary paths.
 
 After that, there are 3 steps that should be follow:
 \li Define the function to optimize.
 model. Also, the key to nonlinear optimization is to include as much
 knowledge as possible about the target function or about the
 problem. Or, if the knowledge is not available, keep the model as
-general as possible (to avoid bias). 
+general as possible (to avoid bias).
 
-For that reason, the parameters are bundled in a structure or dictionary, depending on the API that we use. This is a brief explanation of every parameter
+For that reason, the parameters are bundled in a structure or
+dictionary, depending on the API that we use. This is a brief
+explanation of every parameter
 
 \subsection budgetpar Budget parameters
-\li \b n_iterations: Maximum number of iterations of BayesOpt. Each iteration corresponds with a target function evaluation. This is related with the budget of the application [Default 300]
-\li \b n_inner_iterations: Maximum number of iterations of the inner optimization process. Each iteration corresponds with a criterion evaluation. The inner optimization results in the "most interest point" to run evaluate the target function. This is also used for the kernel hyperparameter computation. In order to scale the process for increasing dimensional spaces, the actual number of iterations is this number times the number of dimensions. [Default 500]
-\li \b n_init_samples: BayesOpt requires an initial set of samples to learn a preliminary model of the target function. Each sample is also a target function evaluation. [Default 30]
-\li \b n_iter_relearn: Although most of the parameters of the model are updated after every iteration, the kernel parameters cannot be updated continuously as it might crash the convergence. This represents the number of iterations between recomputing the kernel parameters. If it is set to 0, they are only learned after the initial set of samples. [Default 0]
+
+\li \b n_iterations: Maximum number of iterations of BayesOpt. Each
+iteration corresponds with a target function evaluation. This is
+related with the budget of the application [Default 300]
+\li \b n_inner_iterations: Maximum number of iterations of the inner
+optimization process. Each iteration corresponds with a criterion
+evaluation. The inner optimization results in the "most interest
+point" to run evaluate the target function. This is also used for the
+kernel hyperparameter computation. In order to scale the process for
+increasing dimensional spaces, the actual number of iterations is this
+number times the number of dimensions. [Default 500]
+\li \b n_init_samples: BayesOpt requires an initial set of samples to
+learn a preliminary model of the target function. Each sample is also
+a target function evaluation. [Default 30]
+\li \b n_iter_relearn: Although most of the parameters of the model
+are updated after every iteration, the kernel parameters cannot be
+updated continuously as it might crash the convergence. This
+represents the number of iterations between recomputing the kernel
+parameters. If it is set to 0, they are only learned after the initial
+set of samples. [Default 0]
 
 \subsection logpar Logging parameters
-\li \b verbose_level: (integer value) Verbose level 0,3 -> warnings, 1,4 -> general information, 2,5 -> debug information, any other value -> only errors. Levels < 3 send the messages to stdout. Levels > 4 send them to a log file. [Default 1].
-\li \b log_filename: Name of the log file (if applicable, verbose > 4)[Default "bayesopt.log"] 
+
+\li \b verbose_level: (integer value) Verbose level 0,3 -> warnings,
+1,4 -> general information, 2,5 -> debug information, any other value
+-> only errors. Levels < 3 send the messages to stdout. Levels > 4
+send them to a log file. [Default 1].
+\li \b log_filename: Name of the log file (if applicable, verbose >
+4)[Default "bayesopt.log"]
 
 \subsection surrpar Surrogate model parameters
-\li \b surr_name: Name of the hierarchical surrogate function (nonparametric process and the hyperpriors on sigma and w). See section ??? for a detailed description. [Default S_GaussianProcess]
+
+\li \b surr_name: Name of the hierarchical surrogate function
+(nonparametric process and the hyperpriors on sigma and w). See
+section ??? for a detailed description. [Default S_GaussianProcess]
 \li \b sigma_s: Signal variance (if known) [Default 1.0]
-\li \b noise: Observation noise. For computer simulations or deterministic functions, it should be close to 0. However, to avoid numerical instability due to model inaccuracies, do not make it 0. [Default 0.0001]
-\li \b alpha, \b beta: Inverse-Gamma prior hyperparameters (if applicable) [Default 1.0, 1.0] 
-\li \b l_type: Learning method for the kernel hyperparameters. See section ??? for a detailed description [Default L_MAP]
+\li \b noise: Observation noise. For computer simulations or
+deterministic functions, it should be close to 0. However, to avoid
+numerical instability due to model inaccuracies, do not make it
+0. [Default 0.0001]
+\li \b alpha, \b beta: Inverse-Gamma prior hyperparameters (if
+applicable) [Default 1.0, 1.0]
+\li \b l_type: Learning method for the kernel hyperparameters. See
+section ??? for a detailed description [Default L_MAP]
 
 \subsection critpar Exploration/exploitation parameters
-\li \b epsilon: According to some authors, it is recommendable to include an epsilon-greedy strategy to achieve near optimal convergence rates. Epsilon is the probability of performing a random (blind) evaluation of the target function [Default 0.0 (disabled)]
-\li \b crit_name: Name of the sample selection criterion or a combination of them. It is used to select which points to evaluate for each iteration of the optimization process. Could be a combination of functions like "cHedge(cEI,cLCB,cPOI,cThompsonSampling)". See section ??? for the different possibilities. [Default: "cEI]"
-\li \b crit_params, \b n_crit_params: Array with the set of parameters for the selected criteria. If there are more than one, the parameters are split among them according to the number of parameters required for each criterion. [Default: 1, 1]
+
+\li \b epsilon: According to some authors, it is recommendable to
+include an epsilon-greedy strategy to achieve near optimal convergence
+rates. Epsilon is the probability of performing a random (blind)
+evaluation of the target function [Default 0.0 (disabled)]
+\li \b crit_name: Name of the sample selection criterion or a
+combination of them. It is used to select which points to evaluate for
+each iteration of the optimization process. Could be a combination of
+functions like "cHedge(cEI,cLCB,cPOI,cThompsonSampling)". See section
+??? for the different possibilities. [Default: "cEI]"
+\li \b crit_params, \b n_crit_params: Array with the set of parameters
+for the selected criteria. If there are more than one, the parameters
+are split among them according to the number of parameters required
+for each criterion. [Default: 1, 1]
 
 \subsection kernelpar Kernel parameters
-\li \b kernel.name: Name of the kernel function. Could be a combination of functions like "kSum(kSEARD,kMaternARD3)". See section for the different posibilities. [Default: "kMaternISO3"]
-\li \b kernel.hp_mean, \b kernel.hp_std, \b kernel.n_hp: Kernel hyperparameters prior. Any "ilegal" standard deviation (std<=0) results in a maximum likelihood estimate. Depends on the kernel selected. If there are more than one, the parameters are split among them according to the number of parameters required for each criterion. [Default: "1.0, 10.0, 1" ]
+
+\li \b kernel.name: Name of the kernel function. Could be a
+combination of functions like "kSum(kSEARD,kMaternARD3)". See section
+for the different posibilities. [Default: "kMaternISO3"]
+\li \b kernel.hp_mean, \b kernel.hp_std, \b kernel.n_hp: Kernel
+hyperparameters prior. Any "ilegal" standard deviation (std<=0)
+results in a maximum likelihood estimate. Depends on the kernel
+selected. If there are more than one, the parameters are split among
+them according to the number of parameters required for each
+criterion. [Default: "1.0, 10.0, 1" ]
 
 \subsection meanpar Mean function parameters
-\li \b mean.name: Name of the mean function. Could be a combination of functions like "mSum(mOne, mLinear)". See section for the different posibilities. [Default: "mOne"]
-\li \b mean.coef_mean, \b kernel.coef_std, \b kernel.n_coef: Mean function coefficients. The standard deviation is only used if the surrogate model assumes a normal prior. If there are more than one, the parameters are split among them according to the number of parameters required for each criterion. [Default: "1.0, 30.0, 1" ]
+
+\li \b mean.name: Name of the mean function. Could be a combination of
+functions like "mSum(mOne, mLinear)". See section for the different
+posibilities. [Default: "mOne"]
+\li \b mean.coef_mean, \b kernel.coef_std, \b kernel.n_coef: Mean
+function coefficients. The standard deviation is only used if the
+surrogate model assumes a normal prior. If there are more than one,
+the parameters are split among them according to the number of
+parameters required for each criterion. [Default: "1.0, 30.0, 1" ]
 
 
 \section usage Using the library

File include/bayesoptbase.hpp

 
     int setSurrogateModel();    
     int setCriteria();
+    bopt_params* getParameters() {return &mParameters;};
 
   protected:
     /** 

File include/gaussian_process_normal.hpp

     int precomputePrediction();
 
   private:
-    vectord mWML;           //!< GP ML parameters
-    
-    /// Precomputed GP prediction operations
-    vectord mAlphaF;
-    matrixd mKF, mL2;
+    vectord mWMap;                      //!< GP posterior parameters
+    double mSigma;
+    vectord mW0;
+    vectord mInvVarW;
+    //! Precomputed GP prediction operations
+    vectord mVf;
+    matrixd mKF, mD;     
 
     GaussianDistribution* d_;      //!< Predictive distributions
   };

File optimization.bib

-% This file was created with JabRef 2.6.
+% This file was created with JabRef 2.9.2.
 % Encoding: UTF-8
 
+@BOOK{Atkinson2007,
+  title = {Optimum Experimental Design with SAS},
+  publisher = {Oxford University Press},
+  year = {2007},
+  author = {Anthony C. Atkinson and Alexander N. Donev and Randall D. Tobias},
+  owner = {rmcantin},
+  timestamp = {2013.02.25}
+}
+
+@ARTICLE{Auger2010,
+  author = {Anne Auger and Olivier Teytaud},
+  title = {Continuous Lunches Are Free Plus the Design of Optimal Optimization
+	Algorithms},
+  journal = {Algorithmica},
+  year = {2010},
+  volume = {57},
+  pages = {121-146},
+  number = {1},
+  month = {May},
+  owner = {rmcantin},
+  timestamp = {2013.03.11}
+}
+
+@INPROCEEDINGS{Baier2007,
+  author = {Tim Baier and Jianwei Zhang},
+  title = {Learning to Grasp Everyday Objects using Reinforcement-Learning with
+	Automatic Value Cut-Off},
+  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems
+	(IROS 2007)},
+  year = {2007},
+  owner = {rmcantin},
+  timestamp = {2013.05.07}
+}
+
 @INPROCEEDINGS{Balcan2008,
   author = {Balcan, Maria Florina and Hanneke, Steve and Wortman, Jennifer},
   title = {The True Sample Complexity of Active Learning.},
   timestamp = {2010.02.15}
 }
 
+@ARTICLE{Bernardo1979,
+  author = {Bernardo, Jose M.},
+  title = {Reference posterior distributions for Bayesian inference (with discussion)},
+  journal = {Journal of the Royal Statistical Society. Series B (Methodological)},
+  year = {1979},
+  volume = {41},
+  pages = {113–147},
+  owner = {rmcantin},
+  timestamp = {2013.04.11}
+}
+
+@BOOK{Bishop2006,
+  title = {Pattern Recognition and Machine Learning},
+  publisher = {Springer},
+  year = {2006},
+  author = {Christipher M. Bishop},
+  owner = {rmcantin},
+  timestamp = {2012.11.21}
+}
+
 @TECHREPORT{Brochu:2010c,
-  author = {Eric Brochu and Vlad M Cora and Nando {de Freitas}},
+  author = {Eric Brochu and Vlad M. Cora and Nando{de Freitas}},
   title = {A Tutorial on Bayesian Optimization of Expensive Cost Functions,
 	with Application to Active User Modeling and Hierarchical Reinforcement
 	Learning},
 }
 
 @INPROCEEDINGS{Brochu07NIPS,
-  author = {E. Brochu and N. de Freitas and A. Ghosh},
+  author = {Eric Brochu and Nando {de Freitas} and Abhijeet Ghosh},
   title = {Active Preference Learning with Discrete Choice Data},
   booktitle = {Advances in Neural Information Processing Systems},
   year = {2007},
   timestamp = {2010.06.18}
 }
 
-@TECHREPORT{Bull2011,
-  author = {A. Bull},
+@ARTICLE{Bull2011,
+  author = {Adam D. Bull},
   title = {Convergence Rates of Efficient Global Optimization Algorithms},
-  institution = {arXiv.org},
+  journal = {Journal of Machine Learning Research},
   year = {2011},
-  journal = {arXiv.org},
+  volume = {12},
+  pages = {2879-2904},
   owner = {rmcantin},
   timestamp = {2011.10.24},
   url = {http://arxiv.org/abs/1101.3501}
 }
 
 @ARTICLE{Chaloner:1995,
-  author = {K Chaloner and I Verdinelli},
-  title = {Bayesian experimental design: A review},
-  journal = {J. of Statistical Science},
+  author = {Kathryn Chaloner and Isabella Verdinelli},
+  title = {{Bayesian experimental design: A review}},
+  journal = {Journal of Statistical Science},
   year = {1995},
   volume = {10},
   pages = {273--304},
   timestamp = {2010.06.18}
 }
 
+@INPROCEEDINGS{Diaconis1988,
+  author = {Persi Diaconis},
+  title = {Bayesian Numerical Analysis},
+  booktitle = {Statistical Decision Theory and Related Topics IV},
+  year = {1988},
+  editor = {S.S. Gupta and J.O. Berger},
+  volume = {1},
+  pages = {163--175},
+  owner = {rmcantin},
+  timestamp = {2013.03.15}
+}
+
+@INPROCEEDINGS{Garnett2010,
+  author = {Roman Garnett and Michael A. Osborne and Stephen J. Roberts},
+  title = {Bayesian optimization for sensor set selection},
+  booktitle = {Information Processing in Sensor Networks (IPSN 2010)},
+  year = {2010},
+  owner = {rmcantin},
+  timestamp = {2013.05.07}
+}
+
+@ARTICLE{Gramacy2012,
+  author = {Gramacy, Robert B. and Lee, Herbert K.H.},
+  title = {Cases for the nugget in modeling computer experiments},
+  journal = {Statistics and Computing},
+  year = {2012},
+  volume = {22},
+  pages = {713-722},
+  doi = {10.1007/s11222-010-9224-x},
+  issn = {0960-3174},
+  issue = {3},
+  keywords = {Computer simulator; Surrogate model; Gaussian process; Interpolation;
+	Smoothing},
+  language = {English},
+  owner = {rmcantin},
+  publisher = {Springer US},
+  timestamp = {2012.10.05},
+  url = {http://dx.doi.org/10.1007/s11222-010-9224-x}
+}
+
 @ARTICLE{Gramacy_Polson_2009,
   author = {Gramacy, Robert B and Polson, Nicholas G},
   title = {Particle learning of Gaussian process models for sequential design
 
 @ARTICLE{Handcock1993,
   author = {Mark S. Handcock and Michael L. Stein},
-  title = {A Bayesian Analysis of Kriging},
+  title = {A {Bayesian} Analysis of {Kriging}},
   journal = {Technometrics},
   year = {1993},
   volume = {35},
 }
 
 @INPROCEEDINGS{Hoffman2011,
-  author = {Matthew Hoffman and Eric Brochu and Nando de Freitas},
+  author = {Matthew Hoffman and Eric Brochu and Nando {de Freitas}},
   title = {Portfolio Allocation for {B}ayesian Optimization},
   booktitle = {27th Conference on Uncertainty in Artificial Intelligence (UAI2011)},
   year = {2011},
 }
 
 @ARTICLE{Huang06,
-  author = {D. Huang and T. T. Allen and W. I. Notz and N. Zeng},
+  author = {D. Huang and T. T. Allen and W. I. Notz and N. Zheng},
   title = {Global Optimization of Stochastic Black-Box Systems via Sequential
 	Kriging Meta-Models},
   journal = {Journal of Global Optimization},
   timestamp = {2010.03.10}
 }
 
+@INPROCEEDINGS{HutHooLey11-smac,
+  author = {Frank Hutter and Holger H. Hoos and Kevin Leyton-Brown},
+  title = {Sequential Model-Based Optimization for General Algorithm Configuration},
+  booktitle = {Proceedings of Learning and Intelligent Optimization (LION-5)},
+  year = {2011},
+  pages = {507–523},
+  owner = {rmcantin},
+  timestamp = {2013.05.07}
+}
+
 @ARTICLE{Igel04,
   author = {Christian Igel and Marc Toussaint},
   title = {A No-Free-Lunch theorem for non-uniform distributions of target functions},
 }
 
 @ARTICLE{Jeffreys1946,
-  author = {Jeffreys, H.},
+  author = {Jeffreys, Harold.},
   title = {An Invariant Form for the Prior Probability in Estimation Problems},
   journal = {Proceedings of the Royal Society of London. Series A, Mathematical
 	and Physical Sciences},
 
 @MISC{Johnson,
   author = {Steven G. Johnson},
-  title = {The {NLopt} nonlinear-optimization package, http://ab-initio.mit.edu/nlopt},
+  title = {The {NLopt} nonlinear-optimization package},
+  howpublished = {http://ab-initio.mit.edu/nlopt},
+  year = {2013},
   owner = {rmcantin},
   timestamp = {2011.10.24},
   url = {http://ab-initio.mit.edu/nlopt}
 }
 
-@ARTICLE{Jones:2001,
-  author = {D.R. Jones},
-  title = {A Taxonomy of Global Optimization Methods Based on Response Surfaces},
-  journal = {Journal of Global Optimization},
-  year = {2001},
-  volume = {21},
-  pages = {345--383},
-  keywords = {Gaussian processes, optimization, active learning, fornips},
-  owner = {rmcantin},
-  timestamp = {2010.06.18}
-}
-
 @ARTICLE{Jones:1993,
   author = {D.R. Jones and C.D. Perttunen and B.E. Stuckman},
   title = {Lipschitzian Optimization Without the {L}ipschitz Constant},
   timestamp = {2010.06.18}
 }
 
+@ARTICLE{Jones:2001,
+  author = {Donald R. Jones},
+  title = {A Taxonomy of Global Optimization Methods Based on Response Surfaces},
+  journal = {Journal of Global Optimization},
+  year = {2001},
+  volume = {21},
+  pages = {345--383},
+  keywords = {Gaussian processes, optimization, active learning, fornips},
+  owner = {rmcantin},
+  timestamp = {2010.06.18}
+}
+
 @ARTICLE{Jones:1998,
-  author = {D.R. Jones and M. Schonlau and W.J. Welch},
+  author = {Donald R. Jones and Matthias Schonlau and William J. Welch},
   title = {Efficient Global Optimization of Expensive Black-Box Functions},
   journal = {Journal of Global Optimization},
   year = {1998},
   timestamp = {2010.06.18}
 }
 
+@ARTICLE{Kaelbling96jmlr,
+  author = {Leslie P. Kaelbling and Michael L. Littman and Andrew W. Moore},
+  title = {{Reinforcement learning: A survey}},
+  journal = {Journal of Artificial Intelligence Research},
+  year = {1996},
+  volume = {4},
+  pages = {237-285},
+  owner = {rmcantin},
+  timestamp = {2013.03.12}
+}
+
+@ARTICLE{kleijnen2012expected,
+  author = {Kleijnen, Jack PC and van Beers, Wim and Van Nieuwenhuyse, Inneke},
+  title = {Expected improvement in efficient global optimization through bootstrapped
+	kriging},
+  journal = {Journal of Global Optimization},
+  year = {2012},
+  volume = {54},
+  pages = {59--73},
+  number = {1},
+  publisher = {Springer}
+}
+
 @ARTICLE{Krause08JMLR,
-  author = {A. Krause and A. Singh and C. Guestrin},
+  author = {Andreas Krause and Ajit Singh and Carlos Guestrin},
   title = {Near-optimal Sensor Placements in {Gaussian} Processes: Theory, Efficient
 	Algorithms and Empirical Studies},
   journal = {Journal of Machine Learning Research},
   timestamp = {2010.06.18}
 }
 
+@ARTICLE{KroemerJRAS_66360,
+  author = {Kroemer, Oliver and Detry, Renaud and Piater, Justus and Peters,
+	Jan},
+  title = {Combining Active Learning and Reactive Control for Robot Grasping},
+  journal = {Robotics and Autonomous Systems},
+  year = {2010},
+  volume = {58},
+  pages = {1105-1116},
+  number = {9},
+  owner = {rmcantin},
+  timestamp = {2013.05.07},
+  url = {http://robot-learning.de/pmwiki/uploads/Publications/KroemerJRAS_6636[0].pdf}
+}
+
 @ARTICLE{Kushner:1964,
-  author = {H.J. Kushner},
+  author = {Harold J. Kushner},
   title = {A New Method of Locating the Maximum of an Arbitrary Multipeak Curve
 	in the Presence of Noise},
-  journal = {{J. of Basic Eng.}},
+  journal = {Journal of Basic Engineering},
   year = {1964},
   volume = {86},
   pages = {97--106},
   timestamp = {2011.10.24}
 }
 
+@INPROCEEDINGS{Lizotte2007,
+  author = {Daniel Lizotte and Tao Wang and Michael Bowling and Dale Schuurmans.},
+  title = {Automatic gait optimization with gaussian process regression},
+  booktitle = {Proceedings of the Twentieth International Joint Conference on Artificial
+	Intelligence (IJCAI)},
+  year = {2007},
+  pages = {944--949},
+  owner = {rmcantin},
+  timestamp = {2013.05.06}
+}
+
+@INPROCEEDINGS{Marchant2012,
+  author = {Marchant, Roman and Ramos, Fabio},
+  title = {Bayesian optimisation for Intelligent Environmental Monitoring},
+  booktitle = {Intelligent Robots and Systems (IROS), 2012 IEEE/RSJ International
+	Conference on},
+  year = {2012},
+  pages = {2242-2249},
+  doi = {10.1109/IROS.2012.6385653},
+  issn = {2153-0858},
+  keywords = {Bayes methods;environmental monitoring (geophysics);mean square error
+	methods;mobile robots;optimisation;Bayesian optimisation;RMSE error;autonomous
+	moving robot;intelligent environmental monitoring;sensing location;sensor
+	network;spatial-temporal model;utility function;Gases;Optimization;Robot
+	sensing systems},
+  owner = {rmcantin},
+  timestamp = {2013.05.06}
+}
+
+@TECHREPORT{Martinez-Cantin2012,
+  author = {Ruben Martinez-Cantin},
+  title = {Efficient computation of stable grasps through {Bayesian} optimization},
+  institution = {ISR-Vislab},
+  year = {2012},
+  owner = {rmcantin},
+  timestamp = {2013.05.07}
+}
+
 @ARTICLE{MartinezCantin09AR,
-  author = {R. Martinez-Cantin and N. de Freitas and E. Brochu and J.A. Castellanos
-	and A. Doucet},
+  author = {Ruben Martinez-Cantin and Nando {de Freitas} and Eric Brochu and
+	Jose A. Castellanos and Arnoud Doucet},
   title = {A {Bayesian} Exploration-Exploitation Approach for Optimal Online
 	Sensing and Planning with a Visually Guided Mobile Robot.},
   journal = {Autonomous Robots - Special Issue on Robot Learning, Part B},
 }
 
 @INPROCEEDINGS{MartinezCantin07RSS,
-  author = {R. Martinez-Cantin and N. de Freitas and J.A. Castellanos},
+  author = {Ruben Martinez-Cantin and Nando {de Freitas} and Arnoud Doucet and
+	Jose A. Castellanos},
   title = {Active Policy Learning for Robot Planning and Exploration under Uncertainty},
   booktitle = {Robotics: Science and Systems},
   year = {2007},
 }
 
 @INCOLLECTION{Mockus78,
-  author = {J. Mockus and V. Tiesis and A. Zilinskas},
+  author = {Jonas Mockus and Vytautas Tiesis and Antanas Zilinskas},
   title = {The application of {Bayesian} methods for seeking the extremum},
   booktitle = {Towards Global Optimisation 2},
   publisher = {Elsevier},
 }
 
 @INPROCEEDINGS{Moore:1996,
-  author = {A.W. Moore and J. Schneider},
+  author = {Andrew W. Moore and Jeff Schneider},
   title = {Memory-based Stochastic Optimization},
   booktitle = {Advances in Neural Information Processing Systems},
   year = {1996},
   timestamp = {2010.06.18}
 }
 
-@ARTICLE{O'Hagan1992,
-  author = {A. O'Hagan},
-  title = {Some Bayesian Numerical Analysis},
+@ARTICLE{OHagan1992,
+  author = {Anthony O'Hagan},
+  title = {Some {B}ayesian Numerical Analysis},
   journal = {Bayesian Statistics},
   year = {1992},
   volume = {4},
   timestamp = {2011.10.24}
 }
 
-@BOOK{Rasmussen:2006,
-  title = {Gaussian Processes for Machine Learning},
-  publisher = {The {MIT} Press},
+@INPROCEEDINGS{Peters_PIICIRS_2006,
+  author = {Peters, J. and Schaal, S.},
+  title = {Policy gradient methods for robotics},
+  booktitle = {Proceedings of the IEEE International Conference on Intelligent Robotics
+	Systems (IROS 2006)},
   year = {2006},
-  author = {C.E. Rasmussen and C.K.I. Williams},
-  address = {Cambridge, Massachusetts},
-  booktitle = {Gaussian Processes for Machine Learning},
-  keywords = {Gaussian processes, fornips},
+  key = {policy gradient methods, reinforcement learning, robotics},
   owner = {rmcantin},
-  timestamp = {2010.06.18}
+  timestamp = {2013.05.07},
+  url = {http://www-clmc.usc.edu/publications/P/peters-IROS2006.pdf}
 }
 
 @ARTICLE{Rasmussen2010,
-  author = {Carl Edward Rasmussen and Hannes Nickisch},
-  title = {Gaussian Processes for Machine Learning (GPML) Toolbox},
+  author = {Carl E. Rasmussen and Hannes Nickisch},
+  title = {{G}aussian Processes for Machine Learning {(GPML)} Toolbox},
   journal = {Journal of Machine Learning Research},
   year = {2010},
   volume = {11},
   timestamp = {2011.09.09}
 }
 
+@BOOK{Rasmussen:2006,
+  title = {{G}aussian Processes for Machine Learning},
+  publisher = {The {MIT} Press},
+  year = {2006},
+  author = {Carl E. Rasmussen and Christopher K.I. Williams},
+  address = {Cambridge, Massachusetts},
+  booktitle = {Gaussian Processes for Machine Learning},
+  keywords = {Gaussian processes, fornips},
+  owner = {rmcantin},
+  timestamp = {2010.06.18}
+}
+
 @ARTICLE{Sacks89SS,
-  author = {J. Sacks and W.J. Welch and T.J. Mitchell and H.P. Wynn},
+  author = {Jerome Sacks and William J. Welch and Toby J. Mitchell and Henry
+	P. Wynn},
   title = {Design and Analysis of Computer Experiments},
   journal = {Statistical Science},
   year = {1989},
   title = {The Design and Analysis of Computer Experiments},
   publisher = {Springer-Verlag},
   year = {2003},
-  author = {T.J. Santner and B. Williams and W. Notz},
+  author = {Thomas J. Santner and Brian J. Williams and William I. Notz},
   owner = {rmcantin},
   timestamp = {2010.06.18}
 }
 
 @PHDTHESIS{Sasena:2002,
-  author = {M.J. Sasena},
+  author = {Michael J. Sasena},
   title = {Flexibility and Efficiency Enhancement for Constrained Global Design
 	Optimization with {K}riging Approximations},
   school = {University of Michigan},
 }
 
 @INCOLLECTION{Schonlau98,
-  author = {Schonlau, M. and Welch, W. and Jones, D. },
+  author = {Matthias Schonlau and William J. Welch and Donald R. Jones},
   title = {Global Versus Local Search in Constrained Optimization of Computer
 	Models},
   booktitle = {New Developments and Applications in Experimental Design},
   timestamp = {2010.06.18}
 }
 
-@INBOOK{Schumacher01,
-  pages = {565--570},
+@INPROCEEDINGS{Schumacher01,
+  author = {Schumacher, Courtney and Vose, Michael D. and Whitley, L Darrell},
   title = {The No Free Lunch and Problem Description Length},
-  publisher = {Morgan Kaufmann},
-  year = {2001},
-  author = {Schumacher, C and Vose, M D and Whitley, L D},
   booktitle = {Proceedings of the Genetic and Evolutionary Computation Conference
 	GECCO2001},
+  year = {2001},
+  pages = {565--570},
+  publisher = {Morgan Kaufmann},
   owner = {rmcantin},
   timestamp = {2011.10.24},
   url = {http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.13.892}
 }
 
+@ARTICLE{singh09efficient,
+  author = {Amarjeet Singh and Andreas Krause and Carlos Guestrin and William
+	Kaiser},
+  title = {Efficient Informative Sensing using Multiple Robots},
+  journal = {Journal of Artificial Intelligence Research (JAIR)},
+  year = {2009},
+  volume = {34},
+  pages = {707-755},
+  owner = {rmcantin},
+  timestamp = {2013.05.07}
+}
+
+@INPROCEEDINGS{singh09nonmyopic,
+  author = {Amarjeet Singh and Andreas Krause and William Kaiser},
+  title = {Nonmyopic Adaptive Informative Path Planning for Multiple Robots},
+  booktitle = {Proc. International Joint Conference on Artificial Intelligence (IJCAI)},
+  year = {2009},
+  owner = {rmcantin},
+  timestamp = {2013.05.07}
+}
+
+@INPROCEEDINGS{Snoek2012,
+  author = {Jasper Snoek and Hugo Larochelle and Ryan Adams},
+  title = {Practical {B}ayesian Optimization of Machine Learning Algorithms},
+  booktitle = {Advances in Neural Information Processing Systems 25},
+  year = {2012},
+  editor = {P. Bartlett and F.C.N. Pereira and C.J.C. Burges and L. Bottou and
+	K.Q. Weinberger},
+  pages = {2960--2968},
+  owner = {rmcantin},
+  timestamp = {2013.02.22}
+}
+
+@ARTICLE{Song2012,
+  author = {Le Song and Alex Smola and Arthur Gretton and Justin Bedo and Karsten
+	Borgwardt},
+  title = {Feature Selection via Dependence Maximization},
+  journal = {Journal of Machine Learning Research},
+  year = {2012},
+  volume = {13(May)},
+  pages = {1393−1434},
+  owner = {rmcantin},
+  timestamp = {2013.02.25}
+}
+
 @INPROCEEDINGS{Srinivas10,
-  author = {N. Srinivas and A. Krause and S. Kakade and M. Seeger},
-  title = {Gaussian Process Optimization in the Bandit Setting: No Regret and
-	Experimental Design},
+  author = {Niranjan Srinivas and Andreas Krause and Sham Kakade and Matthias
+	Seeger},
+  title = {{G}aussian Process Optimization in the Bandit Setting: No Regret
+	and Experimental Design},
   booktitle = {Proc. International Conference on Machine Learning (ICML)},
   year = {2010},
   owner = {rmcantin},
   timestamp = {2010.06.18}
 }
 
-@INCOLLECTION{Streeter03,
+@INPROCEEDINGS{Streeter03,
   author = {Streeter, Matthew},
   title = {Two Broad Classes of Functions for Which a No Free Lunch Result Does
 	Not Hold},
   booktitle = {Genetic and Evolutionary Computation — GECCO 2003},
-  publisher = {Springer Berlin / Heidelberg},
   year = {2003},
-  editor = {Cantú-Paz, Erick and Foster, James and Deb, Kalyanmoy and Davis,
-	Lawrence and Roy, Rajkumar and O’Reilly, Una-May and Beyer, Hans-Georg
-	and Standish, Russell and Kendall, Graham and Wilson, Stewart and
-	Harman, Mark and Wegener, Joachim and Dasgupta, Dipankar and Potter,
-	Mitch and Schultz, Alan and Dowsland, Kathryn and Jonoska, Natasha
-	and Miller, Julian},
   volume = {2724},
   series = {Lecture Notes in Computer Science},
-  pages = {210-210},
+  pages = {1418–1430},
+  publisher = {Springer Berlin / Heidelberg},
   owner = {rmcantin},
   timestamp = {2011.10.24}
 }
 
+@INPROCEEDINGS{Tesch_2012_7375,
+  author = {Matthew Tesch and Jeff Schneider and Howie Choset},
+  title = {Expensive Multiobjective Optimization and Validation with a Robotics
+	Application},
+  booktitle = {Neural Information Processing Systems: Workshop on Bayesian Optimization
+	\& Decision Making},
+  year = {2012},
+  month = {December},
+  owner = {rmcantin},
+  timestamp = {2013.05.07}
+}
+
+@INPROCEEDINGS{Tesch_2011_7370,
+  author = {Matthew Tesch and Jeff Schneider and Howie Choset},
+  title = {Adapting Control Policies for Expensive Systems to Changing Environments
+	},
+  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems
+	(IROS)},
+  year = {2011},
+  month = {September},
+  owner = {rmcantin},
+  timestamp = {2013.05.07}
+}
+
+@INCOLLECTION{ToussaintBSG,
+  author = {Marc Toussaint},
+  title = {The {B}ayesian Search Game},
+  booktitle = {Theory and Principled Methods for Designing Metaheuristics},
+  publisher = {Springer},
+  year = {in press},
+  editor = {Yossi Borenstein, Alberto Moraglio},
+  owner = {rmcantin},
+  timestamp = {2013.02.25}
+}
+
 @ARTICLE{Vazquez10,
   author = {Emmanuel Vazquez and Julien Bect},
   title = {Convergence properties of the expected improvement algorithm with
   timestamp = {2010.06.18}
 }
 
+@INPROCEEDINGS{Veiga2013,
+  author = {Filipe Veiga and Alexandre Bernardino},
+  title = {Active Tactile Exploration for Grasping},
+  booktitle = {ICRA Workshop on Autonomous Learning},
+  year = {2013},
+  owner = {rmcantin},
+  timestamp = {2013.05.07}
+}
+
 @ARTICLE{Villemonteix2009,
-  author = {Villemonteix, J. and Vazquez, E. and Walter, E.},
+  author = {Villemonteix, Julien and Vazquez, Enmanuel and Walter, Eric},
   title = {An informational approach to the global optimization of expensive-to-evaluate
 	functions},
   journal = {Journal of Global Optimization},
   timestamp = {2010.06.22}
 }
 
+@INPROCEEDINGS{ZiyuWang2013,
+  author = {Ziyu Wang and Masrour Zoghi and David Matheson and Frank Hutter and
+	Nando {de Freitas}},
+  title = {Bayesian Optimization in a Billion Dimensions via Random Embeddings},
+  booktitle = {International Joint Conference on Artificial Intelligence (IJCAI).
+	Technical Report arXiv:1301.1942.},
+  year = {2013},
+  owner = {rmcantin},
+  timestamp = {2013.04.08}
+}
+
 @ARTICLE{Williams_Santner_Notz_2000,
-  author = {Williams, B J and Santner, T J and Notz, W I},
+  author = {Williams, Brian J. and Santner, Thomas J. and Notz, William I.},
   title = {Sequential design of computer experiments to minimize integrated
 	response functions},
   journal = {Statistica Sinica},
   timestamp = {2011.10.24}
 }
 
+@TECHREPORT{Wingate2013,
+  author = {David Wingate and Carlos Diuk and Timothy O’Donnell and
+	
+	Joshua Tenenbaum and Samuel Gershman},
+  title = {Compositional Policy Priors},
+  institution = {MIT-CSAIL},
+  year = {2013},
+  number = {MIT-CSAIL-TR-2013-007},
+  owner = {rmcantin},
+  timestamp = {2013.05.06}
+}
+
 @ARTICLE{Wolpert97,
-  author = {Wolpert, David H. and Macready William G.},
+  author = {Wolpert, David H. and Macready, William G.},
   title = {No free lunch theorems for optimization},
   journal = {IEEE Transactions on Evolutionary Computation},
   year = {1997},

File src/nonparametricprocess.cpp

 
 #include "gaussian_process.hpp"
 #include "gaussian_process_ml.hpp"
+#include "gaussian_process_normal.hpp"
 #include "student_t_process_jef.hpp"
 #include "student_t_process_nig.hpp"
 
       s_ptr = new GaussianProcess(dim,parameters);
     else  if(!name.compare("GAUSSIAN_PROCESS_ML"))
       s_ptr = new GaussianProcessML(dim,parameters);
+    else  if(!name.compare("GAUSSIAN_PROCESS_NORMAL"))
+      s_ptr = new GaussianProcessNormal(dim,parameters);
     else if (!name.compare("STUDENT_T_PROCESS_JEFFREYS"))
       s_ptr = new StudentTProcessNIG(dim,parameters); 
     else if (!name.compare("STUDENT_T_PROCESS_NORMAL_INV_GAMMA"))