Commits

Ruben Martinez-Cantin  committed c20fdd2

Improved documentation.

  • Participants
  • Parent commits 07574b6
  • Tags v0.5

Comments (0)

Files changed (5)

File doxygen/contribute.dox

 used in the library. Python would only find the library if it is
 called exactly \c bayesopt.so
 
+\section relsoft Related software
+
+\li Spearmint (Python): A library based on \cite Snoek2012. It is more oriented to cluster computing https://github.com/JasperSnoek/spearmint
+\li Hyperopt (Python): A library mainly based on \cite Bergstra2011. It uses different models for the surrogate model https://github.com/jaberg/hyperopt
+\li Perk (Fortran+Shell): A library for surrogate modelling. It was mainly used in \cite Santner03. http://www.stat.osu.edu/~comp_exp/book.html
+\li SUMO (Matlab): A library for surrogate modelling. Its main purpose it is not optimization but it also includes the Expected Imporvement algorithm. http://www.sumowiki.intec.ugent.be/Main_Page
+\li GPML (Matlab): The most popular library for Gaussian Process. BayesOpt uses some of its design principles. http://www.gaussianprocess.org/gpml/code/matlab/doc/
+\li NLOPT (C-Many languages): One of the best libraries for general purpose nonlinear optimization. http://ab-initio.mit.edu/wiki/index.php/NLopt
+
 */

File doxygen/install.dox

 
 The most important options/variables are explained in \ref confinst.
 
-\subsection instpythonwin Python interface:
-
-The Python interface has not been tested in \b Windows because getting
-the correct dependencies is highly involved. You might need to
-download and install:
-\li Python (binary and \b sources): http://www.python.org
-\li Numpy: http://new.scipy.org/download.html 
-
-Also, read this article about how to link everything:
-http://docs.python.org/2/extending/windows.html#building-on-windows
-
-
 \subsection instmatlabwin MATLAB/Octave interface:
 
 Make sure the library is compiled with the MATLAB_COMPATIBLE option
 might need to change the \c MATLABROOT path with the root folder of
 your Matlab install and copy the dlls. Then, run \c mingw32-make. Note
 that \c mingw32-make only supports 32 bits libraries, so you need a 32
-bit Matlab install.
+bit version of Matlab. There is a fork of MinGW with 64 bit support
+under development, but it has not been tested here.
+
+\subsection instpythonwin Python interface:
+
+The Python interface has not been tested in \b Windows because getting
+the correct dependencies is highly involved. You might need to
+download and install:
+\li Python (binary and \b sources): http://www.python.org
+\li Numpy: http://new.scipy.org/download.html 
+
+Also, read this article about how to link everything:
+http://docs.python.org/2/extending/windows.html#building-on-windows
 
 
 <HR>
 
 \section confinst Configure the compilation/install
 
-As we have made to select the install path or to add python bindings,
-CMake allows to configure the compilation using some variables. These
+CMake allows to configure the compilation using some variables (see
+for example how to compile the Python module in Linux). These
 variables can be set in Linux/MacOS from the command line with the -D
 flag:
 \verbatim
 >> cmake -DCMAKE_BUILD_TYPE=Debug .
 \endverbatim
 
-If you use ccmake or CMake for Windows, just modify the value of the
-variable.
-
+If you use ccmake in Linux/MacOS or CMake for Windows, you can access
+to a list of all the variables and their values. Just modify the value
+of the desired variable.
 
 \subsection instshared Compile as shared libraries
 

File doxygen/models.dox

 /*! \page modelopt Models and functions
+\tableofcontents
 
 This library was originally developed for as part of a robotics
 research project \cite MartinezCantin09AR \cite MartinezCantin07RSS,
 
 "cHedge(cSum(cEI,cDistance),cLCB,cPOI,cOptimisticSampling)"
 
-\subsection learnmod Methods for learning the kernel parameters  
+\section learnmod Methods for learning the kernel parameters  
 
 As commented before, we consider that the prior of the kernel
 hyperparameters \f$\theta\f$ --if available-- is independent of other
 assume no prior. Since we assume that the hyperparameters are independent,
 we can apply priors selectively only to a small set.
 
+\section initdes Initial design methods
+
+In order to build a suitable surrogate function, we a need a
+preliminar set of samples. In Bayesian optimization this is typically
+performed using alternative experimental design criteria. In this
+first step, usually the main criteria is space filling. Thus, we have
+implemented the subsequent designs:
+
+\li Latin hypercube sampling: Each dimension of the space is divided
+in several intervals. Samples are then taken according to a
+generalization of the Latin square
+scheme. http://en.wikipedia.org/wiki/Latin_hypercube_sampling
+
+\li Sobol sequences: It is a set of quasi-random low-discrepancy
+sequences. Thus the space is sampled more evenly than with uniform
+sampling. http://en.wikipedia.org/wiki/Sobol_sequence
+
+\li Uniform sampling: The search space is sampled uniformly.
+
+Note: Since we do not assume any struture in the set of discrete
+points during discrete optimization, only uniform sampling of the
+discrete set is available in that case.
+
 */

File doxygen/reference.dox

 not included by default in the linker, Python or Matlab paths by
 default. This is specially critical when building shared libraries
 (mandatory for Python usage). The script \em exportlocalpaths.sh makes
-sure that the folder is included in all the necessary paths.
+sure that the folder with the libraries is included in all the
+necessary paths.
 
 After that, there are 3 steps that should be follow:
 \li Define the function to optimize.
 \section params Understanding the parameters
 
 BayesOpt relies on a complex and highly configurable mathematical
-model. Also, the key to nonlinear optimization is to include as much
-knowledge as possible about the target function or about the
-problem. Or, if the knowledge is not available, keep the model as
-general as possible (to avoid bias).
+model. In theory, it should work reasonably well for many problems in
+its default configuration. However, Bayesian optimization shines when
+we can include as much knowledge as possible about the target function
+or about the problem. Or, if the knowledge is not available, keep the
+model as general as possible (to avoid bias). In this part, knowledge
+about Gaussian process or nonparametric models in general might be
+useful. 
+
+For example, with the parameters we can select the kind of kernel,
+mean or surrogate model that we want to use. With the kernel we can
+play with the smoothness of the function and it's derivatives. The
+mean function can be use to model the overall trend (is it flat?
+linear?). If we know the overall signal variance we better use a
+Gaussian process, if we don't, we should use a Student's t process
+instead.
 
 For that reason, the parameters are bundled in a structure or
 dictionary, depending on the API that we use. This is a brief

File optimization.bib

   timestamp = {2013.02.22}
 }
 
+@INPROCEEDINGS{Bergstra2011,
+  author = {James Bergstra and Remi Bardenet and Yoshua Bengio and Balázs Kégl},
+  title = {Algorithms for Hyper-parameter Optimization. },
+  booktitle = {Advances in Neural Information Processing Systems},
+  year = {2011},
+  pages = {2546–2554},
+  owner = {rmcantin},
+  timestamp = {2013.07.24}
+}
+
 @ARTICLE{Song2012,
   author = {Le Song and Alex Smola and Arthur Gretton and Justin Bedo and Karsten
 	Borgwardt},