libsgd /

Filename Size Date modified Message
89 B
476 B
1.5 KB
4.8 KB
6.3 KB
6 B
109 B
28 B
214 B
1.5 KB

libsgd - Efficient Stochastic Gradient Descent for Machine Learning

Author: Olivier Grisel <>

libsgd is a SIMD optimized C library with Python bindings (using ctypes) using either SSE or OpenCL.

The goal of libsgd is to provide an optimized implementation of backpropagatable modules as a base for efficient machine learning algorithms such as feed forward multi layer perceptron, stacked autoencoders, linear support vector machines solved in the primal and potentially more.

libsgd has a focus on performance and currently needs a POSIX SSE2 enabled compiler such as gcc 4 or llvm-gcc to work correctly. Hence it will not work on non x86 architectures nor build under windows. It should be quite easy to make the code portable to be able to build / run under non POSIX systems with non x86 architectures but this is not a priority right now. If you want to work on making it portable please contact the developers so as to coordinate our efforts.

libsgd also aims to leverage the computing power of SIMD processors such as latest generation GPUs or Cell BE SPUs probably through implementations of the newly published OpenCL standard.

See the included ROADMAP.txt file for a complete list of implemented and planned features.

libsgd can be built as a standalone C library using the cmake build configuration tool or directly as a python package using the script.

Overview of the source tree

  • - Python build / packaging script
  • lib/ - C source code used to build the library
  • src/ - Python source code for the bindings
  • tests/ - Python source code for the tests
  • ctests/ - simple pure C programs to make it easier to debug the C
    library with gdb / nemiver and valgrind
  • doc/ - testable documentation using the Python doctest module (TODO)
  • bench/ - benchmark suite written in Python (TODO)
  • examples/ - sample Python code for common usages (TODO)


libsgd has a build dependency on a blas library, preferably atlas with SSE optimizations.

Alternatively libsgd will feature a OpenCL optimised version as a build configuration parameter.

The Python package uses numpy to handle load and save large multi dimensional arrays from and to the filesystem and matplotlib to plot the results or convergence curbs.

You need nose to run the tests suite.

Building and installing as a standalone C library

Install CMake and then:

% mkdir build
% cd build
% cmake ..
% make
% make test
% sudo make install

Upon successful completion the output should end with:

-- Installing: /usr/local/lib/
-- Installing: /usr/local/include/sgd.h

Please refer to cmake --help for other build / install options.

Building and installing as a Python package

With python 2.5+ (3.x is not supported yet, contributions welcomed):

% python build
% sudo python install

You can then check that the python module is correctly installed with:

% python -m sgd

If the module is imported sucessfully it should not display any error on the screen.

You can also build the python package inplace and be able to import it without performing a system wide install:

% python build_ext -i
% export PYTHONPATH=`pwd`/src/
% python -m sgd

Run the python tests

Adjust the [nosetests] section of setup.cfg to fit your tastes and run:

% nosetests


Thanks to:


Technical papers introducing the main algorihtms implemented in libsgd.

  • Yann LeCun, Leon Bottou, Genevieve Orr and Klaus-Robert Müller - 1998:

      author = {{Le Cun}, Yann and Bottou, L\'{e}on and Orr, Genevieve B. and M{\"{u}}ller, Klaus-Robert},
      title = {Efficient Backprop},
      booktitle = {Neural Networks, Tricks of the Trade},
      series = {Lecture Notes in Computer Science LNCS~1524},
      publisher = {Springer Verlag},
      year = {1998},
      url = {},
  • Shai Shalev-Shwartz, Yoram Singer and Nathan Srebro - Pegasos: Primal Estimated sub-GrAdient SOlver for SVM - 2007

  • John Duchi and Yoram Singer - Online and Batch Learning using Forward Looking Subgradients - 2008