1. ILPS
  2. Miscellaneous
  3. Solr Online Learning to Rank Plug-in

Overview

HTTPS SSH

Description

The Solr Online Learning to Rank plug-in can be used in order to automatically learn the weights of the features of a Solr search engine online, using the interactions with the user (clicks).

It consists of two parts, a Solr Search component and an external experiment server written in python.

The Search component handles the search requests. If the current query is not an experiment query, it just uses the current weight vector to produce the ranking. If it is, it creates the interleaved list and informs the external server.

The external server is responsible for holding and sharing the weight vector among different search requests. It handles one experiment at a time. After an experiment query is made, it waits for the clicks for n seconds and then updates the weight vector if it has to.

One that wants to use the plug-in, just needs to provide the Solr search component with a unique query id and send the clicked document ids with the query id to the external server as they happen.

The plug-in can handle the learning of the text search and of numeric features

The current weight vector is stored in t.wv, and logging with all the interactions with the external server are stored at history.log.* (new file every ~2 Mbytes).

For more information on the algorithms used, see for example : Hofmann, Katja. "Fast and reliable online learning to rank for information retrieval." (2013).

Set-up

Configuration

  • Edit the Solr configuration file (solrconfig.xml). See solrconfigExample.xml for an example. You can use the provided compiled jar (solr-LTR-plugin-SNAPSHOT1.jar) or you can build the jar using ant.
  • Edit the external server configuration file. See external_server/server_config.cfg. The external server must be restarted every time a parameter is changed in the configuration file.

Dependencies

Solr Plug-in

  • Apache SOLR 4.x

External server

  • Python (tested with v.2.7)
Python dependencies
  • Numpy (tested with v1.6.2)
  • CherryPy (tested with v3.2.3)

You can install the python dependencies by running:

pip install -r external_server/requirements.txt 

Usage

Run the external server before running the Solr instance using :

python external_server/external_server.py

Example query

localhost:8983/solr/learningquery?qid=1&q=grand+theft

Example click

localhost:8983/solr/click?qid=1&docid=123456789

Contributors

Developed by Nikos Voskarides and Anne Schuth at ILPS, University of Amsterdam.