HTTPS SSH

CoMeRDA

============

CoMeRDA is an aggregated search system that provides a webbased user interface with multiple options for visualizing search results. Users are able to search, filter and bookmark results form different document collections. A user's interactions with the system are logged with the help of ILPS Logging.

CoMeRDA was used in experiments for the SIGIR 2013 paper

Aggregated Search Interface Preferences in Multi-Session Search Tasks Bron M, van Gorp J, Nack F, Baltussen LB, de Rijke M.

Requirements


  • Python > 2.6

  • pip <https://pypi.python.org/pypi/pip>_

  • virtualenv <https://pypi.python.org/pypi/virtualenv>_ (optional)

  • Relational database (e.g. SQLite, MySQL or PostgreSQL) for storing user bookmarks

  • Apache Solr <http://lucene.apache.org/solr/>_ containing indexes with documents in the CoMeRDA format
  • A webserver with WSGI or proxy capabilities (e.g. nginx <http://nginx.org/> or Apache httpd <http://httpd.apache.org/>)
  • A working instance of ILPS Logging if you wish to log user interactions

Installation


  1. Clone the Git repo::
    $ Git clone git@git.dispectu.com:dispectu/comerda.git
    $ cd comerda/
  1. Create a virtualenv, activate it and install the required Python packages::
    $ virtualenv ~/my_pyenvs/comerda
    $ source ~/my_pyenvs/comerda/bin/activate 
    $ pip install -r requirements.txt
  1. Create a local settings file to override the default settings specified in settings.py. In the next steps we describe to mininal number of settings that should be changed to get the application up-and-running. Please have a look at the comments in settings.py to get an overview of all possible settings::
    $ vim server/local_settings.py
  1. When running the application in a production environment, set DEBUG to False
  2. Set the SECRET_KEY for the installation (this key is used to sign cookies). A good random key can be generated as follows::
    >>> import os
    >>> os.urandom(24)
    '\x86\xb8f\xcc\xbf\xd6f\x96\xf0\x08v\x90\xed\xad\x07\xfa\x01\xd0\\L#\x95\xf6\xdd'
  1. Set the correct URLs for:
  • SITE_URL; the URL on which the user interface and assets are reachable
  • WEB_API_URL; the URL used by the client side to obtain data
  • SOLR_URL; the URL of the Solr instance used for searching collections
  • SCHEMA_DIR; the absolute path to the folder that contains the Solr schema XML files (see the repository's schemas directory)
  1. Provide the URI of the database. The SQLAlchemy documentation provides inforamation on how to structure the URI <http://docs.sqlalchemy.org/en/rel_0_8/core/engines.html#database-urls>_ for different databases. To use an SQLite database named comderda.db set DATABASE_URI to sqlite:///comerda.db.
  2. Load the schema in the database configured in the previous step (this will create all the required tables)::
    >>> from server import models
    >>> models.db.create_all()
  1. Use WSGI server (like uWSGI or Gunicorn) to run the Flask application. Use a webserver as a proxy to the WSGI container. Also make sure to serve static assets directly through the webserver.

Optional settings


User authentication

CoMeRDA is used in combination with an existing system that handles user registration and authentication. After successful authentication this system adds a record to the sessions table and redirects the user to the CoMeRDA interface. With the authentication token send as a GET parameter CoMeRDA verifies that the user is indeed authenticated and marks the token as 'used'.

The configuration options REDIRECT_LOGIN_URL and REDIRECT_LOGOUT_URL can be set to redirect users to a specific page if they are not logged in or when they log out of the system.

If you don't want to use CoMeRDA's user specific features (bookmarking and logging usage on a user level) set ENABLE_USER_AUTH to False.

Enable/disable usage logging

ILPS Logging is used to log interactions of users with the interface. To enable logging set ENABLE_ILPSLOGGING to True. ILPSLOGGING_API_URL should be the URL of the ILPS Logging instance and ILPSLOGGING_PROJECT_KEY the API key with log access to your project. More detailed log settings can be found in JavaScript on the bottom of the layout template (server/templates/layout.html).

ILPS Logging Events


In addition to the default ILPS Logging events (such as mouse_click, mouse_move, query and paginate) the following 'custom' events are being logged:

Event name Event description Event details
query_input_submit when the user submits a new keyword query via the input box
switch_to_collection when the user switches between collections while in the 'basic' search screen the name of the target collection
switch_screen when the user switches between the 'basic', 'combined' and 'similarity' search screens name of the target screen
document_open when the user opens a document collection and document id
document_close when the user closes a document collection and document id
filter_add when the user adds a filter to the current query collection, facet name and facet value
filter_remove when the user removes a filter from the current query collection, facet name and facet value
hit_hover when the user hovers over a search results with the mouse rank of the document within the result set, collection, document id and the duration of the hover (in ms)
bookmark_add when the user bookmarks a document collection and document id
bookmark_delete when the user removes a bookmarked document collection and document id
filters_show when the user click the 'filters' button while on the collection 'combined' or 'similarity' search screen
filters_show when the user click the 'filters' button while on the collection 'combined' or 'similarity' search screen
disable_facets when the JS function for disabling faceting is executed name of the target screen
enable_facets when the JS function for enabling faceting is executed name of the target screen

Manipulating the interface with external JavaScript


To dynamical modify behavior of the CoMeRDA interface, it is possible to load an external JavaScript file that can execute code on the CoMeRDA pages. This can be a useful feature when conducting experiments with multiple conditions.

To load an external JavaScript file, set the EXTERNAL_JS configuration option to the absolute and publicly reachable URL of this file.

The following JavaScript functions can be used to programmatically disable or enabling CoMeRDA's faceting functionality::

    // Possible values for screen_name: 'basic', 'combined', 'similar'
    disable_facets(screen_name, collection);

    enable_facets(screen_name, collection);

When facets are disabled or enabled, an event is logged and the ILPS Logging state object will automatically change as well.

Use the following function to switch between screens::

    // Remove the 'aggregated' screen button
    $('#aggregated').remove();

    // Switch to the similarity search screen
    switch_search_screen('similarity');

It is advisable to wrap the externally loaded code within jQuery's $( document ).ready( handler ) function in order to be sure that all actions are executed after the page is loaded.

References


Bron M, van Gorp J, Nack F, Baltussen LB, de Rijke M. 2013.
Aggregated Search Interface Preferences in Multi-Session Search Tasks. SIGIR'13: 36th international ACM SIGIR conference on Research and development in information retrieval.

License


Copyright 2013 ILPS group UvA (Information and Language Processing Systems group University of Amsterdam)

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.