Requirements

• 32G memory
• 256G disk

Prerequisites

• docker
• docker-compose
• aws s3 cli

Setup directories and config

mkdir -p /opt/milvus/conf 
wget https://bitbucket.org/%21api/2.0/snippets/jhn-ngo/zXBqoz/027a763525b9741688f8ce51e151ef72233b9ffc/files/server_config.yaml /opt/milvus/conf

Download embeddings (about 75G)

aws s3 cp --profile wasabi --endpoint-url https://s3.eu-central-1.wasabisys.com  --recursive  s3://jhn01/milvus_laser_300/ /opt/milvus/

Download application configuration file and edit needed values:

wget https://bitbucket.org/%21api/2.0/snippets/jhn-ngo/9XEpgp/ea42f520be750a3cca65bbc4b5868d6b5063ef22/files/recsy_config_europeana.ini

This file contain 3 sections:

• [app:main] - for API parameters configuration

  • port: set up port on which application will run (int). Default is 5090
  • no_milvus: true/false value which sets if vector engine should be used. Default is true
  • milvus_host: host where milvus vector engine is runnng. Default is localhost
  • milvus_db_path: path to the vector engine database (mapped inside docker-compose)
  • milvus_index_name: name of the index to be used inside vector engine. Default is europeana_prod_laser_300

• [app:search] - parameters for calling external europeana search API's

  • RECSY_SEARCH_KEY: API key for Europeana Search API
  • RECSY_SEARCH_API: endpoint URL for Europeana Search API. Default is https://api.europeana.eu/record
  • RECSY_USER_SETS_API: endpoint URL for Euopeana Sets API. Default is https://api.europeana.eu/set
  • RECSY_EMBEDDINGS_API: url of embeddings API. Default is: http:// 85.214.245.200:4997

• [app:events] - parameters for setting up events storage (it could be either elasticsearch or sql)

  • RECSY_EVENTS_PROVIDER: sql or elasticsearch. Default is: sql
  • RECSY_EVENTS_URL: url of database for hosting events. Default is: localhost
  • RECSY_EVENTS_PORT: port of database for hosting events. Default is 3200 for elasticsearch or 5432 for sql
  • RECSY_EVENTS_SQL_DIALECT: sql dialect. Default is: postgres (used for sql provider only)
  • RECSY_EVENTS_DB_USER: sql database dialect. Default is: europeana (used for sql provider only)
  • RECSY_EVENTS_DB_PASSWORD: sql database password. (used for sql provider only)
  • RECSY_EVENTS_DB_NAME: sql database name. Default is: events (used for sql provider only)

You could read more on configuration file format here

Run API and Milvus

Create directory for volume of postgres data:

mkdir /home/recsy/events_postgres_data

Then download docker-compose and run it:

wget https://bitbucket.org/%21api/2.0/snippets/jhn-ngo/G9nzpj/ac2892b5e04f477c9268e56570994de571fd140c/files/docker-compose.yml
docker-compose up

Basic test

curl -XGET  http://localhost:5090/recommend/status

You should get list of IDs