HTTPS SSH

Metadata Hopper is a tool built by the University of Illinois at Chicago Library in cooperation with Chicago Collections. It is designed to work with the eXtensible Text Framework digital library platform from California Digital Library.

Metadata Hopper allows users to contribute content to an XTF repository and to enhance that content through shared navigation facets or 'tags.' It also generates a Dublin Core metadata file that works alongside the original metadata file to create a consistent search and browse interface.

Hopper

Installation

This tool was built against Django 1.6, Python 3.3, Postgres 9.2.7

System Level Packages

virtualenv
libxml2
libjpeg
libjpeg-dev
zlib
libtiff
libtiff-dev
tcl/tk
redis

Python 3.3 & VirtualEnv

$ virtualenv -p /usr/bin/python3.3 ve
$ source ve/bin/activate

Commands shown with (ve)$ require the activated virtualenv.

The Repository

Clone the git repo

$ git clone https://<user>@bitbucket.org/uiclibrary/portal-admin.git

Python Packages

(ve)$ pip install -r requirements.txt

Define local settings

Before creating the database create a local_settings.py file and enter the database connection
information and set the following local configuration:

XTF_INDEXER: The location of the XTF textIndexer associated with an XTF installation
FILE_ROOT: The base directory where all imported metadata will be stored and imported images if you choose to store them locally
MEDIA_ROOT: If you are using LocalFileBackend Media root should be set to FILE_ROOT + "Imported/media"

if it is a production server:
DEBUG = False
ALLOWED_HOSTS = ["<server>"]
PRODUCTION_SERVER = True

Optional settings:

To enable email notifications:
https://docs.djangoproject.com/en/1.6/topics/email/

EMAIL_NOTIFICATIONS = True
EMAIL_HOST
EMAIL_PORT
EMAIL_USE_TLS
EMAIL_HOST_USER
EMAIL_HOST_PASSWORD

To enable EZID ark minting use your account information in the following settings:

EZID_URL
EZID_ARK
EZID_USERNAME
EZID_PASSWORD
EZID_ARK_URL = EZID_URL + "id/" + EZID_ARK
EZID_MINT_URL = EZID_URL + "shoulder/" + EZID_ARK

if not set Hopper will create local ids

If you would like to use the AmazonS3Backend use your account information in the following settings:

IMPORT_TRANSFER_BACKEND = "portal_admin.import_backends.AmazonS3Backend"

AWS_STORAGE_BUCKET_NAME
AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY

AWS_S3_CUSTOM_DOMAIN = '%s.s3.amazonaws.com' % AWS_STORAGE_BUCKET_NAME
MEDIAFILES_LOCATION = 'media'
MEDIA_URL = "https://%s/%s/" % (AWS_S3_CUSTOM_DOMAIN, MEDIAFILES_LOCATION)

Create the database

(ve)$ ./manage.py syncdb

If this is a new database you will be asked to create the super user account. Do that.

(ve)$>./manage.py migrate

Fixtures

If initial data does not load, it may be loaded manually via:

(ve)$>./manage.py loaddata portal_admin/fixtures/initial_data.json

Be sure to load any context-specific fixtures your instance of the portal requires.

Locally testing asynchronous processing

In addition to the running the test server you will need to run the redis server and a worker.

To start the redis server:

$ redis-server

To start the worker:

(ve)$ celery --app=portal_admin.celery_setup:app worker --loglevel=INFO

Creating dc.xml en masse

There are three commands that force Hopper to create all related dc.xml files

$ ./manage.py create_dcxml <context_id>

makes all the dc.xml files for all digobjects in the specified context

$ ./manage.py write_institutions

makes all the dc.xml files for all the institutions in Hopper

$ ./manage.py write_locations

makes all the dc.xml files for all the institutions in Hopper

Code-Level Documentation

Code level documentation supports pydoc, the standard python documentation system.

To view the documentation for a specific module, use ./manage shell

For example:
$>./manage shell
Python 3.3.5 (v3.3.5:62cf4e77f785, Mar 9 2014, 01:12:57)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

help('portal_admin.ingestion')
help('portal_admin.processor')

note : It is necessary to use Django's ./manage.py shell command to generate help text.
This is because the system relies on various django-level imports.