Wiki

Clone wiki

WADQC / Troubleshooting for installation

Configuration

Check if the following minimal configuration is present:

Environment variables

  • WAD-QC and its services need to have WADROOT defined in the environment (e.g. in .bashrc: export WADROOT="/home/wad/WADROOT").
  • the executables pg_config and Orthanc should be findable from the user's PATH

Folder structure

The WADROOT folder has the following skeleton structure (when the server is up and running, more items are added to this structure):

WADROOT/
    orthanc/
        bin/
        config/
            orthanc.json
        db/
        lua/
            wad_onstablestudy.lua
            wadselector.py
        plugins/
    pgsql/
        data/
    WAD_QC/
        wadconfig.ini
        wadsetup.ini
        Logs/

In a production installation the folder WADROOT/WAD_QC/systemd will also be present when running in a virtualenv.

The purposes of these items:

  • orthanc.json describes all settings for the Orthanc server, including plugin locations and access credentials.
  • wad_onstablestudy.lua instructs Orthanc to call wadselector.py when a new study has arrived.
  • wadselector.py is a wrapper that starts the selector which tries to match the new study to exisiting rules, and creates new processes to be processed by the wadprocessor service.
  • orthanc/db will contain all DICOM images stored in the Orthanc PACS.
  • pgsql/data will contain the databases of WAD-QC and Orthanc.
  • wadconfig.ini contains the access credentials for the WAD-QC database.
  • wadsetup.ini contains parameters needed for the initial creation of the WAD-QC database.
  • WADROOT/WAD_QC/Logs will contain the logs of the various services.
  • WADROOT/WAD_QC/systemd contains wrappers that are called by systemd to start the WAD-QC services from a virtualenv.

Permissions

If PostgreSQL is installed from the standard Ubuntu repositories, the ownership of the some directories in /var might be reset to postgres on reboot, if the wad_setup.py installer is not used to add a systemd service for PostgreSQL. Fix it manually for user wad by:

sudo chown -R wad:wad /var/run/postgresql /var/log/postgresql

Apache time-out

For time consuming jobs, like checking which studies in the PACS have no matching selector, a server time-out can occur. It is therefore recommended to change the default timeout of 5 minutes to something like 50 minutes: edit apache2 config file (e.g. /etc/apache2/apache2.conf) and change or add a line

Timeout 3000

This fix is implemented in wad_setup platform_fixes as "apache2timeout" for Ubuntu and "httpdtimeout" for CentOS7. But according to several reports, apache2 2.4.6 which is used in CentOS7 has a hardcoded 30s timeout, which cannot be fixed in this way.

PostgreSQL database does not shrink after delete

PostgreSQL does not return disk space automatically if the database shrinks. Instead it will keep the allocated space and reuse it. The autovacuum process only optimizes the database. In order to free the disk space of the Orthanc database, issue

vacuumdb -U orthanc -d orthanc_db --full

Be warned that this process may take a long time.

DICOM connection verification to Orthanc fails

This can happen when sending directly to Orthanc from a modality. It is related to a response time-out that is hard encoded in dcmtk. If Orthanc is linked to dcmtk-3.6.0 this issue has been reported, but this problem does not occur is Orthanc is linked to dcmtk-3.6.1. When compiling a static Orthanc from scratch, dcmtk-3.6.0 is used by default. Using Orthanc from apt-get on a recent Ubuntu links to dcmtk-3.6.1. A bugfix has been send to the developers of Orthanc. In the mean time use either Orthanc from the repositories together with dcmtk-3.6.1 installed, or use the precompiled Orthanc versions from the the WADQC developers.

PostgreSQL exits after a while (semctl(xxxxx, 11, SETVAL, 0) failed: Invalid argument) logout of user

Happens when PostgreSQL runs as a normal user. See https://wiki.postgresql.org/wiki/Systemd. Either use a system account to run PostgreSQL, or modify logind.conf:

In /etc/systemd/logind.conf, section[Login] add

RemoveIPC=no

After changing this file, a system reboot is recommended.

This fix is implemented in wad_setup platform_fixes as "removeipc".

Similarly, use the enable-linger flag in loginctl to prevent killing of background processes when the wad user logs out:

loginctl enable-linger waduser

This fix is implemented in wad_setup platform_fixes as "enable-linger".

Port 5432 not available during install

If you ran into errors during wad_setup.sh, you fixed this but when you restart the script you get this error:

[ERROR]: postgresql_install: ERROR. Requested postgresql port (5432) is not available!
This probably means the postgresql server is already running, and you need to stop it. It's also recommended to clean up all installed files in WADROOT.
sudo systemctl stop wadpostgresql
rm -r /home/wad/WADROOT

Manually updated python packages to incompatible versions

The WAD-QC package is just one python package, but it depends on other python packages to run. When using the wad_setup installer, the virtualenv is populated with tested versions of the python dependencies. It is possible that an updated version of one of these dependencies is not compatible with the WAD-QC system. If for whatever reason you did install an upgrade of the python dependencies and want to revert to the tested version, you can reinstall all dependencies from the command line (make sure to activate the proper virtual env e.g. workon wad2env3):

pip3 install -r requirements3.txt
where the file requirements3.txt is located in the folder where you unpacked wad_setup.zip

With every new release of WAD-QC, requirements3.txt will be updated to the latest tested versions.

Migrating Orthanc from old SQLite database to PostgreSQL

Orthanc uses SQLite as the fall back database if PostgreSQL cannot be used. Due to a bug, the SQLite database was also used if PostgreSQL was available up to WAD-QC version 2.0.2. To migrate the Orthanc database from SQLite to PostgreSQL, use the following guide (everywhere replace PASSWORD with your own orthanc password and replace "wadqc" by your waduser):

sudo apt install pgloader
sudo service wadorthanc stop
Edit /home/wadqc/WADROOT/orthanc/config/orthanc.json:

  • add path to orthanc-postgresql libraries to the plugins section (ubuntu: “/usr/share/orthanc/plugins”)

Create orthanc_db database structure in postgresql if not present already:

sudo service wadorthanc start 
sudo service wadorthanc stop

Create file called pgloader.conf:

load database
    from '/home/wadqc/WADROOT/orthanc/db/index'
    into postgresql://orthanc:PASSWORD@localhost/orthanc_db
with include no drop, create no tables, create no indexes, reset no sequences, disable triggers
set work_mem to '200MB', maintenance_work_mem to '512 MB';
Rename db columns:
sudo psql postgresql://orthanc:PASSWORD@localhost/orthanc_db -c "alter table attachedfiles rename compressedhash to compressedmd5"
sudo psql postgresql://orthanc:PASSWORD@localhost/orthanc_db -c "alter table attachedfiles rename uncompressedhash to uncompressedmd5"
sudo pgloader pgloader.conf
rename back db columns:
sudo psql postgresql://orthanc:PASSWORD@localhost/orthanc_db -c "alter table attachedfiles rename compressedmd5 to compressedhash"  
sudo psql postgresql://orthanc:PASSWORD@localhost/orthanc_db -c "alter table attachedfiles rename uncompressedmd5 to compressedhash"

Fix inconsistencies in resourcetype definitions between orthanc SDK and internal code:

sudo psql postgresql://orthanc:PASSWORD@localhost/orthanc_db -c "update resources set resourcetype=resourcetype-1"
sudo service wadorthanc start
If migration successful:
sudo rm /home/wadqc/WADROOT/orthanc/db/index*

Proxy

Installation requires access to the internet. If the server is behind a proxy, please change the global proxy settings by adding the following lines to /etc/environment:

export http_proxy="http://yourproxy:your_port"
export https_proxy="http://yourproxy:your_port"
export ftp_proxy="http://yourproxy:your_port"
or if you need to provide authentication:
export http_proxy="http://user:password@yourproxy:your_port"
export https_proxy="http://user:password@yourproxy:your_port"
export ftp_proxy="http://user:password@yourproxy:your_port"
or use a better way to not provide plain text passwords in your /etc/environment file.

After changing the /etc/environment file, you need to reboot for the changes to take effect.
Alternatively, above lines could be added to the .bashrc file of the wadqc user (in which case it won't be system-wide).

Note: the proxy should also be specified under wadadmin>server administration>configuration>factory modules to enable the update check and download of factory modules.
Note2: the factory modules page won't load when behind a proxy, so the proxy settings page currently is not accessible. Temporary workaround is to add the setting to the database manually:

sudo -u postgres psql wadqc_db
wadqc_db=# insert into dbvariables (name,val) values ('proxyserver', 'myproxy.domainname.nl:3128');
wadqc_db=# \q
or if you would like to modify an existing value:
sudo -u postgres psql wadqc_db
wadqc_db=# update dbvariables set val='mynewproxy.domainname.nl:3128' where name='proxyserver';
wadqc_db=# \q

Depending on your configuration other proxy settings might be required, see below.

configure pip proxy:

mkdir -p ~/.config/pip
echo [global] >> ~/.config/pip/pip.conf
echo proxy = http://yourproxy:your_port >> ~/.config/pip/pip.conf

configure APT proxy:

sudo echo "Acquire::http::proxy \"http://yourproxy:your_port/\";" >> /etc/apt/apt.conf.d/proxy.conf
sudo echo "Acquire::https::proxy \"http://yourproxy:your_port/\";" >> /etc/apt/apt.conf.d/proxy.conf

configure wget proxy:

sudo echo "use_proxy=yes" >> /etc/wgetrc
sudo echo "http_proxy=http://yourproxy:your_port" >> /etc/wgetrc
sudo echo "https_proxy=http://yourproxy:your_port" >> /etc/wgetrc

configure git proxy:

sudo git config --global http.proxy http://yourproxy:your_port
sudo git config --global https.proxy http://yourproxy:your_port

LDAP

If the linux host is configured with LDAP based authentication, some extra steps are needed to get a functional wadqc account.

sudo apt install libuser
sudo ladduser wadqc
sudo ladduser wadqc sudo
then edit the file /etc/sudoers to include wadqc in the list of recognized sudoerd. As a user with sudo rights (not yet wadqc!) do:
sudo nano /etc/sudoers
and add a line
%wadqc   ALL=(ALL:ALL) ALL

Updated