Navigation | | | :---: | :---: | :---: | :---: Project Plan | Project Requirements | Project Design | README

Developer's Guide

Welcome and thank you for your interest in the Plant Root Imaging and Data Acquisition (PRIDA) software. This basic guide will help you with the continued development of this program.

Table of Contents

The Development Environment

This section describes how to set up the necessary development environment for running the software on Linux Mint - Rosa. To the best of our knowledge, these steps should be similar, if not the same, for other Debian-based Linux operating systems (e.g., Ubuntu).

The following steps will guide you through creating an optional virtual computer for use on Oracle's VirtualBox virtual machine emulation software, configuring the Linux base package requirements, setting up an optional virtual Python environment, and installing the Python libraries for running the software.

VirtualBox

This section guides you through the steps for creating a virtual machine for running Linux. This is particularly useful for those of you who wish to develop and/or run the software's hardware mode on a Windows PC.

1. Download and install VirtualBox for Windows hosts from this site.
2. Download the Linux Mint 17.3 "Rosa" ISO from here.
   • We recommend using the MATE (64-bit) version; however, you may select your desktop environment of choice
   • Select a local mirror for downloading the ISO (e.g., advancedhosters.com)
3. Following these steps
   • Start VirtualBox and click the "New" button
   • Type "Mint" in the Name field and "Linux," "Ubuntu (64-bit)" should default in the Type and Version boxes and click "Continue"
   • Assign a reasonable amount of memory (this can be changed later if more is needed)
   • Select the radio button next to "Create a virtual hard disk now" and click "Create"
   • Check the format of virtual hard disk (we recommend using "VDI") and click "Next"
   • Check the type of virtual hard disk (we recommend "dynamically allocated" file type)
   • Allocate at least 20 GB of space for the virtual hard disk and click "Create"
   • Select your new virtual machine and click "Start"
   • Select your start up disk (i.e., the Linux Mint ISO you downloaded) by clicking the folder icon and navigating to the ISO file you downloaded
   • Follow the Linux Mint start up guide prompts for setting up your new computer
   • Once your computer has finished its setup, you may wish to run optional system updates and install the VirtualBox Guest Additions (from the VirtualBox toolbar, click on "Devices" and select "Insert Guest Additions CD Image" and follow the installation instructions)
   • shutdown
4. Optionally, for connecting USB digital cameras and Arduino motor controllers, download and open the VirtualBox Extension Pack (for all supported platforms) with VirtualBox and follow the prompts for installation.

Linux Base Packages

This section guides you through the steps for installing the base Linux packages for configuring the software development environment.

In Linux Mint, click on the Menu Button (i.e., Start button) and type "Update" in the search bar to locate the "Update Manager" and click on it to open. Install any updates that are suggested. Close Update Manager when finished.

Click on Menu again and open the Package Manager (Synaptic); type in your password when prompted. In the Quick Filter, search and check the boxes next to the following packages (please note that the package version numbers are based on the latest available in October 2016 and may have been since updated):

• python3-pip (v.8.1.1; Python package installer); include dependency (e.g., python-pip-whl)
• libjpeg-dev (v.8c-2ubuntu8; jpg libraries); include dependencies (e.g., libjpeg-turbo8-dev and libjpeg8-dev)
• zlib1g-dev (v.1.2.8; gzip libraries)
• libgphoto2-dev (v.2.5.9; GPhoto2 libraries); include dependencies (e.g., libexif-dev and libjs-jquery)
• python3.5-dev (v.3.5.2-2; Python development headers)
• libhdf5-dev (v.1.8.16; HDF5 library development files); include dependencies (e.g., libhdf5-10, hdf5-helpers, libaec-dev, libaec0, libhdf5-cpp-11, libsz2)
• libraw-dev (v.0.17.1; NEF/raw image support); include dependencies (e.g., liblcms2-dev)
• build-essential (v.12.1ubuntu2; development tools); include dependencies (e.g., g++, g++-5, libstdc++-5-dev)
• qt5-default (v.5.5.1; GNU GPL Qt5 libraries); include dependencies and updates

NOTICE

Prida compilation and execution will work under the latest release of Python 3.6; however, PyInstaller is not compatible with this release (yet) and therefore, if you are planning to compile a binary for Windows or Mac, please choose another version of Python (e.g., Python 3.5.3).

Click "Apply" to begin download and installation.

Python VirtualEnv

The virtual environment is a tool to keep the software dependencies for a specific project in a separate location rather than installing them on your system natively. If you are not using the virtual machine or if you are working on multiple projects on this machine, it is recommended that you set up a virtual environment so as to not conflict with your other projects. If you are on a virtual machine, are not using Python for any other projects, or are just feeling cavalier, feel free to skip this configuration.

Please follow this excellent resource for configuring Python virtual environments: http://python-guide-pt-br.readthedocs.io/en/latest/dev/virtualenvs/.

For your own convenience, the virtualenvwrapper allows multiple virtual environments to be easily switched between.

Python Libraries

This section guides you through the installation process for the Python packages required for running this software.

If you are using a Python virtual environment (as described here), you will have to use pip to install the following packages whilst the virtual environment is sourced. Otherwise, feel free to install these Python packages using the Package Manager.

*For virtual environments, the pyuic5 tool is installed under ~/VirtualEnv/prida/bin directory

+Note that the version numbers in the above table are from 2016 and may not reflect the latest available versions

PyQt

The user interface (UI) of this software is written in PyQt, a Python extension of Qt. We have chosen to use Qt5, the latest version of Qt at the time of this writing, rather than the more generally accepted Qt4, in an effort to future-proof the continued development of this project, which is still (again at the time of this writing) in an early development phase, as well as to take advantage of the latest Qt features.

The Qt UI design is handled by Qt Creator, an integrated development environment (IDE) for creating program-specific UI files. The software's UI is defined by these UI files (see Design Guide for details). The conversion of these UI files to Python modules is accomplished via the PyUIC utility (see Python Libraries for installation).

The compilation of UI files to Python modules is handled by the setup.py script, which can be found with the releases (v.1.3 and above) and working source code.

For the Windows Developer

For the few of you who are brave enough to develop code on Microsoft Windows, the following describes how to prepare the development environment in Windows 7. Please note that the hardware modes (i.e., 2D Mode and 3D Mode) are not currently supported on Windows due to the dependency on gPhoto2, which is a Unix-only library.

For the following instructions, you will need an unzipping software tool. If you do not have one, may we suggest 7-Zip (http://www.7-zip.org), an open-source file archiver by Igor Pavlov.

The following instructions have been successfully tested on Windows 7 (32-bit) and Windows 10 (64-bit).

Download and install the following:

• Git (Free Software Foundation)
  • Install using default options
• Python 3.5.3 (Python Software Foundation)
  • Select "Customize Installation"
    • Optional Features:
      • check all
    • Advanced Features:
      • check "Install for all users"
      • check "Add Python to environment variables"
• PyQt5.6 (Riverbank Computing Limited)
  • Install using default options

Test that Python is installed. Open a command prompt and type python -V and hit enter. You should see something similar to:

C:\Users\user> python -V
Python 3.5.3

Check to make certain the Python version shown in the command prompt is the same as the version you just installed (i.e., Python 3.5.3). If not, check to see if you have another version of Python installed (e.g., Python 2.7). If so, try typing python3 -V.

Test that PyQt5 is installed. Open a command prompt and type python -c "import PyQt5.QtCore; print(PyQt5.QtCore.QT_VERSION_STR)" and hit enter. Note, if you had to use python3 in the test above, use it again here. You should see something similar to:

C:\Users\user> python -c "import PyQt5.QtCore; print(PyQt5.QtCore.QT_VERSION_STR)"
5.6.0

If these two tests are passed, please continue.

Download the following Python wheel files. Special thanks to Christoph Gohlke at the Laboratory for Fluorescence Dynamics, University of California, Irvine for providing the Windows binaries for NumPy, SciPy and h5py.

WHEN DOWNLOADING WHEEL FILES, MAKE CERTAIN YOUR PYTHON VERSION AND OPERATING SYSTEM MATCH THE WHEEL FILE NAME! FOR EXAMPLE: numpy-1.11.3+mkl-cp35-cp35m-win32.whl FOR PYTHON 3.5 (cp35) AND WINDOWS 32-BIT (win32).

• numpy (NumPy Developers)
  • AVOID NumPy v.1.13 IF YOU INTEND TO USE THE IMAGE ANALYSIS COMMAND LINE TOOL; CURRENTLY, THERE IS AN ISSUE WITH SAVING BINARY IMAGES! USE VERSION 1.11.3.
• scipy (SciPy Develpers)
• h5py (Andrew Collette and contributors)
• rawpy (Maik Riechert)

Install Python packages. Open a command prompt as administrator (Start -> type "cmd" -> right click Command Prompt program -> select "Run as Administrator"). Click "Yes" to allow command prompt to make changes to your computer.

Test that pip is installed. Type pip3 -V. You should see something like the following:

C:\Windows\system32> pip3 -V
pip 9.0.1 from c:\program files\python35\lib\site-packages (python 3.5)

Move to the folder where you downloaded the wheel files above (e.g., in your Downloads folder). Folder navigation is done using the cd command and folder contents are displayed using the dir command, for example:

C:\Windows\system32> cd C:\Users\user\Downloads
C:\Users\user\Downloads> dir

 Directory of C:\Users\user\Downloads

06/12/2017  03:31 PM    <DIR>   .
06/12/2017  03:31 PM    <DIR>   ..
06/12/2017  05:13 PM    87,009,341  numpy-1.11.3+mkl-cp35-cp35m-win32.whl
06/12/2017  05:16 PM    11,250,472  scipy-0.19.0-cp35-cp35m-win32.whl
06/12/2017  05:16 PM     1,874,847  h5py-2.7.0-cp35-cp35m-win32.whl
06/12/2017  05:16 PM       449,347  rawpy-0.10.0a1-cp35-cp35m-win32.whl
        4 Files(s)     100,584,007 bytes
        2 Dir(s)    11,700,191,232 bytes free

Run the following commands to install Python packages with pip (Note: replace the wheel file names below with the files you downloaded; use the tab key to autofill names on the command line):

• pip install Pillow
• pip install "numpy-1.11.3+mkl-cp35-cp35m-win32.whl"
• pip install scipy-0.19.0-cp35-cp35m-win32.whl
• pip install h5py-2.7.0-cp35-cp35m-win32.whl
• pip install rawpy-0.10.0a1-cp35-cp35m-win32.whl
• pip install exifread

Exit the elevated-privilege command prompt by typing exit and hitting return.

Open a command prompt and navigate to a directory for storing your local repositories (e.g., C:\Users\user\Repositories). Clone the Bitbucket repository by typing git clone https://bitbucket.org/usda-ars/prida_one.git and hitting enter. This command will create a new folder (e.g., C:\Users\user\Repositories\prida_one) and download the repository files to your computer.

Test the setup. From the command prompt, navigate to a folder with a PRIDA setup.py script (e.g., C:\Users\user\Repositories\prida_one\working). Type python setup.py and hit enter. Note, if you had to use python3 in the previous tests, use it again here. You should see something like the following:

C:\Users\user\Repositories\prida_one\working> python setup.py
---------------------- Prida setup ----------------------
Checking for Python 3 (running v.3.5.3) .......... OK
Checking for pyuic ............................... OK
Checking for PyQt5 (found v.5.6.0) ............... OK
Checking for numpy (found v.1.11.3) .............. OK
Checking for scipy (found v.0.19.0) .............. OK
Checking for PIL (found v.1.1.7) ................. OK
Checking for h5py (found v.2.7.0) ................ OK
Checking for rawpy (found v.0.10.0a1) ............ OK
Checking for exifread (found v.2.1.2) ............ OK
Checking for gphoto2  ............................ N/A
Checking for Prida (found v.1.6.0-dev) ........... OK
Checking for main.py ............................. OK
Checking for UI files ............................ OK
Compiling UI files ...
> about.ui ....................................... OK
> config_mode.ui ................................. OK
> export_as.ui ................................... OK
> input_calib.ui ................................. OK
> input_user.ui .................................. OK
> mainwindow.ui .................................. OK
> fixing mainwindow.py ........................... OK
> tags.ui ........................................ OK
Checking for Prida directory ..................... FAILED
*create local Prida directory now? (Y/N): Y
Creating Prida directory ......................... OK
----------------------- end setup -----------------------
Passed 21/22

Encountered the following warnings:
 1. Python package gphoto2 is not available on this computer.
    2D and 3D imaging modes may not be used.
---------------------------------------------------------

Start application? (Y/N): Y

When prompted to create a Prida directory, type Y and hit enter. You will receive a warning for gphoto2, notifying you that 2D and 3D modes are unavailable. When prompted to start the application, type Y to launch Prida or N to exit the setup.

You should be set to go.

The Desktop App

To make it easier to run the software, a start bar shortcut and desktop app can be created, which is simply defined in a plain text file. This file should be located in "~/.local/share/applications" for the start bar shortcut. If the directory does not already exist, create it (mkdir ~/.local/share/applications). Create a new desktop application file (e.g., from the terminal: nano ~/.local/share/applications/prida.desktop). Copy the following example for the desktop app file contents, making the appropriate changes where necessary:

[Desktop Entry]
Type=Application
Version=1.5.0
Name=Prida
Comment=Plant Root Imaging and Data Acquisition
Exec=/usr/local/share/prida/prida.sh
Icon=/usr/local/share/prida/icon.png
Terminal=false
Categories=Programming

Note the two files and associated paths identified in the "Exec" and "Icon" fields. These file names and paths are arbitrary; however, for the paths, we recommend the Linux standard file location for user shareable scripts, which go in the "/usr/local/share" directory.

If it is not already created, create a "prida" directory in "/usr/local/share" (sudo mkdir /usr/local/share/prida). Change the owner and group on this new directory so you can easily read and write here (sudo chown -R user /usr/local/share/prida and sudo chgrp -R user /usr/local/share/prida, changing the "user" to whatever is your user name). Create a file in the new directory (nano /usr/local/share/prida/prida.sh). Type something similar to the following in the script (don't forget the shebang on the first line and update the paths base on how you set up your system):

#!/bin/sh
# YOU MUST SELECT ONE AND ONLY ONE OF THE FOLLOWING OPTIONS:
# 1. If you are using a Python virtual environment:
source /home/user/VirtualEnv/venv/bin/activate
python /home/user/Repositories/usda-ars/prida_one/releases/v1.5/main.py
# -OR-
# 2. If you are using Python virtualenvwrapper w/ 'prida' virtual environment
export VIRTUALENVWRAPPER_PYTHON=/usr/local/bin/python3
export WORKON_HOME=/home/user/Envs
source /usr/local/bin/virtualenvwrapper.sh
workon prida
python /home/user/Repositories/usda-ars/prida_one/releases/v1.5/main.py
# -OR-
# 3. If you are not using the Python virtual environment:
/usr/bin/python3 /home/user/Repositories/usda-ars/prida_one/releases/v1.5/main.py

NOTE

Make certain you run the setup.py script for releases v.1.3 and later, else the UI files will not be properly compiled and Prida will not start.

The icon file (i.e., icon.png) for the software icons, which look like:

(large)

and

(small)

Note that the icon must be a PNG image (not a JPG).

Log out and back into your desktop environment. In the Application window, Prida should now be available. Right-click to add a shortcut to the Desktop.

Bundling Prida Binaries

To create binary executables for running the software on other operating systems (i.e., Mac OS X and Windows 7) the following provides a guide for bundling the software using PyInstaller. Note that the creation of these bundles requires you to have a working copy of the software running on a different operating system.

Windows 7 (32-bit)

These steps will help you to create a Windows 7 32-bit Prida executable (Prida.exe) using Python 3.4 and PyInstaller.

Assuming a fresh Windows 7 system, complete the steps for the Windows Developer. All installers and wheel files can be found here. Note that the wheel files are specific to both the operating system bitness (i.e., 32-bit) and the version of Python (i.e., 3.4). Package versions are always being updated, therefore it is important to MAKE CERTAIN YOUR PYTHON VERSION (cp34) AND OPERATING SYSTEM (win32) MATCH THE WHEEL FILES!

If you are using a virtual environment, activate it. From the command prompt use pip to install PyInstaller:

pip install PyInstaller

From the Git terminal, clone the Prida repository (located C:\PATH\TO\REPOSITORY):

git clone https://bitbucket.org/usda-ars/prida_one.git

For releases v.1.4 and later, a PyInstaller specification file (win32.spec) and Windows icon (icon.ico) are provided inside the release directory.

From the command line, navigate to the release you want to bundle, for example:

cd C:\PATH\TO\REPOSITORY\prida_one\releases\v1.5\

Make certain you run the setup.py script to ensure the program is working. Once checked, run the following to compile the binary:

pyinstaller --clean --windowed -y -F win32.spec

This command generates a build and dist folder in the working directory. Inside the dist folder resides the newly compiled Prida.exe.

Windows PyInstaller Spec File

The following is the Pyinstaller spec file for Prida v.1.5 in Windows:

# -*- mode: python -*-

block_cipher = None


my_path = '.\\'
img_path = my_path + 'prida\\images\\'

a = Analysis([my_path + 'main.py'],
             pathex=['.'],
             binaries=None,
             datas=None,
             hiddenimports=['h5py.defs',
                            'h5py.utils',
                            'h5py.h5ac',
                            'h5py._proxy',
                            'scipy.linalg',
                            'scipy.linalg.cython_blas',
                            'scipy.linalg.cython_lapack',
                            'scipy.integrate'],
             hookspath=None,
             runtime_hooks=None,
             excludes=None,
             win_no_prefer_redirects=None,
             win_private_assemblies=None,
             cipher=block_cipher)

pyz = PYZ(a.pure,
          a.zipped_data,
          cipher=block_cipher)

a.datas += [('images\\greeter.jpg', img_path + 'greeter.jpg', 'DATA'),
            ('images\\loading.gif', img_path + 'loading.gif', 'DATA'),
            ('images\\calib_dist_center.jpg', img_path + 'calib_dist_center.jpg', 'DATA'),
            ('images\\calib_dist_wall.jpg', img_path + 'calib_dist_wall.jpg', 'DATA'),
            ('images\\calib_tank_depth.jpg', img_path + 'calib_tank_depth.jpg', 'DATA'),
            ('images\\calib_tank_wall.jpg', img_path + 'calib_tank_wall.jpg', 'DATA'),
            ('images\\icon.png', img_path + 'icon.png', 'DATA'),
            ]

# To get rid of the popup warning message in Windows:
a.binaries -= [('.\pywintypes34.dll', None, None)]

exe = EXE(pyz,
          a.scripts,
          a.binaries,
          a.zipfiles,
          a.datas,
          name='Prida.exe',
          debug=False,
          strip=None,
          upx=True,
          console=False,
          icon='icon.ico')

Windows USDA Plants Database Utility

As above, PyInstaller is used to build the Plants Database utility bundle. The following commands and scripts are used to build the bundle.

The utility is located in the C:\PATH\TO\REPOSITORY\prida_one\utilities\plants_util directory. The directory now includes a PyInstaller specification file (win32.spec) and Windows icon (icon.ico).

You will need a copy of the Plants Database (usda_plants.txt.gz), which is available on the Resources page of the Plant Mineral Nutrition website (here). Download the GZIP file and copy it to the plants_util folder (DO NOT UNZIP THIS FILE!).

Open a command prompt. If you have a virtual environment, activate it. Run the following command:

pyinstaller --clean --windowed -y -F win32.spec

This command generates a build and dist folder in the working directory. Inside the dist folder resides the newly compiled plants_db.exe.

Mac OS X Yosemite (64-bit)

These steps will help you to create an Mac OS X Prida executable (Prida.app) using Python 3 and PyInstaller.

Assuming a fresh Mac OS X Yosemite system, complete the following steps to install the required software:

• Download and install Python 3.5.3 (available at Python.org); includes pip
• Install virtualenv with pip3
• Setup Python virtual environment (e.g., see Virtual Environments)
• Inside virtualenv, pip install NumPy, Scipy, Pillow, H5PY and PyInstaller
• Make certain you have setuptools v.19.2 (later versions fail with PyInstaller)
  • pip install setuptools=19.2
• Install Homebrew
• Follow these instructions to install PyQt5
• For enabling 2D Mode, also:
  • Install gphoto2 with Homebrew (brew install gphoto2)
  • Install gphoto2 with pip3 (pip3 install gphoto2)

Clone the PRIDA Bitbucket repository (cloned, for example at the following path: /Users/PATH/TO/REPOSITORY):

git clone https://bitbucket.org/usda-ars/prida_one.git

For releases v.1.4 and later, a PyInstaller specification file (macos.spec) and Mac icon (icon.icns) are provided inside the release directory.

From the terminal, navigate to the release you want to bundle, for example:

cd /Users/PATH/TO/REPOSITORY/prida_one/releases/v1.5/

Make certain you run the setup.py script to ensure the program is working. Once checked, run the following to compile the binary:

pyinstaller --clean --windowed -y -F macos.spec

This command generates a build and dist folder in the working directory. Inside the dist folder resides the newly compiled Prida.app and Prida UNIX executable.

Mac PyInstaller Spec File

The following is the Pyinstaller spec file for Prida v.1.5 in Windows:

# -*- mode: python -*-

block_cipher = None

my_path = "./"
img_path = my_path + "prida/images/"
a = Analysis([my_path + 'main.py'],
             pathex=['.'],
             binaries=[],
             datas=[],
             hiddenimports=['PyQt5',
                            'h5py.defs',
                            'h5py.utils',
                            'h5py.h5ac',
                            'h5py._proxy',
                            'scipy.linalg',
                            'scipy.linalg.cython_blas',
                            'scipy.linalg.cython_lapack',
                            'scipy.integrate'],
             hookspath=[],
             runtime_hooks=[],
             excludes=[],
             win_no_prefer_redirects=None,
             win_private_assemblies=None,
             cipher=block_cipher)
a.datas += [('images/greeter.jpg', img_path + 'greeter.jpg', 'DATA'),
            ('images/loading.gif', img_path + 'loading.gif', 'DATA'),
            ('images/icon.png', img_path + 'icon.png', 'DATA'),
            ('images/calib_dist_center.jpg', img_path + 'calib_dist_center.jpg', 'DATA'),
            ('images/calib_dist_wall.jpg', img_path + 'calib_dist_wall.jpg', 'DATA'),
            ('images/calib_tank_depth.jpg', img_path + 'calib_tank_depth.jpg', 'DATA'),
            ('images/calib_tank_wall.jpg', img_path + 'calib_tank_wall.jpg', 'DATA')]
pyz = PYZ(a.pure, a.zipped_data, cipher=block_cipher)
exe = EXE(pyz,
          a.scripts,
          a.binaries,
          a.zipfiles,
          a.datas,
          name='Prida',
          debug=False,
          strip=None,
          upx=True,
          console=True)
app = BUNDLE(exe,
             name='Prida.app',
             icon='icon.icns',         # or None if unavailable
             bundle_identifier=None,
             info_plist={
                 'CFBundleDevelopmentRegion': 'en_US',
                 'CFBundleShortVersionString': '1.5',
                 'CFBundleVersion': '1.5.0',
                 'CFBundleName': 'Prida',
                 'CFBundleDisplayName': 'Prida',
                 'CFBundleSignature': '????',
                 'NSHighResolutionCapable': 'True',
                 'NSHumanReadableCopyright': 'This software is freely available to the public for use.'
                 }
             )

USDA PLANTS Database Utility Companion App

As above, PyInstaller is used to build the Plants Database utility bundle. The following commands and scripts are used to build the bundle.

The utility is located in the /Users/PATH/TO/REPOSITORY/prida_one/utilities/plants_util directory. The directory now includes a PyInstaller specification file (macos.spec) and Mac icon (icon.icns).

You will need a copy of the Plants Database (usda_plants.txt.gz), which is available on the Resources page of the Plant Mineral Nutrition website (here). Download the GZIP file and copy it to the plants_util folder (DO NOT UNZIP THIS FILE!).

Open a terminal window. If you have a virtual environment, activate it. Run the following command:

pyinstaller --clean --windowed -y -F macos.spec

This command generates a build and dist folder in the working directory. Inside the dist folder resides the newly compiled plants_db.app and UNIX executable.

The HDF5 Hierarchy

At the core, the software stores all user and project data into an HDF5 file. An HDF5 file contains two types of objects: groups (i.e., directories) and datasets (i.e., files). A group may contain another group (i.e., a subdirectory), a dataset, or remain empty. Datasets may exist anywhere within the group hierarchy; however, they are terminal (i.e., no groups under a dataset).

Both object types may also have associated attributes (i.e., metadata), which are saved in a key-value pair format.

The following figure shows how the software organizes its HDF5 data.

The blue boxes indicate groups while red boxes indicate datasets. All HDF5 file hierarchies begin with the root (/) group. Here, we pin the user attributes, which includes the name and email address of the file's owner as well as the user's experiment details—a summary describing the project.

Under the root group are the individual plants that are imaged. Each plant is given a unique identifier (PID) to allow for time course analysis. The individual plants have associated attributes (e.g., genus, species, germination date, and growth conditions). Every time a plant is imaged, the data is collected into a session. Each session has its own attributes (e.g., date, plant age, number of images taken, and camera settings). Each image taken is stored in its own subdirectory. This is to organize the raw images together with any processed data (e.g., 2D root maps). Currently, we set our digital cameras to capture low compression, high resolution JPG images. The software also supports the following image formats: JPEG, TIF and TIFF files. For image files, the software is configured to handle color (RGB), grayscale (0-255), and binary (0-1) image types. Attributes are also associated with the original data (i.e., raw images), for example the color space and pixel dimensions.

Module Structure

The software has been written in Python modules (i.e., separate .py files) for convenience and ease of development. Certain groups of modules have been further divided into their own packages (i.e., folder of modules). The following describes the module/package structure of verion 1.5 of this software:

v1.5/

main.py

setup.py

prida/ (package)

__init__.py

calibutil.py

custom_filter.py

data_worker.py

email_organizer.py

global_conf.py

hard_worker.py

hdf_organizer.py

image_worker.py

imutil.py

prida.py

resources.py

stream_to_logger.py

utilities.py

hardware/ (sub-package)

__init__.py

Adafruit_I2C.py

Adafruit_MotorHAT.py

Adafruit_PWM_Servo_Driver.py

basecontroller.py

camera.py

controller.py

hat.py

imaging.py

led.py

piezo.py

pridaperipheral.py

shield.py

images/

calib_dist_center.jpg

calib_dist_wall.jpg

calib_dist_depth.jpg

calib_tank_wall.jpg

greeter.jpg

icon.png

loading.gif

ui/ (sub-package)

__init__.py

about.ui

config_mode.ui

custom.py

export_as.ui

input_calib_ui

input_user.ui

mainwindow.ui

pilutil.py

tags.ui

Nomenclature

Some confusion was encountered as to the meaning of the word "step" in regards to the motor/bearing movement. This is due to the fact that the stepper motor, the Adafruit library, and Prida all have different ways of defining a "step." For clarity, the following nomenclature is used to distinguish between the three different meanings of the word.

Git Basics

Prida, during development, is stored on a git repository hosted by BitBucket.org. As such, the suggested workflow as well as the basics of interacting with the git repository are detailed below.

Starting a Repository

The Prida central repository already exists and needs no initialization; however, any local machine wishing to access the repository will need to clone the git repository.

git clone <repository>

Using the https://<user>@bitbucket.org/usda-ars/prida.git provided by BitBucket, where user is your BitBucket username, copies the remote repository and all its tracked branches into a local git repository. Typically, this is the most straightforward way to access the repository on your local machine. Note that this command only ever need be used once.

Development - Branching the Master Repository

With access to the remote repository on a local machine, you are now able to begin contributing to development. Below are a list of commands and their uses as pertaining to this process.

git pull

Copies any recently committed changes to the remote repository onto the local directory. Do this as often as possible, especially before committing any changes of your own. This will help avoid any merge conflicts due to concurrent development of the same files.

git branch <branchname>

Creates a new branch on the local repository. This allows you to make edits to the files on your local machine without losing the master files. It is recommended that any new development towards implementing new features or correcting issues be done on a separate branch designated for that particular issue, rather than the main development branch. This also helps avoid any merge conflicts with your fellow developers.

Helpful Hint: calling git branch without any arguments lists all branches on the current directory level in the local directory.

git checkout <branchname>

A multi-purpose function, checkout with the name of the branch as an argument grants the user access to a particular branch for editing. Any changes made henceforth only effect the currently checked out branch.

git checkout -b <branchname>

A shorthand version of running the above two commands (i.e., create a new branch and checkout into it).

git push -u origin <branchname>

Adds the newly created branch to the remote repository.

(make edits)

Now you are ready to make your changes! Note that functions git add and git commit below apply to your local repository and may be used during your own development without effecting the development of others connected to the same remote repository.

(git status)

A useful but non critical function that shows the current status of the local repository as compared to the most recent commit. Red text indicates that the file has been modified and is not scheduled for tracking. Green indicates modified and scheduled for tracking, but not yet committed.

git add <filename>

Stages the changed content in <filename> to be ready for the next commit. Any files to be committed, locally or remotely, must first be added.

Helpful Hint: type git add -u to add all tracked files that have been edited.

git commit

Commits the added changes to the local repository. Upon use, the command requests a password for write privileges, redirects the user to a vim-window (Helpful Hint: export EDITOR=nano from the command line to avoid vim) where they may add comments to the commit (strongly recommended!), which following a write-quit command finally commits the changes to the local repository.

Helpful Hint: use the -m flag followed by a quoted string with your commit (e.g., git commit -m "message") to add your commit message from the command line.

git push

Merges any committed changes on the local directory to the remote repository.

git checkout master

Returns the user to the master branch to prepare for a branch merge.

git pull

Here again, the user should pull any new updates to the master branch to insure that the local repository is up to date before merging any new changes to it.

git merge <branchname>

Merges the specified branch back into the master branch. This will add any changes implemented in the branch into the master branch. Note: merge conflicts may arise here! If users have been editing the same files and committing concurrently, the second user to commit must resolve any conflicting changes at this point.

Congratulations on your successful merge!

git push

Adds the changes from the branch merge to the remote repository. The issue being addressed is now resolved in the working development copy of the system. Note: merge conflicts may arise here! If users have been editing the same files and committing concurrently, the second user to commit must resolve any conflicting changes at this point.

Repository Maintenance - Undoing Changes

Theoretically, these functions should never be needed. Committing a change is exactly that, a commitment. Any commits made to the remote repository are entered into the commit history, which serves as the basis for all development. Each developer is dependent on having the same commit history as everyone else; deleting a commit is very dangerous in this sense as it puts the developers out of sync. However, there are git commands to do this should an unforgivable mistake be committed. In general, these commands should not be used with regards to the remote repository. However, on one's local machine, before the code is pushed, they can be quite useful.

git checkout <commit hashcode>

Used in this manner, the checkout command rolls back the local copy of the repository to the called out commit in read-only mode. This is useful for tracing changes and viewing previous versions, and carries no associated risks. However, no changes can be made in this state.

git revert <commit hashcode>

This is the safest method of "undoing" a erroneous commit. The system reverses the changes of the specified previous commit (and no others, the revert command only effects a single commit at a time!) as itself a new commit. This helps prevent and commit history mis-matches, as it is a new commit and can be treated as such rather than a modification of the common commit history.

git reset <commit hashcode>

This is a destructive command and should rarely, if ever be used. Rolls back the local repository to the commit before the one called out, deleting along the way. While it has other uses as well, in general this command should be left alone unless one is an advanced and experienced git user who fully understands the implications of their actions.

Prida - Git Workflow

As it stands, there are two types of branches in the Prida workflow, both with their own distinct role. Fully functional releases are copied to their own directory within the Master branch.

The Master Branch

The master branch is the main development branch which contains the most up-to-date, potentially unstable version of the system. There is only one master branch on the remote repository, and it is the main branch that all other branches stem from.

The iss# Branches

The issue branches are where development happens. Each developer takes on one of the issues outlined on the BitBucket Prida page, and attempts to design an effective solution. Frequently the code in these branches is non-functional. These branches are merged back into the master branch upon fully addressing the related issue number. These branches are managed and created in the manner of the development workflow detailed in the Git Basics section above.

The V#.# Directories

In the releases directory are the version folders, each with a name following the v#.# naming convention where # indicates the release number. These are snapshots of official, fully functional releases. While incomplete in the sense that they are missing features, they are versions of the system with only entirely implemented working features of the system.