Islandora Batch Uploader (IBU)
A desktop batch uploader app for WWU's MABEL system, which is based on Islandora.
This is a cross-platform (Windows, MacOS, Linux) desktop app built on Electron, Node and VueJS that is designed to make it easy to upload batches of images and video to Islandora.
If you just want to use the app, please visit https://mabel.wwu.edu/ibu for the non-technical video overview and download links.
Once the user has downloaded the desktop app, it will prompt them to login, which attempts to SSH into the Islandora server, and if successful, creates a private/public key pair, and copies the public key to the server via SCP. After that initial login/setup process, the user should not be prompted to login again.
From the point, the user would drag and drop their photos or videos into the target area, and then thumbnails are created of each photo and send to Microsoft Cognitive Services and Clarifai for analysis.
Temporary files are created (in a folder named 'islandora_batch_loader_temp_files' which is created in the user's local computer's 'temp' folder), and those files are deleted when the app is closed.
Metadata/keywords/tags and color information are sent back from those services and presented to the user as checkboxes that they can click to add them to their metadata.
When the user is ready, the click the 'Publish' button, and the app will attempt to generate MODS XML that will also be SCP'd to the server. Once those files are in place, then the app calls the drush commands to initiate the Islandora batch ingest process. A few seconds later, the user should receive feedback that their files are now in Islandora.
Who to contact?
Have questions about this app?
Contact David Bass (david.bass at wwu dot edu) or the MABEL team (mabel at wwu dot edu)
- see "Issues" for the list of todos
- you can only process one video at a time
- you cannot mix images and videos in the same batch yet; that will be addressed in future updates.
- Islandora (we are using version 7.x.-1.*)
- Drush 6+
- two folders on your Drupal site to handle uploaded files:
- the users of this app will need to have SSH write permissions to those folders
- Clarifai and Microsoft Cognitive Services API keys
- optional (recommended): a cron job that removes files (older than 24 hours, or whatever time frame you prefer) from the /var/www/drupal/sites/default/files/desktopapp/temp/ folder (until that is baked into IBU as a feature)
- visit https://mabel.wwu.edu/ibu for a video tutorial and links to the executable versions of IBU.
- once you've seen the video and downloaded the version you need, right-click on the zip file and choose "Extract All" or "unzip". On Windows, you'll probably encounter the "Windows protected your PC" popup. Click "More Info" as shown in this screenshot and then "Run Anyway" to install IBU.
- on macOS, you may see a popup warning that says you cannot run this file because it's from an unidentified developer. Click "cancel" and then right-click the IBU app, and click "open". Then you can click "open" again and the app will run.
Once the app is up and running, right-click on the icon in the taskbar and "pin to taskbar". You can delete the IBU Setup.exe (or dmg) file, because the app will have been installed onto your computer (right-click and choose Properties to see where it is installed).
Once you run the IBU app, you should see the setup / configuration screen:
Islandora Virtual Machine
You should be able to use IBU with the Islandora Virtual Machine (VM) by using the following configuration settings:
hostname: localhost port number: 2222 domainName: http://localhost:8000 username: vagrant password: vagrant
Clarifai and Microsoft need access to your photos to be able to analyze them, which can be tricky when using a virtual machine. To make this easier when using the VM, create a free account at https://cloudinary.com and enter those credentials in the IBU settings/configuration page.
See https://wiki.duraspace.org/display/ISLANDORA/Virtual+Machine for more info on the VM.
Windows-specific Build Issues
On Windows, node-gyp will look for Python 2.7 in your path, and if it doesn't find it, you will encounter errors. The following command did the trick for me:
$ npm --add-python-to-path='true' --debug install --global windows-build-tools
Please note that the command above will not work if your Windows username has a space in it. Check the output of that script to see where Python was installed. For me, it was installed into c:\Python27 (or c:\Python\27). You might also fix the path problem by running this command (assuming you have Python 2.7):
$ npm config set python "c:\Python27\python.exe"
$ npm install
- More info for Windows builders can be found at https://simulatedgreg.gitbooks.io/electron-vue/content/en/getting_started.html#a-note-for-windows-users
# install dependencies npm install (or 'yarn') # serve with hot reload at localhost:9080 npm run dev # build electron application for production npm run build # remove builds npm run build:clean
If you encounter errors related to sharp, this usually fixes it:
Thank you to all of the people at WWU who contributed to this app in one way or another: Tony Kurtz, Ruth Steele, Max Bronsema, Jesse Sturgis, Rebecca Marrall, Nigel Packer, Tim McLaughlin, Rhys Logan, Casey Mullin, Tom McNeely, Brent Carey, Izaac Post, Clarissa Mansfield, Chris Casquilho, Jon Dillon, Kim Marsicek, Jared @ Clarifai, Amélie Bass (cookies), and to the people who wrote the software it is built on (VueJS, Electron, Node, Electron-Vue, Islandora, Islandora Batch, Clarifai, Microsoft Cognitive Services, pHash and many more...). -- David Bass
Photos by Charles Deluvio & Trevor Cole via Unsplash