Islandora Batch Uploader (IBU)
A desktop batch uploader app for WWU's MABEL system, which is based on Islandora.
This is a cross-platform (Windows, MacOS, Linux) desktop app built on Electron, Node and VueJS that is designed to make it easy to upload batches of images and video to Islandora.
Once the user has downloaded the desktop app, it will prompt them to login, which attempts to SSH into the Islandora server, and if successful, creates a private/public key pair, and copies the public key to the server via SCP. After that initial login/setup process, the user should not be prompted to login again.
From the point, the user would drag and drop their photos or videos into the target area, and then thumbnails are created of each photo and send to Microsoft Cognitive Services and Clarifai for analysis.
Temporary files are created (in a folder named 'islandora_batch_loader_temp_files' which is created in the user's local computer's 'temp' folder), and those files are deleted when the app is closed.
Metadata/keywords/tags and color information are sent back from those services and presented to the user as checkboxes that they can click to add them to their metadata.
When the user is ready, the click the 'Publish' button, and the app will attempt to generate MODS XML that will also be SCP'd to the server. Once those files are in place, then the app calls the drush commands to initiate the Islandora batch ingest process. A few seconds later, the user should receive feedback that their files are now in Islandora.
Who to contact?
Have questions about this app?
Contact David Bass (david.bass at wwu dot edu) or the MABEL team (mabel at wwu dot edu)
- see "Issues" for the list of todos
- large videos (> 100MB) can cause problems; future updates should resolve that issue.
- you cannot mix images and videos in the same batch yet; that will be addressed in future updates.
- Islandora (we are using version 7.x.-1.*)
- Drush 7+
- two folders on your Drupal site to handle uploaded files:
- the users of this app will need to have SSH write permissions to those folders
- Clarifai and Microsoft Cognitive Services API keys
- a cron job that removes files (older than 24 hours, or whatever time frame you prefer) from the /var/www/drupal/sites/default/files/desktopapp/temp/ folder (until that is baked into IBU as a feature)
- go to the ./compiled/ folder and then click on the .zip version for your operating system, and then click on the "View Raw" to download it to your local machine. Once downloaded, right-click on the zip file and choose "Extract All" or "unzip". On Windows, you'll probably encounter the "Windows protected your PC" popup. Click "More Info" as shown in this screenshot and then "Run Anyway" to install IBU. Once the app is up and running, right-click on the icon in the taskbar and "pin to taskbar". You can delete the IBU Setup.exe (or dmg) file, because the app will have been installed onto your computer (right-click and choose Properties to see where it is installed).
Windows-specific Build Issues
On Windows, node-gyp will look for Python 2.7 in your path, and if it doesn't find it, you will encounter errors. The following command did the trick for me:
$ npm --add-python-to-path='true' --debug install --global windows-build-tools
Please note that the command above will not work if your Windows username has a space in it.
You might also fix the path problem by running this command (assuming you have Python 2.7):
$ npm config set python "c:\Python\27\python.exe"
$ npm install
- More info for Windows builders can be found at https://simulatedgreg.gitbooks.io/electron-vue/content/en/getting_started.html#a-note-for-windows-users
# install dependencies npm install (or 'yarn') # serve with hot reload at localhost:9080 npm run dev # build electron application for production npm run build # remove builds npm run build:clean
If you encounter errors related to sharp, this usually fixes it:
Thank you to all of the people at WWU who contributed to this app in one way or another: Tony Kurtz, Ruth Steele, Max Bronsema, Jesse Sturgis, Rebecca Marrall, Nigel Packer, Tim McLaughlin, Rhys Logan, Casey Mullin, Tom McNeely, Brent Carey, Izaac Post, Clarissa Mansfield, Chris Casquilho, Jon Dillon, Kim Marsicek, Jared @ Clarifai, Amélie Bass (cookies), and to the people who wrote the software it is built on (VueJS, Electron, Node, Electron-Vue, Islandora, Islandora Batch, Clarifai, Microsoft Cognitive Services, pHash and many more...). -- David Bass
Photo by Charles Deluvio 🇵🇭🇨🇦 on Unsplash