Image Capture

This page describes working with images in !JavaRosa. All image management classes can be found in the org.javarosa.media.image package. The chatterbox/shell implementation that works with images can be found in the org.javarosa.ccptz.

Overview

Image acquisition is currently modeled as an Activity. The !ImageChooserActivity class is the activity that deals with capturing images. The image chooser has three means of getting images:

1. Using the phone's camera to capture an image. Due to limitations in the Java Media API's and memory issues this currently only captures low-resolution (640x480, although this can be changed in the code) images. 1. Browsing the file system for images. This launches a text-based browser where the user can choose what image they want. 1. Automatically sniffing a directory for images. This is most useful when you want to use the phone's native camera control to capture images, but want them to show up in the image chooser immediately after they are taken.

'''All of these options currently require the !FileConnection API.''' This is because the extremely limited heap space available in j2me required storing the images (even those taken within Java) on the file system. These images could instead be saved to the phone's RMS, however this has not currently been implemented.

UI Workflows

The Image Choosing Activity

The following are screenshots of the basic image chooser activity:

Image(ImageList.png) Image(ImageListMenu.png)

At the top you see what directory is being searched (if any). In the menu you can see the options that are available, which are described below.

• Cancel - returns control to the shell with a "cancel" return argument.
• Mark - (also mapped to the "Select" button) marks the current image for selection.
• Camera - opens the J2ME camera control for an image capture.
• Browse - opens the file browser. See [wiki:FileManagement] for more information.
• View - opens the highlighted image in the phone's native image viewer. This has been verified to work on Nokia S60 devices, but is not tested outside of that.
• Delete - removes the image from the list and deletes it from the file system.
• Change Search Directory - changes the search directory for image auto-discovery. This is not an option if searching is turned off.
• Return - returns control back to the shell, including references to any checked images as part of the return arguments for processing.

NOTE:

• We have code to generate thumbnails for images, but it fails on high resolution objects because of heap limitations, and has thus been disabled.

Image Capture

The following shows a screenshot of the Image Capture Activity. This is very simple - with "Capture" taking a snapshot and returning it to the shell, and "Cancel" returning control to the shell with the cancel command set:

Image(ImageCapture.png)

Images in Forms

The following shows how images are represented in a form within the Chatterbox UI:

Image(ImageInForm.png) Image(ImageInFormMenu.png)

When the "Get Images" menu option is pressed - the user is taken to the Image Choosing Activity and the experience is the same.

When the user comes back with a "return" of selected images from the choosing activity the chatterbox form screen moves on to the next question. If they come back to the image question, they will see the following:

Image(ImageInFormPopulated.png)

Again, we have thumbnail code, but it has been disabled due to memory limitations.

'''NOTE: While the image capture is integrated into the Chatterbox UI for handling forms, the serialization and sending of image data has NOT been completed. This is because of some quirks that will require a MultiPart submission and possibly revising the serialization code to work with streams. Development on this is currently in the works. '''

Code Workflow

Image Choosing

Both browsing for files and using the phone's camera are also activities. Therefore when the !ImageChooserActivity needs to do a file browse or a native image capture it passes control back to the shell with the return code "ACTIVITY_NEEDS_RESOLUTION". Additionally, the !ImageChooserActivity passes back a return argument with the special key of !ImageChooserActivity.ACTIVITY_KEY which tells the shell the next activity it should launch. This could be either the !ImageCaptureActivity, which creates a camera control for taking pictures, or the !FileBrowseActivity, which is used for setting the search directory and browsing for images. The code to handle this in the shell looks something like this:

                if (returningActivity instanceof ImageChooserActivity) 
		{ 
			if (returnCode.equals(Constants.ACTIVITY_NEEDS_RESOLUTION)) {
		 		IActivity next = (IActivity) returnVals.get(ImageChooserActivity.ACTIVITY_KEY);
		 		launchActivity(next, context);
		 	} else {
		                // custom shell handling code
                        }
                }

The Capture and Browsing activities return the image in an IDataPointer object, which represents a pointer to an underlying blob of data. Developers should review !ImageCaptureActivity and [wiki:FileManagement] for more information.

The !ImageChooserActivity exposes APIs to turn sniffing on and off, set the sniffing directory, and have other classes add images to the list (used by the sniffing thread). Please see the [http://docs.javarosa.org/javadoc/org/javarosa/media/image/activity/ImageChooserActivity.html JavaDoc] for more information.

Images in Forms

Images are represented in xforms via the "xsd:anyURI" data type, and an "upload" element with mediatype="image/*". The following snippet shows how an image can be represented in an XForm. This can be found in the imageTest.xhtml file in the org.javarosa.ccptz project.

  <h:head>
    <h:title>CCPTZ Demo</h:title>
    <model>
    <instance>
        <data>
          <Subject>
            <Picture />
            <Age />
            <!-- rest of model goes here.... -->
         <bind id="Picture" nodeset="/data/Subject/Picture" type="xsd:anyURI" />
         <bind id="Age" nodeset="/data/Subject/Age" type="xsd:string" />
         <!-- rest of binds go here.... -->
      </model>
  </h:head>           
  <h:body>
    <upload bind="Picture" mediatype="image/*">
      <label>Take a Picture:</label>
      <hint>This will take a picture.</hint>
      <filename ref="@filename" />
      <mediatype ref="@mediatype" />
    </upload>
    <select1 bind="Age">
      <label>How old are you?</label>
      <hint>Select your relevent age group</hint>
      <!-- rest of form goes here.... -->
  </h:body>

The logic for how this has been integrated to the chatterbox can be found in !ImageChooserWidget and !FormEntryController. Developers are encouraged to look there for more guidance.