HTTPS SSH

Procedure

The procedure for creating the offline portion of this app is as follows:

  1. Create the appcache manifest file. For this project, the appcache is associated with the /manifest route. You can view the appcache file for this project as a reference.

    1. For each static resource that you want to serve (your javascript and css files for example), add the url of that file into your appcache under the CACHE section. These files will be explicitly cached from now on.
    2. For each online route that you will have an offline component for, add a fallback route in the appcache. For example, for the /myphotos/create route in this example application, we add

      FALLBACK:
      /myphotos/create /myphotos/ocreate
      
    3. For everything else, it's OK to serve from the network, so add the following to the bottom of your appcache manifest:

      NETWORK:
      *
      
  2. For each online route you have that you want to be accessible offline do the following:

    1. Create a controller for this new route
    2. Create a view for this route and implement any data handling in JavaScript as apposed to using Grails. For example, if you are submitting a form in one of your views, the form should store data using localforage or a similar API instead of submitting a post request to your server. If you have to use URL parameters (as this sample application does using the id parameter), use a # instead of ?=. For more explination on this, see the "Why the #?" section below.
  3. For each online route you have where you could view either online or offline data depending on the context, do the following:

    1. Add a section somewhere in your UI for viewing data that was submitted offline. For example, in this sample application we can view topics that have been submitted offline through the /topics route when we are online and also allow the user to submit those topics to the server. /myphotos/topic is a good example of this.
  4. For each view that allows the user to submit data to be stored locally, ensure that the user has a way of submitting this data to the server when they come back online (see /myphotos/topic/view and the associated views and js files for an example of this).

  5. Take a look in online.js and offline.js and consider mimicking the structure found there if you need it. Online.js falls back to offline.js and all they do is toggle a single global variable. This gives you an easy way to tell if you are online or offline at any given time using Javascript. You may not need this functionality at all for your application, but if you find yourself needing to disable or enable some features depending if you are online or offline, consider this approach. To implement it, simply copy the online.js and offline.js files directly and then add something like:

    FALLBACK:
    online.js offline.js
    

to your appcache manifest depending on the path to your js files.

Localforage Schema

In order to store data locally when offline this application uses localforage. You can learn more about localforage at https://github.com/mozilla/localForage.

The localforage schema used here is as follows:

  1. There is a single key topics which maps to a list of keys. These keys in turn map to Topic objects (Javascript objects).

  2. Topic keys map to individual topic objects which hold onto a name, their key, and a list of keys that map to photos (Blobs).

  3. Photo keys map to the Blob storing their photo data.

Examples of using this schema

To create a new topic, the essential localforage operations that are necessary are as follows:

  1. Create a new topic object in Javascript. This is done by using the Topic object specified in topic.js.

  2. Store each photo object that the user wants to upload into Blobs. Store these Blobs in localforage by generating a guid for each Blob and storing them in key,value pairs. Add each of these keys that are generated into the topic object.

  3. Save the topic object with it's photo keys under a new guid in localforage.

  4. Push the newly generated topic key onto the topics list in localforage and create it if necessary (see topic.js function Topics.push).

To view a topic, the localforage operations that are necessary are as follows:

  1. Fetch the topic object given it's id from localforage

  2. For each photo key the topic object has, load its corresponding photo in from localforage so it can be rendered.

Once we have the topic object and its photos loaded from localforage, we can render everything.

Why the #?

When visiting the offline portion of this app, you may see a URL that looks something like the following:

/myphotos/OTopic/view#id=aaaa...

This is required because if we were to use URL parameters in the usual way:

/myphotos/OTopic/view?id=aaaa...

The appcache wouldn't know which file to serve up when the /OTopic/view route is requested by the user. Appcache serves files as long as the URL matches, but the URL parameter defeats appcache's strategy, thus we need another way to specify parameters in our URL. Browsers ignore the hashtag in the URL string, which means that when we request a URL that looks like:

/myphotos/OTopic/view#id=aaaa...

all appcache sees is:

/myphotos/OTopic/view

So appcache will happily serve up the cached copy of this page. In addition, we can still have our URL parameter be accessible to our Javascript, so using the hashtag in the browser string to pass parameters is an ideal solution.

Javascript Table of Contents

File Name Description
addPhotos.js Contains code to be used on the addPhotos offline page. Adds photos to an existing offline topic.
create.js Creates a new topic and saves it to localforage. Utilizes topic.js mostly.
edit.js Used in edit topic offline page. Changes a topic's title and saves it back to local forage.
offline.js Sets the online global to false (see above).
online.js Sets the online global to true (see above).
submitTopic.js Contains a single function to submit a topic stored in local forage to the server.
topic.js Contains Topic and Topics objects. Topic represents a single topic in our Javascript and Topics contains functions for working with groups of topics.
util.js Contains guid, barrier, and getId global functions. getId parses a URL for a guid and returns it.
viewTopic.js Code for extracting a single topic from localforage and rendering the photos in HTML.
viewTopics.js Render all topics and a representative image for each one in rows of four (same as the online version). Also used when online for viewing topics that are stored locally.

Javascript Async Control

Javascript works with a single threaded model, yet sometimes in the code you may see something like

var sync = barrier(callsRemaining, callback)
...
sync()

Although Javascript is single threaded, we often cannot be sure when the callbacks for things like localforage, jQuery, or other async services will come back. If we have multiple callbacks, we cannot be sure in what order they will come back either. With this in mind, we clearly need some sort of synchronization mechanism to keep our callbacks under control.

Suppose you have the following lines of code:

for (var i = 0; i < 10; i++) {
    localforage.get(i, function(err, data) {
        // Do something in here
    });
}

You want to process some data that you get from localforage, and after all of the data is processed, you would like to execute a final operation. For our purposes, maybe once you've gotten all the data from localforage you want to render something on the page, or process it all at once.

In any case, the first thing you might try is the following:

for (var i = 0; i < 10; i++) {
    (function(i) {
        localforage.get(i, function(err, data) {
            // Do something here

            if (i == 10) {
                // Do our final callback
                callback();
            }
        });
    })(i);
}

This may look fine on the surface, but there is a problem. We don't necessarily know that the callbacks will execute in order because there order is dependant the order our localforage callbacks return.

What we need is something that will trigger after exactly n callbacks from localforage have triggered. This is where our barrier function comes in. This code is taken directly from util.js.

function barrier(n, callback) {
    return function() {
        n--;
        if (n == 0)
            callback();
    }
}

When we call barrier, we get back a function that we can call a certain number of times (that we set with the parameter n) and will execute a final callback once we're finished (the parameter callback).

Here's how we would use barrier for our example:

var sync = barrier(10, callback)
for (var i = 0; i < 10; i++) {
    localforage.get(i, function(err, data) {
        // Do something in here
        sync();
    });
}

sync will be called 10 times (the length of our loop) and we are guaranteed to only execute the final callback once sync has been called the appropriate amount of times.

As a general note, there are other ways to solve this particular problem, this method has been noted as this is what was used for this demo application but your millage may vary.

Appcache Gotchas

  1. Cached files always come from the cache even if you are online
  2. The appcache only updates if the contents of the manifest file change
  3. Files the reference the manifest file are themselves cached

Development Tips

  1. When writing the Javascript for the offline portion of the webapp, turn off the appcache file completely. This can be done by moving the appcache file to a different directory, or commenting out the first line of the file. Turning off the appcache makes sure that the you will always get the lastest Javascript so you can see your changes. During early development this helps to keep from running into weird bugs that may not actually exist

  2. Test in incognito mode. Once enough functionality exists for testing the offline portion, test in incognito mode in Chrome or private browsing mode in Firefox. Testing in these environments will keep your application cache from persisting beyond the session. This way you don't have to continually clear the cache manually, you can just open a new browser window after you make a change.

Git Strategy

The repository uses a tag to indicate which commit marks the end of online development and one branch, master, to commit code to. Changes to the offline part of the codebase should be committed directly to master. Changes to the online part of the codebase follow this procedure:

  1. Checkout the online tag to a new branch

    git checkout -b online-branch online
    
  2. Make the necessary changes

  3. Commit changes`
  4. Make a snapshot of the repository

    git archive -o online-snapshot-complete.zip HEAD
    
  5. Merge the changes in

    git checkout master
    git rebase online-branch
    git branch -D online-branch
    
  6. Move the online tag to the appropriate commit

    git tag -a online ref -f
    
  7. Push the tags and commits to the server

    git push origin master --tags -f
    

There may be conflicts with pulling after the online part of the codebase has been updated. In this case, use the following commands to get re-synched with the remote branch:

git fetch --all
git checkout --hard origin/master

Note that this will destroy local files, so if you have any changes locally that are not already on the remote server, you will have to take other measures to re-synchronize your repository.