HTTPS SSH

Quick Reload - Stealing time back from Maven since 2013. Just press up arrow!

You be all like...

QuickReload is an Atlassian P2 plugin to help you build Atlassian P2 plugins quicker. (Meta!)

It works by watching output directories for P2 .jar files to be create, then uploads them into the running Atlassian host application, such as Jira or Confluence.

It aims to provide the smallest possible time between a piece of code being compiled and linked, to it being loaded and run in the host application.

The ideal workflow is to have a command line terminal open, type in mvn package -DskipTests and as soon as the output
.jar is created, it will be loaded and run. To rinse and repeat, just press up arrow!

Installing Quick Reload

You can manually install QuickReload by copying it into <apphome>/plugins/installed or by uploading it via the Universal Plugin Manager (UPM) within the host application.

The most scalable way is to have it present when you run with the Atlassian SDK and atlas-debug.

In the plugin's POM where the product-specific plugin is configured, add a pluginArtifact entry like this:

            <plugin>
                <groupId>com.atlassian.maven.plugins</groupId>
                <artifactId>jira-maven-plugin</artifactId>
                <extensions>true</extensions>
                <configuration>
                    <productVersion>${jira.version}</productVersion>

                    <!-- make AMPS faster -->
                    <enableDevToolbox>false</enableDevToolbox>
                    <enablePde>false</enablePde>
                    <skipRestDocGeneration>true</skipRestDocGeneration>
                    <allowGoogleTracking>false</allowGoogleTracking>
                    <skipManifestValidation>true</skipManifestValidation>
                    <extractDependencies>false</extractDependencies>

                    <!-- On AMPS 6.3 and older; must be omitted on AMPS 8 and newer -->
                    <enableFastdev>false</enableFastdev>

                    ...
                    ...
                    <pluginArtifact>
                        <groupId>com.atlassian.labs.plugins</groupId>
                        <artifactId>quickreload</artifactId>
                        <version>${quick.reload.version}</version>
                    </pluginArtifact>

This will cause Jira to load the QuickReload plugin when it starts.

You might also notice the various "disable" config to AtlasSDK helpers. This will ensure your environment is as fast as possible. When using AMPS 6.3 or earlier, the one you really want to disable is FastDev. (AMPS 8 removed FastDev support completely.) FastDev was an early version of the same idea as QuickReload; it would watch directories and automatically run Maven upon a source code change. In practice this is not a good idea, but the intention was right.

QuickReload and FastDev share the same library for file watching, and hence can clash with each other. It's best to turn off FastDev if you want to use QuickReload.

A brief history of slow plugin reloads

Reloading plugins inside Atlassian is not as fast as it could be. Atlassian SDK tries to help you here by given you the atlas-cli and atlas-install-plugin
commands for dynamic reloading. But they themselves are slow because starting Maven takes forever; you have burnt 3-5 seconds just to get Maven out of bed.

These tools use UPM's HTTP uploading to transfer the plugin to the host application. A series of slow operations happen to slow down the plugin reload:
1. The generation of the security crypto token for admin:admin is really, really slow.
2. This is in fact one of the biggest culprits of plugin reload.
3. The HTTP upload takes time and it puts the file into a temporary location anyway.
4. The install itself now takes that temporary file and uses the Atlassian plugin system to install the plugin.

In short, these tools work by being external to the host application and hence they incur a lot of time costs just to get things happening.

QuickReload inverts this idea. It sits inside the host application as a plugin and it watches directories for changes.

As soon as a jar file that contains an atlassian-plugin.xml changes on disk (say via a mvn package or a gradle build), it kicks in and causes it to be installed,
right there from your <dir>/target directory. No HTTP transfer, no security token generation, no time wasted. It's as fast as the plugin system can install the plugin.

So your development time will reduce down to the time it takes to compile and package the change, plus the time it takes to install the plugin from the inside.

How it works

The idea is remarkably simple. Firstly there is a file change watching library called atlassian-watch-service that is a simple wrapper for the Java 7 file watch APIs or the jwatchservice code that runs on JDK 6.

https://bitbucket.org/atlassianlabs/atlassian-watchservice

It turns out that the JDK 7 implementation is rubbish and polls directories for changes. So in fact it uses jwatchservice code to get near-realtime updates for
directory changes. This is how the code knows that a jar has changed on disk.

It then examine the jar to see if it's an Atlassian plugin (by having a quick look inside it) and then calls on the plugin system to install that jar.

It turns out that file watching service is too efficient, because it raises a changed event when the jar is first created but not yet filled with zip files. So the code has to have some "retry" time to wait until the jar is fully formed. This turns out to be 10-100ms, so it's not a big problem.

The plugin system doesn't care where plugin files are installed from. It can be <home>/plugins/installed-plugins, or any other directory. So this code uses the <dir>/target directory directly as the place from which to install the plugin.

No copying and no HTTP transfer. Speedy!

Configuring QuickReload

The pom.xml convention

By default, you should not have to configure anything for QuickReload to work, because it uses a pom.xml strategy on what directories to
watch for changes.

  • QR starts from the current running directory
  • QR ascends up directories until it find a pom.xml that has no parent pom.xml
  • QR then descends down from there to find all pom.xml files and tracks their peer <dir>/target directories for changed .jar files

This will allow QuickReload to watch many plugins in a multi-module Maven project.

The quickreload.properties convention

You can drop a quickreload.properties file into the project to tell QuickReload what to explicitly watch, along with the pom.xml convention.

  • QR ascends up directories until it finds a quickreload.properties file
  • QR reads the entries and begins to watch those directories

The directories can be absolute or relative to the quickreload.properties file. Also IF the directory contains a pom.xml and then <dir>/target is assumed as the right place to watch for .jar changes.

The file looks like this:

    #
    # Add directories to this file, one on each line, and the quick reload support will monitor them
    # and automatically load them as soon as they are compiled and assembled.
    #
    # You can use -Dquickreload.dirs=dir1,dir2,dir3 on the JVM command line to also specify directories
    # to monitor.
    #
    # relative path to plugin
    some/path/test-plugin

    # absolute path to plugin
    /Users/me/src/atlassian/servicedesk/master/plugin

    # non-filepath directive to quickreload to turn off batching (note the qr: prefix)
    qr:webresourcebatching=false

The quickreload.properties $HOME convention

QuickReload will look for a $HOME/quickreload.properties file as configuration. You could put very common entries in here, but in general the pom.xml convention is the most powerful and the least amount of work to configure.

Other (legacy) conventions

QuickReload will also look for a quickreload.properties style file in the following locations:

  • $HOME/.quickreload
  • $HOME/.quickreload/quickreload

QuickReload will also traverse up directories to find .quickreload files in a similar manner to quickreload.properties files.

Front End Alternate Resource Directories

QuickReload also enables a front end development dev speed enabler called "Alternate Resource Directories". These cause the web plugin system to load your JavaScript/CSS/LESS from source directories instead of from a package .jar plugin. Hence you can edit for example a .js file and press refresh in the browser, and it changes. It uses the pom.xml strategy and add <dir>/src/main/resources to the plugin system alternate resource directory loading mechanism for you. There is no more need to specify -Dplugin.resource.directories as it will be done for you.

Finally you can specify the directories to track on the command line, via

-Dquickreload.tracked=dir1,dir2,dir3

If you really must but in general, it "just works" for Maven projects.

This allows you to have two or more plugins being developed at the same time with jar and resource monitoring in place automatically. Cool, right?

You can put properties before a line entry via a : character. Multiple properties can be specified via the ; character

    prop1=123;prop2=xyz;prop3 : /some/path

The current properties are :

  • resource - marks the path as a resource entry and it will be placed into the -Dplugin.resource.directories system directory

Controlling web batching

The Atlassian Web Resource system can batch together files for faster loading. But this means its harder to debug. So there is a atlassian.dev.mode system property to control that.

But it's a one time thing that is set at JVM startup right? Not any more.

Press 'B' when QuickReload is running and you can toggle between speed of web batching being enabled and the debug ability of when web batching is disabled.

Who are you calling a half wit?

Since you won't have to wait for plugin uploads to happen so slowly, surely you will need something else to occupy you and your team.

Simply drop a file called .halfwit into your team's project directory, and QuickReload will pick it up and randomly display a line from it.

I call this technique Social Smartarseism.

Don't forget to check this into git and get the team in on it. (smile)

    [INFO] [talledLocalContainer] 2014-05-09 15:36:36,864 http-bio-2990-exec-5 INFO admin 936x20x1 1p8941i 0:0:0:0:0:0:0:1 /rest/quickreload/1.0/plugin [labs.plugins.quickreload.PluginInstaller]
    [INFO] [talledLocalContainer]                       ^
    [INFO] [talledLocalContainer]                       |
    [INFO] [talledLocalContainer]                       |
    [INFO] [talledLocalContainer]                       |
    [INFO] [talledLocalContainer]                       |
    [INFO] [talledLocalContainer]                       |
    [INFO] [talledLocalContainer]
    [INFO] [talledLocalContainer]               “The only thing worse than being talked about is not being talked about.”
    [INFO] [talledLocalContainer]
    [INFO] [talledLocalContainer] Quick Reload Finished (3593 ms) - '/Users/bbaker/src/atlassian/plugins/quickreload/test-plugin/target/quickreload-test-plugin-1.3-SNAPSHOT.jar'

Log Output

The quick reload spits out some extra log output with spacings so that you can pick the plugin reload logs lines from the wall of log text. Trust me, you won't miss the spot where you caused a plugin reload.

    [INFO] [talledLocalContainer] 2014-05-08 18:10:14,332 http-bio-2990-exec-26 INFO anonymous 1090x6739x1 - 0:0:0:0:0:0:0:1 /rest/quickreload/1/install [labs.plugins.quickreload.PluginInstaller]
    [INFO] [talledLocalContainer]                       |
    [INFO] [talledLocalContainer]                       |
    [INFO] [talledLocalContainer]                       |
    [INFO] [talledLocalContainer]                       |
    [INFO] [talledLocalContainer]                       |
    [INFO] [talledLocalContainer]                       v
    [INFO] [talledLocalContainer]
    [INFO] [talledLocalContainer] Starting Quick Reload - '/Users/bbaker/src/atlassian/plugins/quickreload/test-plugin/target/quickreload-test-plugin-1.3-SNAPSHOT.jar'....
    [INFO] [talledLocalContainer]
    [INFO] [talledLocalContainer]
    [INFO] [talledLocalContainer] 2014-05-08 18:10:14,335 http-bio-2990-exec-26 INFO anonymous 1090x6739x1 - 0:0:0:0:0:0:0:1 /rest/quickreload/1/install [atlassian.plugin.loaders.ScanningPluginLoader] No plugins found to be installed
    [INFO] [talledLocalContainer] 2014-05-08 18:10:16,752 http-bio-2990-exec-26 INFO anonymous 1090x6739x1 - 0:0:0:0:0:0:0:1 /rest/quickreload/1/install [atlassian.plugin.manager.DefaultPluginManager] Found dependent enabled plugins for uninstalled plugin 'com.atlassian.labs.plugins.quickreload.test': [].  Disabling...
    [INFO] [talledLocalContainer] 2014-05-08 18:10:16,752 http-bio-2990-exec-26 INFO anonymous 1090x6739x1 - 0:0:0:0:0:0:0:1 /rest/quickreload/1/install [atlassian.plugin.manager.DefaultPluginManager] Updating plugin 'com.atlassian.labs.plugins.quickreload.test' to 'com.atlassian.labs.plugins.quickreload.test'
    [INFO] [talledLocalContainer] 2014-05-08 18:10:16,752 http-bio-2990-exec-26 INFO anonymous 1090x6739x1 - 0:0:0:0:0:0:0:1 /rest/quickreload/1/install [atlassian.plugin.manager.DefaultPluginManager] Disabling com.atlassian.labs.plugins.quickreload.test
    [INFO] [talledLocalContainer] 2014-05-08 18:10:16,757 Timer-1 WARN      [plugins.quickreload.test.Launcher]
    [INFO] [talledLocalContainer]
    [INFO] [talledLocalContainer] Quick Reload Test Plugin is going out of existence!!
    [INFO] [talledLocalContainer]
    [INFO] [talledLocalContainer]
    [INFO] [talledLocalContainer] 2014-05-08 18:10:16,760 http-bio-2990-exec-26 INFO anonymous 1090x6739x1 - 0:0:0:0:0:0:0:1 /rest/quickreload/1/install [atlassian.plugin.loaders.ScanningPluginLoader] Removed plugin com.atlassian.labs.plugins.quickreload.test
    [INFO] [talledLocalContainer] 2014-05-08 18:10:16,777 ThreadPoolAsyncTaskExecutor::Thread 56 WARN anonymous 1090x6739x1 - 0:0:0:0:0:0:0:1 /rest/quickreload/1/install [plugins.quickreload.test.Launcher]
    [INFO] [talledLocalContainer]
    [INFO] [talledLocalContainer] Quick Reload Test Plugin is coming into existence!!
    [INFO] [talledLocalContainer]
    [INFO] [talledLocalContainer]
    [INFO] [talledLocalContainer] 2014-05-08 18:10:16,785 http-bio-2990-exec-26 INFO anonymous 1090x6739x1 - 0:0:0:0:0:0:0:1 /rest/quickreload/1/install [labs.plugins.quickreload.PluginInstaller]
    [INFO] [talledLocalContainer]                       ^
    [INFO] [talledLocalContainer]                       |
    [INFO] [talledLocalContainer]                       |
    [INFO] [talledLocalContainer]                       |
    [INFO] [talledLocalContainer]                       |
    [INFO] [talledLocalContainer]                       |
    [INFO] [talledLocalContainer]
    [INFO] [talledLocalContainer]           A watched plugin never boils!
    [INFO] [talledLocalContainer]
    [INFO] [talledLocalContainer] Quick Reload Finished (2453 ms) - '/Users/bbaker/src/atlassian/plugins/quickreload/test-plugin/target/quickreload-test-plugin-1.3-SNAPSHOT.jar'

Security

There is none. That's the point. The cost of crypto encoding admin:admin is a significant cost in terms of time for plugin reload. So it doesn't do it.

Don't use this in production. Just don't!

HTML Control Panel

QuickReload has a control panel you can visit to see whats in play.

http://localhost:2990/jira/plugins/servlet/qr

REST API

QuickReload has a REST API to help you be a better developer

Visit this to begin discovering it

http://localhost:2990/jira/rest/qr/api

Currently you can

  • See all OSGi bundles in the system

    http://localhost:2990/jira/rest/qr/bundles
    
  • See a specific OSGi bundle

    http://localhost:2990/jira/rest/qr/bundles/{bundleId}
    
  • See all the OSGi services in the system

    http://localhost:2990/jira/rest/qr/services
    
  • See all Atlassian Plugins in the system

    http://localhost:2990/jira/rest/qr/plugins
    
  • See a specific Atlassian Plugin, including its modules and internal Spring bean context

    http://localhost:2990/jira/rest/qr/plugins/{pluginKey}
    
  • See the state of a current state of a plugin, and with POST / DELETE to enable or disable the plugin

    http://localhost:2990/jira/rest/qr/plugin/enabled/{pluginKey}
    
  • See the state of a current state of a plugin module, and with POST / DELETE to enable or disable the plugin module

    http://localhost:2990/jira/rest/qr/plugin/enabled/{moduleKey}
    
  • View the last 100 events in the application

    http://localhost:2990/jira/rest/qr/events
    
  • View (or set) the system properties

    http://localhost:2990/jira/rest/qr/systemproperties
    
  • See the list of QR tracked directories

    http://localhost:2990/jira/rest/qr/tracked
    
  • See whether web batching is enabled

    http://localhost:2990/jira/rest/qr/batching
    

Troubleshooting

java.lang.UnsatisfiedLinkError

If you see this "java.lang.UnsatisfiedLinkError: Native Library" error it means that the QuickReload plugin was re-loaded while running but it could not be. It's ironic that a reloading plugin can't be cleanly re-loaded, but it's more to do with the underlying native file watching library than QuickReload itself. Java does not like to unload native libraries, and the underlying name.pachler.nio.file does not expect to be reloaded. Trust me, you won't see this much. It's seen more in developing this plugin itself, or when you are running FastDev at the same time as QuickReload.

FastDev Plugin Clash

The FastDev plugin from AMPS (prior to 8.0) also uses the same ideas as the QuickReload plugin, but with a different design. It watches your source and then calls Maven to build and plugin install based on source changes.

QuickReload takes a different design tack. It doesn't watch source but nominated jar directories and it's up to you to build the jar, typically via mvn package by pressing up arrow in your terminal. But they do share the same jPathWatch Library and hence will compete on loading the native libraries AND you will be double watching. I recommend that you do not use both at the same time.

    <enableFastdev>false</enableFastdev>

AMPS 8.0 removed support for FastDev completely, so <enableFastdev>false</enableFastdev> is not necessary (and, in fact, is not supported) on AMPS 8.0+.