Support for Jack and Jill

Issue #33 wontfix
Remco Mokveld created an issue

I know it is still in early development, but with the gradle-plugin version 1.0.0 Google introduced a new compiler Jack. Using this breaks the apt plugin because the task compileJava does not exist. Also in the documentation, here, Google states that the new Jack compiler does not support annotation processing.

As long as the javac compiler can still be used this won't be an issue. But i can imagine Google will start gently forcing the Jack compiler onto developers at some point.

Will it be possible to add support for this compiler in the future?

Comments (28)

  1. Hugo Visser repo owner

    Though one. I haven't looked much into Jack yet, but currently it's not android-apt running the processors, it merely configures javac and adds the generated source path to the Gradle model. I assume that at some point Google will add annotation processing to their compiler, and I think at that point I'd be able to support that through android-apt as well.

  2. jug6ernaut

    With Google officially launching Jack with Android Studios 2.1-preview, is this issue something to be revisited? Or are we at the mercy of Google to support annotation processors?

  3. Hugo Visser repo owner

    I still need API support from Google. I'll take a look and file a bug if appropriate.

  4. marcel s

    I have dived into Jack's internals and would like to share my findings, so that we will be able to support both Javac and Jack with android-apt in the near future. I'm afraid that the current Jack revision won't allow us to add annotation processing. Warning, lengthy text incoming:

    First of all, the Jack executable JAR provides arguments for both --processor and --processorpath just like Javac does, which is neat! There is also -A <option>=<value> for annotation processor options. However, I don't see any way into the JackProcessBuilder class, which is responsible for setting up Jack on the command line:

    JackProcessBuilder.java (excerpt):

    @NonNull
    public JavaProcessInfo build(@NonNull BuildToolInfo buildToolInfo) throws ProcessException {
        // (...)
        ProcessInfoBuilder builder = new ProcessInfoBuilder();
        builder.addEnvironments(mEnvironment);
    
        String jackLocation = System.getenv("USE_JACK_LOCATION");
        String jackJar = jackLocation != null
                ? jackLocation + File.separator + SdkConstants.FN_JACK
                : buildToolInfo.getPath(BuildToolInfo.PathId.JACK);
        if (jackJar == null || !new File(jackJar).isFile()) {
            throw new IllegalStateException("Unable to find jack.jar at " + jackJar);
        }
    
        builder.setClasspath(jackJar);
        builder.setMain("com.android.jack.Main");
    
        if (mJavaMaxHeapSize != null) {
            builder.addJvmArg("-Xmx" + mJavaMaxHeapSize);
        } else {
            builder.addJvmArg("-Xmx1024M");
        }
    
        if (mDebugLog) {
            builder.addArgs("--verbose", "debug");
        } else if (mVerbose) {
            builder.addArgs("--verbose", "info");
        }
    
        // (...)
    }
    

    builder.addArgs() is what we would like to get ahold of in order to add our custom arguments to the task. However, the class doesn't offer a hook for custom values (akin to CompileOptions#compilerArgs in Javac). Furthermore, the configuration for Jack "in process mode" is stored in the Api02Config class, which is a sub-class of Api01Config with the available methods setProcessorNames(@Nonnull List<String> processorNames), setProcessorPath(@Nonnull List<File> processorPath) and setProcessorOptions(@Nonnull Map<String, String> processorOptions). At the moment, these aren't actually used when setting up the Jack task, though:

    AndroidBuilder.java (excerpt):

    public void convertByteCodeUsingJackApis(/* A lot of parameters... */) throws /* a lot of Exceptions... */ {
        // (...)
        Optional<JackProvider> jackProvider = buildToolServiceLoader.getSingleService(getLogger(), BuildToolsServiceLoader.JACK);
        if (jackProvider.isPresent()) {
            Api02Config config;
    
            // Get configuration object
            try {
                config = jackProvider.get().createConfig(Api02Config.class);
    
                config.setClasspath(new ArrayList<File>(classpath));
                config.setOutputDexDir(dexOutputFolder);
                config.setOutputJackFile(jackOutputFile);
                config.setImportedJackLibraryFiles(new ArrayList<File>(packagedLibraries));
                config.setAndroidMinApiLevel(minSdkVersion);
                (...)
            } catch (/* ... */) {
                // ...
            }
        }
    }
    

    Let's file a feature request for the Jack tool with the following contents:

    1. Add List<String> processorNames with accessors to JackTask
    2. Add List<File> processorPath with accessors to JackTask
    3. Add Map<String, String> processorOptions with accessors to JackTask
    4. In JackTask#doMinification, pass these three options to both convertByteCodeUsingJackApis and convertByteCodeWithJack
    5. In AndroidBuilder#convertByteCodeUsingJackApis, actually call the Api02Config methods and pass in processor names, paths and options
    6. In JackProcessBuilder, add fields and setters for processorNames, processorPath and processorOptions
    7. In AndroidBuilder#convertByteCodeWithJack, call the JackProcessBuilder methods and pass in processor names, paths and options
    8. In JackProcessBuilder#build, add loops and conditionals as necessary to append processor names, paths and options to the JavaProcessInfo builder using builder.addArgs()

    With these things in place, the AndroidAptPlugin can distinguish between Javac and Jack since the latter is being passed in as the 'javaCompiler' property, so the plugin already recognizes the compilation task. We should be able to create an abstract layer over the actual variant configuration with an interface and two implementations (JavacVariantConfiguration and JackVariantConfiguration, or something along those lines)!

  5. Hugo Visser repo owner

    I rather not detect Jack vs traditional javac, that was the whole point of renaming the compile task in the Android Gradle plugin a while ago. Wouldn't it be sufficient if the Android plugin would just allow for setting the compiler options as it does now for the compiler task? The actual arguments are the same as for javac so I think that would always have been the intention anyway.

  6. marcel s

    That's kind of what I meant with the "abstraction layer", and JackTask being extended with custom compiler options, even though that might not be the correct description. For the APT plugin, it would mean a change along the lines of (assuming that support for arbitrary compile args is added to the Jack API as "compilerArgs"):

    def javaCompile = variant.hasProperty('javaCompiler') ? variant.javaCompiler : variant.javaCompile
    
    // (...)
    
    def processors = aptExtension.processors()
    
    def compileOptions = (javaCompile instanceof JackTask) ? javaCompile.compilerArgs : javaCompile.options.compilerArgs
    compileOptions += [
            '-s', aptOutput // '-A' in Jack
    ]
    
    if (processors) {
        compileOptions += [
                '-processor', processors // '--processor' in Jack
        ]
    }
    
    if (!(processors && aptExtension.disableDiscovery())) {
        compileOptions += [
                '-processorpath', processorPath // '--processorpath' in Jack
        ]
    }
    
  7. Hugo Visser repo owner

    According to Xav from the tools team, annotation support for Jack will be coming soon. That could take several forms, so I'll try to sync up later and plan accordingly. I haven't had time to actually try android-apt on a project compiling for N with the new tools yet. If somebody can supply a sample project, that would be helpful.

    My preferred option to solving this would be either the Android Gradle plugin handling the options transparently so that android-apt and other tools plugging into the compile task are agnostic to it, add an api so that android-apt can do the right thing for Jack, or include the functionality that android-apt into the Android Gradle plugin in some shape or form so that it just works without android-apt.

  8. marcel s

    Including apt functionality into a new version of the Android Gradle plugin directly would definitely be my preferred option as well. Regarding the sample project, here you go! It tries to use PermissionsDispatcher to generate some basic code, but fails if Jack is enabled.

  9. Matias Radzinski

    @hvisser, Any news regarding this issue? Android 2.1 is right around the corner, Preview 5 has been already released and as per some external info I gathered the 2.1 will exit preview phase earlier than everyone thought. I wonder what will happen once 2.1 gets published on the stable channel. APT is widely used, so this issue might end up being a show-stopper.

  10. Hugo Visser repo owner

    Please star the bug mentioned above. There is nothing I can do without the proper API in the tools.

  11. Fabian

    @hvisser looks like the new build tools 2.2.0-alpha1 with AS2.2 supporting annotation processing with annotationProcessor '...'.

  12. Hugo Visser repo owner

    Looks pretty interesting, I'm at I/O now but will have some time to look at the changes that are in that branch. I'll also talk to the tools team here to get some more details.

  13. Hugo Visser repo owner

    Looks like the support in the latest Gradle plugin is almost there, so I don't think it would make sense to keep the scopes and processor config that android-apt has, if there's still a reason for android-apt to exist. If all that is missing is setting the source path, it would be best if they add that, or as an alternative android-apt could supply only that functionality.

  14. Nicholas Wong

    Just tried the latest build tool release 2.2.0-alpha4, link's here. Looks like the classes are now generating to the correct location -> build/generated/source/apt. Seems like Google is paying tribute to you @hvisser! 😃

  15. Hugo Visser repo owner

    If anybody can test it to see what they have actually done, that would be great! My guess is that they still aren't adding the generated sources to the Gradle model and only moved the output directory.

  16. Remco Mokveld reporter

    Seems like it is working. Android Studio recognises the the classes generated by butterknife and they are also added to the APK. Note that Butterknife 8.2.+ does not work with jack because it uses classes from the sun compiler which are not available in the classpath when using jack. 8.1.0 works fine though

  17. Hugo Visser repo owner

    It looks like in the Android 2.2 Gradle plugin will contain all functionality that android-apt adds for older versions, while it's still in the -alpha range, it will mean that there's probably no reason for android-apt to exist anymore when it becomes stable, so that means that with that this issue can be closed :)

  18. Log in to comment