Support for JPMS (Java Platform Module System)

Issue #114 open
Markus KARG
created an issue

Using Babbler with JDK 9 is not an easy task, since Java 9 comes with JPMS enabled -- hence, all Babbler packages are reported as missing. The problem is that people might like to migrate their projects to Java 9, so even if Babbler stays in JDK 8 there will be problems we have to solve. I tried to get it right without modifying Babbler in a first step, but it is really hell, simply spoken, so I gave up after one hour. We have to modify it and provide some help ontop.

Hence I'd like to propose a plan for Java 9 support:

Step 0 -- New git branch jpms

Step 1 -- In MANIFEST.MF we provide a new Automatic-Module-Name entry so the current Babbler JARs will still run on JRE 8 and JRE 9 without any change, but the module name will be fixed right now, so it will not change when later modularizing Babbler. Possibly we can skip this step if all JARs have names exactly reading like the wanted module names (to be checked).

Step 2 -- Provide a public example how to use Babbler in a JDK 9 project. This essentially contains a list of the fixed module names provided in Step 1 and possibly a short exemplary POM.

Step 3 -- While still compiling for JRE 8 using JDK 8 tools, replace Automatic-Module-Name by module-info.java compiled using JDK 9 tools (there is a Maven example on the Maven Compiler Plugin web site). Using this step, JRE 8 and JRE 9 will still be able to use Babbler rather unchanged, but we are not forced to build Babbler completely using JDK 9, which might take some time to get it right, apparently.

Step 4 -- Build using JDK 9 tools for target JRE 8, so we can upgrade our build machines without forcing users to upgrade their target runtimes.

Step 5 -- Build using JDK 9 tools for target JRE 9. IMHO this step cannot be done in the Babbler 1.x line, but we have to wait with this step until we start a Babbler 2.x line -- or at least, we have to start that line with this sole commit inside (and possibly more Off-Topics like support for Flows API).

Comments (28)

  1. Markus KARG reporter

    Had a deeper look into the existing structure of Babbler discovered some more problems to fix, unfortunately, so the above plan is not enough. In particular, Babbler's JARs produce split packages. This means, there are packages defined in more than one JAR, which is essentially not allowed in JPMS. All current solutions I did find either enforce the consumer to explicitly hack a lot of command line options -- or enforce to fix that. In the end, split packages never had been a good idea, even back in Java 8. So it might be better to sort out this issue in the current master branch already, before even thinking of Java 9...?

  2. Christian Schudt repo owner

    I didn't spend too much time with JDK 9 yet, but I thought JDK 9 would allow "old" Jars to be on the module path as "automatic module", where the module name equals the Jar name.

    I am open to do a migration to JDK 9 as long as it runs on JDK 8 as well (Step 3 ensures that?).

    What split packages are there? I tried to prevent them actually, because I knew they were troublesome.

  3. Markus KARG reporter

    Yes, JDK 9 allows "old" JARs to be on the module path as Automatic Modules, but the name of the module then is the name of the JAR. If that fits all cases for Babbler, then we can skip Step 1.

    Step 3's sole purpose is to guarantee that the JARs still will be targetting JRE 8, but additionally will work as modules on JRE 9.

    Anyways, one restriction of Modules is that they must not split packages among several JARs -- which Babbler does. At least for all (!) sub-packages of rocks.xmpp.extensions, because these sub-packages are found in both, xmpp.extensions and xmpp.extensions.client. This was reported by javac (v9), and I assume that there might be more cases when manually checking it (or using jdeps).

    I checked several tutorials, but I do not see any other solutions that either consolidating the JARs or renaming these packages.

  4. Christian Schudt repo owner

    Hm... there's the package rocks.xmpp.extensions.chatstates in the xmpp-extensions.jar and the rocks.xmpp.extensions.chatstates.model package in the xmpp-extensions-client.jar.

    Is this an issue? I assumed that both packages are distinct packages. Each have their own package-info.java. I really thought they were not "split packages". In fact I read into the topic a few years ago and decided to put all "model" sub-packages in the other jar to avoid exactly this issue.

  5. Markus KARG reporter

    Seems like a misunderstanding. Yes, there is no package-hierarchy in Java but the package namespace is flat. But what I actually meant was the fact that the folder xmpp-extensions/src/main/java/rocks/xmpp/extensions/ seems to contain virtually the exact same folders (e. g. filetransfer) than the folder xmpp-extensions-client/src/main/java/rocks/xmpp/extensions/. As JPMS treates both folders as separate modules, JDK 9 complains that e. g. the package rocks.extensions.filetransfer is exported by both automatic modules.

  6. Christian Schudt repo owner

    If you look at the folders you are right. On the other hand, why doesn't it complain, that the package rocks isn't exported by multiple modules then? In your example, rocks.extensions.filetransfer was a mistake, which should be fixed. It is indeed a true split package. But the majority of packages should be fine, like the chatstates package.

  7. Markus KARG reporter

    Thank you, this clearly explains why JDK 9 complained exactly about that -and only about that- particular package! So introducing /model/ in filetransfer should definitively fix it! Great. :-) I will check that immediately...

  8. Markus KARG reporter

    @Christian Schudt Question: What JMPS module names do you want me to use? My proposed commit currently would use the following (reversed DNS names just we are used with Java package names):

    • rocks.xmpp.addr

    • rocks.xmpp.core

    • rocks.xmpp.core.client

    • rocks.xmpp.debug

    • rocks.xmpp.extensions

    • rocks.xmpp.extensions.client

    • rocks.xmpp.fx (Maybe you like rocks.xmpp.javafx better?)

    • rocks.xmpp.sample

    • rocks.xmpp.websocket

    A counterproposal would be using babbler.* instead of rocks.*.

    What do you think?

  9. Markus KARG reporter

    pull request #33 filed. I now will start to slightly modify the build system so JDK 9 is needed for only additional module-info.java files (these cannot be created using JDK 8 due to the new module keyword). Hence, an additional JDK 9 is needed from now on to build Babbler, but not for its usage.

  10. Markus KARG reporter

    Working on Step 3. This is really hard work. Lots of exports to add... Stay tuned. :-)

    (Also filed PR against Precis project, to prevent using filename-based automatic module neams)

  11. Markus KARG reporter

    Healthy meanwhile. Step 3 is half way done. I can build using javac 9 and provide java 8 class files plus java 9 module-info. Next step is to not fail lots of tests... Stay tuned.

  12. Markus KARG reporter

    Had to skip analyze-only as it internally uses ASM, and ASM throws IllegalArgumentException on JDK 9.

    Had to skip errorprone as it fails on JDK 9.

    Had to consolidate steps 3 and 4, as it is impossible to compile module-info.java on JDK 8.

    @Christian Schudt Steps 3 and 4 are successfully finished, with some limitations. mvn clean install works fine on JDK 9 with JRE 8 + module-info.class as target. You can check the current state if you like. Further work on this issue is on hold, until ASM and errorprone are fixed to work fine on JDK 9, which I suspect is not within 2017. See https://bitbucket.org/mkarg/babbler/branch/jdk9.

  13. Markus KARG reporter

    Regarding errorprone: The problem is not that it fails due to technical problems, but that it simply is unable to compile against Java 9 source code (quote: "Support for using the Java 9 language level (not just running the tool on JDK 9) is work in progress.") -- It simply fails with the message "release version 9 not supported". As there isn't anything we can do here, we just have to wait for google to fix that. I do not see any benefit from skipping errorprone just for module-info.java; instead I would just suspend my prepared work and go on once google is done with JRE 9 language support.

  14. Markus KARG reporter

    @Christian Schudt I don't dare to tell, but there is new PRECIS trouble on JDK 9 since your latest changes. While it was all fine before, today (i. e. including all your latest PRECIS and Babbler changes) testing on JDK 9 now fails:

    [ERROR] testLocalPart(rocks.xmpp.addr.JidTest)  Time elapsed: 0.024 s  <<< FAILURE!
    rocks.xmpp.precis.InvalidCodePointException: Invalid code point at position 0: 0x149
            at rocks.xmpp.addr.JidTest.testLocalPart(JidTest.java:387)
    
  15. Log in to comment