Constructor.getClassForName() can't find Class

[Former user]

A bug report in the JBoss Resteasy project (https://issues.jboss.org/browse/RESTEASY-1236) has led me to the method Constructor.getClassForName(), which uses the YAML classloader. In the reproducer submitted to RESTEASY-1236, a class is bundled into a WAR and sent to the server (Wildfly 8.1.0.Final), but YAML is unable to find the class:

20:00:30,421 ERROR [org.jboss.resteasy.resteasy_jaxrs.i18n] (default task-1) RESTEASY002005: Failed executing POST /yaml: org.jboss.resteasy.spi.ReaderException: Failed to decode Yaml at org.jboss.resteasy.plugins.providers.YamlProvider.readFrom(YamlProvider.java:65) [resteasy-yaml-provider-3.0.8.Final.jar:] ... Caused by: Can't construct a java object for tag:yaml.org,2002:org.jboss.resteasy.yaml.YamlProviderObject; exception=Class not found: org.jboss.resteasy.yaml.YamlProviderObject in 'reader', line 1, column 1: !!org.jboss.resteasy.yaml.YamlPr ... ^

at org.yaml.snakeyaml.constructor.Constructor$ConstructYamlObject.construct(Constructor.java:336)
at org.yaml.snakeyaml.constructor.BaseConstructor.constructObject(BaseConstructor.java:182)
at org.yaml.snakeyaml.constructor.BaseConstructor.constructDocument(BaseConstructor.java:141)
at org.yaml.snakeyaml.constructor.BaseConstructor.getSingleData(BaseConstructor.java:127)
at org.yaml.snakeyaml.Yaml.loadFromReader(Yaml.java:481)
at org.yaml.snakeyaml.Yaml.load(Yaml.java:412)
at org.jboss.resteasy.plugins.providers.YamlProvider.readFrom(YamlProvider.java:59) [resteasy-yaml-provider-3.0.8.Final.jar:]
... 45 more

Caused by: org.yaml.snakeyaml.error.YAMLException: Class not found: org.jboss.resteasy.yaml.YamlProviderObject at org.yaml.snakeyaml.constructor.Constructor.getClassForNode(Constructor.java:636) at org.yaml.snakeyaml.constructor.Constructor$ConstructYamlObject.getConstructor(Constructor.java:322) at org.yaml.snakeyaml.constructor.Constructor$ConstructYamlObject.construct(Constructor.java:332) ... 51 more

I have a workaround:

     InputStreamToByteArray istba = new InputStreamToByteArray(entityStream);
     InputStream is = new SequenceInputStream(new ByteArrayInputStream(istba.toByteArray()), entityStream);
     Class<?> clazz = getClass(istba);
     Yaml yaml = new Yaml(new CustomClassLoaderConstructor(clazz, Thread.currentThread().getContextClassLoader()));
     return yaml.load(is);

where getClass() parses the serialized content of the InputStream, and the fiddling with InputStreamToByteArray and SequenceInputStream lets me read the InputStream and then reuse it.

It works, at least of the reproducer example, but it's messy, and I don't trust that I understand YAML serialization well enough to parse it properly.

I haven't tried it, but I'm guessing that

    protected Class<?> getClassForName(String name) throws ClassNotFoundException {
     ClassLoader cl = Thread.currentThread().getContextClassLoader();
     Class<?> clazz = Class.forName(tag, true, cl);
     if (clazz != null) {
         return clazz;
     }
     return Class.forName(name);
}

would work.

What do you think?

[Andrey Somov]

Is this the solution ? https://bitbucket.org/asomov/snakeyaml/src/tip/src/main/java/org/yaml/snakeyaml/constructor/CustomClassLoaderConstructor.java?fileviewer=file-view-default

Basically, if you know that the class should be found by another classloader that just give the proper classloader to SnakeYAML. You can find the usages in tests.

[Alexander Maslov]

This is actually a long standing "issue". There was even small dialog in google group.

I do not understand why we are not using ContextClassLoader... I guess because nobody REALLY complained before.

JBoss classloading was a bit more simple before AS7. And even now I guess if you load snakeyaml from .war/WEB-INF/lib/ it will work.

But in WildFly and EAP6.x if you are using provided resteasy, SnakeYAML is in "org.yaml.snakeyaml" module and loaded as a dependency of "org.jboss.resteasy.resteasy-yaml-provider" module. That is most probably the reason SnakeYAML cannot find classes from deployed WAR/EAR (module's classloader knows nothing about war's classloader).

I think we should switch to ContextClassLoader. But I am also interested if it influences usage of SnakeYAML in OSGi environments.

BTW, @ronsigal, I expected $ mvn clean test to fail because of this "bug", but it actually fails because of

org.jboss.arquillian.container.spi.client.container.LifecycleException: Could not start container
...
Caused by: java.lang.IllegalArgumentException: WFLYLNCHR0001: The path 'null' does not exist
    at org.wildfly.core.launcher.Environment.validateWildFlyDir(Environment.java:318)
    at org.wildfly.core.launcher.AbstractCommandBuilder.validateWildFlyDir(AbstractCommandBuilder.java:628)
    at org.wildfly.core.launcher.StandaloneCommandBuilder.of(StandaloneCommandBuilder.java:91)
    at org.jboss.as.arquillian.container.managed.ManagedDeployableContainer.startInternal(ManagedDeployableContainer.java:85)

Since I am not really in "Arquillian" tests, could you be so kind to check it out? I really would like to have that project as working one.

[Former user]

Hey guys,

First of all, I want to thank you for getting on this case so quickly. You're definitely faster than I am. ;-)

re: "Basically, if you know that the class should be found by another classloader that just give the proper classloader to SnakeYAML."

In fact, I was using CustomClassloaderConstructor, but, for some reason, it didn't work for me unless I passed a Class as well as a classloader. I just tried it again without passing in a Class, and the test passed. That's a great relief, since my parsing of the serialized input was totally half baked. Now, I don't have to do that.

re: "This is actually a long standing "issue". There was even small dialog in google group."

Ah. My first inclination was to post a question rather than create an issue, but I couldn't find any discussion groups.

re: "I guess if you load snakeyaml from .war/WEB-INF/lib/ it will work."

I thought so too, but with

resteasy-yaml.war:

/WEB-INF/
/WEB-INF/lib/
/WEB-INF/lib/snakeyaml-1.16.jar
/WEB-INF/web.xml
/WEB-INF/classes/
/WEB-INF/classes/org/
/WEB-INF/classes/org/jboss/
/WEB-INF/classes/org/jboss/resteasy/
/WEB-INF/classes/org/jboss/resteasy/yaml/
/WEB-INF/classes/org/jboss/resteasy/yaml/YamlProviderNestedObject.class
/WEB-INF/classes/org/jboss/resteasy/yaml/YamlProviderObject.class
/WEB-INF/classes/org/jboss/resteasy/yaml/JApplication.class
/WEB-INF/classes/org/jboss/resteasy/yaml/YamlProviderResource.class

I still got the class not found exception. Maybe org/yaml/snakeyaml/main//snakeyaml-1.16.jar was already loaded?

re: "I expected $ mvn clean test to fail because of this "bug","

I'm sorry. I didn't say anything about running the example.

mvn test -Djboss.home=.../path/to/Wildfly

Without -Djboss.home, I get that same WFLYLNCHR0001 message.

[Former user]

So, now I have to decide what to do. I'm thinking it would be safe to pass in ContextClassLoader. In Wildfly it would be the WAR classloader, and in a standalone environment, ContextClassLoader and Yaml.getClass().getClassLoader() are likely to be the same. Does that make sense?

[Alexander Maslov]

Actually I think there is no easy fix for your problem.

And it all goes back to AS7 Modules ClassLoading and Implicit module dependencies for deployments...

Even if we start using ThreadContextClassloader, reasteasy-yaml project will fail.

I guess you can dump/load in your code all those YamlProviderObjects without any problem event with current version (try to load back the dump in the test). It's because Yaml instance you are using is loaded via YourWARModule Classloader.

But when you do HTTP/POST, WildFly uses Yaml loaded from "org.yaml.snakeyaml" module (via chain of resteasy-* modules dependencies...). That module in WildFly 8.1.0 has snakeyaml-1.13.jar. 1.13 does not use ThreadContextClassLoader - so it will not find classes from YourWARModule.

But even when you put ContextClassLoader-patched version of snakeyaml in WEB-INF/lib it will not help, because "org.snakeyaml.module" jars (with version 1.13) are be used by WildFly's RestEasy jars.

The easiest way (but I hate it because you need to change files outside of your WAR) is to update "org.yaml.snakeyaml" module in WildFly installation modules/system/layers/base/org/yaml/snakeyaml/main Either by changing xml+jar or simply copying new jar instead of old one (keeping old name).

There is another jboss-deployment-structure.xml way. But you need to override all RestEasy modules and provide all needed classes in your war (Pros of this one - you can use latest resteasy and snakeyaml). Unfortunately I didn't find the way just to override resteasy-yaml-provider or org.yaml.snakeyaml most probably because of javaee.api->javax.ws.rs.api->resteasy-* implicit dependencies chain in WildFly.

[Former user]

re: "The easiest way (but I hate it because you need to change files outside of your WAR) is to update "org.yaml.snakeyaml" module in WildFly installation modules/system/layers/base/org/yaml/snakeyaml/main Either by changing xml+jar or simply copying new jar instead of old one (keeping old name)."

Actually, we have a nice way of doing that. The Resteasy distribution ships with a file called resteasy-jboss-modules-wf8-<version>.zip which includes the current Resteasy jars plus a few non-Resteasy jars that we need to upgrade. For example, Wildfly 8.2.1.Final ships with org.bouncycastle:bcpkix-jdk15on-1.50.jar, but resteasy-jboss-modules-3.0.13.Final.zip includes org.bouncycastle:bcpkix-jdk15on-1.52.jar and updates module.xml to

#!
    <module xmlns="urn:jboss:module:1.1" name="org.bouncycastle">
        <resources>
              <resource-root path="bcprov-jdk15on-1.52.jar"/>
              <resource-root path="bcmail-jdk15on-1.52.jar"/>
              <resource-root path="bcpkix-jdk15on-1.52.jar"/>
              ...

So when you unzip resteasy-jboss-modules-3.0.13.Final.zip into the modules directory, those dependencies get overridden.

The most direct solution for me would be to update resteasy-yaml-provider so that it passes the context classloader into snakeyaml. I can get that into the next Resteasy release, and resteasy-jboss-modules-3.0.14.Final.zip will include an updated org/jboss/resteasy/resteasy-yaml-provider/main/resteasy-yaml-provider-3.0.14.Final.jar.

On the other hand, if you guys update snakeyaml to use the context classloader, I could just upgrade to a new version of snakeyaml.

Are we agreed, though, that using the context classloader is a reliable solution? It's working for me in wildfly and in standalone tests. Am I missing anything? I should try it with Jetty.

[Former user]

Ok, the updated resteasy-yaml-provider is working with Jetty as well.

[Andrey Somov]

Do you have enough information to continue ? Can we close the issue ?

[Alexander Maslov]

@asomov , I think we should use the patch from the first post. But I don't really know how we work when classloader is provided. I need to check the code. Let's keep this open for sometime, but I think we will take this patch in:

protected Class<?> getClassForName(String name) throws ClassNotFoundException {
     ClassLoader cl = Thread.currentThread().getContextClassLoader();
     Class<?> clazz = Class.forName(tag, true, cl);
     if (clazz != null) {
         return clazz;
     }
     return Class.forName(name);
}

Maybe same test with totally independent ClassLoaders would work great (if we don't have one already :)

[Alexander Maslov]

Finally I have invented test which fails without this change. Hope to get it in shape :) on weekend and push this change to the repo.

[Alexander Maslov]

So now we first try for ContextClassLoader. Let's see if it causes any troubles.

[Former user]

Thank you all for fixing this. We will update the dependency in Resteasy when you do the next release. Will that be snakeyaml 1.17? Do you know when that will be available?

[Andrey Somov]

Yes, it will be part of version 1.17: https://bitbucket.org/asomov/snakeyaml/wiki/Changes

[Former user]

Thanks, Andrey.

[Andrey Somov]

I have not done anything. Thanks to Alex. He is going to release version 1.17 very soon.

[Former user]

Ok, well, thanks to everyone. ;-)