Is this an instance of class leakage that was mentioned in a
previous
post?
Yes, it is. Welcome to the jungle :/
In my reading of LinkageError it sounds like both the bootloader
and the pluginclassloader have loaded JAXBContext and because of that
the signatures don't match.
It sounds like you have a bead on what the problem is. I'll explain what I think
happened for the benefit of others. Based on the unusually specific and helpful exception
message:
1) the plugin classloader loaded
"com.sun.xml.internal.ws.client.WSServiceDelegate" (which is what it sounds like
you fixed from before - this class is probably now in some library you packaged in your
plugin)
2) from looking at what I think is the source for that class, it extends
com.sun.xml.ws.api.WSService which in turn extends javax.xml.ws.spi.ServiceDelegate.
3) One or both of those superclasses was loaded by the bootclassloader (the classloader at
the very very top of the classloader hierachy) - this is the "leakage" I was
saying was a possibility.
4) Both of the superclasses import and use javax.xml.bind.JAXBContext; thus, the
bootloader also loaded this JAXBContext class (take special note - this is a JAXB API!)
5) WSServiceDelegate overrides that createDispatch method - but its loaded by the plugin
classloader, which means that method needed to load JAXBContext and got the plugin
classloader to load it too
6) boom - you have one method overriding another, but the signatures don't
"match" because the two methods reference two different classes (the names might
be the same, but they are really two different classes because they have two different
loaders)
This is one way leaking classes can cause problems - this is what I was talking about when
I mentioned to be on the lookout for this. Essentially, JAXBContext leaked from the
bootstrap classloader and tried to mix itself with classes loaded by the plugin
classloader.
The other possibility is that the plugin
was build using a newer version of jaxb than is present in the jdk.
I don't think that's it. The exception seems pretty clear that its this class
problem.
As soon as I noticed the leaked class in question was a JAXB class, I asked myself,
"How is your agent getting the JAXB classes into the bootstrap classloader I
wonder?". I immediately went and looked at how we are deploying the JAXB libraries in
the agent. We may have a problem here. But it may be easy to fix.
I'll claim ignorance to this Web Services API and all its dependencies - I've not
used that stuff in years. But I believe this javax.xml.ws stuff (and its com.sun.xml.ws
impl classes) are built into the JRE. However, JAXB was NOT shipping with the JRE when we
were building/running on JRE 5. IT IS SHIPPING NOW WITH JDK6. I will therefore assume you
are running the agent on JRE 6.
We actually put our own JAXB API in the agent/lib/endorsed directory because we needed
JAXB and we needed to support JRE 5 (which didn't ship with JAXB). We install those
JAXB libraries as endorsed libraries via -Djava.endorsed.dirs (see rhq-agent.sh for where
we do that). In our root classloader regex, we hide those JAXB classes because we
didn't want to clash with other plugins that also want to use their own external JAXB
impl. This should allow other plugins to ship their own JAXB libraries and be able to use
those (again, because back on JRE5 we didn't have the JRE to supply them to us).
But when you load the Web Services class, they come from the bootstrap classloader - and
when their signatures reference that JAXBContext API, its ALSO comes out of the bootstrap
classloader because that is now in the the built-in JRE libs! This is probably causing the
problem. But we shouldn't be doing this "JAXB installed as an endorsed
library" now anymore. We are moving to JRE 6, and since it has JAXB built-in we
should not be shipping these JAXB libraries and installing them ourselves (which was only
necessary for JRE5 runtimes)
How can you solve this? First, don't have the agent pass in that endorsed dir sysprop.
you can do this by:
export RHQ_AGENT_JAVA_ENDORSED_DIRS=/bogus
rhq-agent.sh will use that instead of its own endorsed default and thus its JAXB libraries
won't load - it'll fall back to the JRE's built-in JAXB API (and let's
cross our fingers we stuff will still work, we were using JAXB 2.1, unsure how compatible
that is with the built-in JRE version of JAXB).
Alternatively, if you don't want to set that env var, you can just delete the
libraries that you have in agent/lib/endorsed (which I believe are all related to JAXB).
Since the JRE 6 should already have them, you won't miss them.
Now we have to tell the agent to no longer hide those javax.xml.bind and related JAXB
classes. You have to change the root classloader regex - you need to take the default
regex and strip out the JAXB phrases. From the code, those JAXB related regex phrases
are:
1200 // Hide our version of JAXB.
1201 // If the plugins want these, they should include their own implementations.
1202 defaultRegex.append("(javax\\.xml\\.bind\\..*)|");
1203 defaultRegex.append("(com\\.sun\\.activation\\..*)|");
1204 defaultRegex.append("(com\\.sun\\.istack\\..*)|");
1205 defaultRegex.append("(com\\.sun\\.xml\\..*)|");
For the record, those line numbers/code come from:
http://git.fedorahosted.org/git/?p=rhq/rhq.git;a=blob;f=modules/enterpris...
That's the class that defines the default regex (see getPluginContainerConfiguration).
you can either tweek that class and rebuild the agent jar or set the regex preference with
the tweeked regex in agent-configuration.xml and restart the agent clean (--cleanconfig)
to pick up the new regex.
Let us know if that works. If so, we can make those changes to the agent (i.e. no longer
ship with the jaxb libs) and the changes to the default regex and commit those changes to
the master branch.
----- "Bala Nair" <bnairtm(a)comcast.net> wrote:
Ok, we are officially in classloader hell. Our original problem was
a
class not found exception when trying to use a class defined in an
external jar in one of our agent plugins. After some good advice and
an explanation of how resource classloaders work, we decided to do the
following.
1. declare an instance classloader for our plugin's server class (in
the rhq-plugin.xml, server element, classloader="instance" attribute)
2. implement the classloaderfacet in the server discovery component
3. add the relevant jars to the classpath for the server resource
component
So now we no longer see the class not found exception. instead we get
this:
java.util.concurrent.ExecutionException: java.lang.Exception:
java.lang.LinkageError: loader constraint violation: when resolving
overridden method
"com.sun.xml.internal.ws.client.WSServiceDelegate.createDispatch(Ljavax/xml/namespace/QName;Ljavax/xml/bind/JAXBContext;Ljavax/xml/ws/Service$Mode;)Ljavax/xml/ws/Dispatch;"
the class loader (instance of
org/rhq/core/pc/plugin/PluginClassLoader) of the current class,
com/sun/xml/internal/ws/client/WSServiceDelegate, and its superclass
loader (instance of <bootloader>), have different Class objects for
the type javax/xml/bind/JAXBContext used in the signature
Is this an instance of class leakage that was mentioned in a previous
post? In my reading of LinkageError it sounds like both the bootloader
and the pluginclassloader have loaded JAXBContext and because of that
the signatures don't match. The other possibility is that the plugin
was build using a newer version of jaxb than is present in the jdk.
Question: are the steps we took above correct to implement the
classloaderfacet.
Bala Nair
_______________________________________________
rhq-devel mailing list
rhq-devel(a)lists.fedorahosted.org
https://fedorahosted.org/mailman/listinfo/rhq-devel