Hello Everybody,
I had a lengthy discussion yesterday with Ian about a good strategy to resolve this issue.
Here is the plan:
1) Ian will update the mc4j library to have a new method to allow a reload of the bean
information from the mbean server. Currently the bean information is requested only when
the mbean is initialized (the information is then cached) and there is no way to request a
reload afterwards.
The new method will have the same name as the original but one additional param. Here is
some pseudo-code for this method:
void loadSynchronous(boolean reload) {
if( reload == true) {
this.loaded = false;
}
this.loadSynchronous();
}
As you can see, the implementation is very trivial. It will force the mbean to refresh the
cached information if the reload flag is true. With this minimal change, there is no
impact on any existing components that use mc4j. The only drawback to this new method is
that it could be inefficient if called too often since it will transfer over the wire the
entire bean information every time. However, this should be mitigated in the components
that call it by having good use strategies.
2) I will then use this new method in the mod_cluster component during discovery and
availability checks. The method will be called every time the discovery process takes
place to make sure that the correct components are discovered. However, to improve the
performance, this method will be called only once every 15 minutes on availability checks.
The for the rest of the time, the only regular JMX availability will be used.
Mike,
I am not sure how often users would want to change between regular and HA services for
mod_cluster. However, when they do make these kinds of changes (and if they go back and
forth a couple of times for a short period of time) the resources will have incorrect
discovery and availability until one of the two events I mentioned below takes places. The
un-inventory and re-inventory of EAP servers is a solid workaround, but it is neither
efficient nor convenient.
As you can see the code to reload the mbean information is minimal. The discovery process
of these mod_cluster services will require just one extra line of code (the call to this
new method). The availability check will require just a couple of extra lines, to take
into account the 15 minutes intervals between two calls to this new method.
Thank you,
Stefan Negrea
Software Engineer
----- Original Message -----
From: "Michael Foley" <mfoley(a)redhat.com>
To: rhq-devel(a)lists.fedorahosted.org
Sent: Monday, August 29, 2011 3:54:38 PM
Subject: Re: JMX and Class Names
Is changing the mod-cluster configuration with <inject bean=....> a
primary use-case? Or an edge case?
If it is a real edge case ... would this reload issue go away with a
re-inventory of the EAP resource? If this is a risky fix ... is "no
action" a valid course of action here?
Just asking ...
From: "Ian Springer" <ian.springer(a)redhat.com>
To: rhq-devel(a)lists.fedorahosted.org
Sent: Monday, August 29, 2011 4:21:28 PM
Subject: Re: JMX and Class Names
It looks like DMBean caches the MBeanInfo the first time it loads the
MBean metadata but doesn't provide a way to reload the MBeanInfo. It
would not be hard to add a reloadSynchronous method to EmsBean/DMBean
that reloads all metadata. Would this suffice? If so, when would you
call the method - whenever discovery for mod_cluster Resources runs,
or
is there some way for you to detect if the microcontainer has been
restarted and only do the reload then?
If this is something you really need, I can make the change to EMS and
tag and publish a new version of EMS that you can use. Let me know.
-Ian
On 08/29/2011 12:57 PM, Stefan Negrea wrote:
> Hello Everybody,
>
> I think an e-mail is much better to capture and explain the JMX
> problem I am running into.
>
> mod_cluster is configured through a bean configuration file.
>
> Here is the configuration for the top service:
> 1<bean
> class="org.jboss.modcluster.catalina.CatalinaEventHandlerAdapter"
> name="ModClusterListener">
> 2<constructor>
> 3<parameter class="org.jboss.modcluster.ContainerEventHandler">
> 4<inject bean="HAModClusterService"/>
> 5</parameter>
> 6<parameter class="javax.management.MBeanServer">
> 7<inject bean="JMXKernel" property="mbeanServer"/>
> 8</parameter>
> 9</constructor>
> 10</bean>
>
> The default configuration is set with HA version of the mod_cluster
> service. To change it to the regular version, one just need to
> remove the HA moniker for the injected bean on line 4.
> 4<inject bean="ModClusterService"/>
>
> Here is a snippet of configuration for the regular bean:
> <bean class="org.jboss.modcluster.ModClusterService" mode="On
> Demand" name="ModClusterService">
>
<annotation>@org.jboss.aop.microcontainer.aspects.jmx.JMX(name="jboss.web:service=ModCluster",exposedInterface=org.jboss.modcluster.ModClusterServiceMBean.class)</annotation>
>
> And here is snippet of configuration for the HA bean:
> <bean class="org.jboss.modcluster.ha.HAModClusterService"
mode="On
> Demand" name="HAModClusterService">
>
<annotation>@org.jboss.aop.microcontainer.aspects.jmx.JMX(name="jboss.web:service=ModCluster",exposedInterface=org.jboss.modcluster.ha.HAModClusterServiceMBean.class)</annotation>
>
> Both of them use the same JMX name, jboss.web:service=ModCluster, so
> the only way to differentiate between them the two is the class
> name.
>
> The current JMX RHQ plugin does discovery only based on the service
> name. So I had to write a custom discovery method that first runs
> the regular JMX discovery and then checks the class name of each
> bean discovered. If the className attribute of the service does not
> match emsBean.getClassTypeName(), then the bean is discarded from
> the collection of services discovered by original JMX procedure
> (link to the class name based discovery component:
>
http://bit.ly/mYyL9x). Here is snippet of code:
>
> 1 String className =
> context.getDefaultPluginConfiguration().getSimple(CLASS_NAME).getStringValue();
> 2 return className.equals(emsBean.getClassTypeName());
>
> I wrote a similar implementation for the availability of these
> services; check first if the bean is available and then check if the
> class of the bean is the one expected.
>
> And here is where the problem starts. At any point the user can
> change from using the regular service to using HA service (and
> vice-versa). As soon as the change is saved to file, the
> microcontainer reloads the beans and the new resources are
> available. The current JMX RHQ implementation does not return the
> class name changes after the microcontainer reloads. So, if the HA
> service was originally loaded but swapped with regular service, the
> microcontainer will report correctly the new class name. However,
> the JMX RHQ code will continue to report the old class name until
> the RHQ agent is restarted or the JBoss server is un-inventoried and
> inventoried (basically disconnect completely from the JMX provider
> and reconnect).
>
> I had a couple of attempts to fix, the most notable ones are:
> * Refresh the bean attributes before getting the class name:
> 1. emsBean.refreshAttributes();
> 2. emsBean.getClassTypeName();
>
> * Refresh the connection provider before getting the class name:
> 1.emsConnection.refresh()
> 2.emsBean.getClassTypeName();
>
>
> But none of these approaches worked. The class name was not updated
> until the JBoss server was re-inventoried or the agent restarted. I
> also checked the source code for mc4j but I cannot see anything that
> could help.
>
> Does anybody know how I can do a full reload of the JMX connection
> to pick such changes? If there is a way, how costly it is in terms
> of performance (this would be used not only in the discovery but
> also to check the availability the services)? Any other ideas of
> what I can do?
>
>
> Thank you,
> Stefan Negrea
>
> Software Engineer
> _______________________________________________
> rhq-devel mailing list
> rhq-devel(a)lists.fedorahosted.org
>
https://fedorahosted.org/mailman/listinfo/rhq-devel
--
Ian Springer
Principal Software Developer
JBoss Operations Network
Red Hat
ian.springer(a)redhat.com
_______________________________________________
rhq-devel mailing list
rhq-devel(a)lists.fedorahosted.org
https://fedorahosted.org/mailman/listinfo/rhq-devel
_______________________________________________
rhq-devel mailing list
rhq-devel(a)lists.fedorahosted.org
https://fedorahosted.org/mailman/listinfo/rhq-devel