RHQ AS7 plugin with httpclient (important if you rely on this plugin for your development)
Thomas Segismont
tsegismo at redhat.com
Mon Jan 21 13:42:18 UTC 2013
Hi Michael,
Please find my answers below.
Le 21/01/2013 14:21, Michael Foley a écrit :
> Just to get meta on this ....
>
> I am curious:
>
> * How was this memory leak detected? was it just the error message?
> did the error appear instantly....or only after a period of time?
I found it when trying the child resource creation on AS 7 instance. The
error message indicates OutOfMemoryError but it is shown in the server
log even if the error actually occurred on the agent. It's always
reproducible if you upload a non trivial WAR/EAR (say around 40MB).
> * How do you know it is fixed? The error goes away? Or was there
> some heap analysis with a memory analysis tool?
I'm sure it's fixed because it's a reproducible bug, not an error which
occurs randomly after some time or some running conditions.
> * Do you think this error ...and others like it ... can be detected
> with a static analysis tool such as Coverity?
>
I don't know Coverity so I can't say for sure. But I doubt this one
would have been detected. That doesn't mean a static analysis tool
wouldn't be useful of course :)
>
> Regards,
>
> Michael Foley
> QE Lead, JBoss Operations Network
>
> ------------------------------------------------------------------------
> *From: *"Thomas Segismont" <tsegismo at redhat.com>
> *To: *rhq-devel at lists.fedorahosted.org
> *Sent: *Monday, January 21, 2013 8:16:21 AM
> *Subject: *RHQ AS7 plugin with httpclient (important if you rely on this
> plugin for your development)
>
> Hi all,
>
> Following my email on BZ 887320 ( Agent raises OutOfMemoryError when
> adding new Deployment child resource on AS7 ).
>
> A fix has been made and the AS7 plugin in master branch now uses commons
> httpclient to upload content.
>
> All httpclient code is encapsulated in the ASUploadConnection so if
> you're a simple user of the class your code should not be broken (but
> please read the new documentation as some methods are now deprecated).
>
> Some package protected fields which are no longer relevant have been
> removed but this should not impact your code (unless you extended the
> class in same package).
>
> This change was very important as RHQ agents generally run with less
> resources than other Java programs (128M max heap) and we cannot rely on
> a tool which holds full EARs in memory.
>
> If you have any concern with this changeset please email me.
>
> Thanks and regards
> Thomas Segismont
>
> Le 18/12/2012 15:13, Thomas Segismont a écrit :
> > Hi all,
> >
> > I'm writing to you about this bug:
> > https://bugzilla.redhat.com/show_bug.cgi?id=887320
> >
> > For those not aware of it, an OutOfMemoryError is raised on the agent
> > when you try to add a child resource of type deployment to a managed AS7
> > instance.
> >
> > It happens because the AS7 plugin uses HttpUrlConnection to communicate
> > with AS7 HTTP management interface.
> >
> > Basically, HttpUrlConnection waits for the request payload to be fully
> > loaded in memory before it tries to contact the server. It does that to
> > be able to resend the request payload if the server ask for redirection
> > or start an authentication challenge.
> >
> > While this is acceptable for small payloads, it's not suitable for large
> > file uploads (in our case a 42M WAR upload triggers the
> OutOfMemoryError).
> >
> > For large file uploads, HttpUrlConnection provides a streaming mode. But
> > in streaming mode it does neither support redirection nor
> > authentication. I tried to use streaming mode to make a fix yesterday
> > and it's really not working (BTW, sorry for the patch revert waltz).
> >
> > I can see two options:
> >
> > #1 Keep HttpUrlConnection as communication layer
> > In this case, the streaming mode is mandatory. This means we should
> > maintain our own authentication challenge code and set authentication
> > headers manually.
> > #2 Use HttpClient as communication layer
> >
> > Personally, I prefer #2
> >
> > Importing HttpClient in AS7 plugin will increase the plugin dependencies
> > list. But I think it's a small price to pay compared to the efforts we
> > would spend maintaining our own authentication and redirection code.
> >
> > What do you think?
> >
> > Thanks
> > Thomas
> >
>
> _______________________________________________
> rhq-devel mailing list
> rhq-devel at lists.fedorahosted.org
> https://lists.fedorahosted.org/mailman/listinfo/rhq-devel
>
>
>
> _______________________________________________
> rhq-devel mailing list
> rhq-devel at lists.fedorahosted.org
> https://lists.fedorahosted.org/mailman/listinfo/rhq-devel
>
Cheers
Thomas
More information about the rhq-devel
mailing list