Scalability issues in RHQ 4.9 - A summary

James Strachan jstracha at redhat.com
Mon Mar 10 09:20:41 UTC 2014


On 8 Mar 2014, at 12:10, Heiko W.Rupp <hrupp at redhat.com> wrote:
> Am 08.03.2014 um 02:59 schrieb Elias Ross <genman at noderunner.net>:
>> 
>> I wouldn't mind working on some Cassandra features either. I could
>> work on moving events to Cassandra, but I'm not sure how interested I
>> am in dealing with the data migration steps or whatnot.
> 
> 
> As you probably know is Fuse Fabric8 also part of the Red Hat family and
> we are checking how we can work together so that they can use parts of
> what we have an vice versa.
> Fabric8 is (iirc) using ELK (Elasticsearch, Logstash Kibana)

FWIW we’re just using EK in fabric8 ;) We’re not using logstash right now; mostly as inside a fabric8 JVM we know what each log event looks like and all its fields; so we just log it directly to ElasticSearch. i.e. we don’t have to parse a log file and use some brittle regular expressions configuration to try guess which bit of text is the category, which the timestamp, which the thread name & which the message etc. Given users can configure the logging configuration and layout at any point; we prefer to not have to rely on brittle regex configurations; so prefer to write directly into ElasticSearch from the JVM. Though logstash is cool and for non-Java systems (like syslogs etc) is very useful.

In fabric8 we can also log camel messages to ElasticSearch; for generating audit logs of message flows and their payloads which we can then use to correlate messages and to reverse engineer sequence flows and the like. 
https://twitter.com/cmoulliard/status/439410019511963649/photo/1/large
Using MDC / NDC type stuff the same thing could be done with logs/events too really.

Kibana and ElasticSearch are great open source technologies BTW - plus they are lots cheaper than buying Splunk ;)

James
-------
Red Hat

Email: jstracha at redhat.com
Web: http://fusesource.com
Twitter: jstrachan, fusenews
Blog: http://macstrac.blogspot.com/

Open Source Integration



More information about the rhq-devel mailing list