have found the following:
1. I compiled corosync and libqb without '-pthread'
2. thread stack size on Solaris:
platform 32 bit = 1 MB
platform 64 bit = 2 MB
The second case is related to corosync, because it
defines IPC_DISPATCH_SIZE = 1 MB without '--enable-small-memory-footprint'.
Thanks.
2013/4/2 eXeC001er <execooler(a)gmail.com>
2013/4/2 Angus Salkeld <asalkeld(a)redhat.com>
> On 02/04/13 10:57 +0400, eXeC001er wrote:
>
>> 2013/4/2 Angus Salkeld <asalkeld(a)redhat.com>
>>
>> On 29/03/13 01:13 +0400, eXeC001er wrote:
>>>
>>> Hello.
>>>>
>>>> This Q is crosspost from openais-maillist.
>>>>
>>>> I use corosync-2.3.0, libqb-0.14.4 on OpenIndiana:
>>>>
>>>> I tried to create an application that uses corosync via its libraries,
>>>> but
>>>> it seems something wrong.
>>>> On the machine where i debug corosync-tests work fine, but my
>>>> application
>>>> does not work. The application cannot intialize CMAP connection (same
>>>> with
>>>> VOTEQUORUM).
>>>> "*_initialize" function returned CS_ERR_NOT_EXIST error. on
server
>>>> side i
>>>> see "Error in connection setup (7104-7367-0): Broken pipe
(32)".
>>>>
>>>> My appliacation is multi-thread and "*_initialize" is called
not in
>>>> main
>>>> thread.
>>>>
>>>>
>>> Well, I suggest you keep it simple initially.
>>> Do the standard corosync apps work ok?
>>> like:
>>> ./tests/testcpg
>>>
>>>
>> yes. all tests from test directory of corosync sources work fine.
>>
>
> Well then there is no reason why other apps should not work.
> Just have a look at how they are written.
>
Do not forget that tests are single-thread app.
My app is multi-thread and initialization not in main-thread. if i do
initialization in main-thread then everything fine.
>
>
>>
>>
>>> It would be good to try narrow down what is failing.
>>>
>>>
>>>
>>> the stack-trace:
>>>>
>>>> /3@3: -> libvotequorum:votequorum_****initialize(0x8834b2c, 0x0,
>>>> 0x0,
>>>>
>>>> 0xfea63855)
>>>> /2: nanosleep(0xFE99BF48, 0x00000000) = 0
>>>> /3@3: -> libqb:qb_hdb_handle_create()
>>>> /3@3: <- libqb:qb_hdb_handle_create() = 0
>>>> /3@3: -> libcorosync_common:hdb_error_****to_cs()
>>>> /3@3: <- libcorosync_common:hdb_error_****to_cs() = 1
>>>>
>>>> /3@3: -> libqb:qb_hdb_handle_get()
>>>> /3@3: <- libqb:qb_hdb_handle_get() = 0
>>>> /3@3: -> libcorosync_common:hdb_error_****to_cs()
>>>> /3@3: <- libcorosync_common:hdb_error_****to_cs() = 1
>>>>
>>>> /3@3: -> libqb:qb_ipcc_connect()
>>>> /3: so_socket(PF_UNIX, SOCK_STREAM, 0, 0x00000000, SOV_DEFAULT) = 8
>>>> /3: fcntl(8, F_GETFD, 0x00000000) = 0
>>>> /3: fcntl(8, F_SETFD, 0x00000001) = 0
>>>> /3: fcntl(8, F_SETFL, FNONBLOCK) = 0
>>>> /3: connect(8, 0xFE67C13A, 21, SOV_DEFAULT) = 0
>>>> /3: sigaction(SIGPIPE, 0xFE67C0E0, 0xFE67C140) = 0
>>>> /3: send(8, "FFFFFFFF\0\0\0\018\0\0\0".., 24, 0) = 24
>>>> /3: sigaction(SIGPIPE, 0xFE67C0E0, 0x00000000) = 0
>>>> /3: sigaction(SIGPIPE, 0xFE67C0D0, 0xFE67C130) = 0
>>>> /3: recv(8, 0xFE67C210, 3112, 64) Err#11 EAGAIN
>>>>
>>>>
>>> this ^ is the client returngin EAGAIN on the socket...
>>> the problem is probably on the server.
>>>
>>
>>
>> on server side i also see strange case:
>>
>> if i try to run a corosync-* tool, then the tool returned required info,
>> but on server side i see "File not found" error. I see that the
corosync
>> daemon tries to do something with /proc/<client_pid> file:
>>
>> an example:
>>
>> ......... [QB ] recv from client connection failed (10656-12155-22):
>> No
>> such file or directory (2)
>>
>>
> I don't think this is a problem, oindiana/solaris sockets behave a bit
> differently to linux ones and the path to finding a disconnection are
> just a bit different. I think this is just using the wrong errno.
>
> The "/proc/<pid>/" access is also fine, it tries on each connection
> to resolve the process name, and if it can't just uses the pid.
>
>
>
>>
>>>
>>> /3: sigaction(SIGPIPE, 0xFE67C0D0, 0x00000000) = 0
>>>
>>>> /3: shutdown(8, SHUT_RDWR, SOV_DEFAULT) = 0
>>>> /3: close(8) = 0
>>>> /3@3: <- libqb:qb_ipcc_connect() = 0
>>>> /3@3: -> libcorosync_common:qb_to_cs_****error()
>>>> /3@3: <- libcorosync_common:qb_to_cs_****error() = 12
>>>> /3@3: -> libqb:qb_hdb_handle_put(****0xfe9f3070, 0x0, 0xb86775,
>>>>
>>>> 0xfe67ce7c)
>>>> /3@3: <- libqb:qb_hdb_handle_put() = 0
>>>> /3@3: -> libqb:qb_hdb_handle_destroy(****0xfe9f3070, 0x0,
>>>> 0xb86775,
>>>>
>>>> 0x0)
>>>> /3@3: -> libqb:qb_ipcc_disconnect(0x0, 0x8835b88, 0x0, 0x1)
>>>> /3@3: <- libqb:qb_ipcc_disconnect() = 0
>>>> /3@3: <- libqb:qb_hdb_handle_destroy() = 0
>>>> /3@3: <- libvotequorum:votequorum_****initialize() = 12
>>>>
>>>>
>>>> As you can see it returns CS_ERR_NOT_EXIST.
>>>>
>>>> same behaviour with CMAP and another services.
>>>>
>>>> Any ideas?
>>>>
>>>>
>>> Can you turn on debug in corosync.conf and to_stderr
>>> logging {
>>> to_stderr: yes
>>> debug: on
>>> }
>>> and run corosync like this:
>>> "sudo corosync -f"
>>>
>>> then in another window run:
>>> ./tests/testcpg
>>>
>>> then type a couple of words (any junk) - testcpg should send what ever
>>> is typed to corosync. Then Ctrl-C to end.
>>>
>>> Can you send the contents of both shells so I can see what both the
>>> client and server are doing?
>>>
>>>
>> CLIENT:
>>
>> $ sudo ./testcpg
>> Type EXIT to finish
>> Local node id is ac12fe0a
>> membership list
>> nza4-devel:20446 2013-04-02 10:43:38.902
>> ConfchgCallback: group 'GROUP'
>> Local node id is 10.254.18.172/ac12fe0a result 1
>> joined node/pid -1408041462/20446 reason: 1
>> nodes in group now 1
>> node/pid -1408041462/20446
>>
>> nza4-devel:20446 2013-04-02 10:43:38.904
>> TotemConfchgCallback: ringid (2886925834.156)
>> active processors 1: -1408041462
>> param param param param
>> nza4-devel:20446 2013-04-02 10:44:07.682
>> DeliverCallback: message (len=25)from node/pid -1408041462/20446: 'param
>> param param param
>> '
>>
>>
>> SERVER:
>>
>> see attached file or
http://pastie.org/7276133
>>
>>
> I do have access to a oindiana machine if you have a test app that
> fails can you send it?
will try later.
>
> -Angus
>
>
>>
>>> -Angus
>>>
>>> Thanks.
>>>
>>>>
>>>>
>>> ______________________________****_________________
>>>
>>>> quarterback-devel mailing list
>>>> quarterback-devel(a)lists.**fedo**rahosted.org
<
http://fedorahosted.org>
>>>>
<quarterback-**devel@lists.fedorahosted.org<quarterback-devel(a)lists.fedorahosted.org>
>>>> >
>>>>
https://lists.fedorahosted.****org/mailman/listinfo/****
>>>>
quarterback-devel<https://**lists.fedorahosted.org/**
>>>>
mailman/listinfo/quarterback-**devel<https://lists.fedorahosted.org/ma...
>>>> >
>>>>
>>>>
>>> ______________________________****_________________
>>> quarterback-devel mailing list
>>> quarterback-devel(a)lists.**fedo**rahosted.org
<
http://fedorahosted.org><
>>>
quarterback-**devel@lists.fedorahosted.org<quarterback-devel(a)lists.fedorahosted.org>
>>> >
>>>
https://lists.fedorahosted.****org/mailman/listinfo/****
>>>
quarterback-devel<https://**lists.fedorahosted.org/**
>>>
mailman/listinfo/quarterback-**devel<https://lists.fedorahosted.org/ma...
>>> >
>>>
>>>
>
> ______________________________**_________________
>> quarterback-devel mailing list
>>
quarterback-devel@lists.**fedorahosted.org<quarterback-devel(a)lists.fedorahosted.org>
>>
https://lists.fedorahosted.**org/mailman/listinfo/**quarterback-devel<...
>>
>
> ______________________________**_________________
> quarterback-devel mailing list
>
quarterback-devel@lists.**fedorahosted.org<quarterback-devel(a)lists.fedorahosted.org>
>
https://lists.fedorahosted.**org/mailman/listinfo/**quarterback-devel<...
>