Kernel 4.2.5-1 currently on initiator, was the same on target, just
upgraded target to 4.3.3-2 and no change. This has been happening
since I started with InfiniBand, back about 5 months ago.
Just upgraded from ConnectX to ConnectX-2 cards, and seeing the same thing.
On the initiator, /sys/class/infiniband/mlx4_0/ports/1/gids/0 (same as
/sys/devices/pci0000:00/0000:00:03.0/0000:06:00.0/infiniband/mlx4_0/ports/1/gids)
is:
fe80:0000:0000:0000:0002:c903:004b:5275
Different last 6 digits because it's a different card, but still
showing the fe80 prefix it won't let me use.
On Mon, Jan 4, 2016 at 10:04 AM, Bart Van Assche
<bart.vanassche(a)sandisk.com> wrote:
> On 01/01/2016 04:04 AM, jamespharvey20(a)gmail.com wrote:
>>
>> I'm at a loss on how to get the hex address to go underneath acls when
>> setting up srpt. If I use the fe80 prefix I see everywhere including /sys,
>> it gets rejected. If I use the prefix shown in the kernel rejection method,
>> which I can't find anywhere else, it works fine.
>>
>> targetcli configuration:
>>
>> /> ls
>> o- /
>>
.........................................................................................................................
>> [...]
>> o- backstores
>>
..............................................................................................................
>> [...]
>> | o- block
>>
..................................................................................................
>> [Storage Objects: 3]
>> | | o- kvm1
>> ....................................................................
>> [/dev/disk1/kvm1 (100.0GiB) write-thru activated]
>> | | o- kvm2
>> ....................................................................
>> [/dev/disk2/kvm2 (100.0GiB) write-thru activated]
>> | | o- kvm3
>> ....................................................................
>> [/dev/disk3/kvm3 (100.0GiB) write-thru activated]
>> | o- fileio
>>
.................................................................................................
>> [Storage Objects: 0]
>> | o- pscsi
>>
..................................................................................................
>> [Storage Objects: 0]
>> | o- ramdisk
>>
................................................................................................
>> [Storage Objects: 0]
>> o- iscsi
>>
............................................................................................................
>> [Targets: 0]
>> o- loopback
>>
.........................................................................................................
>> [Targets: 0]
>> o- sbp
>>
..............................................................................................................
>> [Targets: 0]
>> o- srpt
>>
.............................................................................................................
>> [Targets: 1]
>> | o- ib.fe800000000000000002c90300001679
>> ...........................................................................
>> [no-gen-acls]
>> | o- acls
>>
............................................................................................................
>> [ACLs: 1]
>> | | o- ib.fe800000000000000002c90300001e79
>> .................................................................... [Mapped
>> LUNs: 3]
>> | | o- mapped_lun0
>>
....................................................................................
>> [lun0 block/kvm1 (rw)]
>> | | o- mapped_lun1
>>
....................................................................................
>> [lun1 block/kvm2 (rw)]
>> | | o- mapped_lun2
>>
....................................................................................
>> [lun2 block/kvm3 (rw)]
>> | o- luns
>>
............................................................................................................
>> [LUNs: 3]
>> | o- lun0
>>
.....................................................................................
>> [block/kvm1 (/dev/disk1/kvm1)]
>> | o- lun1
>>
.....................................................................................
>> [block/kvm2 (/dev/disk2/kvm2)]
>> | o- lun2
>>
.....................................................................................
>> [block/kvm3 (/dev/disk3/kvm3)]
>> o- vhost
>>
............................................................................................................
>> [Targets: 0]
>> o- xen_pvscsi
>>
.......................................................................................................
>> [Targets: 0]
>>
>> NOTE my InfiniBand cards have similar IDs, and only differ by one
>> character -- compare the end, 1679 vs 1e79.
>>
>> On the target (host):
>> $ cat /sys/class/infiniband/mlx4_0/ports/1/gids/0
>> fe80:0000:0000:0000:0002:c903:0000:1679
>> {It's my understanding this should be used underneath srpt, as the wwn}
>>
>> On the initiator (client):
>> $ cat /sys/class/infiniband/mlx4_0/ports/1/gids/0
>> fe80:0000:0000:0000:0002:c903:0000:1e79
>> {It's my understanding this should be used underneath acls, and mapped
>> with the luns}
>> # ibsrpdm -c
>>
>>
id_ext=0002c90300001678,ioc_guid=0002c90300001678,dgid=fe800000000000000002c90300001679,pkey=ffff,service_id=0002c90300001678
>> {so, its /etc/srp_daemon.conf is just comments and the line}
>> a
>>
id_ext=0002c90300001678,ioc_guid=0002c90300001678,dgid=fe800000000000000002c90300001679,pkey=ffff,service_id=0002c90300001678
>>
>> But, after starting service target on the target, and service srpdaemon on
>> the initiator, dmesg on target shows:
>> [ 849.710278] ib_srpt Received SRP_LOGIN_REQ with i_port_id
>> 0x7916000003c90200:0x2c90300001e79, t_port_id
>> 0x2c90300001678:0x2c90300001678 and it_iu_len 260 on port 1
>> (guid=0xfe80000000000000:0x2c90300001679)
>> [ 849.714528] ib_srpt Session : kernel thread ib_srpt_compl (PID 24481)
>> started
>> [ 849.714601] ib_srpt Rejected login because no ACL has been configured
>> yet for initiator 0x7916000003c902000002c90300001e79.
>> [ 849.714613] ib_srpt Session 0x7916000003c902000002c90300001e79: kernel
>> thread ib_srpt_compl (PID 24481) stopped
>>
>> And, dmesg on initiator shows:
>> [ 976.616221] scsi host11: ib_srp: REJ received
>> [ 976.616229] scsi host11: ib_srp: SRP LOGIN from
>> fe80:0000:0000:0000:0002:c903:0000:1e79 to
>> fe80:0000:0000:0000:0002:c903:0000:1679 REJECTED, reason 0x00010001
>> [ 976.616269] scsi host11: ib_srp: Connection 0/4 failed
>> [ 976.616276] scsi host11: ib_srp: Sending CM DREQ failed
>>
>> Why is the target machine, in the srp context only, seeing the ports as
>> starting with 0x7916000003c90200 and 0x2c90300001678, and the client machine
>> seeing the ports as starting with fe80:0000:0000:0000? What are these
>> 0x7916... and 0x2c90... prefixes, and how can I find them besides getting a
>> rejection and looking at the kernel logs? I haven't seen these prefixes
>> anywhere else on either system.
>>
>> In targetcli, if I use acls with 0x7916000003c902000002c90300001e79, then
>> everything works great.
>
>
> Hello James,
>
> Which kernel version are you using at the target side and at the initiator
> side ? When I start the LIO SRP target driver against a recent kernel I use
> a shell script that I wrote myself. I am not familiar with targetcli. The
> shell code I use to obtain target and Linux initiator port names
> respectively is as follows:
> * Target: sed 's/^/0x/;s/://g' \
> /sys/devices/*/*/*/infiniband/*/ports/*/gids/0
> * Initiator(s): sed 's/^fe8/0x000/;s/://g' \
> /sys/devices/*/*/*/infiniband/*/ports/*/gids/0
>
> Bart.