On 05/19/2013 10:10 AM, Craig Watson wrote:
Hi,
I am attempting to set up SAN access to a RAID on a Linux platform using LIO/TMC with a QLogic qle2562 Fiber Channel HBA. I have attempted to follow the example shown at http://www.linux-iscsi.org/wiki/Fibre_Channel#Enable_target_mode and have run into some problems. Details of my issues are after the debug/system info below suggested at http://linux-iscsi.org/wiki/Support. Sorry it is so long but that's what was asked for.
How do I get this to work? It is apparent I'm missing something rather fundamental in the whole setup. I'd really appreciate any suggestions or pointers to documentation I've missed.
Thanks in advance,
Craig Watson
Hi Craig,
I think you're running into
https://github.com/agrover/rtslib-fb/issues/23
Please update to latest targetcli and rtslib: targetcli-2.1.fb25-1.fc18 python-rtslib-2.1.fb34-1.fc18
via 'yum update targetcli --enablerepo=updates-testing'.
(This should also be in the regular 'updates' repo by tomorrow.)
Fedora ships a modified version of these tools, which accounts for the other differences you noted. Please feel free to use the targetcli-fb list (CCd) if you encounter further issues.
Regards -- Andy
On 05/20/2013 12:38 PM, Andy Grover wrote:
On 05/19/2013 10:10 AM, Craig Watson wrote:
Hi,
I am attempting to set up SAN access to a RAID on a Linux platform using LIO/TMC with a QLogic qle2562 Fiber Channel HBA. I have attempted to follow the example shown at http://www.linux-iscsi.org/wiki/Fibre_Channel#Enable_target_mode and have run into some problems. Details of my issues are after the debug/system info below suggested at http://linux-iscsi.org/wiki/Support. Sorry it is so long but that's what was asked for.
How do I get this to work? It is apparent I'm missing something rather fundamental in the whole setup. I'd really appreciate any suggestions or pointers to documentation I've missed.
Thanks in advance,
Craig Watson
Hi Craig,
I think you're running into
https://github.com/agrover/rtslib-fb/issues/23
Please update to latest targetcli and rtslib: targetcli-2.1.fb25-1.fc18 python-rtslib-2.1.fb34-1.fc18
via 'yum update targetcli --enablerepo=updates-testing'.
(This should also be in the regular 'updates' repo by tomorrow.)
Fedora ships a modified version of these tools, which accounts for the other differences you noted. Please feel free to use the targetcli-fb list (CCd) if you encounter further issues.
Regards -- Andy
Hi Andy,
I'm switching mailing lists as you suggested. Thanks for the quick response. I updated tagetcli/rtslib from fedora-updates-testing as you suggested (yum info shown below). This definitely fixed some things. Note the proper response to qla2xxx's info and status commands now. Unfortunately creating a qla2xxx target is still eluding me. The create command still returns "Could not create Target in configFS."
Any ideas?
---------------------------------------------------------------- root@buickgn ~ # yum info targetcli Loaded plugins: langpacks, presto, refresh-packagekit Installed Packages Name : targetcli Arch : noarch Version : 2.1.fb25 Release : 1.fc18 Size : 197 k Repo : installed From repo : updates-testing Summary : An administration shell for storage targets URL : https://fedorahosted.org/targetcli-fb/ License : AGPLv3 Description : An administration shell for configuring iSCSI, FCoE, and other : SCSI targets, using the TCM/LIO kernel target subsystem. FCoE : users will also need to install and use fcoe-utils.
root@buickgn ~ # yum info python-rtslib Loaded plugins: langpacks, presto, refresh-packagekit Installed Packages Name : python-rtslib Arch : noarch Version : 2.1.fb34 Release : 1.fc18 Size : 277 k Repo : installed From repo : updates-testing Summary : API for Linux kernel LIO SCSI target URL : https://fedorahosted.org/targetcli-fb/ License : AGPLv3 Description : API for generic Linux SCSI kernel target.
---------------------------------------------------------------- root@buickgn ~ # targetcli targetcli shell version 2.1.25 Copyright 2011 by RisingTide Systems LLC and others. For help on commands, type 'help'.
/> ls o- / ....................................................................... [...] o- backstores ............................................................ [...] | o- block ................................................ [Storage Objects: 1] | | o- raid ....................... [/dev/md_d0 (1.6TiB) write-thru deactivated] | o- fileio ............................................... [Storage Objects: 0] | o- pscsi ................................................ [Storage Objects: 0] | o- ramdisk .............................................. [Storage Objects: 0] o- iscsi .......................................................... [Targets: 0] o- loopback ....................................................... [Targets: 0] o- qla2xxx ........................................................ [Targets: 0] o- vhost .......................................................... [Targets: 0] /> cd qla2xxx /qla2xxx> info Fabric module name: qla2xxx ConfigFS path: /sys/kernel/config/target/qla2xxx Allowed WWN types: naa Allowed WWNs list: naa.21000024ff46b7ba, naa.21000024ff46b7bb Fabric module features: acls Corresponding kernel module: tcm_qla2xxx /qla2xxx> status Status for /qla2xxx: Targets: 0 /qla2xxx> create naa.21000024ff46b7ba Could not create Target in configFS. /qla2xxx> create 21:00:00:24:ff:46:b7:ba Could not create Target in configFS. /qla2xxx>
----------------------------------------------------------------
This is what /root/.targetcli/log.txt shows for the create command:
[DEBUG] 2013-05-20 22:44:28 /usr/lib/python2.7/site-packages/configshell/shell.py:932 run_cmdline() Running command line 'create naa.21000024ff46b7ba'. [DEBUG] 2013-05-20 22:44:28 /usr/lib/python2.7/site-packages/configshell/shell.py:837 _parse_cmdline() Parsing commandline. [DEBUG] 2013-05-20 22:44:28 /usr/lib/python2.7/site-packages/configshell/shell.py:844 _parse_cmdline() [('command', 0, 6, None), ('pparam', 7, 27, None)] [DEBUG] 2013-05-20 22:44:28 /usr/lib/python2.7/site-packages/configshell/shell.py:848 _parse_cmdline() Found command token create. [DEBUG] 2013-05-20 22:44:28 /usr/lib/python2.7/site-packages/configshell/shell.py:848 _parse_cmdline() Found pparam token naa.21000024ff46b7ba. [DEBUG] 2013-05-20 22:44:28 /usr/lib/python2.7/site-packages/configshell/shell.py:861 _parse_cmdline() Parse gave path='' command='create' pparams=['naa.21000024ff46b7ba'] kparams={} [DEBUG] 2013-05-20 22:44:28 /usr/lib/python2.7/site-packages/configshell/node.py:1857 get_node() Looking for path '.' [DEBUG] 2013-05-20 22:44:28 /usr/lib/python2.7/site-packages/configshell/node.py:1403 execute_command() Executing command create with pparams ['naa.21000024ff46b7ba'] and kparams {}. [DEBUG] 2013-05-20 22:44:28 /usr/lib/python2.7/site-packages/configshell/node.py:1456 assert_params() Min params: 0 [DEBUG] 2013-05-20 22:44:28 /usr/lib/python2.7/site-packages/configshell/node.py:1457 assert_params() Max params: 1 [DEBUG] 2013-05-20 22:44:28 /usr/lib/python2.7/site-packages/configshell/node.py:1458 assert_params() Required params: [DEBUG] 2013-05-20 22:44:28 /usr/lib/python2.7/site-packages/configshell/node.py:1459 assert_params() Optional params: wwn [DEBUG] 2013-05-20 22:44:28 /usr/lib/python2.7/site-packages/configshell/node.py:1460 assert_params() Got 1 standard params. [DEBUG] 2013-05-20 22:44:28 /usr/lib/python2.7/site-packages/configshell/node.py:1461 assert_params() Got 0 extended params. [DEBUG] 2013-05-20 22:44:28 /usr/lib/python2.7/site-packages/configshell/node.py:1462 assert_params() Variable positional params: None [DEBUG] 2013-05-20 22:44:28 /usr/lib/python2.7/site-packages/configshell/node.py:1463 assert_params() Variable keyword params: None [ERROR] 2013-05-20 22:44:28 Could not create Target in configFS.
Thanks again,
Craig Watson
On 05/20/2013 10:28 PM, Craig Watson wrote:
On 05/20/2013 12:38 PM, Andy Grover wrote:
On 05/19/2013 10:10 AM, Craig Watson wrote:
Hi,
I am attempting to set up SAN access to a RAID on a Linux platform using LIO/TMC with a QLogic qle2562 Fiber Channel HBA. I have attempted to follow the example shown at http://www.linux-iscsi.org/wiki/Fibre_Channel#Enable_target_mode and have run into some problems. Details of my issues are after the debug/system info below suggested at http://linux-iscsi.org/wiki/Support. Sorry it is so long but that's what was asked for.
How do I get this to work? It is apparent I'm missing something rather fundamental in the whole setup. I'd really appreciate any suggestions or pointers to documentation I've missed.
Thanks in advance,
Craig Watson
Hi Craig,
I think you're running into
https://github.com/agrover/rtslib-fb/issues/23
Please update to latest targetcli and rtslib: targetcli-2.1.fb25-1.fc18 python-rtslib-2.1.fb34-1.fc18
via 'yum update targetcli --enablerepo=updates-testing'.
(This should also be in the regular 'updates' repo by tomorrow.)
Fedora ships a modified version of these tools, which accounts for the other differences you noted. Please feel free to use the targetcli-fb list (CCd) if you encounter further issues.
Regards -- Andy
Hi Andy,
I'm switching mailing lists as you suggested. Thanks for the quick response. I updated tagetcli/rtslib from fedora-updates-testing as you suggested (yum info shown below). This definitely fixed some things. Note the proper response to qla2xxx's info and status commands now. Unfortunately creating a qla2xxx target is still eluding me. The create command still returns "Could not create Target in configFS."
Any ideas?
Hi Craig,
Sorry for the delay, still trying to borrow a machine with needed hw to try and repro this myself.
Hopefully soon.
Regards -- Andy
On 05/22/2013 12:40 PM, Andy Grover wrote:
On 05/20/2013 10:28 PM, Craig Watson wrote:
I'm switching mailing lists as you suggested. Thanks for the quick response. I updated tagetcli/rtslib from fedora-updates-testing as you suggested (yum info shown below). This definitely fixed some things. Note the proper response to qla2xxx's info and status commands now. Unfortunately creating a qla2xxx target is still eluding me. The create command still returns "Could not create Target in configFS."
Any ideas?
Hi Craig,
Sorry for the delay, still trying to borrow a machine with needed hw to try and repro this myself.
Got a system finally. Worked for me, after I ensured
cat /sys/module/qla2xxx/parameters/qlini_mode
showed "disabled". Adding a setting in modprobe.d/qla2xxx.conf:
options qla2xxx qlini_mode=disabled
didn't seem to take unless I rmmodded/modprobe'd qla2xxx manually.
Can you verify qlini_mode is disabled when you're trying it?
Thanks -- Andy
On 05/23/2013 02:17 PM, Andy Grover wrote:
On 05/22/2013 12:40 PM, Andy Grover wrote:
On 05/20/2013 10:28 PM, Craig Watson wrote:
I'm switching mailing lists as you suggested. Thanks for the quick response. I updated tagetcli/rtslib from fedora-updates-testing as you suggested (yum info shown below). This definitely fixed some things. Note the proper response to qla2xxx's info and status commands now. Unfortunately creating a qla2xxx target is still eluding me. The create command still returns "Could not create Target in configFS."
Any ideas?
Hi Craig,
Sorry for the delay, still trying to borrow a machine with needed hw to try and repro this myself.
Got a system finally. Worked for me, after I ensured
cat /sys/module/qla2xxx/parameters/qlini_mode
showed "disabled". Adding a setting in modprobe.d/qla2xxx.conf:
options qla2xxx qlini_mode=disabled
didn't seem to take unless I rmmodded/modprobe'd qla2xxx manually.
Can you verify qlini_mode is disabled when you're trying it?
Thanks -- Andy
Hi Andy,
Progress!
I tried your suggestion of rmmod and modprobe and that worked. Even though I have /etc/modprobe.d/qla2xxx.conf set up properly (See my original email. It's buried in there), the initiator mode is not being disabled on boot. There must be something going on in the initramfs/initrd stuff that is setting things up incorrectly prior to reading /etc/modprobe.d/*.conf.
It's not all up and running though. :-( Everything looks right in targetcli now but the initiator is not seeing the target.
I've tried swapping the fiber cable from port to port in case I have misidentified the connection. And, I have set up both of the initiator WWNs in the qla2xxx/<WWN>/acls section.
One of the ports of the initiator (atto FC-82EN) is connected to an external 300GB Seagate FC drive and that works fine. I have had both ports of the initiator tied to both ports of the target machine too, and the initiator didn't see things then either.
Basically, I have executed the following commands: cd /backstores block/ create name=raid dev=/dev/md_d0 cd /qla2xxx create naa.21000024ff46b7ba cd luns create /backstores/block/raid cd ../../acls create naa.2100001086038078 cd .. create naa.2100001086038079
Here's what I have done in detail...
root2buickgn ~ # cat /sys/modules/qla2xxx/parameters/qlini_mode enabled root2buickgn ~ # rmmod qla2xxx root2buickgn ~ # modprobe qla2xxx root2buickgn ~ # cat /sys/modules/qla2xxx/parameters/qlini_mode disabled root2buickgn ~ # targetcli /> clearconfig confirm=true /> ls o- / ................................................................................ [...] o- backstores ..................................................................... [...] | o- block ......................................................... [Storage Objects: 0] | o- fileio ........................................................ [Storage Objects: 0] | o- pscsi ......................................................... [Storage Objects: 0] | o- ramdisk ....................................................... [Storage Objects: 0] o- iscsi ................................................................... [Targets: 0] o- loopback ................................................................ [Targets: 0] o- qla2xxx ................................................................. [Targets: 0] o- vhost ................................................................... [Targets: 0] /> cd backstores /backstores> block/ create name=raid dev=/dev/md_d0 Created block storage object raid using /dev/md_d0. Entering new node /backstores/block/raid /backstores/block/raid> cd / /> ls o- / ................................................................................ [...] o- backstores ..................................................................... [...] | o- block ......................................................... [Storage Objects: 1] | | o- raid ................................ [/dev/md_d0 (1.6TiB) write-thru deactivated] | o- fileio ........................................................ [Storage Objects: 0] | o- pscsi ......................................................... [Storage Objects: 0] | o- ramdisk ....................................................... [Storage Objects: 0] o- iscsi ................................................................... [Targets: 0] o- loopback ................................................................ [Targets: 0] o- qla2xxx ................................................................. [Targets: 0] o- vhost ................................................................... [Targets: 0] /> cd qla2xxx /qla2xxx> info Fabric module name: qla2xxx ConfigFS path: /sys/kernel/config/target/qla2xxx Allowed WWN types: naa Allowed WWNs list: naa.21000024ff46b7ba, naa.21000024ff46b7bb Fabric module features: acls Corresponding kernel module: tcm_qla2xxx /qla2xxx> create naa.21000024ff46b7ba Created target naa.21000024ff46b7ba. Entering new node /qla2xxx/naa.21000024ff46b7ba /qla2xxx/naa.21000024ff46b7ba> ls o- naa.21000024ff46b7ba ......................................................... [enabled] o- acls ....................................................................... [ACLs: 0] o- luns ....................................................................... [LUNs: 0] /qla2xxx/naa.21000024ff46b7ba> cd luns /qla2xxx/naa....ff46b7ba/luns> create /backstores/block/raid Created LUN 0. Entering new node /qla2xxx/naa.21000024ff46b7ba/luns/lun0 /qla2xxx/naa....7ba/luns/lun0> ls o- lun0 ......................................................... [block/raid (/dev/md_d0)] /qla2xxx/naa....7ba/luns/lun0> cd ../../acls /qla2xxx/naa....ff46b7ba/acls> create naa.2100001086038078 Created Node ACL for naa.2100001086038078 Created mapped LUN 0. Entering new node /qla2xxx/naa.21000024ff46b7ba/acls/naa.2100001086038078 /qla2xxx/naa....0001086038078> cd .. /qla2xxx/naa....ff46b7ba/acls> create naa.2100001086038079 Created Node ACL for naa.2100001086038079 Created mapped LUN 0. Entering new node /qla2xxx/naa.21000024ff46b7ba/acls/naa.2100001086038079 /qla2xxx/naa....ff46b7ba/acls> cd / /> ls o- / ................................................................................ [...] o- backstores ..................................................................... [...] | o- block ......................................................... [Storage Objects: 1] | | o- raid .................................. [/dev/md_d0 (1.6TiB) write-thru activated] | o- fileio ........................................................ [Storage Objects: 0] | o- pscsi ......................................................... [Storage Objects: 0] | o- ramdisk ....................................................... [Storage Objects: 0] o- iscsi ................................................................... [Targets: 0] o- loopback ................................................................ [Targets: 0] o- qla2xxx ................................................................. [Targets: 1] | o- naa.21000024ff46b7ba ..................................................... [enabled] | o- acls ................................................................... [ACLs: 2] | | o- naa.2100001086038078 .......................................... [Mapped LUNs: 1] | | | o- mapped_lun0 ........................................... [lun0 block/raid (rw)] | | o- naa.2100001086038079 .......................................... [Mapped LUNs: 1] | | o- mapped_lun0 ........................................... [lun0 block/raid (rw)] | o- luns ................................................................... [LUNs: 1] | o- lun0 ................................................. [block/raid (/dev/md_d0)] o- vhost ................................................................... [Targets: 0] />
Any ideas or suggestions on how I can troubleshoot this?
Thanks again for your help.
Craig Watson
On 05/23/2013 02:13 PM, Craig Watson wrote:
Progress!
I tried your suggestion of rmmod and modprobe and that worked. Even though I have /etc/modprobe.d/qla2xxx.conf set up properly (See my original email. It's buried in there), the initiator mode is not being disabled on boot. There must be something going on in the initramfs/initrd stuff that is setting things up incorrectly prior to reading /etc/modprobe.d/*.conf.
I dug around and found the explanation here:
https://bugzilla.redhat.com/show_bug.cgi?id=873220#c24
so either 1) put file in /usr/lib/modprobe.d run dracut -f or 2) run dracut -f -H and it should get applied (until next time initrd is generated at least).
It's not all up and running though. :-( Everything looks right in targetcli now but the initiator is not seeing the target.
I've tried swapping the fiber cable from port to port in case I have misidentified the connection. And, I have set up both of the initiator WWNs in the qla2xxx/<WWN>/acls section.
One of the ports of the initiator (atto FC-82EN) is connected to an external 300GB Seagate FC drive and that works fine. I have had both ports of the initiator tied to both ports of the target machine too, and the initiator didn't see things then either.
<snip>
Any ideas or suggestions on how I can troubleshoot this?
Thanks again for your help.
Any interesting dmesg output from lio?
Alas I don't really know much more about FC or qla. My colleague says there's not much configuration needed or tools on the initiator side.
When you find the solution, I'd love to know what it was, though :-/
-- Andy
Hi Andy,
Success! I have poked and prodded and I'm not sure what got me over the last hurdle. The only substantive thing I did was copy /etc/modprobe.d/qla2xxx.conf to /usr/lib/modprobe.d and run "dracut -f" followed by a reboot. That shouldn't have fixed the kind of symptoms my last problem (no link seen on the initiator) was showing but the problem went away. Things have survived shutting down and rebooting several times now and I'm not seeing problems.
Actually it's not quite all working. When I try accessing the (unprepared) RAID, I get what appear to be read only type results/errors. This is identical to what I got using iscsi the other day with the same platforms. I don't expect it all to work until I patch target_core_iblock.c and rebuild the target module as Nicholas Bellinger so kindly pointed out is needed. That's a task for tomorrow.
At this point I am confident the LIO/TCM/FC target mode is working. Of course, if rebuilding the target module doesn't fix things I'll let you know but it's looking real good right now.
Thank you for all the help.
Craig Watson
On 05/23/2013 07:12 PM, Andy Grover wrote:
On 05/23/2013 02:13 PM, Craig Watson wrote:
Progress!
I tried your suggestion of rmmod and modprobe and that worked. Even though I have /etc/modprobe.d/qla2xxx.conf set up properly (See my original email. It's buried in there), the initiator mode is not being disabled on boot. There must be something going on in the initramfs/initrd stuff that is setting things up incorrectly prior to reading /etc/modprobe.d/*.conf.
I dug around and found the explanation here:
https://bugzilla.redhat.com/show_bug.cgi?id=873220#c24
so either 1) put file in /usr/lib/modprobe.d run dracut -f or 2) run dracut -f -H and it should get applied (until next time initrd is generated at least).
It's not all up and running though. :-( Everything looks right in targetcli now but the initiator is not seeing the target.
I've tried swapping the fiber cable from port to port in case I have misidentified the connection. And, I have set up both of the initiator WWNs in the qla2xxx/<WWN>/acls section.
One of the ports of the initiator (atto FC-82EN) is connected to an external 300GB Seagate FC drive and that works fine. I have had both ports of the initiator tied to both ports of the target machine too, and the initiator didn't see things then either.
<snip>
Any ideas or suggestions on how I can troubleshoot this?
Thanks again for your help.
Any interesting dmesg output from lio?
Alas I don't really know much more about FC or qla. My colleague says there's not much configuration needed or tools on the initiator side.
When you find the solution, I'd love to know what it was, though :-/
-- Andy
On 05/23/2013 10:26 PM, Craig Watson wrote:
Success! I have poked and prodded and I'm not sure what got me over the last hurdle. The only substantive thing I did was copy /etc/modprobe.d/qla2xxx.conf to /usr/lib/modprobe.d and run "dracut -f" followed by a reboot. That shouldn't have fixed the kind of symptoms my last problem (no link seen on the initiator) was showing but the problem went away. Things have survived shutting down and rebooting several times now and I'm not seeing problems.
Actually it's not quite all working. When I try accessing the (unprepared) RAID, I get what appear to be read only type results/errors. This is identical to what I got using iscsi the other day with the same platforms. I don't expect it all to work until I patch target_core_iblock.c and rebuild the target module as Nicholas Bellinger so kindly pointed out is needed. That's a task for tomorrow.
At this point I am confident the LIO/TCM/FC target mode is working. Of course, if rebuilding the target module doesn't fix things I'll let you know but it's looking real good right now.
Thank you for all the help.
Whew, cool.
BTW if you get kernel-3.9.3 from the updates-testing repo that r/w issue should be fixed there.
Living on the bleedin' edge -- Andy
Hi Andy,
As predicted everything works now. I also downloaded 3.9.3 and that works too. I'll probably stick with that rather than a patched target module in 3.9.2. So, for future reference, here's the list of pieces-parts that are known to work together:
Target mode side: QLogic qle2562 Fiber Channel HBA Fedora Core FC18 kernel 3.9.3-201.fc18.x86_64 w/corresponding qla2xxx module and target LIO targetcli version 2.1.fb25 python-rtslib version 2.1.fb34
Initiator: Atto Celerity FC-82EN CentOS 6.4 (Final) kernel 2.6.32-358.6.1.el6.x86_64
It's always a good sign when comm hardware from different vendors works right.
I noticed that I did not have to run dracut to get the driver loaded properly on boot with 3.9.3. It comes up with "qlini_mode=disabled" so something seems to be fixed in the initramfs stuff too. That might be due to /usr/lib/modprobe.d/qla2xxx.conf already being on the system though. All-in-all this is a big improvement from what we started with.
Thanks again,
Craig Watson
On 05/24/2013 12:03 PM, Andy Grover wrote:
On 05/23/2013 10:26 PM, Craig Watson wrote:
Success! I have poked and prodded and I'm not sure what got me over the last hurdle. The only substantive thing I did was copy /etc/modprobe.d/qla2xxx.conf to /usr/lib/modprobe.d and run "dracut -f" followed by a reboot. That shouldn't have fixed the kind of symptoms my last problem (no link seen on the initiator) was showing but the problem went away. Things have survived shutting down and rebooting several times now and I'm not seeing problems.
Actually it's not quite all working. When I try accessing the (unprepared) RAID, I get what appear to be read only type results/errors. This is identical to what I got using iscsi the other day with the same platforms. I don't expect it all to work until I patch target_core_iblock.c and rebuild the target module as Nicholas Bellinger so kindly pointed out is needed. That's a task for tomorrow.
At this point I am confident the LIO/TCM/FC target mode is working. Of course, if rebuilding the target module doesn't fix things I'll let you know but it's looking real good right now.
Thank you for all the help.
Whew, cool.
BTW if you get kernel-3.9.3 from the updates-testing repo that r/w issue should be fixed there.
Living on the bleedin' edge -- Andy
targetcli-fb-devel@lists.fedorahosted.org