[PATCH 2/2] Change our docs that are close to ReST to proper ReST
Vratislav Podzimek
vpodzime at redhat.com
Wed Aug 13 11:13:10 UTC 2014
This format is recognized by many editors and tools (even many web ones) and
still nicely readable as plain text.
Signed-off-by: Vratislav Podzimek <vpodzime at redhat.com>
---
docs/commit-log.rst | 66 ++++++++++++++++++++
docs/commit-log.txt | 63 --------------------
docs/driverdisc.rst | 126 +++++++++++++++++++++++++++++++++++++++
docs/driverdisc.txt | 118 ------------------------------------
docs/iscsi.rst | 165 ++++++++++++++++++++++++++++++++++++++++++++++++++
docs/iscsi.txt | 169 ----------------------------------------------------
docs/multipath.rst | 138 ++++++++++++++++++++++++++++++++++++++++++
docs/multipath.txt | 143 --------------------------------------------
8 files changed, 495 insertions(+), 493 deletions(-)
create mode 100644 docs/commit-log.rst
delete mode 100644 docs/commit-log.txt
create mode 100644 docs/driverdisc.rst
delete mode 100644 docs/driverdisc.txt
create mode 100644 docs/iscsi.rst
delete mode 100644 docs/iscsi.txt
create mode 100644 docs/multipath.rst
delete mode 100644 docs/multipath.txt
diff --git a/docs/commit-log.rst b/docs/commit-log.rst
new file mode 100644
index 0000000..82f178f
--- /dev/null
+++ b/docs/commit-log.rst
@@ -0,0 +1,66 @@
+Rules for commit messages
+==========================
+
+git commit messages for anaconda should follow a consistent format. The
+following are rules to follow when committing a change to the git repo:
+
+1) The first line of the commit message should be a short summary of the
+ change in the patch. We also place (#BUGNUMBER) at the end of this
+ line to indicate the bugzilla.redhat.com bug number addressed in this
+ patch. The bug number is optional since there may be no bug number,
+ but if you have one you are addressing, please include it on the
+ summary line. Lastly, the summary lines need to be short. Ideally
+ less than 75 characters, but certainly not longer than 80.
+
+ Here are acceptable first lines for git commit messages:
+
+ Check partition and filesystem type on upgrade (#123456)
+ Fix bootloader configuration setup on ppc64 (#987654)
+ Introduce a new screen for setting your preferred email client
+
+ The last one would be a new feature that we didn't have a bug number
+ for.
+
+2) The main body of the commit message should begin TWO LINES below the
+ summary line you just entered (that is, there needs to be a blank line
+ between the one line summary and the start of the long commit message).
+ Please document the change and explain the patch here. Use multiple
+ paragraphs and keep the lines < 75 chars. DO NOT indent these lines.
+ Everything in the git commit message should be left justified. PLEASE
+ wrap long lines. If you don't, the 'git log' output ends up looking
+ stupid on 80 column terminals.
+
+3) For RHEL bugs, all commits need to reference a bug number. You may
+ follow one of two formats for specifying the bug number in a RHEL commit.
+
+ a) Put the bug number on the summary line in (#BUGNUMBER) format. Bugs
+ listed this way are treated as 'Resolves' patches in the RHEL
+ universe.
+
+ b) If you have a patch that is Related to or Conflicts with another bug,
+ you may add those lines to the end of the long commit message in this
+ format::
+
+ Related: rhbz#BUGNUMBER
+ Conflicts: rhbz#BUGNUMBER
+ Resolves: rhbz#BUGNUMBER
+
+ These entries should come at the end of the long commit message and
+ must follow the format above. You may have as many of these lines as
+ appropriate for the patch.
+
+ c) Patches that are 'Resolves' patches have two methods to specify the
+ bug numbers, but Related and Conflicts can only be listed in the long
+ commit message.
+
+ On RHEL branches, the 'bumpver' process will verify that each patch for
+ the release references a RHEL bug number. The scripts/makebumpver script
+ will extract the bug numbers from RHEL branch commits and do two things.
+ First, it verifies that the bug referenced is a RHEL bug and in correct
+ states. Second, it adds the appropriate Resolves/Related/Conflicts line
+ to the RPM spec file changelog.
+
+It is recommended to use the pre-push hook checking commit messages for RHEL bug
+numbers and checking the referenced bugs for all the necessary acks. To make it
+work, just copy the scripts/githooks/pre-push and
+scripts/githooks/check_commit_msg.sh scripts to the .git/hooks/ directory.
diff --git a/docs/commit-log.txt b/docs/commit-log.txt
deleted file mode 100644
index 2e3f227..0000000
--- a/docs/commit-log.txt
+++ /dev/null
@@ -1,63 +0,0 @@
-git commit messages for anaconda should follow a consistent format. The
-following are rules to follow when committing a change to the git repo:
-
-1) The first line of the commit message should be a short summary of the
- change in the patch. We also place (#BUGNUMBER) at the end of this
- line to indicate the bugzilla.redhat.com bug number addressed in this
- patch. The bug number is optional since there may be no bug number,
- but if you have one you are addressing, please include it on the
- summary line. Lastly, the summary lines need to be short. Ideally
- less than 75 characters, but certainly not longer than 80.
-
- Here are acceptable first lines for git commit messages:
-
- Check partition and filesystem type on upgrade (#123456)
- Fix bootloader configuration setup on ppc64 (#987654)
- Introduce a new screen for setting your preferred email client
-
- The last one would be a new feature that we didn't have a bug number
- for.
-
-2) The main body of the commit message should begin TWO LINES below the
- summary line you just entered (that is, there needs to be a blank line
- between the one line summary and the start of the long commit message).
- Please document the change and explain the patch here. Use multiple
- paragraphs and keep the lines < 75 chars. DO NOT indent these lines.
- Everything in the git commit message should be left justified. PLEASE
- wrap long lines. If you don't, the 'git log' output ends up looking
- stupid on 80 column terminals.
-
-3) For RHEL bugs, all commits need to reference a bug number. You may
- follow one of two formats for specifying the bug number in a RHEL commit.
-
- a) Put the bug number on the summary line in (#BUGNUMBER) format. Bugs
- listed this way are treated as 'Resolves' patches in the RHEL
- universe.
-
- b) If you have a patch that is Related to or Conflicts with another bug,
- you may add those lines to the end of the long commit message in this
- format:
-
- Related: rhbz#BUGNUMBER
- Conflicts: rhbz#BUGNUMBER
- Resolves: rhbz#BUGNUMBER
-
- These entries should come at the end of the long commit message and
- must follow the format above. You may have as many of these lines as
- appropriate for the patch.
-
- c) Patches that are 'Resolves' patches have two methods to specify the
- bug numbers, but Related and Conflicts can only be listed in the long
- commit message.
-
- On RHEL branches, the 'bumpver' process will verify that each patch for
- the release references a RHEL bug number. The scripts/makebumpver script
- will extract the bug numbers from RHEL branch commits and do two things.
- First, it verifies that the bug referenced is a RHEL bug and in correct
- states. Second, it adds the appropriate Resolves/Related/Conflicts line
- to the RPM spec file changelog.
-
-It is recommended to use the pre-push hook checking commit messages for RHEL bug
-numbers and checking the referenced bugs for all the necessary acks. To make it
-work, just copy the scripts/githooks/pre-push and
-scripts/githooks/check_commit_msg.sh scripts to the .git/hooks/ directory.
diff --git a/docs/driverdisc.rst b/docs/driverdisc.rst
new file mode 100644
index 0000000..dfde6ee
--- /dev/null
+++ b/docs/driverdisc.rst
@@ -0,0 +1,126 @@
+Brief description of DriverDisc version 3
+==========================================
+
+For a new major release we decided to introduce a new version of DriverDisc
+feature to ensure the smoothest vendor and user experience possible. We had
+many reasons for it:
+
+- the old DD didn't support multiple architectures
+- the old DD wasn't particulary easy to create
+- the old DD had two copys of modules, one for anaconda and one for
+ instalation
+- the modules in old DD weren't checked for kernel version
+
+We also changed the feature internal code to enable some functionality that
+was missing from the old version. More about it below.
+
+
+Devices which can contain DDs
+-----------------------------
+
+The best place to save your DriverDisc to is USB flash device. We also support
+(or plan to) IDE and SATA block devices with or without partitions, DriverDisc
+image stored on block device, initrd overlay (see documentation below) and for
+special cases even network retrieval of DriverDisc image.
+
+
+What can be updated using DDs?
+------------------------------
+
+All drivers for block devices, which weren't used for retrieving DriverDiscs,
+the same applies also for network drivers eg. you cannot upgrade network
+driver for device, which was used prior the DriverDisc extraction.
+
+RPMs for installation. If the DriverDisc repo contains newer package, than the
+official repository, the newer package will get used.
+
+We also plan to support anaconda's updates.img placement on the DriverDisc to
+update stage2 behaviour of anaconda.
+
+
+Selecting DD manually
+---------------------
+
+Use the 'inst.dd' kernel command line option to trigger DD mode.
+If no argument is specified, the UI will prompt for the location of the driver
+rpm. Otherwise, the rpm will be fetched from the specified location.
+
+Please consult the appropriate Installer Guide for further information.
+
+
+Automatic DriverDisc detection
+------------------------------
+
+Anaconda automatically looks for driverdiscs during startup.
+
+The DriverDisc has to be on partition or filesystem which has been labeled
+with 'OEMDRV' label.
+
+
+DDv3 structure
+--------------
+
+The new DriverDisc format uses simple layout which can be created on top of
+any anaconda's supported filesystem (vfat, squashfs, ext2 and ext3).
+
+::
+
+ /
+ |rhdd3 - DD marker, contains the DD's description string
+ /rpms
+ | /i386 - contains RPMs for this arch and acts as Yum repo
+ | /i586
+ | /x86_64
+ | /ppc
+ | /... - any other architecture the DD provides drivers for
+
+There is a special requirement for the RPMs used to update drivers. Anaconda
+picks up only RPMs which provide "kernel-modules = <running kernel version>".
+
+
+Initrd overlay driverdisc image
+-------------------------------
+
+We have designed another possible way of providing updates in network boot
+environments. It is possible to update all modules this way, so if special
+storage module (which gets used early) needs to be updated, this is the
+preffered way.
+
+This kind of driverdisc image is applied over the standard initrd and so has
+to respect some rules.
+
+- All updated modules belong to /lib/modules/<kernel version>/.. according to
+ their usual location
+- All new modules belong to /lib/modules/<kernel version>/updates
+- All new firmware files belong to /lib/firmware
+- The rpm repo with updated packages belongs to /tmp/DD-initrd/
+- The (empty) trigger file /.rundepmod must be present
+
+
+Firmware and module update
+--------------------------
+
+The firmware files together with all .ko files from the RPMs are exploded to
+special module location, which has preference over built-in Anaconda modules.
+
+Anaconda doesn't use built-in modules (except some storage modules needed for
+the DD to function properly) during the DriverDisc mode, so even in case when
+you are updating some modules with second (or later) DriverDisc, the updated
+modules will be loaded. There is one exception though, if your module depends
+on a module which is only present in built-in module directory, that built-in
+module gets also loaded.
+
+
+Package installation
+--------------------
+
+It is also possible to include arbitrary packages on the DriverDisc media and
+mark them for installation. You just have to include the package name in the
+Yum repo for correct architecture and mark it as mandatory.
+
+
+Summary
+-------
+
+This new DriverDisc format should simplify the DD creation and usage a lot. We
+will gladly hear any comments as this is partially still work in progress.
diff --git a/docs/driverdisc.txt b/docs/driverdisc.txt
deleted file mode 100644
index 1e28c39..0000000
--- a/docs/driverdisc.txt
+++ /dev/null
@@ -1,118 +0,0 @@
-Brief description of DriverDisc version 3
-==========================================
-
-For a new major release we decided to introduce a new version of DriverDisc
-feature to ensure the smoothest vendor and user experience possible. We had
-many reasons for it:
-
-- the old DD didn't support multiple architectures
-- the old DD wasn't particulary easy to create
-- the old DD had two copys of modules, one for anaconda and one for
- instalation
-- the modules in old DD weren't checked for kernel version
-
-We also changed the feature internal code to enable some functionality that
-was missing from the old version. More about it below.
-
-Devices which can contain DDs
------------------------------
-
-The best place to save your DriverDisc to is USB flash device. We also support
-(or plan to) IDE and SATA block devices with or without partitions, DriverDisc
-image stored on block device, initrd overlay (see documentation below) and for
-special cases even network retrieval of DriverDisc image.
-
-What can be updated using DDs?
-------------------------------
-
-All drivers for block devices, which weren't used for retrieving DriverDiscs,
-the same applies also for network drivers eg. you cannot upgrade network
-driver for device, which was used prior the DriverDisc extraction.
-
-RPMs for installation. If the DriverDisc repo contains newer package, than the
-official repository, the newer package will get used.
-
-We also plan to support anaconda's updates.img placement on the DriverDisc to
-update stage2 behaviour of anaconda.
-
-Selecting DD manually
----------------------
-
-Use the 'inst.dd' kernel command line option to trigger DD mode.
-If no argument is specified, the UI will prompt for the location of the driver
-rpm. Otherwise, the rpm will be fetched from the specified location.
-
-Please consult the appropriate Installer Guide for further information.
-
-Automatic DriverDisc detection
-------------------------------
-
-Anaconda automatically looks for driverdiscs during startup.
-
-The DriverDisc has to be on partition or filesystem which has been labeled
-with 'OEMDRV' label.
-
-
-DDv3 structure
---------------
-
-The new DriverDisc format uses simple layout which can be created on top of
-any anaconda's supported filesystem (vfat, squashfs, ext2 and ext3).
-
-/
-|rhdd3 - DD marker, contains the DD's description string
-/rpms
-| /i386 - contains RPMs for this arch and acts as Yum repo
-| /i586
-| /x86_64
-| /ppc
-| /... - any other architecture the DD provides drivers for
-
-There is a special requirement for the RPMs used to update drivers. Anaconda
-picks up only RPMs which provide "kernel-modules = <running kernel version>".
-
-Initrd overlay driverdisc image
--------------------------------
-
-We have designed another possible way of providing updates in network boot
-environments. It is possible to update all modules this way, so if special
-storage module (which gets used early) needs to be updated, this is the
-preffered way.
-
-This kind of driverdisc image is applied over the standard initrd and so has
-to respect some rules.
-
-- All updated modules belong to /lib/modules/<kernel version>/.. according to
- their usual location
-- All new modules belong to /lib/modules/<kernel version>/updates
-- All new firmware files belong to /lib/firmware
-- The rpm repo with updated packages belongs to /tmp/DD-initrd/
-- The (empty) trigger file /.rundepmod must be present
-
-Firmware and module update
---------------------------
-
-The firmware files together with all .ko files from the RPMs are exploded to
-special module location, which has preference over built-in Anaconda modules.
-
-Anaconda doesn't use built-in modules (except some storage modules needed for
-the DD to function properly) during the DriverDisc mode, so even in case when
-you are updating some modules with second (or later) DriverDisc, the updated
-modules will be loaded. There is one exception though, if your module depends
-on a module which is only present in built-in module directory, that built-in
-module gets also loaded.
-
-Package installation
---------------------
-
-It is also possible to include arbitrary packages on the DriverDisc media and
-mark them for installation. You just have to include the package name in the
-Yum repo for correct architecture and mark it as mandatory.
-
-
-Summary
--------
-
-This new DriverDisc format should simplify the DD creation and usage a lot. We
-will gladly hear any comments as this is partially still work in progress.
-
diff --git a/docs/iscsi.rst b/docs/iscsi.rst
new file mode 100644
index 0000000..847078d
--- /dev/null
+++ b/docs/iscsi.rst
@@ -0,0 +1,165 @@
+iSCSI and Anaconda
+==================
+
+:Authors:
+ Ales Kozumplik <akozumpl at redhat.com>
+
+Introduction
+------------
+
+iSCSI device is a SCSI device connected to your computer via a TCP/IP
+network. The communication can be handled either in hardware or in software, or
+as a hybrid --- part software, part hardware.
+
+The terminology:
+
+- 'initiator', the client in the iscsi connection. The computer we are running
+ Anaconda on is typically an initiator.
+- 'target', the storage device behind the Network. This is where the data is
+ physically stored and read from. You can turn any Fedora/RHEL machine to a
+ target (or several) via scsi-target-utils.
+- 'HBA' or Host Bus Adapter. A device (PCI card typically) you connect to a
+ computer. It acts as a NIC and if you configure it properly it transparently
+ connects to the target when started and all you can see is a block device on
+ your system.
+- 'software initiator' is what you end up with if you emulate most of what HBA is
+ doing and just use a regular NIC for the iscsi communication. The modern Linux
+ kernel has a software initiator. To use it, you need the Open-ISCSI software
+ stack [1, 2] installed. It is known as iscsi-initiator-utils in Fedora/RHEL.
+- 'partial offload card'. Similar to HBA but needs some support from kernel and
+ iscsi-initiator-utils. The least pleasant to work with, particularly because
+ there is no standardized amount of the manual setting that needs to be done
+ (some connect to the target just like HBAs, some need you to bring their NIC
+ part up manually etc.). Partial offload cards exist to get better performing
+ I/O with less processor load than with software initiator.
+- 'iBFT' as in 'Iscsi Boot Firmware Table'. A table in the card's bios that
+ contains its network and target settings. This allows the card to configure
+ itself, connect to a target and boot from it before any operating system or a
+ bootloader has the chance. We can also read this information from
+ /sys/firmware/ibft after the system starts and then use it to bring the card
+ up (again) in Linux.
+- 'CHAP' is the authentication used for iSCSI connections. The authentication
+ can happen during target discovery or target login or both. It can happen in
+ both directions too: the initiator authenticates itself to the target and the
+ target is sometimes required to authenticate itself to the initiator.
+
+
+What is expected from Anaconda
+------------------------------
+
+We are expected to:
+
+- use an HBA like an ordinary disk. It is usually smart enough to bring itself
+ up during boot, connect to the target and just act as an ordinary disk.
+- allow creating new software initiator connections in the UI, both IPv4 and IPv6.
+- facilitate bringing up iBFT connections for partial offload cards.
+- install the root and/or /boot filesystems on any iSCSI initiator known to us
+- remember to install dracut-network if we are booting from an iSCSI initiator that
+ requires iscsi-initiator-utils in the ramdisk (most of them do)
+- boot from an iSCSI initiator using dracut, this requires generating an
+ appropriate set of kernel boot arguments for it [3].
+
+
+How Anaconda handles iscsi
+--------------------------
+
+iSCSI comes into play several times while Anaconda does its thing:
+
+In loader, when deciding what NIC we should setup, we check if we have iBFT
+information from one of the cards. If we do we set that card up with what we
+found in the table, it usually boils down to an IPv4 static or IPv4
+DHCP-obtained address. [4][5]
+
+Next, after the main UI startup during filtering (or storage scan, whatever
+comes first) we startup the iscsi support code in Anaconda [6]. This currently
+involves:
+- manually modprobing related kernel modules
+- starting the iscsiuio daemon (required by some partial offload cards)
+- most importantly, starting the iscsid daemon
+
+All iBFT connections are brought up next by looking at the cards' iBFT data, if
+any. The filtering screen has a feature to add advanced storage devices,
+including iSCSI. Both connection types are handled by libiscsi (see below). The
+brought up iSCSI devices appear as /dev/sdX and are treated as ordinary block
+devices.
+
+When DeviceTree scans all the block devices it uses the udev data (particularly
+the ID_BUS and ID_PATH keys) to decide if the device is an iscsi disk. If it is,
+it is represented with an iScsiDiskDevice class instance. This helps Anaconda
+remember that:
+
+- we need to install dracut-network so the generated dracut image is able to
+ bring up the underlying NIC and establish the iscsi connection.
+- if we are booting from the device we need to pass dracut a proper set of
+ arguments that will allow it to do so.
+
+
+Libiscsi
+--------
+
+How are iSCSI targets found and logged into? Originally Anaconda was just
+running iscsiadm as an external program through execWithRedirect(). This
+ultimately proved awkward especially due to the difficulties of handling the
+CHAP passphrases this way. That is why Hans de Goede <hdegoede at redhat.com>, the
+previous maintainer of the Anaconda iscsi subsystem decided to write a better
+interface and created libiscsi (do not confuse this with the libiscsi.c in
+kernel). Currently libiscsi lives as a couple of patches in the RHEL6
+iscsi-initiator-utils CVS (and in Fedora package git, in somewhat outdated
+version). Since Anaconda is libiscsi's only client at the moment it is
+maintained by the Anaconda team.
+
+The promise of libiscsi is to provide a simple C/Python API to handle iSCSI
+connections while being somewhat stable and independent of the changes in the
+underlying initiator-utils (while otherwise being tied to it on the
+implementation level).
+
+And at the moment libiscsi does just that. It has a set of functions to discover
+and login to targets software targets. It supports making connections through
+partial offload interfaces, but the only discovery method supported at this
+moment is through firmware (iBFT). Its public data structures are independent of
+iscsi-initiator-utils. And there is some python boilerplate that wraps the core
+functions so we can easily call those from Anaconda.
+
+To start nontrivial hacking on libiscsi prepare to spend some time familiarizing
+yourself with the iscsi-initiator-utils internals (it is complex but quite
+nice).
+
+
+Debugging iSCSI bugs
+--------------------
+
+There is some information in anaconda.log and storage.log but libiscsi itself is
+quite bad at logging. Most times useful information can be found by sshing onto
+the machine and inspecting the output of different iscsiadm commands [2][7],
+especially querying the existing sessions and known interfaces.
+
+If for some reason the DeviceTree fails at recognizing iscsi devices as such,
+'udevadm info --exportdb' is of interest.
+
+The booting problems are either due to incorrectly generated dracut boot
+arguments or they are simply dracut bugs.
+
+Note that many of the iscsi adapters are installed in different Red Hat machines
+and so the issues can often be reproduced and debugged.
+
+
+Future of iSCSI in Anaconda
+---------------------------
+
+- extend libiscsi to allow initializing arbitrary connections from a partial
+ offload card. Implement the Anaconda UI to utilize this. Difficulty hard.
+- extend libiscsi with device binding support. Difficulty hard.
+- work with iscsi-initiator-utils maintainer to get libiscsi.c upstream and then
+ to rawhide Fedora. Then the partial offload patches in the RHEL6 Anaconda can
+ be migrated there too and partial offload can be tested. This is something
+ that needs to be done before RHEL7. Difficulty medium.
+- improve libiscsi's logging capabilities. Difficulty easy.
+
+.. [1] http://www.open-iscsi.org/
+.. [2] /usr/share/doc/iscsi-initiator-utils-6.*/README
+.. [3] man 7 dracut.kernel
+.. [4] Anaconda git repository, anaconda/loader/ibft.c
+.. [5] Anaconda git repository, anaconda/loader/net.c, chooseNetworkInterface()
+.. [6] Anaconda git repository, anaconda/storage/iscsi.py
+.. [7] 'man 8 iscsiadm'
+
diff --git a/docs/iscsi.txt b/docs/iscsi.txt
deleted file mode 100644
index ee41582..0000000
--- a/docs/iscsi.txt
+++ /dev/null
@@ -1,169 +0,0 @@
-==================
-iSCSI and Anaconda
-==================
-
-
-Introduction
-------------
-
-iSCSI device is a SCSI device connected to your computer via a TCP/IP
-network. The communication can be handled either in hardware or in software, or
-as a hybrid --- part software, part hardware.
-
-The terminology:
-
-- 'initiator', the client in the iscsi connection. The computer we are running
- Anaconda on is typically an initiator.
-- 'target', the storage device behind the Network. This is where the data is
- physically stored and read from. You can turn any Fedora/RHEL machine to a
- target (or several) via scsi-target-utils.
-- 'HBA' or Host Bus Adapter. A device (PCI card typically) you connect to a
- computer. It acts as a NIC and if you configure it properly it transparently
- connects to the target when started and all you can see is a block device on
- your system.
-- 'software initiator' is what you end up with if you emulate most of what HBA is
- doing and just use a regular NIC for the iscsi communication. The modern Linux
- kernel has a software initiator. To use it, you need the Open-ISCSI software
- stack [1, 2] installed. It is known as iscsi-initiator-utils in Fedora/RHEL.
-- 'partial offload card'. Similar to HBA but needs some support from kernel and
- iscsi-initiator-utils. The least pleasant to work with, particularly because
- there is no standardized amount of the manual setting that needs to be done
- (some connect to the target just like HBAs, some need you to bring their NIC
- part up manually etc.). Partial offload cards exist to get better performing
- I/O with less processor load than with software initiator.
-- 'iBFT' as in 'Iscsi Boot Firmware Table'. A table in the card's bios that
- contains its network and target settings. This allows the card to configure
- itself, connect to a target and boot from it before any operating system or a
- bootloader has the chance. We can also read this information from
- /sys/firmware/ibft after the system starts and then use it to bring the card
- up (again) in Linux.
-- 'CHAP' is the authentication used for iSCSI connections. The authentication
- can happen during target discovery or target login or both. It can happen in
- both directions too: the initiator authenticates itself to the target and the
- target is sometimes required to authenticate itself to the initiator.
-
-
-What is expected from Anaconda
-------------------------------
-
-We are expected to:
-
-- use an HBA like an ordinary disk. It is usually smart enough to bring itself
- up during boot, connect to the target and just act as an ordinary disk.
-- allow creating new software initiator connections in the UI, both IPv4 and IPv6.
-- facilitate bringing up iBFT connections for partial offload cards.
-- install the root and/or /boot filesystems on any iSCSI initiator known to us
-- remember to install dracut-network if we are booting from an iSCSI initiator that
- requires iscsi-initiator-utils in the ramdisk (most of them do)
-- boot from an iSCSI initiator using dracut, this requires generating an
- appropriate set of kernel boot arguments for it [3].
-
-
-How Anaconda handles iscsi
---------------------------
-
-iSCSI comes into play several times while Anaconda does its thing:
-
-In loader, when deciding what NIC we should setup, we check if we have iBFT
-information from one of the cards. If we do we set that card up with what we
-found in the table, it usually boils down to an IPv4 static or IPv4
-DHCP-obtained address. [4][5]
-
-Next, after the main UI startup during filtering (or storage scan, whatever
-comes first) we startup the iscsi support code in Anaconda [6]. This currently
-involves:
-- manually modprobing related kernel modules
-- starting the iscsiuio daemon (required by some partial offload cards)
-- most importantly, starting the iscsid daemon
-
-All iBFT connections are brought up next by looking at the cards' iBFT data, if
-any. The filtering screen has a feature to add advanced storage devices,
-including iSCSI. Both connection types are handled by libiscsi (see below). The
-brought up iSCSI devices appear as /dev/sdX and are treated as ordinary block
-devices.
-
-When DeviceTree scans all the block devices it uses the udev data (particularly
-the ID_BUS and ID_PATH keys) to decide if the device is an iscsi disk. If it is,
-it is represented with an iScsiDiskDevice class instance. This helps Anaconda
-remember that:
-
-- we need to install dracut-network so the generated dracut image is able to
- bring up the underlying NIC and establish the iscsi connection.
-- if we are booting from the device we need to pass dracut a proper set of
- arguments that will allow it to do so.
-
-
-Libiscsi
---------
-
-How are iSCSI targets found and logged into? Originally Anaconda was just
-running iscsiadm as an external program through execWithRedirect(). This
-ultimately proved awkward especially due to the difficulties of handling the
-CHAP passphrases this way. That is why Hans de Goede <hdegoede at redhat.com>, the
-previous maintainer of the Anaconda iscsi subsystem decided to write a better
-interface and created libiscsi (do not confuse this with the libiscsi.c in
-kernel). Currently libiscsi lives as a couple of patches in the RHEL6
-iscsi-initiator-utils CVS (and in Fedora package git, in somewhat outdated
-version). Since Anaconda is libiscsi's only client at the moment it is
-maintained by the Anaconda team.
-
-The promise of libiscsi is to provide a simple C/Python API to handle iSCSI
-connections while being somewhat stable and independent of the changes in the
-underlying initiator-utils (while otherwise being tied to it on the
-implementation level).
-
-And at the moment libiscsi does just that. It has a set of functions to discover
-and login to targets software targets. It supports making connections through
-partial offload interfaces, but the only discovery method supported at this
-moment is through firmware (iBFT). Its public data structures are independent of
-iscsi-initiator-utils. And there is some python boilerplate that wraps the core
-functions so we can easily call those from Anaconda.
-
-To start nontrivial hacking on libiscsi prepare to spend some time familiarizing
-yourself with the iscsi-initiator-utils internals (it is complex but quite
-nice).
-
-
-Debugging iSCSI bugs
---------------------
-
-There is some information in anaconda.log and storage.log but libiscsi itself is
-quite bad at logging. Most times useful information can be found by sshing onto
-the machine and inspecting the output of different iscsiadm commands [2][7],
-especially querying the existing sessions and known interfaces.
-
-If for some reason the DeviceTree fails at recognizing iscsi devices as such,
-'udevadm info --exportdb' is of interest.
-
-The booting problems are either due to incorrectly generated dracut boot
-arguments or they are simply dracut bugs.
-
-Note that many of the iscsi adapters are installed in different Red Hat machines
-and so the issues can often be reproduced and debugged.
-
-
-Future of iSCSI in Anaconda
----------------------------
-
-- extend libiscsi to allow initializing arbitrary connections from a partial
- offload card. Implement the Anaconda UI to utilize this. Difficulty hard.
-- extend libiscsi with device binding support. Difficulty hard.
-- work with iscsi-initiator-utils maintainer to get libiscsi.c upstream and then
- to rawhide Fedora. Then the partial offload patches in the RHEL6 Anaconda can
- be migrated there too and partial offload can be tested. This is something
- that needs to be done before RHEL7. Difficulty medium.
-- improve libiscsi's logging capabilities. Difficulty easy.
-
-
-
-[1] http://www.open-iscsi.org/
-[2] /usr/share/doc/iscsi-initiator-utils-6.*/README
-[3] man 7 dracut.kernel
-[4] Anaconda git repository, anaconda/loader/ibft.c
-[5] Anaconda git repository, anaconda/loader/net.c, chooseNetworkInterface()
-[6] Anaconda git repository, anaconda/storage/iscsi.py
-[7] 'man 8 iscsiadm'
-
-
----
-Red Hat Author(s): Ales Kozumplik <akozumpl at redhat.com>
diff --git a/docs/multipath.rst b/docs/multipath.rst
new file mode 100644
index 0000000..eb2910d
--- /dev/null
+++ b/docs/multipath.rst
@@ -0,0 +1,138 @@
+Multipath and Anaconda
+======================
+
+:Authors:
+ Ales Kozumplik <akozumpl at redhat.com>
+
+Introduction
+------------
+
+If there are two block devices in your /dev for which udev reports the same
+'ID_SERIAL' then you can create a certain device mapper device which arbitrarily
+uses those devices to access the physical device. And that is Multipath [1].
+
+For instance, suppose there are::
+
+ /dev/sda, with ID_SERIAL of 20090ef12700001d2, and
+ /dev/sdb, with the same ID_SERIAL.
+
+Those are probably some adapters in the system that just connect your box to a
+storage area network (SAN) somewhere. There are perhaps two cables, one for sda,
+one for sdb, and if one of the cables gets cut the other can still transmit
+data. Normally the system won't recognize that sda and sdb have this special
+relation to each other, but by creating a suitable device map using multipath
+tools [2] we can create a DM device /dev/mapper/mpatha and use it for storing
+and retrieving data.
+
+The device mapper then automatically routes IO requests to /dev/mapper/mpatha to
+either sda or sdb depending on the load of the line or network congestion on the
+particular network etc.
+
+The nomenclature I will use here is:
+- 'multipath device' for the smart /dev/mapper/mpathX device.
+- 'multipath member device' for the '/dev/sdX' devices. Also 'a path'.
+
+
+What is expected from Anaconda
+------------------------------
+
+Anaconda is expected to:
+- detect that there are multipath devices present
+- coalesce all relevant (e.g. exclusiveDisks) multipath devices.
+- only let the user interact with the multipath devices in filtering,
+ cleardiskssel and partition screen, that is once we know 'sdc' and 'sdd' are
+ part of 'mpathb' show only 'mpathb' and never the paths.
+- install bootloader and boot from an mpath device
+- make it happen so all the multipath devices (carrying or not the root
+ filesystem) we used for installation are correctly coalesced in the booted
+ system. This is achieved by generating a suitable /etc/multipath.conf and
+ writing it into sysroot.
+- be able to refer to mpath devices from kickstart, either by name like 'mpatha'
+ or by their id like 'disk/by-id/scsi-20090ef12700001d2'
+
+
+How Anaconda handles multipath
+------------------------------
+
+To detect presence of multipath devices we rely on multipath tools. The same we
+do for coalescing, see pyanaconda/storage/devicelibs/mpath.py, the file that
+provides some abstraction from mpath tools. During the device scan we use the
+'multipath -d' output to find out what devices are going to end up as multipath
+members. The MultipathTopology object also enhances the multipath member's udev
+dictionaries with 'ID_FS_TYPE' set to 'multipath_member' (yes, this is a hack
+surviving from the original mpath implementation, and righteous is he who
+eradicates it). This information is picked up by DeviceTree when populating
+itself. Meaning, if 'sda' and 'sdb' are multipath member devices DeviceTree
+gives them MultipathMember format and creates one MultipathDevice for them (we
+know its name from 'multipath -d'). We end up with:
+
+DiskDevice 'sda', format 'MultipathMember'
+DiskDevice 'sdb', format 'MultipathMember'
+MultipathDevice 'mpatha', parents are 'sda' and 'sdb'.
+
+From then on, Anaconda only deals with the MultipathDevice and generally leaves
+anything with 'MultipathMember' format alone (understand, this is an inert
+format that really is not there but we use it just to mark the device as
+"useless beyond a multipath member", kind of like MDRaidMember).
+
+Partition happens over the multipath device and during the preinstallconfig step
+/mnt/sysimage/etc/multipath.conf is created and filled with information about
+the coalesced devices. This is handled in the Storage.write() method. It is
+important this file and /etc/multipath/wwids (autogenerated by mpath tools)
+make it to the sysimage before the dracut image is generated.
+
+
+Debugging multipath bugs
+-------------------------
+
+Unlike with iSCSI, to reproduce a multipath bug one does not need the same
+specific hardware as the reporter. Just found any box connected to a multipathed
+SAN and you are fine (at the moment, connecting to the same iSCSI target through
+its IPv4 and IPv6 address also produces a multipathed device).
+
+On top of that, much of the necessary information is already included in the
+anaconda logs or can be easily extracted from the reporter. The things to
+particularly look at are:
+
+- storage.log, the output around 'devices to scan for multipath' and 'devices
+ post multipath scan'. The latter shows a triple with regular disks, disks
+ comprising multipath devices and partitions. This helps you quickly find out
+ what the target system is about.
+
+- this information is also in program.log's calls to 'multipath' [3]. If mpath
+ devices are mysteriously appearing/disappearing between filtering and
+ partitioning screens look at those. 'multipath -ll' is called to display
+ currently coalesced mpath devices, 'multipath -d' is called to show the mpath
+ devices that would be coalesced if we ran 'multipath' now. This is exploited
+ by the device filtering screen.
+
+
+Future of multipath in Anaconda
+--------------------------------
+
+Overall as of RHEL6.2, the shape of multipath in Anaconda is good and what's
+more important it is flexible enough to sustain new RFEs and bugs. Those are
+however bugs that I expect to appear sometime soon:
+
+- enable or disable mpath_friendly_names in kickstart. Disabling friendly names
+ just means the mpath devices are called by their wwid,
+ e.g. /dev/mapper/360334332345343234, not '/dev/mapper/mpathc'. This is
+ straightforward to implement.
+- extend support for mpath devices in kickstart in general. Currently mpath
+ devices should be accepted in most commands but I am sure there will be corner
+ cases. Difficulty medium.
+- [rawhide] stop extending the udev info dictionary with 'ID_FS_TYPE' and
+ 'ID_MPATH_NAME'. Doing it this way is asking for the trouble if a dictionary
+ of particular mpath device is reloaded from udev without running it through
+ the MultipathTopology object as it will miss those entries (and DeviceTree
+ depends on them a lot). Difficulty hard, but includes a lot of pleasant
+ refactoring.
+- Improve support for multipathing iSCSI devices. Someone might ask for it one
+ day (in fact, with the NIC bounding they already did), and it will make mpath
+ debugging possible on any virt machine with multiple virt NICs.
+
+
+.. [1] http://akozumpl.fedorapeople.org/archive/Multipass.jpg
+.. [2] http://christophe.varoqui.free.fr/
+.. [3] 'man 8 multipath'
+
diff --git a/docs/multipath.txt b/docs/multipath.txt
deleted file mode 100644
index e8af24e..0000000
--- a/docs/multipath.txt
+++ /dev/null
@@ -1,143 +0,0 @@
-======================
-Multipath and Anaconda
-======================
-
-
-Introduction
-------------
-
-If there are two block devices in your /dev for which udev reports the same
-'ID_SERIAL' then you can create a certain device mapper device which arbitrarily
-uses those devices to access the physical device. And that is Multipath [1].
-
-For instance, suppose there are
-
-/dev/sda, with ID_SERIAL of 20090ef12700001d2, and
-/dev/sdb, with the same ID_SERIAL.
-
-Those are probably some adapters in the system that just connect your box to a
-storage area network (SAN) somewhere. There are perhaps two cables, one for sda,
-one for sdb, and if one of the cables gets cut the other can still transmit
-data. Normally the system won't recognize that sda and sdb have this special
-relation to each other, but by creating a suitable device map using multipath
-tools [2] we can create a DM device /dev/mapper/mpatha and use it for storing
-and retrieving data.
-
-The device mapper then automatically routes IO requests to /dev/mapper/mpatha to
-either sda or sdb depending on the load of the line or network congestion on the
-particular network etc.
-
-The nomenclature I will use here is:
-
-- 'multipath device' for the smart /dev/mapper/mpathX device.
-- 'multipath member device' for the '/dev/sdX' devices. Also 'a path'.
-
-
-What is expected from Anaconda
-------------------------------
-
-Anaconda is expected to:
-- detect that there are multipath devices present
-- coalesce all relevant (e.g. exclusiveDisks) multipath devices.
-- only let the user interact with the multipath devices in filtering,
- cleardiskssel and partition screen, that is once we know 'sdc' and 'sdd' are
- part of 'mpathb' show only 'mpathb' and never the paths.
-- install bootloader and boot from an mpath device
-- make it happen so all the multipath devices (carrying or not the root
- filesystem) we used for installation are correctly coalesced in the booted
- system. This is achieved by generating a suitable /etc/multipath.conf and
- writing it into sysroot.
-- be able to refer to mpath devices from kickstart, either by name like 'mpatha'
- or by their id like 'disk/by-id/scsi-20090ef12700001d2'
-
-
-How Anaconda handles multipath
-------------------------------
-
-To detect presence of multipath devices we rely on multipath tools. The same we
-do for coalescing, see pyanaconda/storage/devicelibs/mpath.py, the file that
-provides some abstraction from mpath tools. During the device scan we use the
-'multipath -d' output to find out what devices are going to end up as multipath
-members. The MultipathTopology object also enhances the multipath member's udev
-dictionaries with 'ID_FS_TYPE' set to 'multipath_member' (yes, this is a hack
-surviving from the original mpath implementation, and righteous is he who
-eradicates it). This information is picked up by DeviceTree when populating
-itself. Meaning, if 'sda' and 'sdb' are multipath member devices DeviceTree
-gives them MultipathMember format and creates one MultipathDevice for them (we
-know its name from 'multipath -d'). We end up with:
-
-DiskDevice 'sda', format 'MultipathMember'
-DiskDevice 'sdb', format 'MultipathMember'
-MultipathDevice 'mpatha', parents are 'sda' and 'sdb'.
-
-From then on, Anaconda only deals with the MultipathDevice and generally leaves
-anything with 'MultipathMember' format alone (understand, this is an inert
-format that really is not there but we use it just to mark the device as
-"useless beyond a multipath member", kind of like MDRaidMember).
-
-Partition happens over the multipath device and during the preinstallconfig step
-/mnt/sysimage/etc/multipath.conf is created and filled with information about
-the coalesced devices. This is handled in the Storage.write() method. It is
-important this file and /etc/multipath/wwids (autogenerated by mpath tools)
-make it to the sysimage before the dracut image is generated.
-
-
-Debugging multipath bugs
-------------------------
-
-Unlike with iSCSI, to reproduce a multipath bug one does not need the same
-specific hardware as the reporter. Just found any box connected to a multipathed
-SAN and you are fine (at the moment, connecting to the same iSCSI target through
-its IPv4 and IPv6 address also produces a multipathed device).
-
-On top of that, much of the necessary information is already included in the
-anaconda logs or can be easily extracted from the reporter. The things to
-particularly look at are:
-
-- storage.log, the output around 'devices to scan for multipath' and 'devices
- post multipath scan'. The latter shows a triple with regular disks, disks
- comprising multipath devices and partitions. This helps you quickly find out
- what the target system is about.
-
-- this information is also in program.log's calls to 'multipath' [3]. If mpath
- devices are mysteriously appearing/disappearing between filtering and
- partitioning screens look at those. 'multipath -ll' is called to display
- currently coalesced mpath devices, 'multipath -d' is called to show the mpath
- devices that would be coalesced if we ran 'multipath' now. This is exploited
- by the device filtering screen.
-
-
-Future of multipath in Anaconda
--------------------------------
-
-Overall as of RHEL6.2, the shape of multipath in Anaconda is good and what's
-more important it is flexible enough to sustain new RFEs and bugs. Those are
-however bugs that I expect to appear sometime soon:
-
-- enable or disable mpath_friendly_names in kickstart. Disabling friendly names
- just means the mpath devices are called by their wwid,
- e.g. /dev/mapper/360334332345343234, not '/dev/mapper/mpathc'. This is
- straightforward to implement.
-- extend support for mpath devices in kickstart in general. Currently mpath
- devices should be accepted in most commands but I am sure there will be corner
- cases. Difficulty medium.
-- [rawhide] stop extending the udev info dictionary with 'ID_FS_TYPE' and
- 'ID_MPATH_NAME'. Doing it this way is asking for the trouble if a dictionary
- of particular mpath device is reloaded from udev without running it through
- the MultipathTopology object as it will miss those entries (and DeviceTree
- depends on them a lot). Difficulty hard, but includes a lot of pleasant
- refactoring.
-- Improve support for multipathing iSCSI devices. Someone might ask for it one
- day (in fact, with the NIC bounding they already did), and it will make mpath
- debugging possible on any virt machine with multiple virt NICs.
-
-
-
-[1] http://akozumpl.fedorapeople.org/archive/Multipass.jpg
-[2] http://christophe.varoqui.free.fr/
-[3] 'man 8 multipath'
-
-
-
----
-Red Hat Author(s): Ales Kozumplik <akozumpl at redhat.com>
--
1.9.3
More information about the anaconda-patches
mailing list