I just got a Trimslice Pro (a friend wasn't using it), and I booted the Fedora-Minimal-armhfp-20-Beta-TC2-sda image from an SD card and started copying it to the internal SSD, and started getting errors on the serial console:
[ 864.153171] hub 1-0:1.0: cannot reset port 1 (err = -110) [ 864.382165] hub 1-0:1.0: cannot reset port 1 (err = -110) [ 864.611136] hub 1-0:1.0: cannot reset port 1 (err = -110) [ 864.616551] hub 1-0:1.0: Cannot enable port 1. Maybe the USB cable is bad? ... [ 923.473666] hub 1-0:1.0: cannot reset port 1 (err = -110) [ 923.702667] hub 1-0:1.0: cannot reset port 1 (err = -110) [ 923.931671] hub 1-0:1.0: cannot reset port 1 (err = -110) [ 924.574479] end_request: I/O error, dev sda, sector 240082 [ 924.579969] Buffer I/O error on device sda1, logical block 119017 [ 924.586047] Buffer I/O error on device sda1, logical block 119018 [ 924.592124] Buffer I/O error on device sda1, logical block 119019 ... [ 1422.769804] Buffer I/O error on device sda1, logical block 135410 [ 1422.977894] EXT4-fs error (device sda1) in ext4_writepages:2540: IO failure [ 1423.163072] EXT4-fs error (device sda1): ext4_journal_check_start:56: Detected aborted journal [ 1423.171705] EXT4-fs (sda1): Remounting filesystem read-only [ 1423.880858] EXT4-fs (sda1): ext4_writepages: jbd2_start: 1024 pages, ino 31; err -30 [ 1433.542568] hub 1-0:1.0: cannot reset port 1 (err = -110)
This is using the 3.11.3-301.fc20.armv7hl kernel.
Searching the list archives, I see this has come up before, but there doesn't seem to be a resolution:
https://lists.fedoraproject.org/pipermail/arm/2013-January/005114.html
Is this a kernel bug? Or is the SSD starting to fail?
Thanks, Jeff
Jeff:
Would you please try adjusting the read ahead higher? See if the problem is reproduced after.
Thanks On Oct 9, 2013 3:49 PM, "Jeffrey Bastian" jbastian@redhat.com wrote:
I just got a Trimslice Pro (a friend wasn't using it), and I booted the Fedora-Minimal-armhfp-20-Beta-TC2-sda image from an SD card and started copying it to the internal SSD, and started getting errors on the serial console:
[ 864.153171] hub 1-0:1.0: cannot reset port 1 (err = -110) [ 864.382165] hub 1-0:1.0: cannot reset port 1 (err = -110) [ 864.611136] hub 1-0:1.0: cannot reset port 1 (err = -110) [ 864.616551] hub 1-0:1.0: Cannot enable port 1. Maybe the USB cable is bad? ... [ 923.473666] hub 1-0:1.0: cannot reset port 1 (err = -110) [ 923.702667] hub 1-0:1.0: cannot reset port 1 (err = -110) [ 923.931671] hub 1-0:1.0: cannot reset port 1 (err = -110) [ 924.574479] end_request: I/O error, dev sda, sector 240082 [ 924.579969] Buffer I/O error on device sda1, logical block 119017 [ 924.586047] Buffer I/O error on device sda1, logical block 119018 [ 924.592124] Buffer I/O error on device sda1, logical block 119019 ... [ 1422.769804] Buffer I/O error on device sda1, logical block 135410 [ 1422.977894] EXT4-fs error (device sda1) in ext4_writepages:2540: IO failure [ 1423.163072] EXT4-fs error (device sda1): ext4_journal_check_start:56: Detected aborted journal [ 1423.171705] EXT4-fs (sda1): Remounting filesystem read-only [ 1423.880858] EXT4-fs (sda1): ext4_writepages: jbd2_start: 1024 pages, ino 31; err -30 [ 1433.542568] hub 1-0:1.0: cannot reset port 1 (err = -110)
This is using the 3.11.3-301.fc20.armv7hl kernel.
Searching the list archives, I see this has come up before, but there doesn't seem to be a resolution:
https://lists.fedoraproject.org/pipermail/arm/2013-January/005114.html
Is this a kernel bug? Or is the SSD starting to fail?
Thanks, Jeff _______________________________________________ arm mailing list arm@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/arm
On Wed, Oct 09, 2013 at 06:07:18PM -0500, Jon wrote:
Would you please try adjusting the read ahead higher? See if the problem is reproduced after.
I doubled it and unfortunately the problem still happened within a few seconds:
[root@trimslice ~]# blockdev --getra /dev/sda 256 [root@trimslice ~]# blockdev --setra 512 /dev/sda [root@trimslice ~]# blockdev --getra /dev/sda 512 [root@trimslice ~]# mount /dev/sda1 /mnt/boot [root@trimslice ~]# rsync -xav /boot/ /mnt/boot/
Over on the serial console:
[ 1635.044729] hub 1-0:1.0: cannot reset port 1 (err = -110) [ 1635.273814] hub 1-0:1.0: cannot reset port 1 (err = -110) [ 1635.279207] hub 1-0:1.0: Cannot enable port 1. Maybe the USB cable is bad?
I tried slowing down the rsync with nice -n19 ionice -c3 rsync --bwlimit=1024K ... but that didn't help.
The good news is the SSD is ok according to the badblocks program.
Jeff
On Thu, Oct 10, 2013 at 08:06:23AM -0500, Jeffrey Bastian wrote:
I tried slowing down the rsync with nice -n19 ionice -c3 rsync --bwlimit=1024K ... but that didn't help.
I tried cgroups is a bigger hammer to throttle I/O, but it didn't help either.
[root@trimslice ~]# ls -l /dev/sda brw-rw----. 1 root disk 8, 0 Dec 31 1969 /dev/sda [root@trimslice ~]# echo "8:0 131072" > /sys/fs/cgroup/blkio/blkio.throttle.write_bps_device [root@trimslice ~]# cat /sys/fs/cgroup/blkio/blkio.throttle.write_bps_device 8:0 131072
I then started with a fresh file system and copied /boot:
[root@trimslice ~]# mkfs.ext3 -L boot /dev/sda1 [root@trimslice ~]# mount /dev/sda1 /mnt/boot [root@trimslice ~]# rsync -xav /boot/ /mnt/boot/
It took much longer with the throttled writes, but eventually the kernel started spewing errors:
[ 9949.502220] hub 1-0:1.0: cannot reset port 1 (err = -110) [ 9949.731333] hub 1-0:1.0: cannot reset port 1 (err = -110) [ 9949.736725] hub 1-0:1.0: Cannot enable port 1. Maybe the USB cable is bad? [ 9949.968686] hub 1-0:1.0: cannot reset port 1 (err = -110) [ 9950.197669] hub 1-0:1.0: cannot reset port 1 (err = -110) [ 9950.426677] hub 1-0:1.0: cannot reset port 1 (err = -110) [ 9950.655722] hub 1-0:1.0: cannot reset port 1 (err = -110) [ 9951.298181] end_request: I/O error, dev sda, sector 165122 [ 9951.303676] Buffer I/O error on device sda1, logical block 81537 [ 9951.309667] Buffer I/O error on device sda1, logical block 81538 [ 9951.315657] Buffer I/O error on device sda1, logical block 81539 [ 9951.321647] Buffer I/O error on device sda1, logical block 81540 [ 9951.327676] Buffer I/O error on device sda1, logical block 81541 [ 9951.333670] Buffer I/O error on device sda1, logical block 81542 [ 9951.339661] Buffer I/O error on device sda1, logical block 81543 [ 9951.345651] Buffer I/O error on device sda1, logical block 81544 [ 9951.351687] Buffer I/O error on device sda1, logical block 81545 [ 9951.357680] Buffer I/O error on device sda1, logical block 81546 [10069.419378] hub 1-0:1.0: cannot reset port 1 (err = -110) [10069.648520] hub 1-0:1.0: cannot reset port 1 (err = -110)
Oh well. I guess I'll just keep /boot on the SD card.
Hopefully the Utilite works a little better.
Thanks! Jeff
On Thu, Oct 10, 2013 at 10:24:57AM -0500, Jeffrey Bastian wrote:
I tried cgroups as a bigger hammer to throttle I/O, but it didn't help either.
I take it back: it did help a bit. Although I still saw lots of I/O errors from the kernel, the rsync was able to finish successfully.
Previously after enough errors accumulated, the kernel would re-mount /mnt/sysimage/boot as read-only and my rsync would fail, but with the cgroup throttling down writes, it was able to finish copying and the mount remained read-write the entire time.
With a quick update of the UUIDs in /etc/fstab and /boot/boot.{cmd,scr} the Trimslice is booting F20-Beta-TC2 from the SSD now.
Jeff
On 10/10/2013 05:28 PM, Jeffrey Bastian wrote:
On Thu, Oct 10, 2013 at 10:24:57AM -0500, Jeffrey Bastian wrote:
I tried cgroups as a bigger hammer to throttle I/O, but it didn't help either.
I take it back: it did help a bit. Although I still saw lots of I/O errors from the kernel, the rsync was able to finish successfully.
Previously after enough errors accumulated, the kernel would re-mount /mnt/sysimage/boot as read-only and my rsync would fail, but with the cgroup throttling down writes, it was able to finish copying and the mount remained read-write the entire time.
With a quick update of the UUIDs in /etc/fstab and /boot/boot.{cmd,scr} the Trimslice is booting F20-Beta-TC2 from the SSD now.
Just out of interest, what make/model SSD are you using? My past experience shows that, contrary to what you might expect from supposedly standardized technologies, serious compatibility issues are not at all uncommon.
Gordan
On Thu, Oct 10, 2013 at 05:34:35PM +0100, Gordan Bobic wrote:
Just out of interest, what make/model SSD are you using? My past experience shows that, contrary to what you might expect from supposedly standardized technologies, serious compatibility issues are not at all uncommon.
It's whatever Trimslice put in their Pro model; it's entirely internal and not end-user-accessible. This is different from the H model which lets you add your own hard drive or SSD.
According to this forum, it's a Sandisk P4, PN SDSA4AH-032G. http://www.utilite-computer.com/forum/viewtopic.php?f=29&t=87
On Wed, Oct 9, 2013 at 9:49 PM, Jeffrey Bastian jbastian@redhat.com wrote:
I just got a Trimslice Pro (a friend wasn't using it), and I booted the Fedora-Minimal-armhfp-20-Beta-TC2-sda image from an SD card and started copying it to the internal SSD, and started getting errors on the serial console:
[ 864.153171] hub 1-0:1.0: cannot reset port 1 (err = -110) [ 864.382165] hub 1-0:1.0: cannot reset port 1 (err = -110) [ 864.611136] hub 1-0:1.0: cannot reset port 1 (err = -110) [ 864.616551] hub 1-0:1.0: Cannot enable port 1. Maybe the USB cable is bad? ... [ 923.473666] hub 1-0:1.0: cannot reset port 1 (err = -110) [ 923.702667] hub 1-0:1.0: cannot reset port 1 (err = -110) [ 923.931671] hub 1-0:1.0: cannot reset port 1 (err = -110) [ 924.574479] end_request: I/O error, dev sda, sector 240082 [ 924.579969] Buffer I/O error on device sda1, logical block 119017 [ 924.586047] Buffer I/O error on device sda1, logical block 119018 [ 924.592124] Buffer I/O error on device sda1, logical block 119019 ... [ 1422.769804] Buffer I/O error on device sda1, logical block 135410 [ 1422.977894] EXT4-fs error (device sda1) in ext4_writepages:2540: IO failure [ 1423.163072] EXT4-fs error (device sda1): ext4_journal_check_start:56: Detected aborted journal [ 1423.171705] EXT4-fs (sda1): Remounting filesystem read-only [ 1423.880858] EXT4-fs (sda1): ext4_writepages: jbd2_start: 1024 pages, ino 31; err -30 [ 1433.542568] hub 1-0:1.0: cannot reset port 1 (err = -110)
This is using the 3.11.3-301.fc20.armv7hl kernel.
Searching the list archives, I see this has come up before, but there doesn't seem to be a resolution:
https://lists.fedoraproject.org/pipermail/arm/2013-January/005114.html
Is this a kernel bug? Or is the SSD starting to fail?
I suspect neither is actually the answer. We've had a quite regular issue with timeouts of the SSD on the trimslice pros that we've never managed to nail down well. It could be a problem with the SSD, the USB HW, or the kernel drivers for the tegra USB controller. The issue seems to come and go. The way I use my TS pro is to use the SSD as /home and have root on a SD card but that's primarily as it allows me to swap out the SD card for different releases and easy testing. In that config the SSD seems pretty stable but the few times I tried to install the OS onto the SSD it had time out issues but that's back in ther 3.4 - 3.6 kernel days so not recently.
Peter
On 10/10/2013 10:59 AM, Peter Robinson wrote:
On Wed, Oct 9, 2013 at 9:49 PM, Jeffrey Bastian jbastian@redhat.com wrote:
I just got a Trimslice Pro (a friend wasn't using it), and I booted the Fedora-Minimal-armhfp-20-Beta-TC2-sda image from an SD card and started copying it to the internal SSD, and started getting errors on the serial console:
[ 864.153171] hub 1-0:1.0: cannot reset port 1 (err = -110) [ 864.382165] hub 1-0:1.0: cannot reset port 1 (err = -110) [ 864.611136] hub 1-0:1.0: cannot reset port 1 (err = -110) [ 864.616551] hub 1-0:1.0: Cannot enable port 1. Maybe the USB cable is bad? ... [ 923.473666] hub 1-0:1.0: cannot reset port 1 (err = -110) [ 923.702667] hub 1-0:1.0: cannot reset port 1 (err = -110) [ 923.931671] hub 1-0:1.0: cannot reset port 1 (err = -110) [ 924.574479] end_request: I/O error, dev sda, sector 240082 [ 924.579969] Buffer I/O error on device sda1, logical block 119017 [ 924.586047] Buffer I/O error on device sda1, logical block 119018 [ 924.592124] Buffer I/O error on device sda1, logical block 119019 ... [ 1422.769804] Buffer I/O error on device sda1, logical block 135410 [ 1422.977894] EXT4-fs error (device sda1) in ext4_writepages:2540: IO failure [ 1423.163072] EXT4-fs error (device sda1): ext4_journal_check_start:56: Detected aborted journal [ 1423.171705] EXT4-fs (sda1): Remounting filesystem read-only [ 1423.880858] EXT4-fs (sda1): ext4_writepages: jbd2_start: 1024 pages, ino 31; err -30 [ 1433.542568] hub 1-0:1.0: cannot reset port 1 (err = -110)
This is using the 3.11.3-301.fc20.armv7hl kernel.
Searching the list archives, I see this has come up before, but there doesn't seem to be a resolution:
https://lists.fedoraproject.org/pipermail/arm/2013-January/005114.html
Is this a kernel bug? Or is the SSD starting to fail?
I suspect neither is actually the answer. We've had a quite regular issue with timeouts of the SSD on the trimslice pros that we've never managed to nail down well. It could be a problem with the SSD, the USB HW, or the kernel drivers for the tegra USB controller. The issue seems to come and go. The way I use my TS pro is to use the SSD as /home and have root on a SD card but that's primarily as it allows me to swap out the SD card for different releases and easy testing. In that config the SSD seems pretty stable but the few times I tried to install the OS onto the SSD it had time out issues but that's back in ther 3.4 - 3.6 kernel days so not recently.
Maybe I'm not quite comparing like-for-like here, but I suspect it's more to do with the SATA controller than the Tegra USB controller. I am using a heavily modified Toshiba AC100 (Tegra2 based, same as TrimSlice) with a SuperTalent RC8 USB SSD (internally containing USB->SATA adapter, and a Sandforce controller) and have never seen any disk dropout issues. I am also NOT using a vanilla kernel.
Gordan
On Thu, Oct 10, 2013 at 10:59:54AM +0100, Peter Robinson wrote:
The way I use my TS pro is to use the SSD as /home and have root on a SD card
Hmm, that's interesting. I noticed that I only had the USB errors when copying the /boot filesystem to sda1. When I copied the root filesystem to sda3, it copied over 1 GiB with no errors at all.
Maybe I'll keep /boot on an SD card also. I'm doing something similar on my Pandaboard with /boot on the SD card and the rest on a USB thumb drive.
Jeff
On Wed, Oct 09, 2013 at 03:49:23PM -0500, Jeffrey Bastian wrote:
I just got a Trimslice Pro (a friend wasn't using it), and I booted the Fedora-Minimal-armhfp-20-Beta-TC2-sda image from an SD card
One last comment: I was really impressed with how easy it was to boot the F20 Beta! I just wrote the image to the SD card, inserted it into the Trimslice and turned it on! A few minutes later the firstboot step let me set a root password and I was on my way.
It's come a long way since I first tried booting Fedora 13 on my Pandaboard.
Thank you for all the hard work and polish on the distro!
Jeff