On Mon, 2023-10-30 at 22:58 +1100, fedora(a)eyal.emu.id.au wrote:
On 30/10/2023 22.41, Patrick O'Callaghan wrote:
> On Mon, 2023-10-30 at 22:35 +1100, fedora(a)eyal.emu.id.au wrote:
> > On 30/10/2023 21.40, Patrick O'Callaghan wrote:
> > > On Mon, 2023-10-30 at 18:22 +1100, fedora(a)eyal.emu.id.au wrote:
> > > > Should I blame rsync (or the way I use it)?
> > > >
> > > > Next, I will do a plain 'cp -a' between the original source
> > > > (sdh
> > > > on
> > > > /sata) and target(md127 on /data1) to see how it goes.
> > >
> > > If you say how you use it, someone might be able to answer
> > > that.
> > > Give
> > > the exact command line with all options (e.g. if you have
> > > strict
> > > change
> > > detection you might be checksumming every file on the source
> > > before
> > > deciding to copy it).
> >
> > First, I already did this test and it still chokes after a few
> > GB.
> > The problem is still there
> > and it is not only an rsync problem. But why does rsync use 100%
> > CPU?
> >
> > I thought that I gave the command but I did not initially, yet
> > when I
> > show the output of 'ps' it says:
> > sudo rsync -aHSK --stats --progress --checksum-
> > choice=none --
> > no-compress -W /sata/backups/tapes /data1/no-backup/old-backups/
> > so there it is.
>
> Nothing obvious at first glance. The '-W' option should mean that
> rsync
> is simply copying files without it's usual delta detection, which
> could
> have been the culprit.
The culprit of rsync being slow.
I lost you here. 'the culprit' of what? It was very slow
initially.
I then added '-W' hoping to speed it up but it did not help.
I also added '--checksum-choice=none --no-compress' for the same
reason (probably
not needed) but none had an effect.
That's what I assumed.
I suspect this may be related to another thread I started earlier
'kworker consumes 100% CPU on degraded RAID6'
Possibly. A good test would be to repeat the experiment but with a
different target destination, such as /dev/null. This would indicate
whether the problem is with reading or with writing.
poc