I received this test failure https://taskotron.fedoraproject.org/artifacts/all/ad295f84-7690-11e7-a988-5254008e42f6/task_output/rpmgrill.json:
{ "module" : "VirusCheck", "order" : 2, "results" : [ { "arch" : "src", "code" : "ClamAV", "context" : { "path" : "/libwbxml-0.11.5.tar.bz2" }, "diag" : "ClamAV <b>Win.Trojan.Ramnit-5550</b> subtest triggered (ClamAV 0.99.2/21723/Mon Jun 13 13:53:00 2016)", "subpackage" : "libwbxml" } ], "run_time" : 9, "status" : "completed" },
I checked it locally in Fedora 27 with the same result. The tarball contains win32/expat/libexpat.dll that triggers the antivirus.
But look at the clamav database version (21723/Mon Jun 13 13:53:00 2016). It's more than a year old. After updating the database (you need to install clamav-update package and wait for a cronjob or run /usr/bin/freshclam as root manually), the hit went away. It's a false positive caused by Tasktron not updating the virus database.
Please update the virus database regularly.
-- Petr
On Tue, Aug 1, 2017 at 11:18 AM, Petr Pisar ppisar@redhat.com wrote:
I received this test failure https://taskotron.fedoraproject.org/artifacts/ all/ad295f84-7690-11e7-a988-5254008e42f6/task_output/rpmgrill.json:
{ "module" : "VirusCheck", "order" : 2, "results" : [ { "arch" : "src", "code" : "ClamAV", "context" : { "path" : "/libwbxml-0.11.5.tar.bz2" }, "diag" : "ClamAV <b>Win.Trojan.Ramnit-5550</b> subtest triggered (ClamAV 0.99.2/21723/Mon Jun 13 13:53:00 2016)", "subpackage" : "libwbxml" } ], "run_time" : 9, "status" : "completed" },
I checked it locally in Fedora 27 with the same result. The tarball contains win32/expat/libexpat.dll that triggers the antivirus.
But look at the clamav database version (21723/Mon Jun 13 13:53:00 2016). It's more than a year old. After updating the database (you need to install clamav-update package and wait for a cronjob or run /usr/bin/freshclam as root manually), the hit went away. It's a false positive caused by Tasktron not updating the virus database.
Please update the virus database regularly.
-- Petr
Hi Petr,
thanks for the report. In the task, we just install rpmgrill and run it. If rpmgrill reports outdated clamav results, it seems that's something that should be fixed in rpmgrill itself (it could depend on clamav-update and update the virus db before running the virus check). Can you please report a bug against rpmgrill and post the link here? (We could do it ourselves before running rpmgrill, but it seems cleaner to me if rpmgrill itself performs this, because that's the tool responsible for running the virus check - we're just a wrapper around it that schedules in on every koji build. Furthermore, having it in rpmgrill will fix this for all users of rpmgrill, regardless of execution). Thanks a lot.
Kamil
On Tue, Aug 01, 2017 at 11:59:41AM +0200, Kamil Paral wrote:
thanks for the report. In the task, we just install rpmgrill and run it. If rpmgrill reports outdated clamav results, it seems that's something that should be fixed in rpmgrill itself (it could depend on clamav-update and update the virus db before running the virus check). Can you please report a bug against rpmgrill and post the link here?
https://bugzilla.redhat.com/show_bug.cgi?id=1477130
-- Petr
Dear Petr,
On Tue, Aug 01, 2017 at 01:05:34PM +0200, Petr Pisar wrote:
On Tue, Aug 01, 2017 at 11:59:41AM +0200, Kamil Paral wrote:
thanks for the report. In the task, we just install rpmgrill and run it. If rpmgrill reports outdated clamav results, it seems that's something that should be fixed in rpmgrill itself (it could depend on clamav-update and update the virus db before running the virus check). Can you please report a bug against rpmgrill and post the link here?
Many thanks for the report. I haven't looked into the specifics, but I'm not sure what we envision rpmgrill should be doing here. Should it run a freshclam every invokation of rpmgrill? From what I see, that can take quite a bit of time.
Maybe what could be done tho is a systemd timer installed with the package which runs freshclam every now and than?
Kind Regards,
Excerpts from Róman Joost's message of 2017-08-02 10:18 +10:00:
Dear Petr,
On Tue, Aug 01, 2017 at 01:05:34PM +0200, Petr Pisar wrote:
On Tue, Aug 01, 2017 at 11:59:41AM +0200, Kamil Paral wrote:
thanks for the report. In the task, we just install rpmgrill and run it. If rpmgrill reports outdated clamav results, it seems that's something that should be fixed in rpmgrill itself (it could depend on clamav-update and update the virus db before running the virus check). Can you please report a bug against rpmgrill and post the link here?
Many thanks for the report. I haven't looked into the specifics, but I'm not sure what we envision rpmgrill should be doing here. Should it run a freshclam every invokation of rpmgrill? From what I see, that can take quite a bit of time.
Maybe what could be done tho is a systemd timer installed with the package which runs freshclam every now and than?
This might not work well if rpmgrill is invoked as part of some system which creates a fresh VM from scratch and then deletes it shortly after (think single-use slaves with Jenkins OpenStack Cloud plugin, for example). The freshclam timer will likely never trigger before rpmgrill is run.
One possible thing which might help is if rpmgrill could warn or even fail, if it detects that it's being run with "outdated" ClamAV definitions. Not sure how old you want to consider "outdated", or if there is an easy way to check it... At worst I guess something like, if modtime of ClamAV definitions is more than 4 weeks in the past give an error? Do we know how frequently the definitions are updated?
The other thing is that this idea of "download some data from the internet in order to make this package work" is not a good approach. It breaks in exactly the scenario I mentioned above, where a freshly installed copy of the package is not actually usable. The pciids and usbids database used to be like this too (shipping some old version of the data, plus a cron job to pull down updates from the internet) but nowadays we have the hwdata package which just gets updated with the latest definitions once per month. This is a much nicer solution because it means you can install a machine using only Fedora packages (or a freshly built disk image) and it already has the data it needs, without then going back to some random server on the internet.
So maybe the ClamAV definitions should be treated similarly? In a separate package which gets updated on a regular interval to pull in the latest data?
On Wed, Aug 2, 2017 at 2:44 AM, Dan Callaghan dcallagh@redhat.com wrote:
Excerpts from Róman Joost's message of 2017-08-02 10:18 +10:00:
Dear Petr,
On Tue, Aug 01, 2017 at 01:05:34PM +0200, Petr Pisar wrote:
On Tue, Aug 01, 2017 at 11:59:41AM +0200, Kamil Paral wrote:
thanks for the report. In the task, we just install rpmgrill and run
it. If
rpmgrill reports outdated clamav results, it seems that's something
that
should be fixed in rpmgrill itself (it could depend on clamav-update
and
update the virus db before running the virus check). Can you please
report
a bug against rpmgrill and post the link here?
Many thanks for the report. I haven't looked into the specifics, but I'm not sure what we envision rpmgrill should be doing here. Should it run a freshclam every invokation of rpmgrill? From what I see, that can take quite a bit of time.
That was my initial idea, yes. But now that I tried it, I see two problems: a) freshclam needs to run as root, therefore rpmgrill executed under a standard user can't update it anyway. It could warn about it, and trigger the update if running under root (taskotron case), but it's not a generic solution which makes sure the results are up-to-date. b) The database refresh is SLOW. I expected just a few seconds. But here's my experience: https://paste.fedoraproject.org/paste/3oGdfMpOP84~rhQm34AZvw/raw Three minutes full of timing out. This can probably be much longer, if servers are in even a worse condition some day.
Maybe what could be done tho is a systemd timer installed with the package which runs freshclam every now and than?
This might not work well if rpmgrill is invoked as part of some system which creates a fresh VM from scratch and then deletes it shortly after (think single-use slaves with Jenkins OpenStack Cloud plugin, for example). The freshclam timer will likely never trigger before rpmgrill is run.
That's Taskotron case, yes. After looking more into this, I guess I wouldn't object updating clamav database before running rpmgrill. The problem I have with it is how slow it is. We run rpmgrill on every new koji builds, that means very frequently. Each run is performed on a clean machine, meaning each rpmgrill execution takes 3+ minutes longer just because of clamav servers being horrible. I'd definitely add a timeout and kill the process if it did not finish in 5 minutes or so, but even the usual e.g. 3 minutes of extra execution time (mostly idle time) is something I don't like.
One possible thing which might help is if rpmgrill could warn or even fail, if it detects that it's being run with "outdated" ClamAV definitions. Not sure how old you want to consider "outdated", or if there is an easy way to check it... At worst I guess something like, if modtime of ClamAV definitions is more than 4 weeks in the past give an error? Do we know how frequently the definitions are updated?
The other thing is that this idea of "download some data from the internet in order to make this package work" is not a good approach. It breaks in exactly the scenario I mentioned above, where a freshly installed copy of the package is not actually usable. The pciids and usbids database used to be like this too (shipping some old version of the data, plus a cron job to pull down updates from the internet) but nowadays we have the hwdata package which just gets updated with the latest definitions once per month. This is a much nicer solution because it means you can install a machine using only Fedora packages (or a freshly built disk image) and it already has the data it needs, without then going back to some random server on the internet.
Very much agreed.
So maybe the ClamAV definitions should be treated similarly? In a separate package which gets updated on a regular interval to pull in the latest data?
That would be the best solution here, yes. Could someone please file an RFE against clamav?
On Wed, Aug 02, 2017 at 04:53:20PM +0200, Kamil Paral wrote:
On Wed, Aug 2, 2017 at 2:44 AM, Dan Callaghan dcallagh@redhat.com wrote: [...]
The other thing is that this idea of "download some data from the internet in order to make this package work" is not a good approach. It breaks in exactly the scenario I mentioned above, where a freshly installed copy of the package is not actually usable. The pciids and usbids database used to be like this too (shipping some old version of the data, plus a cron job to pull down updates from the internet) but nowadays we have the hwdata package which just gets updated with the latest definitions once per month. This is a much nicer solution because it means you can install a machine using only Fedora packages (or a freshly built disk image) and it already has the data it needs, without then going back to some random server on the internet.
Very much agreed.
So maybe the ClamAV definitions should be treated similarly? In a separate package which gets updated on a regular interval to pull in the latest data?
That would be the best solution here, yes. Could someone please file an RFE against clamav?
Done:
https://bugzilla.redhat.com/show_bug.cgi?id=1477777
PS: I like the bug number :))
Kind Regards,
So maybe the ClamAV definitions should be treated similarly? In a separate package which gets updated on a regular interval to pull in the latest data?
That would be the best solution here, yes. Could someone please file an
RFE
against clamav?
Done:
https://bugzilla.redhat.com/show_bug.cgi?id=1477777
The clamav maintainer doesn't seem to have the time needed to provide a regularly updated clamav-data package. So today I tried to update the rpmgrill task with running freshclam before running rpmgrill, using clamav.upjs.sk mirror. That mirror is very fast when I try it (as opposed to the default mirror). However, when running it in our infra, I hit "WARNING: Mirror 158.197.16.70 is not synchronized." errors with clamav.upjs.sk mirror [1] and extremely long times with the default mirror (my patience ran out after 5 minutes).
So, sorry, I give up. Unless we can convince someone to maintain a regularly updated clamav-data package, or provide a reliable and fast clamav mirror (that is willing to be hit very often from our infra), I'm afraid we can't offer an updated clamav database for our tests.
[1] https://paste.fedoraproject.org/paste/hqewIGsDK8XFn9O6B0bPPQ/raw
qa-devel@lists.fedoraproject.org