Is there any objection to making the ResultsDB database dumps public under https://infrastructure.fedoraproject.org/infra/db-dumps/ ?
I wanted to grab a database dump today, to populate a local database for development purposes. I was able to scp it from db-qa02 since I'm in sysadmin-qa, but I figured it would be nice to have it available publicly alongside those other applications' db dumps.
I can't think of any reason why the ResultsDB dumps need to be kept secret. There should be no sensitive data in them, right?
So if there are no objections I'll file an infra ticket for adding the dumps into the public-db-copy script.
Assuming that the schema [1] is correct, there should be no personal or secret information. Though it would be good to validate that there aren't any unexpected tables.
[1]:https://fedoraproject.org/wiki/AutoQA_resultsdb_schema
________________________________________
Harold Dost | @hdost
On Sep 25, 2017 01:30, "Dan Callaghan" dcallagh@redhat.com wrote:
Is there any objection to making the ResultsDB database dumps public under https://infrastructure.fedoraproject.org/infra/db-dumps/ ?
I wanted to grab a database dump today, to populate a local database for development purposes. I was able to scp it from db-qa02 since I'm in sysadmin-qa, but I figured it would be nice to have it available publicly alongside those other applications' db dumps.
I can't think of any reason why the ResultsDB dumps need to be kept secret. There should be no sensitive data in them, right?
So if there are no objections I'll file an infra ticket for adding the dumps into the public-db-copy script.
-- Dan Callaghan dcallagh@redhat.com Senior Software Engineer, Products & Technologies Operations Red Hat
_______________________________________________ qa-devel mailing list -- qa-devel@lists.fedoraproject.org To unsubscribe send an email to qa-devel-leave@lists.fedoraproject.org
On Mon, 25 Sep 2017 05:37:03 -0400 Harold Dost harolddost@gmail.com wrote:
Assuming that the schema [1] is correct, there should be no personal or secret information. Though it would be good to validate that there aren't any unexpected tables.
That schema was correct in the past but has not been kept up to date AFAIK.
That being said, none of the changes I can think of would cause a problem in making the database dump public.
Tim
_______________________________________
Harold Dost | @hdost
On Sep 25, 2017 01:30, "Dan Callaghan" dcallagh@redhat.com wrote:
Is there any objection to making the ResultsDB database dumps public under https://infrastructure.fedoraproject.org/infra/db-dumps/ ?
I wanted to grab a database dump today, to populate a local database for development purposes. I was able to scp it from db-qa02 since I'm in sysadmin-qa, but I figured it would be nice to have it available publicly alongside those other applications' db dumps.
I can't think of any reason why the ResultsDB dumps need to be kept secret. There should be no sensitive data in them, right?
So if there are no objections I'll file an infra ticket for adding the dumps into the public-db-copy script.
-- Dan Callaghan dcallagh@redhat.com Senior Software Engineer, Products & Technologies Operations Red Hat
qa-devel mailing list -- qa-devel@lists.fedoraproject.org To unsubscribe send an email to qa-devel-leave@lists.fedoraproject.org
On 09/25/2017 07:13 AM, Tim Flink wrote:
On Mon, 25 Sep 2017 05:37:03 -0400 Harold Dost harolddost@gmail.com wrote:
Assuming that the schema [1] is correct, there should be no personal or secret information. Though it would be good to validate that there aren't any unexpected tables.
That schema was correct in the past but has not been kept up to date AFAIK.
That being said, none of the changes I can think of would cause a problem in making the database dump public.
If you all are ok with it, I can make these public after the freeze is over.
kevin
On Mon, 2 Oct 2017 14:12:54 -0700 Kevin Fenzi kevin@scrye.com wrote:
On 09/25/2017 07:13 AM, Tim Flink wrote:
On Mon, 25 Sep 2017 05:37:03 -0400 Harold Dost harolddost@gmail.com wrote:
Assuming that the schema [1] is correct, there should be no personal or secret information. Though it would be good to validate that there aren't any unexpected tables.
That schema was correct in the past but has not been kept up to date AFAIK.
That being said, none of the changes I can think of would cause a problem in making the database dump public.
If you all are ok with it, I can make these public after the freeze is over.
Sounds good to me. Thanks for getting that taken care of.
Tim
On 10/03/2017 06:29 AM, Tim Flink wrote:
On Mon, 2 Oct 2017 14:12:54 -0700 Kevin Fenzi kevin@scrye.com wrote:
On 09/25/2017 07:13 AM, Tim Flink wrote:
On Mon, 25 Sep 2017 05:37:03 -0400 Harold Dost harolddost@gmail.com wrote:
Assuming that the schema [1] is correct, there should be no personal or secret information. Though it would be good to validate that there aren't any unexpected tables.
That schema was correct in the past but has not been kept up to date AFAIK.
That being said, none of the changes I can think of would cause a problem in making the database dump public.
If you all are ok with it, I can make these public after the freeze is over.
Sounds good to me. Thanks for getting that taken care of.
They should be there starting tomorrow.
kevin
On Sun, Oct 8, 2017 at 12:14 AM, Kevin Fenzi kevin@scrye.com wrote:
They should be there starting tomorrow.
I still don't see any dumps in there. Is it still being worked on?
Thanks.
On 10/24/2017 03:36 AM, Kamil Paral wrote:
On Sun, Oct 8, 2017 at 12:14 AM, Kevin Fenzi kevin@scrye.com wrote:
They should be there starting tomorrow.
I still don't see any dumps in there. Is it still being worked on?
I seem to have made a typo in the script that does this. ;(
Sent in a freeze break request with a fix
kevin
On Wed, Nov 1, 2017 at 7:48 PM, Kevin Fenzi kevin@scrye.com wrote:
On 10/24/2017 03:36 AM, Kamil Paral wrote:
On Sun, Oct 8, 2017 at 12:14 AM, Kevin Fenzi kevin@scrye.com wrote:
They should be there starting tomorrow.
I still don't see any dumps in there. Is it still being worked on?
I seem to have made a typo in the script that does this. ;(
Sent in a freeze break request with a fix
I and Josef looked at it today, and it seems it contains some very old data - from Jan 2017 to April 2017. We failed to find the script that generates the dump in infra ansible, so we can't really debug what's wrong. Can you please look at it or give us some pointers?
Thanks.
On 01/11/2018 07:44 AM, Kamil Paral wrote:
On Wed, Nov 1, 2017 at 7:48 PM, Kevin Fenzi <kevin@scrye.com mailto:kevin@scrye.com> wrote:
On 10/24/2017 03:36 AM, Kamil Paral wrote: > On Sun, Oct 8, 2017 at 12:14 AM, Kevin Fenzi <kevin@scrye.com <mailto:kevin@scrye.com>> wrote: > >> They should be there starting tomorrow. >> > > > I still don't see any dumps in there. Is it still being worked on? I seem to have made a typo in the script that does this. ;( Sent in a freeze break request with a fixI and Josef looked at it today, and it seems it contains some very old data - from Jan 2017 to April 2017. We failed to find the script that generates the dump in infra ansible, so we can't really debug what's wrong. Can you please look at it or give us some pointers?
Odd.
It's https://infrastructure.fedoraproject.org/cgit/ansible.git/tree/roles/batcave... which runs on batcave01 and runs: https://infrastructure.fedoraproject.org/cgit/ansible.git/plain/scripts/publ...
scp db-qa01.qa.fedoraproject.org:/resultsdb-$(date +%F).dump.xz /srv/web/infra/db-dumps/resultsdb.dump.xz
and on db-qa01.qa:/backups:
-rw-r--r--. 1 postgres postgres 142589024 Apr 12 2016 resultsdb-2016-04-12.dump.xz -rw-r--r--. 1 postgres postgres 50184088 Jun 20 2017 resultsdb-2017-06-20.dump.xz -rw-r--r--. 1 postgres postgres 50367072 Jan 13 01:59 resultsdb-2018-01-13.dump.xz -rw-r--r--. 1 postgres postgres 50367072 Jan 14 01:12 resultsdb-2018-01-14.dump.xz
It claims to be the dump from today... so not sure whats going on.
kevin
On Sun, Jan 14, 2018 at 11:19 PM, Kevin Fenzi kevin@scrye.com wrote:
Odd.
It's https://infrastructure.fedoraproject.org/cgit/ansible.git/tree/roles/ batcave/files/public-db-copy.sh?id=778dae3 which runs on batcave01 and runs: https://infrastructure.fedoraproject.org/cgit/ansible.git/plain/scripts/ public-db-copy?id=778dae3
scp db-qa01.qa.fedoraproject.org:/resultsdb-$(date +%F).dump.xz /srv/web/infra/db-dumps/resultsdb.dump.xz
That script copies over the dump. But how does the dump get created?
On Sun, 14 Jan 2018 14:19:54 -0800 Kevin Fenzi kevin@scrye.com wrote:
On 01/11/2018 07:44 AM, Kamil Paral wrote:
On Wed, Nov 1, 2017 at 7:48 PM, Kevin Fenzi <kevin@scrye.com mailto:kevin@scrye.com> wrote:
On 10/24/2017 03:36 AM, Kamil Paral wrote: > On Sun, Oct 8, 2017 at 12:14 AM, Kevin Fenzi <kevin@scrye.com > <mailto:kevin@scrye.com>> wrote: >> They should be there starting tomorrow. >> > > > I still don't see any dumps in there. Is it still being > worked on? I seem to have made a typo in the script that does this. ;( Sent in a freeze break request with a fixI and Josef looked at it today, and it seems it contains some very old data - from Jan 2017 to April 2017. We failed to find the script that generates the dump in infra ansible, so we can't really debug what's wrong. Can you please look at it or give us some pointers?
Odd.
It's https://infrastructure.fedoraproject.org/cgit/ansible.git/tree/roles/batcave... which runs on batcave01 and runs: https://infrastructure.fedoraproject.org/cgit/ansible.git/plain/scripts/publ...
scp db-qa01.qa.fedoraproject.org:/resultsdb-$(date +%F).dump.xz /srv/web/infra/db-dumps/resultsdb.dump.xz
and on db-qa01.qa:/backups:
-rw-r--r--. 1 postgres postgres 142589024 Apr 12 2016 resultsdb-2016-04-12.dump.xz -rw-r--r--. 1 postgres postgres 50184088 Jun 20 2017 resultsdb-2017-06-20.dump.xz -rw-r--r--. 1 postgres postgres 50367072 Jan 13 01:59 resultsdb-2018-01-13.dump.xz -rw-r--r--. 1 postgres postgres 50367072 Jan 14 01:12 resultsdb-2018-01-14.dump.xz
It claims to be the dump from today... so not sure whats going on.
The database for production resultsdb is on db-qa02.qa instead of db-qa01.qa.
From my read of those scripts, I suspect that is the cause of the public dumps being old.
Tim
On Mon, Jan 15, 2018 at 3:25 PM, Tim Flink tflink@redhat.com wrote:
On Sun, 14 Jan 2018 14:19:54 -0800 Kevin Fenzi kevin@scrye.com wrote:
On 01/11/2018 07:44 AM, Kamil Paral wrote:
On Wed, Nov 1, 2017 at 7:48 PM, Kevin Fenzi <kevin@scrye.com mailto:kevin@scrye.com> wrote:
On 10/24/2017 03:36 AM, Kamil Paral wrote: > On Sun, Oct 8, 2017 at 12:14 AM, Kevin Fenzi <kevin@scrye.com > <mailto:kevin@scrye.com>> wrote: >> They should be there starting tomorrow. >> > > > I still don't see any dumps in there. Is it still being > worked on? I seem to have made a typo in the script that does this. ;( Sent in a freeze break request with a fixI and Josef looked at it today, and it seems it contains some very old data - from Jan 2017 to April 2017. We failed to find the script that generates the dump in infra ansible, so we can't really debug what's wrong. Can you please look at it or give us some pointers?
Odd.
It's https://infrastructure.fedoraproject.org/cgit/ansible.git/tree/roles/
batcave/files/public-db-copy.sh?id=778dae3
which runs on batcave01 and runs: https://infrastructure.fedoraproject.org/cgit/ansible.git/plain/scripts/
public-db-copy?id=778dae3
scp db-qa01.qa.fedoraproject.org:/resultsdb-$(date +%F).dump.xz /srv/web/infra/db-dumps/resultsdb.dump.xz
and on db-qa01.qa:/backups:
-rw-r--r--. 1 postgres postgres 142589024 Apr 12 2016 resultsdb-2016-04-12.dump.xz -rw-r--r--. 1 postgres postgres 50184088 Jun 20 2017 resultsdb-2017-06-20.dump.xz -rw-r--r--. 1 postgres postgres 50367072 Jan 13 01:59 resultsdb-2018-01-13.dump.xz -rw-r--r--. 1 postgres postgres 50367072 Jan 14 01:12 resultsdb-2018-01-14.dump.xz
It claims to be the dump from today... so not sure whats going on.
The database for production resultsdb is on db-qa02.qa instead of db-qa01.qa.
From my read of those scripts, I suspect that is the cause of the public dumps being old.
Thanks for the fix, it's working properly now (Josef checked).
qa-devel@lists.fedoraproject.org
-
Dan Callaghan -
Harold Dost -
Kamil Paral -
Kevin Fenzi -
Tim Flink