On Wed, 16 Oct 2019 at 12:41, Kevin Fenzi <kevin(a)scrye.com> wrote:
On Wed, Oct 16, 2019 at 10:47:00AM +0200, Pierre-Yves Chibon wrote:
> Good Morning Everyone,
>
> This morning I found out that
https://pagure.io/fedora-infrastructure was not
> available, it was throwing a 500 error on every page/call.
>
> I checked the logs and found:
> GitError: Error performing curl request: (60): Peer certificate cannot be
> authenticated with given CA certificates
>
> The combination and "GitError" and a SSL related error led me to
repoSpanner.
> So with the help of Patrick, we confirmed that the SSL cert for pagure01 was
> expiring on Oct 15th 2019.
> We then regenerated that SSL cert.
>
> We thought the repospanner playbook was going to redeploy that cert so I ran it,
> but it did not change anything (both in its run as well as in the symptoms
> observed).
>
> We then found out that this piece is actually part of the pagure.yml playbook,
> so I've ran it with `-t repospanner/server` to limit its effect.
> Then I've restarted httpd, stunnel and repospanner(a)ansible.service on pagur01.
> The first two were likely not necessary, the last one was to get the new cert in
> use.
>
> So I would like retro-active approval for my actions since the systems I've
> touched are frozen.
So a few things:
1) +1 to the actions... thanks for fixing that!
Agreed +1
2) we need nagios monitoring those certs, or we need to just tear
down that cluster if we aren't going to use it (which we are currently
not).
Yep. These are difficult to monitor and build because we don't use
openssl to build them like other certs.. but the repospanner command
itself.
3) We could also 'unrepospanner' that repo since we
aren't using it
and put the old one back.
Agreed.
4) pagure perhaps should gracefully print 'sorry, the repo is
not
available right now due to a repospanner problem' but otherwise work?
Might be good also
For 2 and 3, perhaps we should discuss and decide.
For 4, can you file a pagure bug (if you agree).
kevin
_______________________________________________
infrastructure mailing list -- infrastructure(a)lists.fedoraproject.org
To unsubscribe send an email to infrastructure-leave(a)lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedora...
--
Stephen J Smoogen.