Huge, huge thanks to everyone who worked on this; it was well communicated, surprisingly non-disruptive given the scope and impact, and it works!

-- 
Gwyn Ciesla
she/her/hers
------------------------------------------------ 
in your fear, seek only peace 
in your fear, seek only love
-d. bowie


Sent with ProtonMail Secure Email.

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Thursday, March 25, 2021 5:55 PM, Aoife Moloney <amoloney@redhat.com> wrote:

Evening/Morning all,

I am beyond pleased to announce that the Fedora Accounts Team has reached the point in our deployment that critical path services have been configured to the new solution and end user impact should be little to none, so we are now officially out of outage!

For a while we thought we may need some additional time to work through some misbehaving apps, but the group of people on the Fedora accounts team, and a number of people from the wider Fedora project gave up countless hours to help our team troubleshoot and test the deployment and it really sped up the outage.


Our status on services is now: GREEN! https://status.fedoraproject.org/

The outage ticket is now: CLOSED! https://pagure.io/fedora-infrastructure/issue/9747

To those of you who helped us, you know who you are, thank you so much! This deployment would not have been possible without your help. It truly was FOSS at its finest and was a privilege to see in action :)

Now to the details. The work is still not finished and there will probably be issues users will run into over the next few days. Our team can still be reached on IRC channel #fedora-aaa if you have any questions for us or need some urgent assistance, but we will be reviewing and actioning tickets tagged 'Noggin' over the coming days and weeks for fixes required.


Here's what work we have outstanding over the next few days: 
  • zodbot supybot-fedora needs adjusting
  • Elections & FMN PR merges
  • Any outstanding OTP for groups scripting
  • Any UI fixes required
  • Host configs & create some maintainer-tests to verify things are working
  • Firewall changes that are necessary
  • Fixes to opened tickets tagged 'Noggin'

And Here's what would we would like you to do if you hit any issues logging in over the next few days:
  • Clear your cookies
  • Clear your cache
  • Retry the login
  • If the above quick fixes dont work, please then file a ticket in https://pagure.io/fedora-infrastructure and tag it 'Noggin' so our team can review and respond

On behalf of the Fedora Accounts Team, once again thank you all so so much for your patience, understanding and most of all help over these last few days as we rolled out this new system to production.


Kindest regards & many thanks,
Aoife

--

Aoife Moloney

Product Owner

Community Platform Engineering Team

Red Hat EMEA

Communications House

Cork Road

Waterford