[PATCH] kojid: arm: fix host triplet on arm builders
by Nicolas Chauvet
armhfp architecture is using GNU Extended ABI to differenciate from
previous armv7 softfp/soft GNU one.
Currently, the fedora infrastructure doesn't maintain this old arm 32bit
abi so we can assume every koji arm* builders to use this "new" ABI.
Also, aarch64 builders are using another aarch64 prefix for the name
which this patch handles right since aarch64 is using redhat-linux-gnu.
This patch should have limited or no impact to most gcc based packages
because gcc knowns the %{_host} triplet for itself. But it will fix
llvm/clang build usability which rely on the appropriate host triplet
to interface with gcc.
This was tested with llvm 3.5.2/3.6.2 to build mesa with clang.
reported as https://fedorahosted.org/fedora-infrastructure/ticket/4858
Signed-off-by: Nicolas Chauvet <kwizart(a)gmail.com>
---
roles/koji_builder/templates/kojid.conf | 4 ++++
1 files changed, 4 insertions(+), 0 deletions(-)
diff --git a/roles/koji_builder/templates/kojid.conf b/roles/koji_builder/templates/kojid.conf
index 9bb9a29..7c1b55b 100644
--- a/roles/koji_builder/templates/kojid.conf
+++ b/roles/koji_builder/templates/kojid.conf
@@ -33,7 +33,11 @@ packager=Fedora Project
distribution=Fedora Project
; The _host string to use in mock
+{% if ansible_hostname.startswith('arm') %}
+mockhost=redhat-linux-gnueabi
+{% else %}
mockhost=redhat-linux-gnu
+{% endif %}
; The URL for the xmlrpc server
server={{koji_server_url}}
--
1.7.2.1
8 years, 8 months
Move to mailman3
by Pierre-Yves Chibon
Good morning everyone,
As you may know, mailman3 has been released and with it the new archiver
bridging mailing lists and forums: HyperKitty.
Some of the automated mailing list have been already migrated to mailman3,
but we still need to migrate 'live' mailing list (ie: having user actually
discussing, not just bot sending emails).
There has been thoughts to use the infrastructure list as a potential first list
to migrate.
Thoughts on this?
Pierre
To give you an idea: HyperKitty for scm-commits:
https://lists.fedoraproject.org/archives/list/scm-commits@lists.fedorapro...
8 years, 8 months
END OF FREEZE: Re: Fedora 23 Alpha Freeze now in effect
by Stephen John Smoogen
The Fedora 23 Alpha freeze is over as we have shipped an alpha. Thank
you to everyone who worked on things.
On 28 July 2015 at 14:02, Kevin Fenzi <kevin(a)scrye.com> wrote:
> Greetings.
>
> we are now in the infrastructure freeze leading up to the Fedora 23
> Alpha release. This is a pre-release freeze.
>
> We do this to ensure that our infrastructure is stable and ready to
> release the Fedora 23 Alpha when it's available.
>
> You can see a list of hosts that do not freeze by checking out the
> ansible repo and running the freezelist script:
>
> git clone https://infrastructure.fedoraproject.org/infra/ansible.git
> scripts/freezelist -i inventory
>
> Any hosts listed as freezes is frozen until 2015-08-12. (or later if
> Alpha slips). Frozen hosts should have no changes made to them
> without a sign-off on the change from at least 2 sysadmin-main or
> rel-eng members, along with (in most cases) a patch of the exact
> change to be made to this list.
>
> Thanks,
>
> kevin
>
> _______________________________________________
> infrastructure mailing list
> infrastructure(a)lists.fedoraproject.org
> https://admin.fedoraproject.org/mailman/listinfo/infrastructure
--
Stephen J Smoogen.
8 years, 8 months
Freeze break request: make only fas01 clear sessions, resolving a
deadlock
by Patrick Uiterwijk
Change in controllers.py is at line 101: "if socket.gethostname()...."
commit d0f4e6f6f956133da4116025eead691d4d96fbb7
Author: Patrick Uiterwijk <puiterwijk(a)redhat.com>
Date: Mon Aug 10 22:29:44 2015 +0000
HOTFIX: Make sure that only fas01clears sessions
This will prevent deadlocks in the SQL server
diff --git a/roles/fas_server/files/controllers.py b/roles/fas_server/files/controllers.py
new file mode 100644
index 0000000..6b15a46
--- /dev/null
+++ b/roles/fas_server/files/controllers.py
@@ -0,0 +1,263 @@
+# -*- coding: utf-8 -*-
+#
+# Copyright © 2008 Ricky Zhou
+# Copyright © 2008-2014 Red Hat, Inc.
+#
+# This copyrighted material is made available to anyone wishing to use, modify,
+# copy, or redistribute it subject to the terms and conditions of the GNU
+# General Public License v.2. This program is distributed in the hope that it
+# will be useful, but WITHOUT ANY WARRANTY expressed or implied, including the
+# implied warranties of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+# See the GNU General Public License for more details. You should have
+# received a copy of the GNU General Public License along with this program;
+# if not, write to the Free Software Foundation, Inc., 51 Franklin Street,
+# Fifth Floor, Boston, MA 02110-1301, USA. Any Red Hat trademarks that are
+# incorporated in the source code or documentation are not subject to the GNU
+# General Public License and may only be used or replicated with the express
+# permission of Red Hat, Inc.
+#
+# Author(s): Ricky Zhou <ricky(a)fedoraproject.org>
+# Mike McGrath <mmcgrath(a)redhat.com>
+# Toshio Kuratomi <toshio(a)redhat.com>
+#
+from bunch import Bunch
+
+from turbogears import expose, config, identity, redirect
+from turbogears.database import session
+from cherrypy import request
+
+import turbogears
+import cherrypy
+import time
+
+from fedora.tg import controllers as f_ctrlers
+from fedora.tg.utils import request_format
+
+from fas import release
+from fas.user import User
+from fas.group import Group
+from fas.configs import Config
+from fas.fpca import FPCA
+from fas.json_request import JsonRequest
+from fas.help import Help
+from fas.model import Session, People
+from fas.model import SessionTable
+
+
+from fas.auth import undeprecated_cla_done
+from fas.util import available_languages
+
+from fas import plugin
+
+import os
+
+import datetime
+
+import socket
+
+try:
+ import cPickle as pickle
+except ImportError:
+ import pickle
+
+class SQLAlchemyStorage:
+ def __init__(self):
+ pass
+
+ def load(self, session_id):
+ s = Session.query.get(session_id)
+ if not s:
+ return None
+ expiration_time = s.expiration_time
+ pickled_data = s.data
+ data = pickle.loads(pickled_data.encode('utf-8'))
+ return (data, expiration_time)
+
+ # This is an iffy one. CherryPy's built in session
+ # storage classes use delete(self, id=None), but it
+ # isn't called from anywhere in cherrypy. I think we
+ # can do this as long as we're careful about how we call it.
+ def delete(self, session_id=None):
+ if session_id is None:
+ session_id = cherrypy.session.id
+ s = Session.query.get(session_id)
+ session.delete(s)
+ session.flush()
+
+ def save(self, session_id, data, expiration_time):
+ pickled_data = pickle.dumps(data)
+ s = Session.query.get(session_id)
+ if not s:
+ s = Session()
+ s.id = session_id
+ s.data = pickled_data
+ s.expiration_time = expiration_time
+ session.flush()
+
+ def acquire_lock(self):
+ pass
+
+ def release_lock(self):
+ pass
+
+ def clean_up(self, sess):
+ # This is to make sure that only one server cleans up sessions
+ if socket.gethostname() != 'fas01.phx2.fedoraproject.org':
+ return
+ result = SessionTable.delete(
+ SessionTable.c.expiration_time.__lt__(datetime.datetime.now())
+ ).execute()
+
+config.update({'session_filter.storage_class': SQLAlchemyStorage})
+
+def get_locale(locale=None):
+ if locale:
+ return locale
+ try:
+ return turbogears.identity.current.user.locale
+ except AttributeError:
+ pass
+ try:
+ return cherrypy.request.simple_cookie['fas_locale'].value
+ except KeyError:
+ pass
+
+ default_language = config.get('default_language',
+ turbogears.i18n.utils._get_locale())
+ return default_language
+
+config.update({'i18n.get_locale': get_locale})
+
+
+def add_custom_stdvars(variables):
+ return variables.update({'gettext': _, "lang": get_locale(),
+ 'available_languages': available_languages(),
+ 'fas_version': release.VERSION,
+ 'webmaster_email': config.get('webmaster_email')})
+turbogears.view.variable_providers.append(add_custom_stdvars)
+
+# from fas import json
+# import logging
+# log = logging.getLogger("fas.controllers")
+
+#TODO: Appropriate flash icons for errors, etc.
+# mmcgrath wonders if it will be handy to expose an encrypted mailer with fas
+# over json for our apps
+
+class Root(plugin.RootController):
+
+ user = User()
+ group = Group()
+ fpca = FPCA()
+ json = JsonRequest()
+ config = Config()
+ help = Help()
+
+ def __init__(self):
+ # TODO: Find a better place for this.
+ os.environ['GNUPGHOME'] = config.get('gpghome')
+ plugin.RootController.__init__(self)
+
+ def getpluginident(self):
+ return 'fas'
+
+ @expose(template="fas.templates.welcome", allow_json=True)
+ def index(self):
+ if turbogears.identity.not_anonymous():
+ if request_format() == 'json':
+ # redirects don't work with JSON calls. This is a bit of a
+ # hack until we can figure out something better.
+ return dict()
+ turbogears.redirect('/home')
+ return dict(now=time.ctime())
+
+ @identity.require(identity.not_anonymous())
+ @expose(template="fas.templates.home", allow_json=True)
+ def home(self):
+ user_name = turbogears.identity.current.user_name
+ person = People.by_username(user_name)
+ (cla_done, undeprecated_cla) = undeprecated_cla_done(person)
+
+ person = person.filter_private()
+ return dict(person=person, memberships=person['memberships'], cla=undeprecated_cla)
+
+ @expose(template="fas.templates.about")
+ def about(self):
+ return dict()
+
+ @expose(template="fas.templates.login", allow_json=True)
+ def login(self, forward_url=None, *args, **kwargs):
+ '''Page to become authenticated to the Account System.
+
+ This shows a small login box to type in your username and password
+ from the Fedora Account System.
+
+ :kwarg forward_url: The url to send to once authentication succeeds
+ '''
+ actual_login_dict = f_ctrlers.login(forward_url=forward_url, *args, **kwargs)
+
+ try:
+ login_dict = Bunch()
+ login_dict['user'] = Bunch()
+ for field in People.allow_fields['complete']:
+ login_dict['user'][field] = None
+ for field in People.allow_fields['self']:
+ login_dict['user'][field] = getattr(actual_login_dict['user'], field)
+ # Strip out things that the user shouldn't see about their own
+ # login
+ login_dict['user']['internal_comments'] = None
+ login_dict['user']['emailtoken'] = None
+ login_dict['user']['security_answer'] = None
+ login_dict['user']['alias_enabled'] = None
+ login_dict['user']['passwordtoken'] = None
+
+ # Add things that are needed by some other apps
+ login_dict['user'].approved_memberships = list(
+ actual_login_dict['user'].approved_memberships)
+ login_dict['user'].memberships = list(actual_login_dict['user'].memberships)
+ login_dict['user'].unapproved_memberships = list(
+ actual_login_dict['user'].unapproved_memberships)
+ login_dict['user'].group_roles = list(actual_login_dict['user'].group_roles)
+ login_dict['user'].roles = list(actual_login_dict['user'].roles)
+ login_dict['user'].groups = [g.name for g in actual_login_dict['user'].approved_memberships]
+ return login_dict
+ except KeyError, e:
+ # No problem, this usually means that we failed to login and
+ # therefore we don't have a user field.
+ login_dict = actual_login_dict
+
+ if not identity.current.anonymous and identity.was_login_attempted() \
+ and not identity.get_identity_errors():
+ # Success that needs to be passed back via json
+ return login_dict
+
+ if identity.was_login_attempted() and request.fas_provided_username:
+ if request.fas_identity_failure_reason == 'status_inactive':
+ turbogears.flash(_('Your old password has expired. Please'
+ ' reset your password below.'))
+ if request_format() != 'json':
+ redirect('/user/resetpass')
+ if request.fas_identity_failure_reason == 'status_account_disabled':
+ turbogears.flash(_('Your account is currently disabled. For'
+ ' more information, please contact %(admin_email)s' %
+ {'admin_email': config.get('accounts_email')}))
+ if request_format() != 'json':
+ redirect('/login')
+
+ return login_dict
+
+ @expose(allow_json=True)
+ def logout(self):
+ return f_ctrlers.logout()
+
+ @expose()
+ def language(self, locale):
+ if locale not in available_languages():
+ turbogears.flash(_('The language \'%s\' is not available.') % locale)
+ redirect(request.headers.get("Referer", "/"))
+ return dict()
+ #turbogears.i18n.set_session_locale(locale)
+ cherrypy.response.simple_cookie['fas_locale'] = locale
+ redirect(request.headers.get("Referer", "/"))
+ return dict()
+
diff --git a/roles/fas_server/tasks/main.yml b/roles/fas_server/tasks/main.yml
index 57370a8..980013d 100644
--- a/roles/fas_server/tasks/main.yml
+++ b/roles/fas_server/tasks/main.yml
@@ -355,3 +355,12 @@
- config
- fas
- hotfixfas
+
+- name: HOTFIX make sure only fas01 cleans up sessions
+ copy: src={{ roles }}/fas_server/files/controllers.py
+ dest=/usr/lib/python2.6/site-packages/fas/controllers.py
+ mode=644 owner=root group=root
+ tags:
+ - config
+ - fas
+ - hotfixfas
8 years, 8 months
Retroactive freeze break request: Add indexes for ipsilon
by Patrick Uiterwijk
Hi,
So it turned out that the slowness of Ipsilon was caused by the lack of keys on the database,
so it got slower over time, as more entries got added.
I just ran the following queries to create new indexes on production, could I get any +1s retroactive?
-----------------------------------------------------
ALTER TABLE association ADD PRIMARY KEY (uuid,name);
CREATE INDEX ON association (uuid);
ALTER TABLE dbinfo ADD PRIMARY KEY (name, option);
CREATE INDEX ON dbinfo (name);
ALTER TABLE openid_data ADD PRIMARY KEY (name, option);
CREATE INDEX ON openid_data (name);
ALTER TABLE sessions ADD PRIMARY KEY (id);
CREATE INDEX ON sessions (expiration_time);
ALTER TABLE transactions ADD PRIMARY KEY (uuid, name);
CREATE INDEX ON transactions (uuid);
ALTER TABLE users ADD PRIMARY KEY (name, option);
CREATE INDEX ON users (name);
-----------------------------------------------------
With kind regards,
Patrick Uiterwijk
Fedora Infra
8 years, 8 months
bodhi2 plans
by Kevin Fenzi
Greetings.
So, in the releng meeting last week we discussed about landing bodhi2
before the alpha freeze started, however there were just too many
things still up in the air to do that, so we held off.
However, I'd like to propose a concrete date to enable it and try and
make sure we have everything we need lined up for making that work. :)
My understanding of the current status:
(Luke: please correct me if anything is wrong or unclear here)
* We have bodhi2 web frontend up and running fine in staging:
https://admin.stg.fedoraproject.org/updates/
(well, right now it's down as Luke is working on a update id bug there
and is repopulating the db, but it was and will be up again soon).
* We have a bodhi2 backend up in staging and it's successfully mashed
updates pushes.
* bodhi2 isn't yet packaged in Fedora (it and it's deps are in a copr):
http://copr.fedoraproject.org/coprs/lmacken/bodhi2/
So, to get that last bit we need:
* API users. I thought we had a list somewhere, but I cannot seem to
find it. At least fedora-easy-karma and fedora-gooey-karma use it.
Are all of these using python-fedora for the api? Do we need to
update that and bodhi-client?
* We need bodhi2 in fedora/epel. Were we going to do it as a seperate
package, or just an update to the existing 'bodhi' package. I assume
maintainers would need the new client to do 'fedpkg update' ?
* Do we need any fedpkg changes? Or anything else? How do we want to
handle the switchover?
* Is taskotron all ready to go with bodhi2? or more changes there?
* We will need an outage to switch over. I assume the bodhi1->bodhi2 db
conversion shouldn't take too long? But once we convert and make live
we are commmited.
* Anything else I missed?
I'd like to propose we schedule the switchover/outage for 2015-08-19.
(Provided alpha doesn't slip). This will allow us a few days to recover
from flock and get anything lined up we need to.
Thoughts? additions?
kevin
8 years, 8 months
Freeze Break Request - fedmsg authz policy
by Ralph Bean
I found a bug! The impact of it is that the releng primary arch 'compose'
fedmsg messages were being considered invalid by our consuming services
(ircbot, etc..) but the s390 compose messages were being let through.
First -- here's the patch, then the explanation:
diff --git a/inventory/host_vars/branched-composer.phx2.fedoraproject.org b/inventory/host_vars/branched-composer.phx2.fedoraproject.org
index bfe9b94..a5d3514 100644
--- a/inventory/host_vars/branched-composer.phx2.fedoraproject.org
+++ b/inventory/host_vars/branched-composer.phx2.fedoraproject.org
@@ -7,12 +7,3 @@ volgroup: /dev/vg_bvirthost08
kojipkgs_url: kojipkgs.fedoraproject.org
kojihub_url: koji.fedoraproject.org/kojihub
kojihub_scheme: https
-
-# These are consumed by a task in roles/fedmsg/base/main.yml
-fedmsg_certs:
-- service: shell
- owner: root
- group: root
-- service: bodhi
- owner: root
- group: masher
diff --git a/inventory/host_vars/rawhide-composer.phx2.fedoraproject.org b/inventory/host_vars/rawhide-composer.phx2.fedoraproject.org
index a0d17a6..9cb3409 100644
--- a/inventory/host_vars/rawhide-composer.phx2.fedoraproject.org
+++ b/inventory/host_vars/rawhide-composer.phx2.fedoraproject.org
@@ -6,12 +6,3 @@ volgroup: /dev/vg_bvirthost06
kojipkgs_url: kojipkgs.fedoraproject.org
kojihub_url: koji.fedoraproject.org/kojihub
kojihub_scheme: https
-
-# These are consumed by a task in roles/fedmsg/base/main.yml
-fedmsg_certs:
-- service: shell
- owner: root
- group: root
-- service: bodhi
- owner: root
- group: masher
It is just *removing* lines from the host_vars files for rawhide-composer and
branched-composer. But why?
First, those fedmsg_certs vars are already defined at the group_vars level here:
https://infrastructure.fedoraproject.org/cgit/ansible.git/tree/inventory/...
That more fully-defined and correct statement at the group level was being overwritten
by the less fully defined statements at the host level (see ansible var
precedence rules). Everything appeared to be working normally while this was
the case because, since no hosts declared that they could send those fedmsg
topics there was no explicit check for who could send them.
It didn't matter until we added the s390 koji hub a week or so ago which is
allowed to broadcast those same topics. It had its fedmsg_certs correctly
defined, and since it declared that it could send those topics -- and no other
hosts made the same declarations -- the primary compose messages suddenly
started being considered invalid (unauthorized).
By removing these old crufty definitions at the host level and letting the
correct definition at the group level prevail -- all those hosts should show up
correctly in the fedmsg authz policy and things should start working again.
This will require a master playbook run on just the fedmsgdconfig tag to push
out making this a "high touch" change to do during freeze, but I'm quite
certain it is correct.
Can I get two +1s?
-Ralph
8 years, 8 months
RE: Move to mailman3
by Sander Hoentjen
Hey yeah I figured as much, haven't touched my procmailrc for as long as I can remember :)
Will go fix it now.
-----Original message-----
From: Aurelien Bompard <abompard(a)fedoraproject.org>
Sent: Friday 7th August 2015 20:18
To: Fedora Infrastructure <infrastructure(a)lists.fedoraproject.org>
Subject: Re: Move to mailman3
> And for me things show up in my inbox instead of in a folder, because I
> filter on X-BeenThere. This disappeared, not sure if it counts as a
> regression though :)
If I understand correctly, the X-BeenThere header has been obsoleted
some time ago (read: years ;-)), you can now filter on List-Id or
List-Post.
https://bugs.launchpad.net/mailman/+bug/883158
Sorry for the trouble, that's clearly something to warn people of.
Aurélien
8 years, 8 months
RE: Move to mailman3
by Sander Hoentjen
And for me things show up in my inbox instead of in a folder, because I filter on X-BeenThere. This disappeared, not sure if it counts as a regression though :)
Sander
-----Original message-----
From: Michael Cronenworth <mike(a)cchtml.com>
Sent: Friday 7th August 2015 19:42
To: infrastructure(a)lists.fedoraproject.org
Subject: Re: Move to mailman3
On 08/07/2015 12:37 PM, Aurelien Bompard wrote:
>> Might be an idea to send one when you start the migration as well :]
>> Just that we are not surprised or that we watch if one of our email doesn't land
>> on the list.
>
> Yeah, that would have been a good idea, but it's too late: your
> message was processed by Mailman3 already ;-)
> https://lists.fedoraproject.org/archives/list/infrastructure%40lists.fedo...
>
> I'm currently checking for bugs & regressions, feel free to ping me :-)
>
> Aurélien
>
Did you use an old subscription list? I have not been subscribed to @infra for about
a month and today I start receiving these from you guys.
Performing the unsubscribe process again does not send me a confirmation e-mail.
8 years, 8 months