Agenda Item: Introduction Larrry Brower
by Brower, Larry
Hi All,
My name is Larry and I would like to request permission to help with the Fedora sysadmin group.
I have been using Linux since the late 90's and have over 15 years of professional experience. I have worked with large web hosting providers as well as multinational oil and gas companies.
I hold the following certifications, RHCSA, CCNA R/S, CCDA, CCNP Voice and have started to study for the RHCE examination. I am willing and happy to learn new things.
I would like to work with the NOC and core administration or security administration teams.
Is there anyone available who is willing to sponsor me?
Thank you,
Larry Brower, RHCSA, CCNA R/S, CCDA, CCNP Voice
Communications Technician
Aramco Services Company
Aramco Research Center - Houston
16300 Park Row | Houston, Tx. 77084
Office: 713.432.4516 | Mobile: 832.570.5416 | Direct: 281.647.2987
larry.brower(a)aramcoservices.com
8 years, 11 months
Dist Git for Copr
by Adam Samalik
Hello everyone,
I would like to make a proposal and start a discussion about Dist Git for Copr.
At the moment, Copr can build packages only from source rpms, which has been uploaded in advance somewhere in the internet - so if you want to build something, you need your own file server. As people often delete the sources from their server after the build process, the builds are no longer easily reproducible. And there is also no history of the sources.
We would like to use the Dist Git project[1] to store the sources. This would make all builds reproducible, there would be no need of having a file server to build and we would have a history of changes to each package.
At first, we would use the Dist Git server as a storage only. Users would not have direct access. Instead, they would either:
a) upload their source rpms somewhere and use Copr as before (no change in work-flow)
b) submit the source rpms directly into Copr
In both cases, Copr would import ($ fedpkg import) the packages into the Dist Git and build them from there.
The design of git repos would be:
Repo name: 'user-name/copr-project/pkg-name'
Branch name: 'epel-7', 'fedora-22'
At that moment, we would need a discussion on how to open the Dist Git server to users.
- What would be the best web frontend?
- Could we use Pagure? (used to be progit)
- Which tool would they use?
- How would they authenticate?
There has been some unfinished discussion on https://fedorahosted.org/fedora-infrastructure/ticket/4564
The main purpouse of this is having a storage for sources, reproducible builds and history of sources. Eventualy, we can extend it with bug tracking, easy forking and merging and the other "Git-Hub-like" stuff.
Thanks!
Adam Samalik
[1] https://github.com/release-engineering/dist-git
8 years, 11 months
Freeze Break Request - reduce number of auto-disabled mirrors
by Adrian Reber
The new MM2 crawler disables mirrors which have failed to be
successfully crawled for 4 consecutive crawls. This seems to be a good
idea to reduce the total number of crawls by removing mirrors which are
just too slow. Unfortunately the current default timeout of 2 hours is
not enough. Especially for mirrors which mirror more than one category
as the timeout is per host and not per category. The problem is also not
network bound but it seems to be related to two crawlers updating the
directories of all mirrors on the same database at the same time. To
workaround this timeout problem I am now starting the crawler on the
second crawler 3 hours later and I have also increased the timeout from
2 hours to 3 hours. Additionally a small fix is included to also crawl
the last mirror in the database which was ignored until now.
After this is applied I would also re-enable the auto-disabled hosts
in the database.
Can I get two +1 for these changes?
Additionally I think we can remove the second crawler and just use the
times when the first crawler is idle to crawl other hosts. So instead of
starting a crawl every 12 hours on two crawlers we could crawl half of the
mirrors every 6 hours. But that is for after the freeze.
Adrian
commit 0dc0b70d9790b95cb6c1f41b4d36fb9aa2c9fbfc
Author: Adrian Reber <adrian(a)lisas.de>
Date: Wed May 13 11:23:21 2015 +0000
Start the crawl later on the second crawler.
Even with rsync as crawl method some hosts are taking a very long time
to be crawled. The network connection with rsync is only open for a
short time, but with both crawlers reading and writing from the database
it takes a very long time until the status of all directories is
updated. Therefore this patch introduces a 3 hour delay of the crawl
on the second crawler. This could also be solved with two different
cron.d files; one for each crawler.
diff --git a/roles/mirrormanager/crawler/files/crawler.cron b/roles/mirrormanager/crawler/files/crawler.cron
index 3d695ca..c74b915 100644
--- a/roles/mirrormanager/crawler/files/crawler.cron
+++ b/roles/mirrormanager/crawler/files/crawler.cron
@@ -1,4 +1,8 @@
# run the crawler twice a day
# logs sent to /var/log/mirrormanager/crawler.log and crawl/* by default
# 32GB of RAM is not enough for 75 threads, 38 seems to work so far
-0 */12 * * * mirrormanager /usr/bin/mm2_crawler --threads 38 `/usr/local/bin/run_crawler.sh 2` > /dev/null 2>&1
+#
+# [ "`hostname -s`" == "mm-crawler02" ] && sleep 3h is used to start the crawl
+# later on the second crawler to reduce the number of parallel accesses to
+# the database
+0 */12 * * * mirrormanager [ "`hostname -s`" == "mm-crawler02" ] && sleep 3h; /usr/bin/mm2_crawler --threads 38 `/usr/local/bin/run_crawler.sh 2` > /dev/null 2>&1
commit 06309516b88ffade6f00c78833bc62aa002d7f56
Author: Adrian Reber <adrian(a)lisas.de>
Date: Wed May 13 11:53:16 2015 +0000
Increase crawler timeout from 2h to 3h.
Since MM2 is in production about 140 mirrors have been auto-disabled
due to crawler timing out after 2 hours (default). Try if it works
better with 3 hours. This in combination with the previous commit
to decrease the load on the database should help to auto disable
less good mirrors. Especially mirrors who mirroring almost
everything can hardly be crawled within the 2 hour limit. Unfortunately
the limit is per host and not category.
diff --git a/roles/mirrormanager/crawler/files/crawler.cron b/roles/mirrormanager/crawler/files/crawler.cron
index c74b915..66801d7 100644
--- a/roles/mirrormanager/crawler/files/crawler.cron
+++ b/roles/mirrormanager/crawler/files/crawler.cron
@@ -5,4 +5,4 @@
# [ "`hostname -s`" == "mm-crawler02" ] && sleep 3h is used to start the crawl
# later on the second crawler to reduce the number of parallel accesses on
# the database
-0 */12 * * * mirrormanager [ "`hostname -s`" == "mm-crawler02" ] && sleep 3h; /usr/bin/mm2_crawler --threads 38 `/usr/local/bin/run_crawler.sh 2` > /dev/null 2>&1
+0 */12 * * * mirrormanager [ "`hostname -s`" == "mm-crawler02" ] && sleep 3h; /usr/bin/mm2_crawler --timeout-minutes 180 --threads 38 `/usr/local/bin/run_crawler.sh 2` > /dev/null 2>&1
commit 78c19a35d1706eb43e4e0c5fa202e0a6549674de
Author: Adrian Reber <adrian(a)lisas.de>
Date: Wed May 13 11:59:37 2015 +0000
Also crawl the last mirror in the database.
The last mirror in the database was not crawled and this adds '1' to
the --stopid if necessary.
diff --git a/roles/mirrormanager/crawler/files/run_crawler.sh b/roles/mirrormanager/crawler/files/run_crawler.sh
index b9d642e..3269dea 100644
--- a/roles/mirrormanager/crawler/files/run_crawler.sh
+++ b/roles/mirrormanager/crawler/files/run_crawler.sh
@@ -26,4 +26,7 @@ for i in `seq 1 ${NUMBER_OF_CRAWLERS}`; do
fi
let STARTID=${STARTID}+${PART}
let STOPID=${STOPID}+${PART}
+ if [ "${STOPID}" -eq "${MAX_HOST}" ]; then
+ let STOPID=${STOPID}+1
+ fi
done
8 years, 11 months
Plan for tomorrow's Fedora Infrastructure meeting (2015-05-14)
by Kevin Fenzi
The infrastructure team will be having it's weekly meeting tomorrow,
2015-05-07 at 18:00 UTC in #fedora-meeting on the freenode network.
This week we are continuing to try something new.
We have a gobby document
(see: https://fedoraproject.org/wiki/Gobby )
fedora-infrastructure-meeting-next is the document.
Please try and review and edit that document before the meeting and we
will use it to have our agenda of things to discuss. A copy as of this
morning is included in this email.
If you have something to discuss, add the topic to the discussion area
with your name. If you would like to teach other folks about some
application or setup in our infrastructure, please add that topic and
your name to the learn about section.
kevin
--
= Introduction =
This shared document is for the next fedora infrastructure meeting.
We will use it over the week before the meeting to gather status and
info and discussion items and so forth, then use it in the irc meeting
to transfer information to the meetbot logs.
= Meeting start stuff =
#startmeeting Infrastructure (2015-05-14)
#meetingname infrastructure
#topic aloha
#chair smooge relrod nirik abadger1999 lmacken dgilmore mdomsch
threebean pingou puiterwijk pbrobinson #topic New folks introductions /
Apprentice feedback
= Status / information / Trivia / Announcements =
(We put things here we want others on the team to know, but don't need
to discuss) (Please use #info <the thing> - your name)
#topic announcements and information
#info Classroom session on new fedora infrastructure private cloud from
msuchy:
http://meetbot.fedoraproject.org/fedora-classroom/2015-05-11/fedora-class...
#info pagure01 is now setup in production - kevin/pingou #info new
cloud nodes are now monitored in nagios - kevin #info we are now in
Fedora 22 Final freeze - kevin #info Wrote up initial migration page
for cloud migration:·
https://fedoraproject.org/wiki/Infrastructure_private_cloud_icehouse_migr...
- kevin #info transient cloud instance playbook reworked and working -
kevin #info
= Things we should discuss =
We use this section to bring up discussion topics. Things we want to
talk about as a group and come up with some consensus or decision or
just brainstorm a problem or issue. If there are none of these we skip
this section. (Use #topic your discussion topic - your username)
#topic What are we going to do about meetbot (whither supybot)? -
threebean #topic
= Learn about some application or setup in infrastructure =
(This section, each week we get 1 person to talk about an application
or setup that we have. Just going over what it is, how to contribute,
ideas for improvement, etc. Whoever would like to do this, just add the
info in this section. In the event we don't find someone to teach about
something, we skip this section and just move on to open floor.)
#topic Learn about:
= Meeting end stuff =
#topic Open Floor
#endmeeting
8 years, 11 months
Freeze Break Request - Declare fedmsg cert for mm2 backend
by Ralph Bean
First one of the season!
We're getting a traceback from fedmsg-logger in the
mm2_get_internet2_netblocks script on mm-backend01. It is complaining
that it can't find the cert it needs.
Here's a patch to our ansible repo that will tell it to use the cert
that's already on the system in that context:
Can I get two +1s to apply this and push it out to mm-backend01?
diff --git a/roles/fedmsg/base/templates/ssl.py.j2 b/roles/fedmsg/base/templates/ssl.py.j2
index aebc481..ab3e6c3 100644
--- a/roles/fedmsg/base/templates/ssl.py.j2
+++ b/roles/fedmsg/base/templates/ssl.py.j2
@@ -182,9 +182,10 @@ config = dict(
("mirrormanager2.mm-frontend01", "mirrormanager2-mm-frontend01.%s" % suffix),
("shell.mm-frontend01", "shell-mm-frontend01.%s" % suffix),
- ("shell.mm-backend01", "shell-mm-backend01.%s" % suffix),
("shell.mm-crawler01", "shell-mm-crawler01.%s" % suffix),
("shell.mm-crawler02", "shell-mm-crawler02.%s" % suffix),
+ ("shell.mm-backend01", "shell-mm-backend01.%s" % suffix),
+ ("mirrormanager.mm-backend01", "shell-mm-backend01.%s" % suffix),
("shell.anitya-frontend01", "shell-anitya-frontend01.fedoraproject.org"),
("anitya.anitya-frontend01", "anitya-anitya-frontend01.fedoraproject.org"),
8 years, 11 months
[release] pagure: 0.1.3
by Pierre-Yves Chibon
Good morning everyone,
Yesterday when I was looking into setting up pagure on EL7 again, I found out
that EL7 has gitolite3 but not gitolite 2 which pagure was relying on so far.
So this morning, with the help of Patrick, pagure has been ported to gitolite3
and I just cut a new pagure release.
Here is the changelog of the new version:
* Wed May 13 2015 Pierre-Yves Chibon <pingou(a)pingoured.fr> - 0.1.3-1
- Update to 0.1.3
- Add support for gitolite3
- Fix unit-tests suite to work on jenkins
Pierre
8 years, 11 months
Update on Statscache and Fedora releng dash
by Ratnadeep Debnath
Hi Ralph,
I have made quite some changes to statscache[1] and
statscache_plugins[2] to empower a faster prototype of releng dash[3],
currently working with only the AMIs plugin.
I have come up with a layout spec for releng plugin[4] to generate the
UI dynamically with minimum client side code, same code working for
all widgets in the dashboard.
How to test the changes:
1. cd into statscache_plugins/statscahe_plugins, and run 'python -m
unittest tests.releng' to seed initial data for AMIs plugin.
2. cd into statscache/ dir and run the flask server: python statscache/app.py
3. cd into fedora-releng-dash repo dir, and checkout to branch
'statscache', and open index.html in browser.
You should see the AMI widgets in the page.
Also, I think, we need to find a better way to pass the 'frequency'
param to instantiate models and plugins. Also, the data fetching part
could be moved to the plugins to allow plugin level optimizations, if
needed.
Please let me know your thoughts on the above changes.
[1]: https://github.com/fedora-infra/statscache/pull/4
[2]: https://github.com/fedora-infra/statscache_plugins/pull/1
[3]: https://github.com/fedora-infra/fedora-releng-dash/pull/20/
[4]: https://github.com/rtnpro/statscache_plugins/commit/f1ca6c85bf2a4cb22af93...
Regards,
rtnpro
--
Ratnadeep Debnath,
https://www.waartaa.com
GPG Fingerprint: 033C 8041 A0E9 CDBA 2E02 B785 2119 5486 F245 DFD6
8 years, 11 months
Fedora Cloud classroom
by Miroslav Suchý
As previously announced, I plan to held classroom lesson.
Mark your calendar:
11th May 2005
#fedora-classroom on irc.freenode.net
15:00 UTC (that is 17:00 CET, 11:00 EDT)
The lesson is intended primary for Fedora Infrastructure members (or
apprentices).
I will try to cover following topics:
* what is Fedora Cloud (spoiler: our OpenStack instance)
* which machines are used
* where is the documentation
* where are the playbooks and which files are important and where it is
stored in ansible.git
* which services are enabled and why
* how they are installed
* where are the weak parts, where you must be careful
Mirek Suchy
8 years, 11 months
[release] MirrorManager2: 0.1.0
by Pierre-Yves Chibon
Hi everyone,
I just cut and push to stg a new MirrorManager2 release: 0.1.0
Here is the changelog:
* Thu May 07 2015 Pierre-Yves Chibon <pingou(a)pingoured.fr> - 0.1.0-1
- Update 0.1.0
- Add the possibilities to delete a site or a host
- Do not only create /var/lock/mirrormanager on installation (Adrian Reber)
That should fix the question from Jeff.
I'll push to prod once I tested it on stg :)
Pierre
8 years, 11 months
May status update for Fedora Infrastructure Apprentices
by Kevin Fenzi
Greetings.
You are getting this email because you are in the 'fi-apprentice' group
in the fedora account system (or are reading this on the
infrastructure list).
Feel free to reply just directly to me, or cc the infrastructure list
for everyone to see and comment on.
https://fedoraproject.org/wiki/Infrastructure_Apprentice
At the first of every month(or so), I am going to be sending out an
email like this one. I would like feedback on how things are going for
you.
I'd like to ask for everyone to send me a quick reply with the
following data or anything related you can think of that might help us
make the apprentice program more useful.
0. Whats your fedora account system login?
1. Have you logged in and used your fi-apprentice membership to look at
our machines/setup in the last month? Do you plan to?
2. Has it helped you decide any area you wish to focus on or contribute
to more?
3. Have you looked at or been able to work on any of the fi-apprentice
'easyfix' tickets?
https://fedorahosted.org/fedora-infrastructure/report/14
4. Do you still wish to be a member of the group? If not (for whatever
reason) could you provide any hints to help others down the road?
5. Is there any help or communication or ideas you have that would help
you do any of the above?
6. What do you find to be the hardest part of getting involved?
Finding things to work on? Getting attention from others to help you?
Finding tickets in your interest area?
7. Have you been able to make any weekly irc meetings? Do you find them
helpful or interesting?
8. Have you logged into our Gobby instance and read/seen/added to our
meeting agenda? https://fedoraproject.org/wiki/Gobby
Any other general feedback is also quite welcome, including
improvements to this email, the wiki page, etc.
Any folks I do not hear from in the next week will be removed from the
group. (Note that it's easy to be readded when you have time or
whatever and it's nothing at all personal, we just want to keep the
group up to date with active folks).
Thanks, and looking forward to your feedback!
kevin
8 years, 11 months