Anonymized access log from a fedora mirror
by Lukas Zapletal
Hello,
I have two students interested in diploma thesis called Yum plugin for
suggesting packages based on usage:
http://bit.ly/18hrHbL
TL;DR - from anonymized access log, create a database of suggested
packages using data mining techniques and provide a Yum plugin that
would suggest "Users of vim also installed: ctags, git, ..."
I am gonna create a Fedora Feature wiki page shortly describing this in
more detail. Our goal is to offer this project for integration into
Fedora later on, at least provide Fedora packages for it.
To do that, we need good source of data. It would be best to collect
access logs from one or two main Fedora mirrors. We would provide short
script in Python that would parse access logs and anonymize the data (IP
address hash-salted) and filtered only relevant data (RPM files from
latest Fedora release or updates repositories). That would be phase one
which should give us a sample data.
Phase two would be to integrate this script with logrotate and for one
Fedora release cycle (Fedora 19) the script would collect relevant
anonymized data into a file. Final suggested package database would be
created from this file (or maybe files to allow us to move them on the
fly out of the stat directory).
The big (legal) question is if we are able to provide this anonymized
data to public, or if we want to sign NDA with all people involved. I am
CCing Tom for this question.
I need your help with connecting to relevant people. Any comments are
appreciated.
Many thanks and I hope this effort will lead to improving user
experience with Fedora packaging.
--
Later,
Lukas "lzap" Zapletal
irc: lzap #theforeman
9 years, 9 months
GitLab packaging progress and discussion about deployment on fedorahosted
by Achilleas Pipinellis
Dear infra team and others that this mail may concern,
As many of you know (I hope :p), one of this year's GSoC projects, is
to package GitLab and all its dependencies for Fedora and later for
EPEL. There have been at least 3 discussion threads about this since
March [0][1][2][3].
I have been in contact with GitLab's core team and we talked about how
we could all work together to make this happen and how GitLab could be
eventually deployed in fedorahosted as an alternative git service. For
the time being there are 2 major show-stoppers:
1) GitLab uses some forked gems.
These are the forked gems by GitLab which add some extra functionality
or fix some bugs of the original gem:
Upstream | GitLab
-------------------------------------
grit | gitlab-grit
grack | gitlab-grack
gollum-lib | gitlab-gollum-lib
omniauth-ldap | gitlab_omniauth-ldap
pygments.rb | gitlab-pygments.rb
-------------------------------------
Vit Ondruch, my mentor, pointed me in these FESCO [4] and FPC [5]
tickets, which pretty much conclude that:
"FESCo is fine with forks as long as they are parallel installable and
don't interfere with each other."
and
"The FPC does not see a need for additional guidelines relating to
forks at this time, they should be treated like any other package."
I also raised this issue in #fedora-devel today and they told me the
same thing FESCo concluded.
I think GitLab's forks don't abide by FESCo's verdict, as both original
and forked gem are called with the same library, eg. require 'grit', so
there is no distinction between them.
I am cc'ing Sytse Sijbrandij from GitLab's core team to talk about what
changes could be made in order for the forks to get accepted.
2) The feature of public browserability for non logged in users is not
yet implemented.
This long awaited feature is the major show-stopper for getting GitLab
deployed on fedorahosted. Quoting Sytse's proposal:
> Of the two improvements I think that 2) is the most necessary. Without
> a public project UI Fedora will not switch to GitLab. There is already
> a public project git clone functionality in GitLab. Opening up more of
> the project such as issues might load to in-advert exposure of issues.
> Therefore we want to do it only on a major version upgrade. We started
> with GitLab 6.0 a week ago and would like to merge this fast so it can
> be included in the beta released on July 22. If Axilleas works on this
> then Dmitriy is prepared to coach him, this will be needed since this
> feature will impact the whole application.
>
> Our proposal would be to:
> A) Start working on the public UI immediately
> B arrange a online meeting between Fedora and GitLab.com people to talk
> about packaging
> C) have beta version of GitLab run on a demo server with Fedora project
> on August 1
> D) aim to have GitLab in production for all Fedora projects on
> September 1
>
> Of course this is just a proposal. We have too little knowledge of the
> Fedora project to make a proper plan.
The initial plan of the GSoC proposal is to package GitLab,
but since I talked about it with Sytse I thought it would be more
productive to bring the discussion here and sort some things out all
together.
Thank you all for your time,
Axilleas
[0]
https://lists.fedoraproject.org/pipermail/infrastructure/2013-March/01263...
[1]
https://lists.fedoraproject.org/pipermail/infrastructure/2013-April/01273...
[2]
https://lists.fedoraproject.org/pipermail/infrastructure/2013-April/01275...
[3]
https://lists.fedoraproject.org/pipermail/infrastructure/2013-May/012885....
[4] https://fedorahosted.org/fesco/ticket/810
[5] https://fedorahosted.org/fpc/ticket/148
--
GPG : 0xABF99BE5
Blog: http://axilleas.github.io
FAS : https://fedoraproject.org/wiki/User:Axilleas
10 years
Plan for tomorrow's Fedora Infrastructure meeting (2013-08-29)
by Kevin Fenzi
The infrastructure team will be having it's weekly meeting tomorrow,
2013-08-29 at 19:00 UTC in #fedora-meeting on the freenode network.
Suggested topics:
#topic New folks introductions and Apprentice tasks.
If any new folks want to give a quick one line bio or any apprentices
would like to ask general questions, they can do so in this part of the
meeting. Don't be shy!
#topic Applications status / discussion
Check in on status of our applications: pkgdb, fas, bodhi, koji,
community, voting, tagger, packager, dpsearch, etc.
If there's new releases, bugs we need to work around or things to note.
#topic Sysadmin status / discussion
Here we talk about sysadmin related happenings from the previous week,
or things that are upcoming.
#topic Upcoming Tasks/Items
https://apps.fedoraproject.org/calendar/list/infrastructure/
#topic Open Floor
Submit your agenda items, as tickets in the trac instance and send a
note replying to this thread.
More info here:
https://fedoraproject.org/wiki/Infrastructure/Meetings#Meetings
Thanks
kevin
10 years, 1 month
Hi
by Sayth Renshaw
First off would just like to say high.
Looking forward to contributing and learning as well.
Bit about myself my name is Sayth, from Newcastle Australia. Used Fedora
off and in since Core 4.
Last year I completed a Diploma in Database Development and Web Design
using ASP MVC.
Been learning a bit if python and ruby to see what I liked. They are both
very similar going with Python for the minute. Know some SQL and basic
MySQL.
Any advice on learning and contributing ideas appreciated.
Sayth
10 years, 1 month
Meeting Agenda Item: Introduction Basil Kurian
by Basil Kurian
Myself Basil. I 'm a fan of Fedora/Centos. For me Fedora is the choice of
operating system for personal use, and in my workplaces I used to deal with
RHEL/Centos boxes. I started using Fedora in 2008 (Fedora 8 : werewolf). I
'm quite willing to join the infrastructure team and I can contribute 4 to
6 hours a week. I 'm interest in handling some infrastructure
automation/troubleshooting tasks.
Full Name : Basil Kurian
IRC handle : basil_kurian_ (freenode)
Country : India
Skills :
Familiar with scalable infrastructures and high traffic production
environments.
Sound knowledge in Configuration management tools like puppet.
Can develop tools/scripts in python/django/flask
Hands on knowledge in Amazon and private cloud platform
Public profile : http://linkedin.com/in/basilkurian
Regards
Basil Kurian
10 years, 1 month
mailman3 / hyperkitty planning
by Kevin Fenzi
Greetings
I thought I would send out my thoughts on mailman3/hyperkitty roll out
based on our discussions at flock.
- We now have a mailman01.stg instance. It's setup to use yum-cron and
apply security updates every night (we should confirm it's working as
we want it to tho)
- We need mailman3 / hyperkitty rpms added to infra repo.
- We need to get mailman3/hyperkitty roles added to this instance.
- Next we need to test importing things. I can get copies of the
existing lists Mailboxes to import with.
- Lots of testing. :) Writing SOPS/docs for list admins, creating new
lists, etc.
- Once testing looks good, we can move to production. Due to needing a
database backend I was thinking we would just leave it in phx2 (and
make 2 of them). smtp-mm hosts can easily be used for incoming
emails and bastion for outgoing.
- We already have a few folks eager to move their lists over and help
once we get to production. We can move this list (infrastructure)
and cloud is very interested too.
- Once those initial lists look ok, we can schedule a flag day and
move all the rest of the lists.
- Down the road, once RHEL7 is out we can migrate over to that.
Some questions:
- Are we going to need to keep the old archives online for people
searching history/google? Or is there any way to redirect them?
- Should we do a seperate db for this in production? It's unclear to me
how much in the way of resources the db side will take.
- Currently lists are at osu and don't go via proxies, but if we have
it at phx2 we could just use the proxies and get caching, etc as
well. Or is there a reason not to?
Any other things we should look at for the roll out?
kevin
10 years, 1 month
Pull Request Reviews
by Ralph Bean
Hi infra team-
We have a handful of pull requests open on our github repo for which
I'd like to ask for help in reviewing:
1) fedbadges 4d PR#22 - Allow lambda expressions in the dat .. http://bit.ly/1aeY7qh
2) tahrir-api 4d PR#19 - Feature/ranking .. http://bit.ly/1dhjI0G
3) fedbadges 3d PR#23 - Publish a fedmsg message when a use .. http://bit.ly/14UUlkP
4) tahrir 3d PR#199 - Feature/publish message on rank cha .. http://bit.ly/14sVXfq
5) fedmsg 2d PR#173 - Feature/idempotent .. http://bit.ly/1643CQX
-
Number 1 is an independent change and is an enhancement to the badges
awarder that will make it so we can perform much more flexible
datanommer queries. (It makes it so we can do flexible operations in
python after making the actual datanommer query but before we
conclude that a badge should or should not be awarded).
Number 2 is a requirement for numbers 3 and 4. It does a number of things:
- it pulls the leaderboard calculation code from tahrir (the web
frontend) into tahrir-api (the underlying lib used by both the
badges awarder backend and the web frontend)
- it caches users' rank in the database which should improve
performance on the webapp significantly.
- it adds a new `notification_callback` which we'll use to publish
fedmsg messages about users' rank changing. We want to award
badges based on this kind of stuff, so its a nice bonus.
Number 3 simply makes use of Number 2. It removes the old hardcoded
fedmsg stuff from the backend badges awarder.
Number 4 simply makes use of Number 2. It removes the old leaderboard
code and the old hardcoded fedmsg stuff from the badges web frontend.
Number 5 is an independent change that will ultimately fix a bug in
the datagrepper web api.
-
Any help providing review would be much appreciated.
Cheers-
-Ralph
10 years, 1 month
[PATCH] use a simpler syntax for the nrpe config
by Michael Scherer
From: Michael Scherer <misc(a)zarb.org>
---
roles/nagios_client/tasks/main.yml | 16 ++++++++--------
1 file changed, 8 insertions(+), 8 deletions(-)
diff --git a/roles/nagios_client/tasks/main.yml b/roles/nagios_client/tasks/main.yml
index a04e3fb..373cd60 100644
--- a/roles/nagios_client/tasks/main.yml
+++ b/roles/nagios_client/tasks/main.yml
@@ -43,15 +43,15 @@
- config
- name: install nrpe client configs
- template: src={{ item.src }} dest=/etc/nrpe.d/{{ item.dest }}
+ template: src={{ item }}.j2 dest=/etc/nrpe.d/{{ item.dest }}
with_items:
- - { src: 'check_mirrorlist_cache.cfg.j2', dest: 'check_mirrorlist_cache.cfg' }
- - { src: 'check_raid.cfg.j2', dest: 'check_raid.cfg' }
- - { src: 'check_cron.cfg.j2', dest: 'check_cron.cfg' }
- - { src: 'check_disk.cfg.j2', dest: 'check_disk.cfg' }
- - { src: 'check_swap.cfg.j2', dest: 'check_swap.cfg' }
- - { src: 'check_postfix_queue.cfg.j2', dest: 'check_postfix_queue.cfg' }
- - { src: 'check_lock.cfg.j2', dest: 'check_lock.cfg' }
+ - check_mirrorlist_cache.cfg
+ - check_raid.cfg
+ - check_cron.cfg
+ - check_disk.cfg
+ - check_swap.cfg
+ - check_postfix_queue.cfg
+ - check_lock.cfg
notify:
- restart nrpe
tags:
--
1.8.3.1
10 years, 1 month
Meeting Agenda Item: Introduction Janez Nemanic
by Janez Nemanic
Hello,
I'm Janez from Slovenia and I would like to join Fedora Infrastructure Team. The main reason that I would like to be part of team is because I would like to contribute to project that enables people to get their operating system and other software completely for free.
My IRC nickname is janeznemanic .
I have been studying Linux system administration and Python for the last four months. Before Python I also studied Java and C#. I would also like to learn more about automation of software systems. For example how to use Puppet or similar tools.
I would like to work as system administrator (maintaining system, writing scripts, etc.) and as web developer on different issues. Being almost fresh to Linux and open source projects means that it is hard for me to decide which particular issue I would like to work on. So I am hoping to get some help from the team.
My goal is to become experienced system administrator in Fedora project.
Janez Nemanic
Best regards
10 years, 1 month