fedrtc.org -> infrastructure?
by Daniel Pocock
Hi all,
At some point, it would be good to see fedrtc.org migrate to Fedora
infrastructure and use the fedoraproject.org domain
I'd be happy to submit the full request for resources[1] but I just want
to see if there is any initial comment on it. Here is a list of what is
involved:
- it uses a PostgreSQL database schema[2]
- it requires some DNS entries (SRV and NAPTR), examples[3]
- it needs a TLS cert for fedoraproject.org on the host(s) where it runs
- it has static HTTP content and PHP that is currently hosted with all
but one problem[4] on a RHEL7 httpd. Content is in Github[5], it could
be presented as an RPM if necessary.
- all packages are in EPEL7, except:
cajun-json in EPEL6, in testing for EPEL7
resiprocate in Fedora, builds from SRPM on RHEL7
- the SIP proxy is a single daemon, managed by systemctl. All settings
in a single file, /etc/repro/repro.config
- the TURN server process is also a single daemon, managed by systemctl.
All settings in a single file, /etc/reTurn/reTurnServer.config
Just to clarify the scope of this: it is not a full telephony service
like Asterisk, just a SIP proxy and TURN server. There is no persistent
state information (as there would be for voicemail, email service, etc)
and no customized routing.
Ongoing maintenance requirements:
- TLS certificate renewals
- monitoring the ports
- package updates from time to time
It currently runs on a lab machine, I'd be happy to arrange SSH access
to the Fedora Infrastructure team to see exactly what is involved and
verify that it is manageable.
Regards,
Daniel
1. https://fedoraproject.org/wiki/Request_For_Resources
2.
https://github.com/resiprocate/resiprocate/blob/master/repro/create_postg...
3. http://rtcquickstart.org/guide/multi/dns.html
4. https://bugzilla.redhat.com/show_bug.cgi?id=1222649
5. https://github.com/opentelecoms-org/fedrtc-org
8 years
ansible csi group vars - dns
by Ali Khalidi
infra -
here is the diffs for csi vars for group: dns
applicable when we come out of freez.
diff --git a/inventory/group_vars/dns b/inventory/group_vars/dns
index 17da9d0..db01b32 100644
--- a/inventory/group_vars/dns
+++ b/inventory/group_vars/dns
@@ -16,3 +16,7 @@ nrpe_procs_warn: 300
nrpe_procs_crit: 500
sudoers: "{{ private }}/files/sudo/sysadmin-dns"
+
+csi_security_category: High
+csi_primary_contact: Fedora Admins - admin(a)fedoraproject.org
+csi_purpose: Domain Name Service
diff --git a/inventory/host_vars/ns02.fedoraproject.org
b/inventory/host_vars/ns02.fedoraproject.org
index ec29b7e..ffc9479 100644
--- a/inventory/host_vars/ns02.fedoraproject.org
+++ b/inventory/host_vars/ns02.fedoraproject.org
@@ -15,3 +15,19 @@ datacenter: ibiblio
ks_url: http://209.132.181.6/repo/rhel/ks/kvm-rhel-7-ext
ks_repo: http://209.132.181.6/repo/rhel/RHEL7-x86_64/
+
+
+csi_relationship: |
+
+ ns02 is a master dns server.
+ It serves about every domain under fedoraproject, fedo*, as well
as others, both forward and reverse.
+
+ * This host relies on:
+ - The virthost it's hosted on (ibiblio03.fedoraproject.org)
+ - batcave for dns git and keys
+ - connectivity to maxmind to create geoIP dns acl
+
+ * Things that rely on this host:
+ - The Internet/Community to resolve everything related to fedora
and reverse-IP for allocated subnets
+ - If this host is down, dns queries will slow down by the portion
of this host to the total name servers responsible for the same domain
set.
+ - secodary/slave dns servers
diff --git a/inventory/host_vars/ns03.phx2.fedoraproject.org
b/inventory/host_vars/ns03.phx2.fedoraproject.org
index 4515d5e..669b5ff 100644
--- a/inventory/host_vars/ns03.phx2.fedoraproject.org
+++ b/inventory/host_vars/ns03.phx2.fedoraproject.org
@@ -10,3 +10,18 @@ ansible_ssh_host: ns03.phx2.fedoraproject.org
vmhost: virthost21.phx2.fedoraproject.org
datacenter: phx2
+
+csi_relationship: |
+
+ ns03 is a master dns server.
+ It serves about every domain under fedoraproject, fedo*, as well
as others, both forward and reverse.
+
+ * This host relies on:
+ - The virthost it's hosted on (virthost21.phx2.fedoraproject.org)
+ - batcave for dns git and keys
+ - connectivity to maxmind to create geoIP dns acl
+
+ * Things that rely on this host:
+ - The Internet/Community to resolve everything related to fedora
and reverse-IP for allocated subnets
+ - If this host is down, dns queries will slow down by the portion
of this host to the total name servers responsible for the same domain
set.
+ - secodary/slave dns servers
diff --git a/inventory/host_vars/ns04.phx2.fedoraproject.org
b/inventory/host_vars/ns04.phx2.fedoraproject.org
index 9c72ca9..1a4089f 100644
--- a/inventory/host_vars/ns04.phx2.fedoraproject.org
+++ b/inventory/host_vars/ns04.phx2.fedoraproject.org
@@ -10,3 +10,18 @@ ansible_ssh_host: ns04.phx2.fedoraproject.org
vmhost: virthost15.phx2.fedoraproject.org
datacenter: phx2
+
+csi_relationship: |
+
+ ns04 is a master dns server.
+ It serves about every domain under fedoraproject, fedo*, as well
as others, both forward and reverse.
+
+ * This host relies on:
+ - The virthost it's hosted on (virthost15.phx2.fedoraproject.org)
+ - batcave for dns git and keys
+ - connectivity to maxmind to create geoIP dns acl
+
+ * Things that rely on this host:
+ - The Internet/Community to resolve everything related to fedora
and reverse-IP for allocated subnets
+ - If this host is down, dns queries will slow down by the portion
of this host to the total name servers responsible for the same domain
set.
+ - secodary/slave dns servers
diff --git a/inventory/host_vars/ns05.fedoraproject.org
b/inventory/host_vars/ns05.fedoraproject.org
index 3955f44..e02f9a9 100644
--- a/inventory/host_vars/ns05.fedoraproject.org
+++ b/inventory/host_vars/ns05.fedoraproject.org
@@ -12,3 +12,18 @@ postfix_group: vpn
vmhost: internetx01.fedoraproject.org
datacenter: internetx
+
+csi_relationship: |
+
+ ns05 is a master dns server.
+ It serves about every domain under fedoraproject, fedo*, as well
as others, both forward and reverse.
+
+ * This host relies on:
+ - The virthost it's hosted on (internetx01.fedoraproject.org)
+ - batcave for dns git and keys
+ - connectivity to maxmind to create geoIP dns acl
+
+ * Things that rely on this host:
+ - The Internet/Community to resolve everything related to fedora
and reverse-IP for allocated subnets
+ - If this host is down, dns queries will slow down by the portion
of this host to the total name servers responsible for the same domain
set.
+ - secodary/slave dns servers
8 years
Fedora 23 Final Freeze now in effect
by Kevin Fenzi
Greetings.
we are now in the infrastructure freeze leading up to the Fedora 23
Final release. This is a final release freeze.
This means that hosts that are marked as freezing should not have any
changes made to them except as part of a freeze break request
(see below)
We do this to make sure that our infrastructure is stable to allow for
building/composing/testing and distributing Fedora 23. This
freeze will end 1 day after Fedora 23 is released.
(Currently scheduled for 2015-10-27)
Freeze breaks should be sent to this list, clearly describing the
planned change and include patches (if applicable). Freeze breaks
should not be applied until they have gotten at least 2 +1's from
members of sysadmin-main and/or releng groups.
You can see a list of hosts that do not freeze by checking out the
ansible repo and running the freezelist script:
git clone http://infrastructure.fedoraproject.org/infra/ansible.git
scripts/freezelist -i inventory
Thanks,
kevin
8 years, 1 month
mdapi in our infrastructure
by Pierre-Yves Chibon
Good Morning everyone,
Over the last few days I have been working on a small app: mdapi.
It is aimed at serving the metadata from our repos simply and *fast*, offering
information from koji, rawhide, all our active branches and epel (you'll have to
specify which you want) and for each it will return you the first hit it finds
in the testing, updates or release repo (it says which in the json returned).
I deployed it for testing in our cloud, at: http://209.132.184.236/
This morning I took a look at what it would take to package mdapi for epel7.
mdapi has 5 dependencies:
- aiohttp -> Has an epel7 branch but was never built, open a ticket on the
bugzilla asking it
- requests -> Exists in RHEL channels but not as py3, so will need a new
package
- simplejson -> Asked the current PoC if it could be built with the py3
sub-package in epel7
- sqlalchemy -> Looked at it (I'm the PoC for epel7), but requires:
- python3-setuptools (same situation as requests)
- requires: python3-pip, python3-wheel, python3-mock,
python3-pytest
- python3-nose (same situation as requests and setuptools
as requests
- werkzeug -> same situation as requests, setuptools...
The dependency list as well as the current situation of the guidelines for
packaging python3 app in epel [1], makes me proposing that we deploy mdapi in
Fedora (at least for now).
The second point I would like to raise is how we deploy this application. This
isn't a standard wsgi application (since it's async) and it cannot run with
apache (afaiu).
So far the other async application we have (in pagure) have been deployed simply
as a systemd service.
I wonder if we want to use the same approach here or if we should investigate
things like gunicorn/nginx or so.
Does someone have experience in this field? Any advice/feedback?
Thanks for your attention,
Pierre
[1] https://fedorahosted.org/fpc/ticket/567
8 years, 1 month
FAS3 status update
by Pierre-Yves Chibon
Dear all,
Over the last couple of weeks Xavier and I managed to catch up a couple of time
to discuss the current status of FAS3.
I would like to summarize a little the output of these discussions.
We can basically, consider 3 types of tasks left.
* Development:
Admin panel
Xavier is working on providing an Admin panel to FAS3. This would provide a
central place for admin tasks such as blocking an user, deleting an account
and so on
-> Due by next week
Group membership model
Xavier is thinking to rework the membership model for FAS groups. There is
currently three levels: user, sponsors and admins while there are groups where
sponsors might not be needed or where another level could be useful.
-> Optional for 3.0, NOTABLOCKER
Unit-tests
FAS3 has currently little tests and needs much more
-> Would be good to have for 3.0 but NOTABLOCKER (at least for a staging
instance)
2FA
2 Factor Authentication is probably the biggest point remaining to develop.
In FAS2, yubikey is integrated into FAS, while gauth isn't. For FAS3 we
probably want to integrate both so that users have a place to set and reset
their tokens.
On the otherside, keeping the yubikey and the gauth servers out of FAS3 also
make sense. So we may want to find a way for both to share some information
where FAS3 would have read/write permissions while the yubikey/gauth servers
would have only read.
* Migration
The idea here is two provide two scripts, a python script that will check the
FAS2 DB and report problematic data (for example multiple users having the same
IRC nick).
Then a SQL script to be used once to convert the FAS2 DB into a FAS3 DB or
migrate from the FAS2 DB to a different FAS3 DB (we'll see what is the easiest
or makes the most sense).
* Integration
There are still quite a few integrations to do.
- Packaging
Xavier checked and said that all dependencies are present in RHEL, EPEL or
Fedora already. We'll need to double-check and request the epel7 branch
where needed.
- Ipsilon
This should likely be the first one to work on, in order to allow login on our
apps in staging.
- fedora-python
Here the client needs to be ported from FAS2 to FAS3. The API has entirely
changed and is not backward compatible but we *may* be able to make the API
of the client backward compatible.
Xavier already started working on this in the FAS3 branch
- All our apps using FAS
The idea is to deploy FAS3 in staging so that we can test/fix all our apps
using FAS one by one and take our time to do this.
As you can see, most of the development will be done by next week, except for
the 2FA aspect which we need to figure out.
I have volunteered to work on the migration scripts.
Xavier and Patrick will need to coordinate for the Ipsilon integration.
Then, we should be ready to push a FAS3 instance to staging.
The idea is that we have FAS3 running in staging by Christmas.
How does this sound? Anyone willing to step in?
Thanks for your attention,
Pierre & Xavier
8 years, 1 month
Freeze break request: developer.fedoraproject.org to production
by Kevin Fenzi
Greetings.
I've gotten the developer portal working in staging a few days ago:
https://developer.stg.fedoraproject.org/
I'd like to move it into production now so it's ready for their
announcement. +1s?
Note that currently the site is built using jekyl on
developer.fedorainfracloud.org and then synced with a github repo which
we pull from. Moving forward we will come up with a better way to do
all this, we just wanted to get things working for release now.
kevin
--
diff --git a/playbooks/groups/sundries.yml b/playbooks/groups/sundries.yml
index 34dab01..c8bc7c0 100644
--- a/playbooks/groups/sundries.yml
+++ b/playbooks/groups/sundries.yml
@@ -49,7 +49,7 @@
- role: membership-map/build
when: master_sundries_node
- role: developer/build
- when: master_sundries_node and env == "staging"
+ when: master_sundries_node
tasks:
- include: "{{ tasks }}/yumrepos.yml"
diff --git a/playbooks/include/proxies-fedora-web.yml b/playbooks/include/proxies-fedora-web.yml
index 250e75a..ce1f7d1 100644
--- a/playbooks/include/proxies-fedora-web.yml
+++ b/playbooks/include/proxies-fedora-web.yml
@@ -41,5 +41,4 @@
website: docs.fedoraproject.org
- role: developer/website
- website: developer.stg.fedoraproject.org
- when: env == "staging"
+ website: developer.fedoraproject.org
diff --git a/playbooks/include/proxies-websites.yml b/playbooks/include/proxies-websites.yml
index 5feef25..0e60bed 100644
--- a/playbooks/include/proxies-websites.yml
+++ b/playbooks/include/proxies-websites.yml
@@ -422,13 +422,12 @@
cert_name: "{{wildcard_cert_name}}"
- role: httpd/website
- name: developer.stg.fedoraproject.org
+ name: developer.fedoraproject.org
server_aliases: [developer.stg.fedoraproject.org]
# Set this explicitly to stg here.. as per the original puppet config.
SSLCertificateChainFile: wildcard-2014.stg.fedoraproject.org.intermediate.cert
sslonly: true
cert_name: "{{wildcard_cert_name}}"
- when: env == "staging"
# Kinda silly that we have two entries here, one for prod and one for stg.
# This is inherited from our puppet setup -- we can collapse them as
soon as
8 years, 1 month
Plan for thursday's Fedora Infrastructure meeting - 2015-10-29
by Kevin Fenzi
The infrastructure team will be having it's weekly meeting tomorrow,
2015-10-15 at 18:00 UTC in #fedora-meeting on the freenode network.
We have a gobby document
(see: https://fedoraproject.org/wiki/Gobby )
NOTE: Gobby has changed, we are now using gobby05 and infinote. :)
fedora-infrastructure-meeting-next is the document.
Please try and review and edit that document before the meeting and we
will use it to have our agenda of things to discuss. A copy as of today
is included in this email.
If you have something to discuss, add the topic to the discussion area
with your name. If you would like to teach other folks about some
application or setup in our infrastructure, please add that topic and
your name to the learn about section.
kevin
--
= Introduction =
This shared document is for the next fedora infrastructure meeting.
We will use it over the week before the meeting to gather status and info and
discussion items and so forth, then use it in the irc meeting to transfer
information to the meetbot logs.
= Meeting start stuff =
#startmeeting Infrastructure (2015-10-29)
#meetingname infrastructure
#topic aloha
#chair smooge relrod nirik abadger1999 lmacken dgilmore mdomsch threebean pingou puiterwijk pbrobinson
#topic New folks introductions / Apprentice feedback
= Status / information / Trivia / Announcements =
(We put things here we want others on the team to know, but don't need to discuss)
(Please use #info <the thing> - your name)
#topic announcements and information
#info Still in f23 final freeze - everyone
#info
= Things we should discuss =
We use this section to bring up discussion topics. Things we want to talk about
as a group and come up with some consensus or decision or just brainstorm a
problem or issue. If there are none of these we skip this section.
(Use #topic your discussion topic - your username)
#topic
= Learn about some application or setup in infrastructure =
(This section, each week we get 1 person to talk about an application or setup
that we have. Just going over what it is, how to contribute, ideas for improvement,
etc. Whoever would like to do this, just add the info in this section. In the
event we don't find someone to teach about something, we skip this section
and just move on to open floor.)
#topic Learn about:
= Meeting end stuff =
#topic Open Floor
#endmeeting
8 years, 1 month