Tackling COPR service performance issues
by Michal Novotny
Hello guys,
we, COPR team, are very sorry for the latest service occasional
unavailability. There were several performance issues that we needed to
sort out:
1) High memory consumption in API1 on a copr detail - resolved by code
optimization
2) Timeouts on search index updating - resolved by employing regular batch
index updates instead of event-triggered ones
3) Memory consumption needed to render RubyGems monitor page - this was
tackled by rewriting the code from SQLAlchemy to pure SQL
The RubyGems and PyPI package/build/monitor pages still take a long time
for their delivery. Mirek Suchy implemented streamed server responses to
prevent page timeouts which should alleviate the situation a little bit. We
are open to any further ideas that could help us make these COPR resources
more available.
After the mentioned updates, the COPR service availability should be much
better. If any further issues should occur, we will react promptly.
clime
7 years, 7 months
copr-cli use python3 now
by Miroslav Suchý
I just built new copr-cli for fedora-rawhide. It will now use python3.
There should be no regression as we have python3-copr for some time. But if you spot something bad, please report it.
--
Miroslav Suchy, RHCA
Red Hat, Senior Software Engineer, #brno, #devexp, #fedora-buildsys
7 years, 8 months
wiki edit
by Benjamin Lefoul
Hello,
According to https://fedorahosted.org/copr/wiki it is possible to
contribute to the wiki via git patches (and probably now through the
regular GitHub pull-request process). But I cannot find the wiki in the
GitHub repo... Am I missing something?
Regards,
Benjamin Lefoul
7 years, 8 months
Disabled copr-dist-git webui
by Miroslav Suchý
I disabled copr-dist-git WebUI. That one with /cgit/ prefix.
This was necessary due performance issue. Once resolved I will enable it again and I will inform you.
--
Miroslav Suchy, RHCA
Red Hat, Senior Software Engineer, #brno, #devexp, #fedora-buildsys
7 years, 8 months
New releases of COPR packages
by Michal Novotny
Hello,
we have released the following packages today:
- python-copr 1.72
- copr-cli 1.53
- copr-frontend 1.97
- copr-backend 1.91
- copr-selinux 1.43
It will take some time until they get into stable repos but if you do not
want to wait, you can find the built packages in our official COPR repo -
https://copr.fedorainfracloud.org/coprs/g/copr/copr/builds/ :).
Some of the most prominent features and bugfixes that the just released
packages offer are:
*Bug 1346288* <https://bugzilla.redhat.com/show_bug.cgi?id=1346288> - Server
returning error 500 after large package upload
- this bug occurred randomly all over the site and was caused by heavy load
on our search index database. We took some of the load away by various
optimizations and the error should not occur anymore.
*Bug 1335237* <https://bugzilla.redhat.com/show_bug.cgi?id=1335237> - copr
create command missing --disable_createrepo
- fixed
*Bug 1335236* <https://bugzilla.redhat.com/show_bug.cgi?id=1335236> - Internet
access during builds flag needed in CLI create/modify commands
- we have added --enable-net parameter to copr-cli create/modify commands
*Bug 1346945* <https://bugzilla.redhat.com/show_bug.cgi?id=1346945> - Project
List filled with automated builds
- we have added --unlisted-on-hp parameter to copr-cli create/modify
commands
*Bug 1344805* <https://bugzilla.redhat.com/show_bug.cgi?id=1344805> - Many
queued tasks from same user can block other user tasks
- we have added --background option to new build in CLI that lowers
priority of the job and which is used to run the automated builds (you can
use it too if you feel like a super nice and charming person ;))
Additionally, we said "NO" to url and upload package types (or package
source types, more specifically) and removed those from copr-cli and
python-copr package manipulation interface. It was a hard decision but, for
the time being, the best possible as the url and upload builds are best to
be done with the old&tried method of manually submitting a single 'New
build' whether it is via UI or copr-cli build command. It kind of makes
sense (on the second look) because source for these package is fixed to a
certain version and release of the package (either you upload package of a
certain version or you specify a link to it) and this limitation cannot
easily be overcome.
BUT for all the other package types (Tito/PyPI/MockSCM/RubyGems), you can
use a new cool command: copr-cli build-package --name pkgname coprname.
To use this, you first need to have a default source defined for the
package and this can be done in UI ('Packages' Tab in your splendid copr)
or through copr-cli's package manipulation interface, e.g.:
copr-cli add-package-tito --name pkgname --git-url someurl coprname.
What else...oh yeah, the version of python-copr 1.70 broke compatibility
with COPR frontend 1.92 and older (it was the 'source_type' error, sorry
for that). Well, python-copr 1.72 is much better in this matter and is
compatible with almost every available copr-frontend except for 1.93 ,
which is history now and was also screwed up to work together with
python-copr 1.70 (and copr-cli 1.51). Well, I guess we were trying to move
a bit too fast at that point...but, heh, let's keep that pace and just try
to avoid silly stones on the road.
clime
P.S.: I forgot to mention that the next release(s) will bring huge amount
of internal COPR improvements, be it multi-threaded copr-dist-git or
copr-backend with some crazy simplifications (+ optimizations) in build &
action task workflow (It all started with Mirek's great new design
proposal).
P.P.S: Related erratas:
https://bodhi.fedoraproject.org/updates/FEDORA-2016-e4496e8bda
https://bodhi.fedoraproject.org/updates/FEDORA-2016-af777aa589
https://bodhi.fedoraproject.org/updates/FEDORA-2016-313a6bfe5a
https://bodhi.fedoraproject.org/updates/FEDORA-2016-570ac5dcaf
7 years, 8 months