[copr/copr] branch main updated (8b2c8fe9 -> 496949a9)
by pagure@pagure.io
This is an automated email from the git hooks/post-receive script.
frostyx pushed a change to branch main
in repository copr/copr.
from 8b2c8fe9 frontend: add link to /user/repositories/ on the user detail page
add 496949a9 frontend: show the batch ID (if assigned) in the build detail
No new revisions were added by this update.
Summary of changes:
frontend/coprs_frontend/coprs/templates/coprs/detail/build.html | 8 ++++++++
1 file changed, 8 insertions(+)
--
To stop receiving notification emails like this one, please contact
the administrator of this repository.
1 year, 7 months
[copr/copr] branch main updated (a426bc8e -> 8b2c8fe9)
by pagure@pagure.io
This is an automated email from the git hooks/post-receive script.
frostyx pushed a change to branch main
in repository copr/copr.
from a426bc8e frontend: stg frontend instance should link to stg FAS instance
add 8b2c8fe9 frontend: add link to /user/repositories/ on the user detail page
No new revisions were added by this update.
Summary of changes:
frontend/coprs_frontend/coprs/templates/coprs/show/user.html | 3 +++
1 file changed, 3 insertions(+)
--
To stop receiving notification emails like this one, please contact
the administrator of this repository.
1 year, 7 months
[copr/copr] branch main updated (8244a626 -> a426bc8e)
by pagure@pagure.io
This is an automated email from the git hooks/post-receive script.
frostyx pushed a change to branch main
in repository copr/copr.
from 8244a626 frontend: test-cover rebuild permission denied
add a426bc8e frontend: stg frontend instance should link to stg FAS instance
No new revisions were added by this update.
Summary of changes:
frontend/coprs_frontend/coprs/config.py | 2 ++
frontend/coprs_frontend/coprs/context_processors.py | 2 +-
2 files changed, 3 insertions(+), 1 deletion(-)
--
To stop receiving notification emails like this one, please contact
the administrator of this repository.
1 year, 7 months
[copr/copr] branch main updated (827c257a -> 8244a626)
by pagure@pagure.io
This is an automated email from the git hooks/post-receive script.
frostyx pushed a change to branch main
in repository copr/copr.
from 827c257a docs: how to release copr fixes
add 806b117b frontend: fix two new pylint issues from #2266
add e99fcc88 frontend: add unit tests for comps
add af6eb23a frontend: we only upload comps as bytes
add 4cdfe1e4 frontend: cover web-UI build cancel action
add 69ef9737 frontend: cover /api_v3/build-chroots/list
add 4e941772 frontend: drop unused method
add 8244a626 frontend: test-cover rebuild permission denied
No new revisions were added by this update.
Summary of changes:
.../coprs_frontend/coprs/logic/builds_logic.py | 12 -----
frontend/coprs_frontend/coprs/models.py | 9 ++--
.../coprs/views/apiv3_ns/__init__.py | 2 +-
frontend/coprs_frontend/tests/coprs_test_case.py | 9 +++-
frontend/coprs_frontend/tests/data/comps.xml | 1 +
frontend/coprs_frontend/tests/request_test_api.py | 53 +++++++++++++++++++---
.../tests/test_apiv3/test_build_chroots.py | 41 +++++++++++++++++
frontend/coprs_frontend/tests/test_comps.py | 29 ++++++++++++
.../tests/test_logic/test_builds_logic.py | 13 +++++-
.../test_views/test_coprs_ns/test_coprs_general.py | 11 ++++-
10 files changed, 151 insertions(+), 29 deletions(-)
create mode 100644 frontend/coprs_frontend/tests/data/comps.xml
create mode 100644 frontend/coprs_frontend/tests/test_comps.py
--
To stop receiving notification emails like this one, please contact
the administrator of this repository.
1 year, 7 months
[copr/copr] branch main updated (a32b52b4 -> 827c257a)
by pagure@pagure.io
This is an automated email from the git hooks/post-receive script.
frostyx pushed a change to branch main
in repository copr/copr.
from a32b52b4 frontend: add page for exploring projects
add 827c257a docs: how to release copr fixes
No new revisions were added by this update.
Summary of changes:
doc/how_to_release_copr.rst | 90 +++++++++++++++++++++++++++++++++++----------
1 file changed, 70 insertions(+), 20 deletions(-)
--
To stop receiving notification emails like this one, please contact
the administrator of this repository.
1 year, 7 months
[copr/copr] branch main updated (36349b2e -> a32b52b4)
by pagure@pagure.io
This is an automated email from the git hooks/post-receive script.
msuchy pushed a change to branch main
in repository copr/copr.
from 36349b2e frontend: add test for updating chroot via APIv3
add a32b52b4 frontend: add page for exploring projects
No new revisions were added by this update.
Summary of changes:
frontend/coprs_frontend/coprs/__init__.py | 2 +
frontend/coprs_frontend/coprs/logic/coprs_logic.py | 16 +++++++
frontend/coprs_frontend/coprs/logic/stat_logic.py | 27 +++++++++++-
frontend/coprs_frontend/coprs/models.py | 16 +++++++
.../coprs_frontend/coprs/templates/explore.html | 50 ++++++++++++++++++++++
frontend/coprs_frontend/coprs/views/explore_ns.py | 31 ++++++++++++++
6 files changed, 141 insertions(+), 1 deletion(-)
create mode 100644 frontend/coprs_frontend/coprs/templates/explore.html
create mode 100644 frontend/coprs_frontend/coprs/views/explore_ns.py
--
To stop receiving notification emails like this one, please contact
the administrator of this repository.
1 year, 7 months
[copr/copr] annotated tag copr-backend-1.159-1 created (now 50374164)
by pagure@pagure.io
This is an automated email from the git hooks/post-receive script.
praiskup pushed a change to annotated tag copr-backend-1.159-1
in repository copr/copr.
at 50374164 (tag)
tagging 51a618ac85ca4046fbc5daf074e647d47d271c49 (commit)
replaces copr-frontend-1.190-1
by Pavel Raiskup
on Tue Aug 16 14:35:14 2022 +0200
- Log -----------------------------------------------------------------
Tagging package [copr-backend] version [copr-backend-1.159-1] in directory [backend/].
-----------------------------------------------------------------------
This annotated tag includes the following new commits:
new 51a618ac Automatic commit of package [copr-backend] release [1.159-1].
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
--
To stop receiving notification emails like this one, please contact
the administrator of this repository.
1 year, 7 months
[copr/copr] branch main updated: Automatic commit of package
[copr-backend] release [1.159-1].
by pagure@pagure.io
This is an automated email from the git hooks/post-receive script.
praiskup pushed a commit to branch main
in repository copr/copr.
The following commit(s) were added to refs/heads/main by this push:
new 51a618ac Automatic commit of package [copr-backend] release [1.159-1].
51a618ac is described below
commit 51a618ac85ca4046fbc5daf074e647d47d271c49
Author: Pavel Raiskup <praiskup(a)redhat.com>
AuthorDate: Tue Aug 16 14:35:14 2022 +0200
Automatic commit of package [copr-backend] release [1.159-1].
Created by command:
/usr/bin/tito tag
---
.tito/packages/copr-backend | 2 +-
backend/copr-backend.spec | 7 ++++++-
backend/setup.py | 2 +-
3 files changed, 8 insertions(+), 3 deletions(-)
diff --git a/.tito/packages/copr-backend b/.tito/packages/copr-backend
index 955d1ba7..12e557b4 100644
--- a/.tito/packages/copr-backend
+++ b/.tito/packages/copr-backend
@@ -1 +1 @@
-1.158-1 backend/
+1.159-1 backend/
diff --git a/backend/copr-backend.spec b/backend/copr-backend.spec
index 34b71d2b..edd8a33e 100644
--- a/backend/copr-backend.spec
+++ b/backend/copr-backend.spec
@@ -9,7 +9,7 @@
%global copr_common_version 0.13.1.dev
Name: copr-backend
-Version: 1.158
+Version: 1.159
Release: 1%{?dist}
Summary: Backend for Copr
@@ -244,6 +244,11 @@ useradd -r -g copr -G lighttpd -s /bin/bash -c "COPR user" copr
%exclude %{_pkgdocdir}/playbooks
%changelog
+* Tue Aug 16 2022 Pavel Raiskup <praiskup(a)redhat.com> 1.159-1
+- count only hits from an appropriate CDN hostname
+- add option for infinite number of attempts to the hitcounter script
+- print more reasonable output from AWS hitcounter script
+
* Tue Aug 16 2022 Jiri Kyjovsky <j1.kyjovsky(a)gmail.com> 1.158-1
- log every request that is sent to frontend
diff --git a/backend/setup.py b/backend/setup.py
index 2078a699..9d7dd49c 100644
--- a/backend/setup.py
+++ b/backend/setup.py
@@ -16,7 +16,7 @@ __url__ = "https://pagure.io/copr/copr"
setup(
name='copr-backend',
- version="1.158",
+ version="1.159",
description=__description__,
long_description=long_description,
author=__author__,
--
To stop receiving notification emails like this one, please contact
the administrator of this repository.
1 year, 7 months
[copr/copr] 03/03: backend: count only hits from an appropriate CDN
hostname
by pagure@pagure.io
This is an automated email from the git hooks/post-receive script.
praiskup pushed a commit to branch main
in repository copr/copr.
commit d29241459b520ee0016ab5dbe1b193be101b1b44
Author: Jakub Kadlcik <frostyx(a)email.cz>
AuthorDate: Wed Aug 10 00:34:39 2022 +0200
backend: count only hits from an appropriate CDN hostname
---
backend/conf/crontab/daily | 3 +++
backend/run/copr-aws-s3-hitcounter | 48 ++++++++++++++++++++++++++++++++++++++
2 files changed, 51 insertions(+)
diff --git a/backend/conf/crontab/daily b/backend/conf/crontab/daily
index 70f467f1..e6403a11 100644
--- a/backend/conf/crontab/daily
+++ b/backend/conf/crontab/daily
@@ -2,3 +2,6 @@
runuser -c "PYTHONUNBUFFERED=1 python3 /usr/bin/copr_prune_results.py &> /dev/null" - copr
runuser -c "PYTHONUNBUFFERED=1 python3 /usr/bin/copr_prune_srpms.py &> /dev/null" - copr
+
+# Optional - Counting downloaded RPMs through Amazon CDN
+# runuser -c "PYTHONUNBUFFERED=1 python3 /usr/bin/copr-aws-s3-hitcounter &> /dev/null" - copr
diff --git a/backend/run/copr-aws-s3-hitcounter b/backend/run/copr-aws-s3-hitcounter
index f7031f4b..5b177b77 100755
--- a/backend/run/copr-aws-s3-hitcounter
+++ b/backend/run/copr-aws-s3-hitcounter
@@ -29,6 +29,10 @@ from copr_backend.helpers import setup_script_logger
# We will allow only this hostname to delete files from the S3 storage
PRODUCTION_HOSTNAME = "copr-be.aws.fedoraproject.org"
+DEVEL_HOSTNAME = "copr-be-dev.aws.fedoraproject.org"
+
+PRODUCTION_CDN_HOSTNAME = "download.copr.fedorainfracloud.org"
+DEVEL_CDN_HOSTNAME = "download.copr-dev.fedorainfracloud.org"
log = logging.getLogger(__name__)
@@ -122,6 +126,37 @@ def parse_access_file(path):
return accesses
+def get_cdn_hostname(args):
+ """
+ The devel and production accesses are mixed together. Which ones do we want
+ to count?
+ """
+ # If a CDN hostname was explicitly specified when calling the script
+ if args.cdn_hostname:
+ return args.cdn_hostname
+
+ # Count hits from devel CDN hostname on devel instance
+ hostname = gethostname()
+ if hostname == DEVEL_HOSTNAME:
+ return DEVEL_CDN_HOSTNAME
+
+ # Default to production hits. Don't worry, we don't accidentally
+ # remove them from any other instance
+ return PRODUCTION_CDN_HOSTNAME
+
+
+def check_different_cdn_hostname(accesses, cdn_hostname):
+ """
+ If a list of HTTP accesses contain any access for a different CDN hostname
+ (e.g. for devel instance when the script is running on production), return
+ its value. Otherwise `None`.
+ """
+ for access in accesses:
+ if access["x-host-header"] != cdn_hostname:
+ return access["x-host-header"]
+ return None
+
+
def get_arg_parser():
"""
Generate argument parser for this script
@@ -147,6 +182,12 @@ def get_arg_parser():
help=("If true, try infinite number of attempts when contacting the "
"frontend. Do not use this option for cron tasks because the "
"number of simultaneously running instances might go up"))
+ parser.add_argument(
+ "--cdn-hostname",
+ help=("By default the devel instance counts only hits from devel, and "
+ "the production instance from production. You can override this "
+ "by explicitly specifying the CDN hostname of interest, e.g. {0}"
+ .format(PRODUCTION_CDN_HOSTNAME)))
return parser
@@ -157,6 +198,7 @@ def main():
parser = get_arg_parser()
args = parser.parse_args()
tmp = tempfile.mkdtemp(prefix="copr-aws-s3-hitcounter-")
+ cdn_hostname = get_cdn_hostname(args)
if args.verbose:
log.setLevel(logging.DEBUG)
@@ -169,6 +211,12 @@ def main():
raw = gunzip(gz)
accesses = parse_access_file(raw)
+ different_cdn = check_different_cdn_hostname(accesses, cdn_hostname)
+ if different_cdn:
+ log.debug("Skipping: %s (different hostname: %s)",
+ s3file, different_cdn)
+ continue
+
log.info("[%s/%s] %s (%s accesses)",
i, len(files), s3file, len(accesses))
--
To stop receiving notification emails like this one, please contact
the administrator of this repository.
1 year, 7 months
[copr/copr] 02/03: backend: add option for infinite number of
attempts to the hitcounter script
by pagure@pagure.io
This is an automated email from the git hooks/post-receive script.
praiskup pushed a commit to branch main
in repository copr/copr.
commit 21701e00f86bbef421ddd6a1b75b91d6e1d4cd5a
Author: Jakub Kadlcik <frostyx(a)email.cz>
AuthorDate: Tue Aug 9 23:40:30 2022 +0200
backend: add option for infinite number of attempts to the hitcounter script
I don't think this will be useful in production for two reasons. We
don't want to use this option for cron tasks. Also, in production,
once we process an access file, we delete it from S3. Therefore
re-running the script won't count anything twice nor waste any time
with already counted files.
But this option is useful when debugging. I don't want the script to
die after hours of counting accesses that would be counted again when
re-running the script.
---
backend/copr_backend/hitcounter.py | 9 +++++++--
backend/run/copr-aws-s3-hitcounter | 9 ++++++++-
2 files changed, 15 insertions(+), 3 deletions(-)
diff --git a/backend/copr_backend/hitcounter.py b/backend/copr_backend/hitcounter.py
index 1c53ebd2..2a2e0327 100644
--- a/backend/copr_backend/hitcounter.py
+++ b/backend/copr_backend/hitcounter.py
@@ -59,7 +59,7 @@ def url_to_key_strings(url):
return []
-def update_frontend(accesses, log, dry_run=False):
+def update_frontend(accesses, log, dry_run=False, try_indefinitely=False):
"""
Increment frontend statistics based on these `accesses`
"""
@@ -86,7 +86,12 @@ def update_frontend(accesses, log, dry_run=False):
"from_backend",
)
if not dry_run:
- SafeRequest(auth=opts.frontend_auth, log=log).post(url, result)
+ request = SafeRequest(
+ auth=opts.frontend_auth,
+ log=log,
+ try_indefinitely=try_indefinitely,
+ )
+ request.post(url, result)
def get_hit_data(accesses, log):
diff --git a/backend/run/copr-aws-s3-hitcounter b/backend/run/copr-aws-s3-hitcounter
index 8b795ae2..f7031f4b 100755
--- a/backend/run/copr-aws-s3-hitcounter
+++ b/backend/run/copr-aws-s3-hitcounter
@@ -141,6 +141,12 @@ def get_arg_parser():
"--verbose",
action="store_true",
help=("Print verbose information about what is going on"))
+ parser.add_argument(
+ "--try-indefinitely",
+ action="store_true",
+ help=("If true, try infinite number of attempts when contacting the "
+ "frontend. Do not use this option for cron tasks because the "
+ "number of simultaneously running instances might go up"))
return parser
@@ -170,7 +176,8 @@ def main():
# a scenario when we increment the accesses on the frontend but then
# leave the s3 file untouched, which would result in parsing and
# incrementing from the same file again in the next run
- update_frontend(accesses, log=log, dry_run=args.dry_run)
+ update_frontend(accesses, log=log, dry_run=args.dry_run,
+ try_indefinitely=args.try_indefinitely)
s3.delete_file(s3file)
# Clean all temporary files
--
To stop receiving notification emails like this one, please contact
the administrator of this repository.
1 year, 7 months