Hi all,
I had about 150-200 of these emails come in over the last few minutes.
Is there any kind of action we can take to quiet these down or get to
the bottom of what's causing them? This also happens intermittently
outside of the past 30 minutes too.
Thanks!
-------- Forwarded Message --------
Subject: fedmsg error log
badges-backend01.phx2.fedoraproject.org
Date: Mon, 10 Apr 2017 23:41:09 -0000
From: fedmsg(a)fedoraproject.org
To: sysadmin-badges-members(a)fedoraproject.org
Message
-------
[2017-04-10 23:41:09][moksha.hub ERROR]
```python
Couldn't talk to pkgdb2 for u'mbs/mbs.fedoraproject.org', 404,
u'<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2
Final//EN">\n<title>404
Not Found</title>\n<h1>Not Found</h1>\n<p>The requested URL was
not
found on the server. If you entered the URL manually please check your
spelling and try again.</p>\n'
```
Process Details
---------------
- host:
badges-backend01.phx2.fedoraproject.org
- PID: 31829
- name: fedmsg-hub
- command: /usr/bin/python /usr/bin/fedmsg-hub
- msg_id: 2017-18d2a3e1-2cb0-45a2-97d8-aba549a4e8fb
Callstack that lead to the logging statement
--------------------------------------------
```python
File "/usr/lib64/python2.7/threading.py", line 784 in __bootstrap
self.__bootstrap_inner()
File "/usr/lib64/python2.7/threading.py", line 811 in __bootstrap_inner
self.run()
File "/usr/lib64/python2.7/threading.py", line 764 in run
self.__target(*self.__args, **self.__kwargs)
File
"/usr/lib64/python2.7/site-packages/twisted/python/threadpool.py", line
167 in _worker
result = context.call(ctx, function, *args, **kwargs)
File "/usr/lib64/python2.7/site-packages/twisted/python/context.py",
line 118 in callWithContext
return self.currentContext().callWithContext(ctx, func, *args, **kw)
File "/usr/lib64/python2.7/site-packages/twisted/python/context.py",
line 81 in callWithContext
return func(*args,**kw)
File "/usr/lib/python2.7/site-packages/moksha/hub/api/consumer.py",
line 191 in _work
self.consume(message)
File "/usr/lib/python2.7/site-packages/fedbadges/consumers.py", line
237 in consume
for recipient in badge_rule.matches(msg):
File "/usr/lib/python2.7/site-packages/fedbadges/rules.py", line 236
in matches
if not self.criteria.matches(msg):
File "/usr/lib/python2.7/site-packages/fedbadges/utils.py", line 94 in
inner
return method(self, *args, **kwargs)
File "/usr/lib/python2.7/site-packages/fedbadges/rules.py", line 368
in matches
child.matches(msg) for child in self.children
File "/usr/lib/python2.7/site-packages/fedbadges/rules.py", line 66 in
<lambda>
"not": lambda x: all([not item for item in x])
File "/usr/lib/python2.7/site-packages/fedbadges/rules.py", line 368
in <genexpr>
child.matches(msg) for child in self.children
File "/usr/lib/python2.7/site-packages/fedbadges/utils.py", line 96 in
inner
log.exception(e)
```
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/fedbadges/utils.py", line 94,
in inner
return method(self, *args, **kwargs)
File "/usr/lib/python2.7/site-packages/fedbadges/rules.py", line 371,
in matches
return self.specialization.matches(msg)
File "/usr/lib/python2.7/site-packages/fedbadges/rules.py", line 418,
in matches
user=expectation['user'],
File "/usr/lib/python2.7/site-packages/fedbadges/utils.py", line 145,
in get_pkgdb_packages_for
return _getter(user)
File "/usr/lib/python2.7/site-packages/dogpile/cache/region.py", line
1053, in decorate
should_cache_fn)
File "/usr/lib/python2.7/site-packages/dogpile/cache/region.py", line
657, in get_or_create
async_creator) as value:
File "/usr/lib/python2.7/site-packages/dogpile/core/dogpile.py", line
158, in __enter__
return self._enter()
File "/usr/lib/python2.7/site-packages/dogpile/core/dogpile.py", line
98, in _enter
generated = self._enter_create(createdtime)
File "/usr/lib/python2.7/site-packages/dogpile/core/dogpile.py", line
149, in _enter_create
created = self.creator()
File "/usr/lib/python2.7/site-packages/dogpile/cache/region.py", line
625, in gen_value
created_value = creator()
File "/usr/lib/python2.7/site-packages/dogpile/cache/region.py", line
1049, in creator
return fn(*arg, **kw)
File "/usr/lib/python2.7/site-packages/fedbadges/utils.py", line 143,
in _getter
return _get_pkgdb2_packages_for(config, user)
File "/usr/lib/python2.7/site-packages/fedbadges/utils.py", line 166,
in _get_pkgdb2_packages_for
data = _get_page(1)
File "/usr/lib/python2.7/site-packages/fedbadges/utils.py", line 159,
in _get_page
username, req.status_code, req.text))
IOError: Couldn't talk to pkgdb2 for u'mbs/mbs.fedoraproject.org', 404,
u'<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2
Final//EN">\n<title>404
Not Found</title>\n<h1>Not Found</h1>\n<p>The requested URL was
not
found on the server. If you entered the URL manually please check your
spelling and try again.</p>\n'