How does it map to apache's ServerLimit, etc. values? Is it possible that there is a much higher value of allowed httpd processes to be alive in the same moment? Having 300 max_connections is quite enough for large-scale deployment, so I would more look into how apache is configured than tuning the postgres.

On Sat, Jan 8, 2022 at 1:20 AM Robby Callicotte <rcallicotte@mailbox.org> wrote:
Greetings all,

I have recently installed koji(-hub/web) 1.27.0 on CentOS 8 stream and I'm
seeing some very odd behavior.  It appears that koji is maxing out the
connections to the postgres database.  I'm running postgres 13 with all
default connection settings (max_connections=100) using unix-sockets instead
of tcp-sockets.

Periodically, clicking around in the koji-web interface yields the message
"Server is offline".  After some time I am able to resume clicking links.

Postgres logs show this message when the "Server offline" message appears:

2022-01-07 15:41:48.449 CST [185432] FATAL: remaining connection slots are
reserved for non-replication superuser connections
2022-01-07 15:43:09.907 CST [185558] FATAL: remaining connection slots are
reserved for non-replication superuser connections

Systemd shows the following:
● postgresql.service - PostgreSQL database server
Loaded: loaded (/usr/lib/systemd/system/postgresql.service; enabled; vendor
preset: disabled)
Active: active (running) since Fri 2022-01-07 17:25:12 CST; 3min 8s ago
Process: 199845 ExecStartPre=/usr/libexec/postgresql-check-db-dir postgresql
(code=exited, status=0/SUCCESS)
Main PID: 199848 (postmaster)
Tasks: 100 (limit: 75288)
Memory: 201.8M
CGroup: /system.slice/postgresql.service
├─199848 /usr/bin/postmaster -D /var/lib/pgsql/data
├─199849 postgres: logger
├─199851 postgres: checkpointer
├─199852 postgres: background writer
├─199853 postgres: walwriter
├─199854 postgres: autovacuum launcher
├─199855 postgres: stats collector
├─199856 postgres: logical replication launcher
├─199867 postgres: koji koji [local] idle
├─199869 postgres: koji koji [local] idle
├─199870 postgres: koji koji [local] idle
├─199871 postgres: koji koji [local] idle
├─199872 postgres: koji koji [local] idle
...

The host that runs koji has only been active for one day and is not configured
with any tags, buildroots, or packages.  There are only two build hosts
connected to the hub. 

I would appreciate any assistance you could give.

PS - I increased the max_connections setting in postgres to 200 and eventually
that limit is also reached.


--
Robby Callicotte
He/Him/His
Timezone: America/Chicago
IRC: c4t3l | Twitter: @robbycl2v_______________________________________________
koji-devel mailing list -- koji-devel@lists.fedorahosted.org
To unsubscribe send an email to koji-devel-leave@lists.fedorahosted.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedorahosted.org/archives/list/koji-devel@lists.fedorahosted.org
Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure


--

            Tomas Kopecek <tkopecek@redhat.com>
            RHEL Build Development, RedHat