Attention: Please don't respond to this mail, but to the GitHub discussion mentioned below.
Hello all,
the Cockpit team is currently investigating a redesign of Cockpit's web server, which becomes slowly but surely more pressing. See [1] for details if you are interested. A very interesting possibility is to replace much of it with a web service worker [2], which would allow us to make the architecture dramatically simpler, more robust, and more secure (see prototype[3]).
The main concern with this approach is that web service workers don't work with unknown TLS certificates ("secure context"), in particular for the self-signed certificates that cockpit creates on startup if you didn't supply your own. I. e. they don't support the usual TOFU model with "click here to accept this unknown certificate anyway" that browsers show for "normal" web pages in this case.
So I would like to do a little survey and request for comments about deprecating or changing the handling of self-signed TLS certs in Cockpit. This is an annoyingly hard problem: On one hand we really want to build a world where TLS works and your services are trustworthy (*especially* if you are typing in your root password as the next thing!), but for many home or even small business situations even LetsEncrypt etc. is impractical/impossible (no DNS, etc.). My questions:
* Are you currently accessing Cockpit with the builtin fallback self-signed certificate?
* If so, would you have a better option like getting a proper cert, or joining the machine to a FreeIPA domain (which requests a valid cert from the IPA server), or running Cockpit behind a reverse proxy which already has a proper certificate? Or using the Cockpit Client flatpak? If any of these don't work for you, can you describe your situation?
* If the above isn't possible, could you live with running the cockpit web server on a machine which has proper TLS setup, and connect to your target machines with SSH?
* As last resort, Cockpit could show an error page (after accepting the invalid certificate in the browser) which explains the situation and offers you to download the CA certificate for importing into your browser as trusted CA. This includes the case that the default self-signed certificate is not valid for the hostname you are using to connect to cockpit (i.e. not "localhost" or the server's unqualified host name) - then you need to regenerate the certificate with the additional name.
This could then be used to actually establish some trust to the certificate, by something like `ssh remote cockpit-show-cert` which would show you the fingerprint or similar -- then you have something actually meaningful to check in the browser.
Would that be acceptable for you?
* Are you working on a different web-based project and already have some experience with handling proper certificates?
Notes:
* This only affects remote connections -- if you connect to your own machine on http://localhost:9090, you can use unencrypted HTTP and service workers are fine.
* We are aware of and use `sscg` if it's installed (cockpkit-ws.rpm Recommends: it). We can change the non-sscg fallback to also create a separate CA and cert, instead of the single self-signed cert. So that's not a problem, just a bit of work.
Please respond on https://github.com/cockpit-project/cockpit/discussions/21695 to keep the responses in one place -- this annoucement goes to several mailing lists.
Thank you!
Martin
[1] https://issues.redhat.com/projects/COCKPIT/issues/COCKPIT-1238 [2] https://developer.mozilla.org/de/docs/Web/API/Service_Worker_API [3] https://issues.redhat.com/browse/COCKPIT-1240
Hi Martin,
Just throwing this out there, I have an idea but also a question
Currently what web server does cockpit use or is it dependent on the server admin what cockpit uses?
Could cockpit get tied into working with lets encrypts cert bot to deploy a properly signed certificate?
Regards, Jonathan ________________________________ From: Martin Pitt via cockpit-devel cockpit-devel@lists.fedorahosted.org Sent: 11 March 2025 13:04 To: server@lists.fedoraproject.org server@lists.fedoraproject.org; cockpit-devel@lists.fedorahosted.org cockpit-devel@lists.fedorahosted.org Cc: Martin Pitt mpitt@redhat.com Subject: RFC: weaning off cockpit web console from self-signed certificates
Attention: Please don't respond to this mail, but to the GitHub discussion mentioned below.
Hello all,
the Cockpit team is currently investigating a redesign of Cockpit's web server, which becomes slowly but surely more pressing. See [1] for details if you are interested. A very interesting possibility is to replace much of it with a web service worker [2], which would allow us to make the architecture dramatically simpler, more robust, and more secure (see prototype[3]).
The main concern with this approach is that web service workers don't work with unknown TLS certificates ("secure context"), in particular for the self-signed certificates that cockpit creates on startup if you didn't supply your own. I. e. they don't support the usual TOFU model with "click here to accept this unknown certificate anyway" that browsers show for "normal" web pages in this case.
So I would like to do a little survey and request for comments about deprecating or changing the handling of self-signed TLS certs in Cockpit. This is an annoyingly hard problem: On one hand we really want to build a world where TLS works and your services are trustworthy (*especially* if you are typing in your root password as the next thing!), but for many home or even small business situations even LetsEncrypt etc. is impractical/impossible (no DNS, etc.). My questions:
* Are you currently accessing Cockpit with the builtin fallback self-signed certificate?
* If so, would you have a better option like getting a proper cert, or joining the machine to a FreeIPA domain (which requests a valid cert from the IPA server), or running Cockpit behind a reverse proxy which already has a proper certificate? Or using the Cockpit Client flatpak? If any of these don't work for you, can you describe your situation?
* If the above isn't possible, could you live with running the cockpit web server on a machine which has proper TLS setup, and connect to your target machines with SSH?
* As last resort, Cockpit could show an error page (after accepting the invalid certificate in the browser) which explains the situation and offers you to download the CA certificate for importing into your browser as trusted CA. This includes the case that the default self-signed certificate is not valid for the hostname you are using to connect to cockpit (i.e. not "localhost" or the server's unqualified host name) - then you need to regenerate the certificate with the additional name.
This could then be used to actually establish some trust to the certificate, by something like `ssh remote cockpit-show-cert` which would show you the fingerprint or similar -- then you have something actually meaningful to check in the browser.
Would that be acceptable for you?
* Are you working on a different web-based project and already have some experience with handling proper certificates?
Notes:
* This only affects remote connections -- if you connect to your own machine on https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Flink.edgep...https://link.edgepilot.com/s/341579e1/LsBBRz6SdkmqXvciAG5HKQ?u=http://localhost:9090/, you can use unencrypted HTTP and service workers are fine.
* We are aware of and use `sscg` if it's installed (cockpkit-ws.rpm Recommends: it). We can change the non-sscg fallback to also create a separate CA and cert, instead of the single self-signed cert. So that's not a problem, just a bit of work.
Please respond on https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Flink.edgep...https://link.edgepilot.com/s/f441fd7c/dVd7ACyJ0E_uCqQAwjjOEg?u=https://github.com/cockpit-project/cockpit/discussions/21695 to keep the responses in one place -- this annoucement goes to several mailing lists.
Thank you!
Martin
[1] https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Flink.edgep...https://link.edgepilot.com/s/5c4501b9/b-TOfmssYEqVZtcTe8V6_w?u=https://issues.redhat.com/projects/COCKPIT/issues/COCKPIT-1238 [2] https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Flink.edgep...https://link.edgepilot.com/s/cf700175/PvZRoO5FXEywBfdSvOt5jw?u=https://developer.mozilla.org/de/docs/Web/API/Service_Worker_API [3] https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Flink.edgep...https://link.edgepilot.com/s/213fd245/JwrzR4WlV0e3GkT7XwsI1Q?u=https://issues.redhat.com/browse/COCKPIT-1240
-- _______________________________________________ cockpit-devel mailing list -- cockpit-devel@lists.fedorahosted.org To unsubscribe send an email to cockpit-devel-leave@lists.fedorahosted.org Fedora Code of Conduct: https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Flink.edgep...https://link.edgepilot.com/s/8b2bde96/WvX5slK96ECjRaCGVQ8O3Q?u=https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Flink.edgep...https://link.edgepilot.com/s/5c91a587/yNN51T5RxUiXOuM9_5Joxw?u=https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Flink.edgep...https://link.edgepilot.com/s/fc52c1c3/yAUlsJ-SDUWQdvuBS019bg?u=https://lists.fedorahosted.org/archives/list/cockpit-devel@lists.fedorahosted.org Do not reply to spam, report it: https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Flink.edgep...https://link.edgepilot.com/s/7f567b0f/ijI3if79hE_ZB05tiRcsYw?u=https://pagure.io/fedora-infrastructure/new_issue
Links contained in this email have been replaced. If you click on a link in the email above, the link will be analyzed for known threats. If a known threat is found, you will not be able to proceed to the destination. If suspicious content is detected, you will see a warning.
Hello Jonathan,
Jonathan Aquilina via cockpit-devel [2025-03-11 12:53 +0000]:
Currently what web server does cockpit use or is it dependent on the server admin what cockpit uses?
cockpit-ws is an independent implementation of a web server. We can't just use a standard one like nginx, as it does a lot of cockpit specific things (authentication, PAM session creation, web socket, bridge connection).
Could cockpit get tied into working with lets encrypts cert bot to deploy a properly signed certificate?
Yes, of course -- I mentioned that below as "getting a proper cert". E.g. we have a recipe how to set up LE and reverse proxy:
https://cockpit-project.org/external/wiki/Proxying-Cockpit-over-Apache-with-...
Martin
Good Afternoon Martin,
Probably a stupid question here but what advantage does having its own custom created web server offer over something like nginx with reverse proxy, apache, or even something super lightweight such as lighttpd or even caddy?
I think my question can be rephrased just in case its not understandable, why reinvent the wheel when out there in FOSS land solutions for web servers already exist? No doubt switching to these alternatives, no doubt there are more that i am not aware of, would greatly reduce complexity of the existing codebase.
If I am deraling the purpose of this thread, do let me know if I am and I will refrain from any further responses or questions.
In my case in regards to the use of cockpit. I love that i can run it locally on my own linux laptop and manage any of my linux servers with it and in a secure manner as well.
Having a proper LE certificate will be amazing to have it out of the box on install.
Regards,
Jonathan Aquilina
________________________________ From: Martin Pitt via cockpit-devel cockpit-devel@lists.fedorahosted.org Sent: 11 March 2025 14:36 To: cockpit-devel@lists.fedorahosted.org cockpit-devel@lists.fedorahosted.org Cc: Martin Pitt mpitt@redhat.com Subject: Re: RFC: weaning off cockpit web console from self-signed certificates
Hello Jonathan,
Jonathan Aquilina via cockpit-devel [2025-03-11 12:53 +0000]:
Currently what web server does cockpit use or is it dependent on the server admin what cockpit uses?
cockpit-ws is an independent implementation of a web server. We can't just use a standard one like nginx, as it does a lot of cockpit specific things (authentication, PAM session creation, web socket, bridge connection).
Could cockpit get tied into working with lets encrypts cert bot to deploy a properly signed certificate?
Yes, of course -- I mentioned that below as "getting a proper cert". E.g. we have a recipe how to set up LE and reverse proxy:
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Flink.edgep...https://link.edgepilot.com/s/f9c09b58/SsGPlQw51kWHnX4g9nj7Ng?u=https://cockpit-project.org/external/wiki/Proxying-Cockpit-over-Apache-with-LetsEncrypt
Martin
-- _______________________________________________ cockpit-devel mailing list -- cockpit-devel@lists.fedorahosted.org To unsubscribe send an email to cockpit-devel-leave@lists.fedorahosted.org Fedora Code of Conduct: https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Flink.edgep...https://link.edgepilot.com/s/3d495c3b/9BbVK5faLU2RiFYiDmd6PA?u=https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Flink.edgep...https://link.edgepilot.com/s/2cd039f7/GF3Iemo08k2e2CuUsJeRag?u=https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Flink.edgep...https://link.edgepilot.com/s/b059ed60/k9IwYel6bkSBWVaqcU5eDg?u=https://lists.fedorahosted.org/archives/list/cockpit-devel@lists.fedorahosted.org Do not reply to spam, report it: https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Flink.edgep...https://link.edgepilot.com/s/6a93d7a6/Zf8H7eSShkW4CbfXRJ0GsA?u=https://pagure.io/fedora-infrastructure/new_issue
Links contained in this email have been replaced. If you click on a link in the email above, the link will be analyzed for known threats. If a known threat is found, you will not be able to proceed to the destination. If suspicious content is detected, you will see a warning.
Hello Jonathan,
changed the Subject to make it clearer that this is a different topic.
Jonathan Aquilina via cockpit-devel [2025-03-11 13:46 +0000]:
Probably a stupid question here but what advantage does having its own custom created web server offer over something like nginx with reverse proxy, apache, or even something super lightweight such as lighttpd or even caddy?
Because none of these can do what cockpit-ws does: authentication, PAM session setup, connecting to cockpit-bridge, exposing and driving the websocket, speaking the JSON wire protocol, etc. But you are right for the portion of being a generic web server to serve the *.html/*.css/*.js files) - this should go away (many thousand lines of C code) and be replaced with a standard Python/Rust/etc. libary, so that we only implement the cockpit specific bits. That's what triggered all this discussion and researching for a better design of the web server to minimize it in the future.
In my case in regards to the use of cockpit. I love that i can run it locally on my own linux laptop and manage any of my linux servers with it and in a secure manner as well.
So for you that's a http://localhost:9090 case then? (no TLS/https). That will be fine.
Having a proper LE certificate will be amazing to have it out of the box on install.
Sure, but that's not going to happen. Setting up certificates is hard, and absolutely not automatable. There's a reason why it has taken decades to even get to where we are now, and it's still far from done. 😅
Thanks,
Pitti
Hi Martin,
On my local machine I would still prefer to run a proper certificate and even on my home network I have looked into this.
The issue is automated renewal and auto deployment to the infrastructure devices from cert bot.
What could be done is that certbot can download but not deploy the certificates and then lets say in the config we specify where the LE certificates get placed /opt/LE for instance and any time they renew cockpit would just look in that location and have the certificates ready. All that would be needed would be an 80 to 443 redirect.
In terms of authentication has modern forms of authentication been considered like logging in with SSO or other platforms to login, just like keycloak offers along side SSO.
I am very interested in contributing to this project in any way that I can. How can someone get started contributing?
Regards,
Jonathan Aquilina
[cid:c51a856e-074a-416f-b203-e61fb16e6ca6]
________________________________ From: Martin Pitt via cockpit-devel cockpit-devel@lists.fedorahosted.org Sent: 11 March 2025 14:55 To: cockpit-devel@lists.fedorahosted.org cockpit-devel@lists.fedorahosted.org Cc: Martin Pitt mpitt@redhat.com Subject: cockpit-ws custom web server [was: Re: RFC: weaning off cockpit web console from self-signed certificates]
Hello Jonathan,
changed the Subject to make it clearer that this is a different topic.
Jonathan Aquilina via cockpit-devel [2025-03-11 13:46 +0000]:
Probably a stupid question here but what advantage does having its own custom created web server offer over something like nginx with reverse proxy, apache, or even something super lightweight such as lighttpd or even caddy?
Because none of these can do what cockpit-ws does: authentication, PAM session setup, connecting to cockpit-bridge, exposing and driving the websocket, speaking the JSON wire protocol, etc. But you are right for the portion of being a generic web server to serve the *.html/*.css/*.js files) - this should go away (many thousand lines of C code) and be replaced with a standard Python/Rust/etc. libary, so that we only implement the cockpit specific bits. That's what triggered all this discussion and researching for a better design of the web server to minimize it in the future.
In my case in regards to the use of cockpit. I love that i can run it locally on my own linux laptop and manage any of my linux servers with it and in a secure manner as well.
So for you that's a https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Flink.edgep...https://link.edgepilot.com/s/90482740/sDKo6z22o0CrJXvS9Kz_EA?u=http://localhost:9090/ case then? (no TLS/https). That will be fine.
Having a proper LE certificate will be amazing to have it out of the box on install.
Sure, but that's not going to happen. Setting up certificates is hard, and absolutely not automatable. There's a reason why it has taken decades to even get to where we are now, and it's still far from done. 😅
Thanks,
Pitti
-- _______________________________________________ cockpit-devel mailing list -- cockpit-devel@lists.fedorahosted.org To unsubscribe send an email to cockpit-devel-leave@lists.fedorahosted.org Fedora Code of Conduct: https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Flink.edgep...https://link.edgepilot.com/s/41b60283/LBnsPY43K0KmvbQnjhKgQg?u=https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Flink.edgep...https://link.edgepilot.com/s/22f1c424/nR6BeJeUA0SHbabb_kDHAw?u=https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Flink.edgep...https://link.edgepilot.com/s/5df21a94/LbvXh2a6qkK5Z-P7yxjBGg?u=https://lists.fedorahosted.org/archives/list/cockpit-devel@lists.fedorahosted.org Do not reply to spam, report it: https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Flink.edgep...https://link.edgepilot.com/s/82db3431/l-I3lrqcrUKUOvtV-f4nnA?u=https://pagure.io/fedora-infrastructure/new_issue
Links contained in this email have been replaced. If you click on a link in the email above, the link will be analyzed for known threats. If a known threat is found, you will not be able to proceed to the destination. If suspicious content is detected, you will see a warning.
Hello Jonathan,
Jonathan Aquilina [2025-03-11 14:04 +0000]:
On my local machine I would still prefer to run a proper certificate and even on my home network I have looked into this.
The issue is automated renewal and auto deployment to the infrastructure devices from cert bot.
What could be done is that certbot can download but not deploy the certificates and then lets say in the config we specify where the LE certificates get placed /opt/LE for instance and any time they renew cockpit would just look in that location and have the certificates ready.
You can already do that: Let certbot create the cert in /etc/cockpit/ws-cert.d/ , or put a symlink there to where LE writes its certificate (as that is often shared by multiple consumers). See https://cockpit-project.org/guide/latest/https.html
In terms of authentication has modern forms of authentication been considered like logging in with SSO or other platforms to login, just like keycloak offers along side SSO.
SSO with FreeIPA or AD have both been supported for a long time: https://cockpit-project.org/guide/latest/sso.html
You can also use custom authentication schemas, but this is much more difficult, underdocumented, and there isn't a lot of precendent. That is an area which could use some love.
I am very interested in contributing to this project in any way that I can.
Thanks for your interest!
How can someone get started contributing?
https://cockpit-project.org/external/wiki/Contributing.html is a good entry point - it has the documentation how to get started with setting up a dev environment, making and testing your first change, how cockpit works, our Matrix channel for asking us (we are there to help!), and for how to make contributions. If anything is unclear on this page, please also let us know, we want to keep it current, useful, and comprehensible.
Thanks,
Pitti
Thanks Martin,
All interesting information 😊
I will head over to the matrix channel, but in a quick question is there an official docker container available for cockpit? I am asking as that would be an ideal dev environment to work with.
Regards, Jonathan Aquilina
-----Original Message----- From: Martin Pitt via cockpit-devel cockpit-devel@lists.fedorahosted.org Sent: 12 March 2025 05:34 To: cockpit-devel@lists.fedorahosted.org Cc: Martin Pitt mpitt@redhat.com Subject: Re: cockpit-ws custom web server
Hello Jonathan,
Jonathan Aquilina [2025-03-11 14:04 +0000]:
On my local machine I would still prefer to run a proper certificate and even on my home network I have looked into this.
The issue is automated renewal and auto deployment to the infrastructure devices from cert bot.
What could be done is that certbot can download but not deploy the certificates and then lets say in the config we specify where the LE certificates get placed /opt/LE for instance and any time they renew cockpit would just look in that location and have the certificates ready.
You can already do that: Let certbot create the cert in /etc/cockpit/ws-cert.d/ , or put a symlink there to where LE writes its certificate (as that is often shared by multiple consumers). See https://link.edgepilot.com/s/046a5968/Ty0pb87TOU2vJ-4iJN_dqQ?u=https://cockp...
In terms of authentication has modern forms of authentication been considered like logging in with SSO or other platforms to login, just like keycloak offers along side SSO.
SSO with FreeIPA or AD have both been supported for a long time: https://link.edgepilot.com/s/859d00af/Gr7uf2r74EynN125WplI9g?u=https://cockp...
You can also use custom authentication schemas, but this is much more difficult, underdocumented, and there isn't a lot of precendent. That is an area which could use some love.
I am very interested in contributing to this project in any way that I can.
Thanks for your interest!
How can someone get started contributing?
https://link.edgepilot.com/s/79ae1b61/RO0H3-Zgo0yJmAqMSxM3_g?u=https://cockp... is a good entry point - it has the documentation how to get started with setting up a dev environment, making and testing your first change, how cockpit works, our Matrix channel for asking us (we are there to help!), and for how to make contributions. If anything is unclear on this page, please also let us know, we want to keep it current, useful, and comprehensible.
Thanks,
Pitti
-- _______________________________________________ cockpit-devel mailing list -- cockpit-devel@lists.fedorahosted.org To unsubscribe send an email to cockpit-devel-leave@lists.fedorahosted.org Fedora Code of Conduct: https://link.edgepilot.com/s/75d2311e/JdDHR9sa1Uu-7atjRWDm6w?u=https://docs.... List Guidelines: https://link.edgepilot.com/s/273f9a06/othnbwCURUucmHhxEJGErQ?u=https://fedor... List Archives: https://link.edgepilot.com/s/6dba911f/oMr5nPn2jk__I1Y4q3039A?u=https://lists... Do not reply to spam, report it: https://link.edgepilot.com/s/7633c27c/gObO7zx0UkeWljsXlFcFgw?u=https://pagur...
Links contained in this email have been replaced. If you click on a link in the email above, the link will be analyzed for known threats. If a known threat is found, you will not be able to proceed to the destination. If suspicious content is detected, you will see a warning.
Hello Jonathan,
Jonathan Aquilina via cockpit-devel [2025-03-12 4:46 +0000]:
I will head over to the matrix channel, but in a quick question is there an official docker container available for cockpit? I am asking as that would be an ideal dev environment to work with.
Yes! Well, toolbox (distrobox works too), not docker, as that's much nicer. That's in fact our prefered and recommended setup: https://github.com/cockpit-project/cockpit/blob/main/HACKING.md#setting-up-d...
Martin
cockpit-devel@lists.fedorahosted.org