Hi,
I am trying to filter traffic between pasta rootless podman
containers, using manual setup without any managed solution, on an
EL9.2 clone.
The simple case I wish to complete is to permit traffic to api only to
it's 8088 port and only from web source, and do not allow web to
connect to anywhere else, and neither api.
For this, I have setup two containers, named web and api, and given
them the addresses
10.1.1.211 web
10.1.1.221 api
I've setup their networking using the pasta software,using podman
create -h api --network=pasta:-4,-a,10.1.1.221,--map-gw,--no-udp,--no-dhcp,--no-ndp,--no-ra,--no-dhcpv6,-I,eth0,-t,10.1.1.221/1882:80
(and similarly web 10.1.1.211/1802:80)
I am running each of them into their own host OS username and I've
choosen the same number for their UID as well as for the TCP listening
port number of their own containers.
$ id -u web
1802
$ id -u api
1882
In firewalld I have configured masquerading each UID into their own
outgoing IP address:
$ sudo firewall-cmd --direct --get-all-rules
ipv4 nat POSTROUTING 0 -m owner --uid-owner 1802 -j SNAT --to-source 10.1.1.211
ipv4 nat POSTROUTING 0 -m owner --uid-owner 1882 -j SNAT --to-source 10.1.1.221
The IP addresses are already assigned on the network interface
$ nmcli c s public | grep ipv4.address
ipv4.addresses: 10.1.1.223/24, 10.1.1.211/24,
10.1.1.221/24
I have created a zone for each source IP address, and two policies:
one web to api, one web to anywhere:
$ sudo firewall-cmd --get-active-zones
fromapi
sources: 10.1.1.221
fromweb
sources: 10.1.1.211
$ sudo firewall-cmd --zone=fromapi --list-all
fromapi (active)
target: default
icmp-block-inversion: no
interfaces:
sources: 10.1.1.221
services:
ports:
protocols:
forward: no
masquerade: no
forward-ports:
source-ports:
icmp-blocks:
rich rules:
$ sudo firewall-cmd --zone=fromweb --list-all
fromweb (active)
target: default
icmp-block-inversion: no
interfaces:
sources: 10.1.1.211
services:
ports:
protocols:
forward: no
masquerade: no
forward-ports:
source-ports:
icmp-blocks:
rich rules:
$ sudo firewall-cmd --get-policies
allow-host-ipv6 web2any web2api
$ sudo firewall-cmd --info-policy=web2any
web2any (active)
priority: -1
target: REJECT
ingress-zones: fromweb
egress-zones: ANY
services:
ports:
protocols:
masquerade: no
forward-ports:
source-ports:
icmp-blocks:
rich rules:
Note above changing the default policy to REJECT
$ sudo firewall-cmd --info-policy=web2api
web2api (active)
priority: -1
target: CONTINUE
ingress-zones: fromweb
egress-zones: fromapi
services:
ports: 8088/tcp
protocols:
masquerade: no
forward-ports:
port=8088:proto=tcp:toport=1882:toaddr=10.1.1.221
source-ports:
icmp-blocks:
rich rules:
Note above I am doing the port forward because I intend to setup a
second host and I want to standardize across hosts what port would the
API run, while allowing to keep using the uid=port pair numbers using
podman
The resulting nft list ruleset is quite big, pasted here:
https://paste.sh/3_ejeq92#bt4HpZy-Gg0IIfN6wxleFU5W
Now, to testing.
First issue:
I enter web shell and run:
[root@web /]# http_proxy='' ALL_PROXY='' curl -m 1 api:1882
api 10.1.1.221 port local 1882 port mapped 8088 # api answer produced
by a echo + netcat script
On tcpdump however, I see the response from api going to HOST's
primary IP address (10.1.1.223) instead of returning to web IP
address. Not sure what is the cause and how this affects the goal, but
mentioning anyway:
$ sudo tcpdump -qnni any not port 22
tcpdump: data link type LINUX_SLL2
dropped privs to tcpdump
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on any, link-type LINUX_SLL2 (Linux cooked v2), snapshot
length 262144 bytes
20:01:46.369277 lo In IP 10.1.1.211.50922 > 10.1.1.221.1882: tcp 0
20:01:46.369321 lo In IP 10.1.1.221.1882 > 10.1.1.223.50922: tcp 0
20:01:46.369345 lo In IP 10.1.1.211.50922 > 10.1.1.221.1882: tcp 0
20:01:46.369478 lo In IP 10.1.1.211.50922 > 10.1.1.221.1882: tcp 72
20:01:46.369496 lo In IP 10.1.1.221.1882 > 10.1.1.223.50922: tcp 0
20:01:46.404545 lo In IP 10.1.1.221.1882 > 10.1.1.223.50922: tcp 97
20:01:46.404584 lo In IP 10.1.1.211.50922 > 10.1.1.221.1882: tcp 0
20:01:46.404798 lo In IP 10.1.1.221.1882 > 10.1.1.223.50922: tcp 0
20:01:46.405043 lo In IP 10.1.1.211.50922 > 10.1.1.221.1882: tcp 0
20:01:46.405066 lo In IP 10.1.1.221.1882 > 10.1.1.223.50922: tcp 0
Attempting however to connect to the mapped port - fails, and I see no
traffic in tcpdump:
[root@web /]# http_proxy='' ALL_PROXY='' curl -m 1 api:8088
curl: (28) Connection timed out after 1001 milliseconds
Second, attempting to connect from web to anywhere, such as to a
nearby proxy, unexpectedly succeeds:
[root@web /]# http_proxy='' ALL_PROXY='' curl -m 1 10.1.1.252:3128
<html><head>
[...]
tcpdump:
20:07:22.747293 ens192 Out IP 10.1.1.211.53902 > 10.1.1.252.3128: tcp 0
20:07:22.747466 ens192 B ARP, Request who-has 10.1.1.211 tell
10.1.1.252, length 46
20:07:22.747478 ens192 Out ARP, Reply 10.1.1.211 is-at
00:0c:36:e2:8a:7d, length 28
20:07:22.747560 ens192 In IP 10.1.1.252.3128 > 10.1.1.211.53902: tcp 0
20:07:22.747595 ens192 Out IP 10.1.1.211.53902 > 10.1.1.252.3128: tcp 0
20:07:22.747872 ens192 Out IP 10.1.1.211.53902 > 10.1.1.252.3128: tcp 83
[...]
Questions:
1) Why does the response from api goes to destination being primary IP
of the host, instead of the IP alias of the web container?
2) Why does web can initiate outgoing to anywhere?
I tried raising the priority for web2api rule to -10 but despite
"success" status, it seems to have not changed:
$ sudo firewall-cmd --permanent --policy=web2api --set-priority=-10
success
$ sudo firewall-cmd --policy=web2api --set-priority=-10
success
$ sudo firewall-cmd --info-policy=web2api
web2api (active)
priority: -1
not even after --reload (maybe is this an issue fixed in a later release?)
changed it by editing the xml:
$ sudo firewall-cmd --info-policy=web2api
web2api (active)
priority: -10
Yet still web can make outgoing requests other than what expected.
I am fairly new at understanding zones and policies, and despite
having used firewalld before and reading and re-reading the docs,
things are still a bit fuzzy for me. For example it took many
re-reading to conclude that egress and ingress mean the direction of
traffic in relation to the firewall itself and not in relation to the
interface (which is exactly in the opposite direction), or that I can
have a zone without an interface and only using a source IP; but can I
use that zone to filter traffic destined _to_ that IP when used in a
policy as an egress zone, or should I explicitly configure that in a
rich rule?
