Thanks Fernando. I will verify it locally, and let you know.
Nobin
-----Original Message-----
From: Fernando Fernandez Mancera <ferferna(a)redhat.com>
Sent: Monday, August 29, 2022 8:13 PM
To: Nobin Mathew <nobin.mathew(a)ericsson.com>
Cc: nmstate-devel(a)lists.fedorahosted.org
Subject: Re: [nmstate-devel] OVS missing some interfaces on node reboot or NetworkManager
retstart
Hi Nobin,
just wanted to let you know that I backported the fix to nm-1-38,
nm-1-36 and nm-1-34 branches. You can check it here:
Let me know if you need anything else.
Thanks,
Fernando.
On Mon, Aug 29, 2022 at 11:08 AM Nobin Mathew <nobin.mathew(a)ericsson.com> wrote:
Thanks Fernando for the quick response...
It will be great if you can provide a backported patch for 1.34, otherwise please share
the patch which fixed this problem, I will try to backport.
-Nobin
-----Original Message-----
From: Fernando Fernandez Mancera <ferferna(a)redhat.com>
Sent: Monday, August 29, 2022 1:06 PM
To: Nobin Mathew <nobin.mathew(a)ericsson.com>
Cc: nmstate-devel(a)lists.fedorahosted.org
Subject: Re: [nmstate-devel] OVS missing some interfaces on node
reboot or NetworkManager retstart
Hi!
On Mon, Aug 29, 2022 at 7:36 AM Nobin Mathew <nobin.mathew(a)ericsson.com> wrote:
>
> Hi,
>
> We r running
>
> ######
>
> /home/ # NetworkManager -V
>
> 1.34.0
>
> /home/ # ovs-vsctl -V
>
> ovs-vsctl (Open vSwitch) 2.14.2
>
> DB Schema 8.2.0
>
>
>
> /home/ # uname -a
>
> Linux 5.3.18-150300.59.87-default SMP Thu Jul 21 14:31:28 UTC 2022
> (cc90276) x86_64 x86_64 x86_64 GNU/Linux
>
>
>
> this is SLES15 sp3 machine.
>
>
>
> pool1-bm25c5:~> nmstatectl --version
>
> 1.2.0
>
> pool1-bm25c5:~>
>
>
>
> ######
>
> when we reboot node or restart NetworkManager(systemctl restart NetworkManager) we
are loosing some of the ovs interfaces (randomly, not on all restarts).
>
I think this issue is related to this bug:
https://bugzilla.redhat.com/show_bug.cgi?id=2080236
It is a NetworkManager issue and it was fixed in NetworkManager-1.39.6, could you check
it out? Would you need an upstream backport to 1.34?
Thanks!
Fernando.
> logs are here:
>
> Missing interfaces:
>
>
https://protect2.fireeye.com/v1/url?k=31323334-501d5122-313273af-454
> 44
> 5555731-93530804397fe1ff&q=1&e=f9884072-555f-4847-9f8b-466f9b806b2a&
> u=
> https%3A%2F%2Fgist.github.com%2Fnmathew%2Fd7d472b938f389c4efbadd01ef
> 78
> d466%23file-missing-interfaces
>
> NetworkManager trace log for restart duration
>
>
https://protect2.fireeye.com/v1/url?k=31323334-501d5122-313273af-454
> 44
> 5555731-cfdd31dcdea77648&q=1&e=f9884072-555f-4847-9f8b-466f9b806b2a&
> u=
> https%3A%2F%2Fgist.github.com%2Fnmathew%2Fd7d472b938f389c4efbadd01ef
> 78 d466%23file-networkmanager-logs-with-trace
>
> /etc/NetworkManager/system-connections
>
>
https://protect2.fireeye.com/v1/url?k=31323334-501d5122-313273af-454
> 44
> 5555731-3474c8a3dfe835ee&q=1&e=f9884072-555f-4847-9f8b-466f9b806b2a&
> u=
> https%3A%2F%2Fgist.github.com%2Fnmathew%2Fd7d472b938f389c4efbadd01ef
> 78
> d466%23file-connection-conf
>
> ovsdb-client-dump for non working case(or when interfaces are
> missing)
>
>
https://protect2.fireeye.com/v1/url?k=31323334-501d5122-313273af-454
> 44
> 5555731-289bfcbe422b0355&q=1&e=f9884072-555f-4847-9f8b-466f9b806b2a&
> u=
> https%3A%2F%2Fgist.github.com%2Fnmathew%2Fd7d472b938f389c4efbadd01ef
> 78 d466%23file-non-working-case-ovsdb-client-dump
>
> ovsdb-client-dump for working case(or when all interfaces are
> present)
>
>
https://protect2.fireeye.com/v1/url?k=31323334-501d5122-313273af-454
> 44
> 5555731-b1efb31f6496469e&q=1&e=f9884072-555f-4847-9f8b-466f9b806b2a&
> u=
> https%3A%2F%2Fgist.github.com%2Fnmathew%2Fd7d472b938f389c4efbadd01ef
> 78 d466%23file-on-working-ovsdb-client-dump
>
> ovs-vsctl-show when interfaces are missing
>
>
https://protect2.fireeye.com/v1/url?k=31323334-501d5122-313273af-454
> 44
> 5555731-75a8fd4d20fe10f6&q=1&e=f9884072-555f-4847-9f8b-466f9b806b2a&
> u=
> https%3A%2F%2Fgist.github.com%2Fnmathew%2Fd7d472b938f389c4efbadd01ef
> 78
> d466%23file-ovs-vsctl-show
>
> ovs-vswitchd-log for the restart duration
>
>
https://protect2.fireeye.com/v1/url?k=31323334-501d5122-313273af-454
> 44
> 5555731-8c324322ae3a8e8f&q=1&e=f9884072-555f-4847-9f8b-466f9b806b2a&
> u=
> https%3A%2F%2Fgist.github.com%2Fnmathew%2Fd7d472b938f389c4efbadd01ef
> 78
> d466%23file-ovs-vswitchd-log
>
> ovsdb-tool-show-log for the restart duration
>
>
https://protect2.fireeye.com/v1/url?k=31323334-501d5122-313273af-454
> 44
> 5555731-c5b663cb0344f371&q=1&e=f9884072-555f-4847-9f8b-466f9b806b2a&
> u=
> https%3A%2F%2Fgist.github.com%2Fnmathew%2Fd7d472b938f389c4efbadd01ef
> 78 d466%23file-sudo-ovsdb-tool-mmmmmm-show-log
>
> In this particular instance of run, we lost two interfaces.
>
> we have a script which sets "autoconnect-slaves" to 0, which was
introduced for an older version of NM. grep -lF -e type=ovs-bridge -e type=ovs-interface
-e type=ovs-port /etc/NetworkManager/system-connections/*|xargs -I % nmcli connection
modify filename % connection.autoconnect-slaves 0 this scripts also runs currently, that
why all "autoconnect-slaves" set to zero.
>
> when we are losing interfaces that port additions are not coming to ovsdb:
>
> 2022-08-26T22:38:26.176Z|00179|bridge|INFO|bridge data_bridge: added
> interface traffic5_iface on port 1
>
> 2022-08-26T22:38:26.348Z|00182|bridge|INFO|bridge data_bridge: added
> interface traffic1_iface on port 2
>
> 2022-08-26T22:38:26.376Z|00183|bridge|INFO|bridge data_bridge: added
> interface bm25c_ext_iface on port 3
>
> 2022-08-26T22:38:26.376Z|00184|bridge|INFO|bridge data_bridge: added
> interface bond0 on port 4
>
> Only 4 interfaces are getting added back, two are missing, "ovsdb-tool -mmmmmm
show-log " shows the same.
>
> Any pointers where it is going wrong?
>
> -Nobin
>
>
>
> _______________________________________________
> nmstate-devel mailing list -- nmstate-devel(a)lists.fedorahosted.org
> To unsubscribe send an email to
> nmstate-devel-leave(a)lists.fedorahosted.org
> Fedora Code of Conduct:
>
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines:
>
https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
>
https://protect2.fireeye.com/v1/url?k=31323334-501d5122-313273af-454
> 44
> 5555731-09b7aeb775896a89&q=1&e=f9884072-555f-4847-9f8b-466f9b806b2a&
> u=
> https%3A%2F%2Flists.fedorahosted.org%2Farchives%2Flist%2Fnmstate-dev
> el
>
https://protect2.fireeye.com/v1/url?k=31323334-501d5122-313273af-45444555...
Do not reply to spam, report it:
>
https://protect2.fireeye.com/v1/url?k=31323334-501d5122-313273af-454
> 44
> 5555731-1df4bd1e5f7cb906&q=1&e=f9884072-555f-4847-9f8b-466f9b806b2a&
> u= https%3A%2F%2Fpagure.io%2Ffedora-infrastructure%2Fnew_issue