After the latest update there are some errors in the log that I haven't seen before.
kernel: xs_tcp_setup_socket: connect returned unhandled error -107
There are two of those per minute continuously. I haven't noticed any other problems with the system, but wondering about those messages are about.
/Jerome
I'm running 5.10.15-200.fc33.x86_64 and have not seen those errors.
On Wed, Feb 17, 2021 at 9:34 AM Jerome Lille jerome.lille@ownbay.net wrote:
After the latest update there are some errors in the log that I haven't seen before.
kernel: xs_tcp_setup_socket: connect returned unhandled error -107
There are two of those per minute continuously. I haven't noticed any other problems with the system, but wondering about those messages are about.
/Jerome _______________________________________________ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
On Wed, Feb 17, 2021 at 10:33 AM Jerome Lille jerome.lille@ownbay.net wrote:
After the latest update there are some errors in the log that I haven't seen before.
kernel: xs_tcp_setup_socket: connect returned unhandled error -107
There are two of those per minute continuously. I haven't noticed any other problems with the system, but wondering about those messages are about.
Error 107 is ENOTCONN (transport endpoint is not connected). See /usr/include/asm-generic/errno.h for error numbers.
The xs_tcp_setup_socket function is part of the RPC code in the kernel. Something on your system is trying to make an RPC call once every 30 seconds, but is failing to connect to the server. If you enable debug-level logging, you should see a log message showing what address it is trying to connect to. See if this does the trick:
$ echo 0xffff > /proc/sys/sunrpc/rpc_debug [Wait for the log messages to appear] $ echo 0 > /proc/sys/sunrpc/rpc_debug
Disclaimer: I haven't actually tried the above, nor do I see these messages on my system.
On Wed, 2021-02-17 at 11:43 -0700, Jerry James wrote:
On Wed, Feb 17, 2021 at 10:33 AM Jerome Lille jerome.lille@ownbay.net wrote:
After the latest update there are some errors in the log that I haven't seen before.
kernel: xs_tcp_setup_socket: connect returned unhandled error -107
There are two of those per minute continuously. I haven't noticed any other problems with the system, but wondering about those messages are about.
Error 107 is ENOTCONN (transport endpoint is not connected). See /usr/include/asm-generic/errno.h for error numbers.
The xs_tcp_setup_socket function is part of the RPC code in the kernel. Something on your system is trying to make an RPC call once every 30 seconds, but is failing to connect to the server. If you enable debug-level logging, you should see a log message showing what address it is trying to connect to. See if this does the trick:
$ echo 0xffff > /proc/sys/sunrpc/rpc_debug [Wait for the log messages to appear] $ echo 0 > /proc/sys/sunrpc/rpc_debug
Thanks for the suggestion. I did get more logging
RPC: 000000005d78dc3a connect status 107 connected 0 sock state 7 xs_tcp_setup_socket: connect returned unhandled error -107 RPC: xs_tcp_state_change client 000000005d78dc3a... RPC: state 7 conn 0 dead 0 zapped 1 sk_shutdown 3 RPC: set up xprt to 192.168.1.101 (port 111) via tcp RPC: xs_connect scheduled xprt 0000000064ac8be5 RPC: worker connecting xprt 0000000064ac8be5 via tcp to 192.168.1.101 (port 111) RPC: 0000000064ac8be5 connect status 115 connected 0 sock state 2 RPC: xs_tcp_state_change client 0000000064ac8be5... RPC: state 1 conn 0 dead 0 zapped 1 sk_shutdown 0 RPC: xs_tcp_send_request(80) = 0 RPC: xs_data_ready... RPC: setting port for xprt 000000005d78dc3a to 44982 RPC: xs_connect scheduled xprt 000000005d78dc3a RPC: destroy backchannel transport RPC: backchannel list empty= true RPC: xs_bind 0.0.0.0:902: ok (0) RPC: xs_destroy xprt 0000000064ac8be5 RPC: xs_close xprt 0000000064ac8be5
192.168.1.101 is the IP of my local NFS server (Centos7), my Fedora desktop that produces these error messages has some shares from it mounted. No errors on the NFS server and the nfs shares seems to work fine.
/Jerome
On Wed, Feb 17, 2021 at 12:08 PM Jerome Lille jerome.lille@ownbay.net wrote:
Thanks for the suggestion. I did get more logging
RPC: 000000005d78dc3a connect status 107 connected 0 sock state 7 xs_tcp_setup_socket: connect returned unhandled error -107 RPC: xs_tcp_state_change client 000000005d78dc3a... RPC: state 7 conn 0 dead 0 zapped 1 sk_shutdown 3 RPC: set up xprt to 192.168.1.101 (port 111) via tcp RPC: xs_connect scheduled xprt 0000000064ac8be5 RPC: worker connecting xprt 0000000064ac8be5 via tcp to 192.168.1.101 (port 111) RPC: 0000000064ac8be5 connect status 115 connected 0 sock state 2 RPC: xs_tcp_state_change client 0000000064ac8be5... RPC: state 1 conn 0 dead 0 zapped 1 sk_shutdown 0 RPC: xs_tcp_send_request(80) = 0 RPC: xs_data_ready... RPC: setting port for xprt 000000005d78dc3a to 44982 RPC: xs_connect scheduled xprt 000000005d78dc3a RPC: destroy backchannel transport RPC: backchannel list empty= true RPC: xs_bind 0.0.0.0:902: ok (0) RPC: xs_destroy xprt 0000000064ac8be5 RPC: xs_close xprt 0000000064ac8be5
192.168.1.101 is the IP of my local NFS server (Centos7), my Fedora desktop that produces these error messages has some shares from it mounted. No errors on the NFS server and the nfs shares seems to work fine.
Hmm. Somebody else will have to step in here. My knowledge of NFS is about 15 years out of date. :-)
On 18/02/2021 03:07, Jerome Lille wrote:
192.168.1.101 is the IP of my local NFS server (Centos7), my Fedora desktop that produces these error messages has some shares from it mounted. No errors on the NFS server and the nfs shares seems to work fine.
Are you using NFSv3 or NFSv4 to mount the shares?
On Thu, 2021-02-18 at 07:43 +0800, Ed Greshko wrote:
On 18/02/2021 03:07, Jerome Lille wrote:
192.168.1.101 is the IP of my local NFS server (Centos7), my Fedora desktop that produces these error messages has some shares from it mounted. No errors on the NFS server and the nfs shares seems to work fine.
Are you using NFSv3 or NFSv4 to mount the shares?
It is version 3.
Also I now notices that after a reboot there was this message in the logs
kernel: lockd: server 192.168.1.101 not responding, still trying
right before the first message of this kind
kernel: xs_tcp_setup_socket: connect returned unhandled error -107
But as I said the nfs shares seem to work fine, I can read and write to files on them.
/Jerome
On 19/02/2021 00:38, Jerome Lille wrote:
On Thu, 2021-02-18 at 07:43 +0800, Ed Greshko wrote:
On 18/02/2021 03:07, Jerome Lille wrote:
192.168.1.101 is the IP of my local NFS server (Centos7), my Fedora desktop that produces these error messages has some shares from it mounted. No errors on the NFS server and the nfs shares seems to work fine.
Are you using NFSv3 or NFSv4 to mount the shares?
It is version 3.
Also I now notices that after a reboot there was this message in the logs
kernel: lockd: server 192.168.1.101 not responding, still trying
right before the first message of this kind
kernel: xs_tcp_setup_socket: connect returned unhandled error -107
But as I said the nfs shares seem to work fine, I can read and write to files on them.
Right.
Any reason for not switching to NFSv4 where rpc isn't used?
On 2/18/21 8:38 AM, Jerome Lille wrote:
But as I said the nfs shares seem to work fine, I can read and write to files on them.
Yep, everything will look fine until an application on your client tries to lock a file, and then it will hang forever. Locking files during editing is pretty common, so I'm surprised you haven't seen problems yet.
You should do one of:
1: Update the client fstab and use NFSv4 instead of v3. In most cases, this will be the simplest and best fix.
2: Update the server firewall rules to allow all traffic from clients. Not recommended.
3: Update /etc/sysconfig/nfs and configure the system to used fixed ports for NFS services. Update your firewall rules to allow clients to reach those ports. More work than #2, but maintains security. This guide should help:
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/htm...
On Thu, 2021-02-18 at 14:23 -0800, Gordon Messmer wrote:
On 2/18/21 8:38 AM, Jerome Lille wrote:
But as I said the nfs shares seem to work fine, I can read and write to files on them.
Yep, everything will look fine until an application on your client tries to lock a file, and then it will hang forever. Locking files during editing is pretty common, so I'm surprised you haven't seen problems yet.
You should do one of:
1: Update the client fstab and use NFSv4 instead of v3. In most cases, this will be the simplest and best fix.
Thanks, for the suggestion.
I tried to simply change type from nfs to nfs4 in the fstab in the client. Unfortunately I then get the response
mount.nfs4: access denied by server while mounting ...
There is nothing in the server logs about this. And it is not the firewall, because even when I turned it off, I get the same access denied.
Anything else I have to change?
My exportfs is very simple, just /path/to/export client(rw)
/Jerome
On 22/02/2021 00:46, Jerome Lille wrote:
I tried to simply change type from nfs to nfs4 in the fstab in the client. Unfortunately I then get the response
mount.nfs4: access denied by server while mounting ...
There is nothing in the server logs about this. And it is not the firewall, because even when I turned it off, I get the same access denied.
Anything else I have to change?
My exportfs is very simple, just /path/to/export client(rw)
Well, it should just "work". I have a centos7 nfs server and using F33 as the client. In this example, the F33 client doesn't have an fstab entry for the mount.
On the server side....
[root@cos7 ~]# cat /etc/exports /home/egreshko 2001:b030:112f:2::0/64(rw,sync,insecure,no_root_squash,no_subtree_check) /home/egreshko 192.168.0.0/16(rw,sync,insecure,no_root_squash,no_subtree_check)
[root@cos7 ~]# firewall-cmd --info-zone=public public (active) target: default icmp-block-inversion: no interfaces: eth0 sources: services: dhcpv6-client mountd nfs nfs3 rpc-bind ssh ports: protocols: masquerade: no forward-ports: source-ports: icmp-blocks: rich rules:
Then on the client side....
[egreshko@meimei ~]$ sudo mount 192.168.122.5:/home/egreshko /mnt [egreshko@meimei ~]$ df -T | grep 122 192.168.122.5:/home/egreshko nfs4 29599744 5197056 24402688 18% /mnt
I have not made any changes to the nfs server configuration file /etc/nfs.conf so the defaults are in effect.
# vers2=n # vers3=y # vers4=y # vers4.0=y # vers4.1=y # vers4.2=y
Check to make sure these haven't been altered.
On 22/02/2021 08:24, Ed Greshko wrote:
have not made any changes to the nfs server configuration file /etc/nfs.conf so the defaults are in effect.
# vers2=n # vers3=y # vers4=y # vers4.0=y # vers4.1=y # vers4.2=y
Check to make sure these haven't been altered.
FWIW, I did do a test and modified that file to have....
vers4=n vers4.0=n vers4.1=n vers4.2=n
I restarted the nfs server and tried mounting again....
[egreshko@meimei ~]$ sudo mount.nfs4 192.168.122.5:/home/egreshko /mnt mount.nfs4: access denied by server while mounting 192.168.122.5:/home/egreshko
On Mon, 2021-02-22 at 08:39 +0800, Ed Greshko wrote:
On 22/02/2021 08:24, Ed Greshko wrote:
have not made any changes to the nfs server configuration file /etc/ nfs.conf so the defaults are in effect.
# vers2=n # vers3=y # vers4=y # vers4.0=y # vers4.1=y # vers4.2=y
Check to make sure these haven't been altered.
FWIW, I did do a test and modified that file to have....
vers4=n vers4.0=n vers4.1=n vers4.2=n
I restarted the nfs server and tried mounting again....
[egreshko@meimei ~]$ sudo mount.nfs4 192.168.122.5:/home/egreshko /mnt mount.nfs4: access denied by server while mounting 192.168.122.5:/home/egreshko
This is really puzzling. I've tried to turn off both Selinux and firewall on both server and client. Didn't make any difference. My nfs.conf is untouched.
When I do the following command mount -v server:/path /path
I get the following
mount.nfs: timeout set for Mon Feb 22 16:07:08 2021 mount.nfs: trying text-based options 'vers=4.2,addr=192.168.1.101,clientaddr=192.168.1.103' mount.nfs: mount(2): Permission denied mount.nfs: trying text-based options 'vers=4,minorversion=1,addr=192.168.1.101,clientaddr=192.168.1.103' mount.nfs: mount(2): Permission denied mount.nfs: trying text-based options 'vers=4,addr=192.168.1.101,clientaddr=192.168.1.103' mount.nfs: mount(2): Permission denied mount.nfs: trying text-based options 'addr=192.168.1.101' mount.nfs: prog 100003, trying vers=3, prot=6 mount.nfs: trying 192.168.1.101 prog 100003 vers 3 prot TCP port 2049 mount.nfs: prog 100005, trying vers=3, prot=17 mount.nfs: trying 192.168.1.101 prog 100005 vers 3 prot UDP port 20048
So it succeeds in mounting on version 3 after failed with the same on version 4.
On 23/02/2021 00:27, Jerome Lille wrote:
On Mon, 2021-02-22 at 08:39 +0800, Ed Greshko wrote:
On 22/02/2021 08:24, Ed Greshko wrote:
have not made any changes to the nfs server configuration file /etc/ nfs.conf so the defaults are in effect.
# vers2=n # vers3=y # vers4=y # vers4.0=y # vers4.1=y # vers4.2=y
Check to make sure these haven't been altered.
FWIW, I did do a test and modified that file to have....
vers4=n vers4.0=n vers4.1=n vers4.2=n
I restarted the nfs server and tried mounting again....
[egreshko@meimei ~]$ sudo mount.nfs4 192.168.122.5:/home/egreshko /mnt mount.nfs4: access denied by server while mounting 192.168.122.5:/home/egreshko
This is really puzzling. I've tried to turn off both Selinux and firewall on both server and client. Didn't make any difference. My nfs.conf is untouched.
You do mean that file on the nfs server side, right?
When I do the following command mount -v server:/path /path
I get the following
I can only get those errors if I disable nfsv4 on the server side.
On the server, what is the output of
systemctl status nfs-server
On Tue, 2021-02-23 at 04:37 +0800, Ed Greshko wrote:
On 23/02/2021 00:27, Jerome Lille wrote:
My nfs.conf is untouched.
You do mean that file on the nfs server side, right?
Exactly
When I do the following command mount -v server:/path /path
I get the following
I can only get those errors if I disable nfsv4 on the server side.
On the server, what is the output of
systemctl status nfs-server
nfs-server.service - NFS server and services Loaded: loaded (/usr/lib/systemd/system/nfs-server.service; enabled; vendor preset: disabled) Drop-In: /run/systemd/generator/nfs-server.service.d └─order-with-mounts.conf Active: active (exited) since Tue 2021-02-23 13:44:19 CET; 1h 9min ago Process: 17823 ExecStopPost=/usr/sbin/exportfs -f (code=exited, status=0/SUCCESS) Process: 17820 ExecStopPost=/usr/sbin/exportfs -au (code=exited, status=0/SUCCESS) Process: 17815 ExecStop=/usr/sbin/rpc.nfsd 0 (code=exited, status=0/SUCCESS) Process: 17854 ExecStartPost=/bin/sh -c if systemctl -q is-active gssproxy; then systemctl reload gssproxy ; fi (code=exited, status=0/SUCCESS) Process: 17837 ExecStart=/usr/sbin/rpc.nfsd $RPCNFSDARGS (code=exited, status=0/SUCCESS) Process: 17835 ExecStartPre=/usr/sbin/exportfs -r (code=exited, status=0/SUCCESS) Main PID: 17837 (code=exited, status=0/SUCCESS) CGroup: /system.slice/nfs-server.service
systemd[1]: Starting NFS server and services... systemd[1]: Started NFS server and services.
/Jerome
On 23/02/2021 21:58, Jerome Lille wrote:
On Tue, 2021-02-23 at 04:37 +0800, Ed Greshko wrote:
On 23/02/2021 00:27, Jerome Lille wrote:
My nfs.conf is untouched.
You do mean that file on the nfs server side, right?
Exactly
OK....
Modify that file to have....
# [mountd] debug=all
And then restart the server.
After doing the mount from the client, check the journal for rpc.mountd messages.
On Tue, 2021-02-23 at 23:16 +0800, Ed Greshko wrote:
[mountd] debug=all
And then restart the server.
After doing the mount from the client, check the journal for rpc.mountd messages.
Ok, what I got when doing 'mount -v server:/path /path' (i.e. it tried v4 first then succeeded on v3) was
rpc.mountd[20558]: auth_unix_ip: inbuf 'nfsd 192.168.1.103' rpc.mountd[20558]: v4root_create: path '/' flags 0x12405 rpc.mountd[20558]: v4root_create: path '/mnt' flags 0x10405 rpc.mountd[20558]: v4root_create: path '/mnt/raid' flags 0x10405 rpc.mountd[20558]: v4root_create: path '/' flags 0x12405 rpc.mountd[20558]: v4root_create: path '/mnt' flags 0x10405 rpc.mountd[20558]: v4root_create: path '/mnt/raid' flags 0x10405 rpc.mountd[20558]: auth_unix_ip: client 0x55a049adcdb0 'desktop' rpc.mountd[20558]: nfsd_fh: inbuf 'desktop 1 \x00000000' rpc.mountd[20558]: nfsd_fh: found 0x55a049ade800 path / rpc.mountd[20558]: nfsd_export: inbuf 'desktop /mnt' rpc.mountd[20558]: nfsd_export: found 0x55a049aded00 path /mnt rpc.mountd[20558]: nfsd_fh: inbuf 'desktop 7 \xa167010400000000a30d293e41ca4977a19736ba24947eb6' rpc.mountd[20558]: nfsd_fh: found 0x55a049aded10 path /mnt rpc.mountd[20558]: nfsd_export: inbuf 'desktop /mnt/raid' rpc.mountd[20558]: nfsd_export: found 0x55a049adf210 path /mnt/raid rpc.mountd[20558]: nfsd_fh: inbuf 'desktop 6 \x0fe5fb05c7864830a8a9dbe250c2f2ed' rpc.mountd[20558]: nfsd_fh: found 0x55a049adf220 path /mnt/raid rpc.mountd[20558]: from_local: updating local if addr list rpc.mountd[20558]: from_local: checked 7 local if addrs; incoming address not found rpc.mountd[20558]: check_default: access by 192.168.1.103 ALLOWED rpc.mountd[20558]: Received NULL request from 192.168.1.103 rpc.mountd[20558]: check_default: access by 192.168.1.103 ALLOWED (cached) rpc.mountd[20558]: Received NULL request from 192.168.1.103 rpc.mountd[20558]: check_default: access by 192.168.1.103 ALLOWED (cached) rpc.mountd[20558]: Received MNT3(/mnt/raid/media) request from 192.168.1.103 rpc.mountd[20558]: authenticated mount request from 192.168.1.103:774 for /mnt/raid/media (/mnt/raid/media) rpc.mountd[20558]: nfsd_fh: inbuf 'desktop 7 \x01005610000000000fe5fb05c7864830a8a9dbe250c2f2ed' rpc.mountd[20558]: nfsd_fh: found 0x55a049addd80 path /mnt/raid/media
Could it be that v4 differs from v3 in how it does IP <-> hostname resolution? On the server I have the IP of the client/desktop in the /etc/hosts file and the /etc/exports has only the hostname
/Jerome
On 24/02/2021 00:27, Jerome Lille wrote:
On Tue, 2021-02-23 at 23:16 +0800, Ed Greshko wrote:
[mountd] debug=all
And then restart the server.
After doing the mount from the client, check the journal for rpc.mountd messages.
Ok, what I got when doing 'mount -v server:/path /path' (i.e. it tried v4 first then succeeded on v3) was
rpc.mountd[20558]: check_default: access by 192.168.1.103 ALLOWED rpc.mountd[20558]: Received NULL request from 192.168.1.103 rpc.mountd[20558]: check_default: access by 192.168.1.103 ALLOWED (cached) rpc.mountd[20558]: Received NULL request from 192.168.1.103 rpc.mountd[20558]: check_default: access by 192.168.1.103 ALLOWED (cached) rpc.mountd[20558]: Received MNT3(/mnt/raid/media) request from 192.168.1.103 rpc.mountd[20558]: authenticated mount request from 192.168.1.103:774 for /mnt/raid/media (/mnt/raid/media) rpc.mountd[20558]: nfsd_fh: inbuf 'desktop 7 \x01005610000000000fe5fb05c7864830a8a9dbe250c2f2ed' rpc.mountd[20558]: nfsd_fh: found 0x55a049addd80 path /mnt/raid/media
That is the output I get when I disable NFSv4 in /etc/nfs.conf.
The other way to disable NFSv4 is to modify /etc/sysconfig/nfs to have the line
# Optional arguments passed to rpc.nfsd. See rpc.nfsd(8) RPCNFSDARGS="--no-nfs-version 4"
You should check that as well.
Bottom line, NFSv4 is set disabled. Those are the 2 ways I know how to do. Maybe there are more?
Could it be that v4 differs from v3 in how it does IP <-> hostname resolution? On the server I have the IP of the client/desktop in the /etc/hosts file and the /etc/exports has only the hostname
No. You see "access by 192.168.1.103 ALLOWE" in all attempts.
On Wed, 2021-02-24 at 02:38 +0800, Ed Greshko wrote:
The other way to disable NFSv4 is to modify /etc/sysconfig/nfs to have the line
# Optional arguments passed to rpc.nfsd. See rpc.nfsd(8) RPCNFSDARGS="--no-nfs-version 4"
You should check that as well.
Bottom line, NFSv4 is set disabled. Those are the 2 ways I know how to do. Maybe there are more?
I checked /etc/sysconfig/nfs and it hadn't any options in RPCNFSDARGS.
I also checked what would happen if I only had version 4, disallowing version 3 on server side. The error message that the client got was different. For version 4 it was "mount(2): Permission denied" and for version 3 it was "portmap query failed: RPC: Program/version mismatch".
I also noticed a log message that comes each time I restart the nfs server, that I have no idea what it is about
polkitd[1513]: Unregistered Authentication Agent for unix- process:23099:167829226 (system bus name :1.6793, object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale en_GB.UTF-8) (disconnected from bus)
Although there was no process with PID 23099 when I looked.
/Jerome
On 24/02/2021 04:41, Jerome Lille wrote:
On Wed, 2021-02-24 at 02:38 +0800, Ed Greshko wrote:
The other way to disable NFSv4 is to modify /etc/sysconfig/nfs to have the line
# Optional arguments passed to rpc.nfsd. See rpc.nfsd(8) RPCNFSDARGS="--no-nfs-version 4"
You should check that as well.
Bottom line, NFSv4 is set disabled. Those are the 2 ways I know how to do. Maybe there are more?
I checked /etc/sysconfig/nfs and it hadn't any options in RPCNFSDARGS.
I also checked what would happen if I only had version 4, disallowing version 3 on server side. The error message that the client got was different. For version 4 it was "mount(2): Permission denied" and for version 3 it was "portmap query failed: RPC: Program/version mismatch".
I also noticed a log message that comes each time I restart the nfs server, that I have no idea what it is about
polkitd[1513]: Unregistered Authentication Agent for unix- process:23099:167829226 (system bus name :1.6793, object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale en_GB.UTF-8) (disconnected from bus)
Although there was no process with PID 23099 when I looked.
That polkitd message occurs here as well. So, not an issue.
What is the output of
cat /run/sysconfig/nfs-utils
On 24/02/2021 05:41, Ed Greshko wrote:
That polkitd message occurs here as well. So, not an issue.
What is the output of
cat /run/sysconfig/nfs-util
Oh, also, do you happen to have a /etc/nfs.conf.d directory?
On Wed, 2021-02-24 at 05:41 +0800, Ed Greshko wrote:
What is the output of
cat /run/sysconfig/nfs-utils
RPCNFSDARGS=" " RPCMOUNTDARGS="" STATDARGS="" SMNOTIFYARGS="" RPCIDMAPDARGS="" GSSDARGS="" BLKMAPDARGS="" GSS_USE_PROXY="yes"
Oh, also, do you happen to have a /etc/nfs.conf.d directory?
No such directory.
I also tested to explicitly disabling version 4 in /etc/nfs.conf, i.e. vers4=n In that case I got a different error message, I got "mount(2): Protocol not supported" in the client. Previously I had gotten "mount(2): Permission denied"
/Jerome
On 24/02/2021 22:22, Jerome Lille wrote:
On Wed, 2021-02-24 at 05:41 +0800, Ed Greshko wrote:
What is the output of
cat /run/sysconfig/nfs-utils
RPCNFSDARGS=" " RPCMOUNTDARGS="" STATDARGS="" SMNOTIFYARGS="" RPCIDMAPDARGS="" GSSDARGS="" BLKMAPDARGS="" GSS_USE_PROXY="yes"
Oh, also, do you happen to have a /etc/nfs.conf.d directory?
No such directory.
I also tested to explicitly disabling version 4 in /etc/nfs.conf, i.e. vers4=n In that case I got a different error message, I got "mount(2): Protocol not supported" in the client. Previously I had gotten "mount(2): Permission denied"
Well, I'm sorry but I'm out of ideas.
I can mount file systems hosted on a Centos7 server from an F33 client with no problems with nfsv4. I can't figure out how to create the problems you're seeing without modification to configuration options.
My only suggestion now would be if you have another F33 system you can use to mount and test to make sure it isn't a client issue.
On Thu, 2021-02-25 at 07:40 +0800, Ed Greshko wrote:
Well, I'm sorry but I'm out of ideas.
I can mount file systems hosted on a Centos7 server from an F33 client with no problems with nfsv4. I can't figure out how to create the problems you're seeing without modification to configuration options.
My only suggestion now would be if you have another F33 system you can use to mount and test to make sure it isn't a client issue.
Thanks anyway for all the suggestions.
I did try to mount the nfs share from a different F33 system and the result was the same.
/Jerome
On Sun, 21 Feb 2021 at 12:46, Jerome Lille jerome.lille@ownbay.net wrote:
I tried to simply change type from nfs to nfs4 in the fstab in the client. Unfortunately I then get the response
mount.nfs4: access denied by server while mounting ...
There is nothing in the server logs about this. And it is not the firewall, because even when I turned it off, I get the same access denied.
Anything else I have to change?
My exportfs is very simple, just /path/to/export client(rw)
I haven't used CentOS7 for several years, so I was ignoring this thread in the hope others could answer.
My not very trustworthy memory is that "/etc/exports" on the server needs the "insecure" option for NFS4 to work, see: https://access.redhat.com/solutions/3773891 for explanation and some diagnostic tests. My take on diagnostics is:
1) find the ports used for nfs4 tcp:
rpcinfo -p <server_IP> | awk '/ 4 /' 100000 4 tcp 111 portmapper 100000 4 udp 111 portmapper 100003 4 tcp 2049 nfs 100021 4 udp 35759 nlockmgr 100021 4 tcp 38583 nlockmgr
2) use telnet on the client to verify that you can connect to ports 111 and 2049 on the server:
% telnet <server_IP> 111 Trying <server_IP> ... Connected to <server_IP>. Escape character is '^]'. ^] telnet> quit Connection closed. % telnet <server_IP> 2049 Trying <server_IP>... Connected to <server_IP>. Escape character is '^]'. ^] telnet> quit Connection closed.
3. verify the exported filesystems:
% showmount -e <server_IP> Export list for <server_IP>: /path/to/export client
Fedora 33 doesn't appear to need the "insecure" option. For CentOS7, see https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/htm... .
On 26/02/2021 01:19, George N. White III wrote:
I haven't used CentOS7 for several years, so I was ignoring this thread in the hope others could answer.
My not very trustworthy memory is that "/etc/exports" on the server needs the "insecure" option for NFS4 to work, see: https://access.redhat.com/solutions/3773891 https://access.redhat.com/solutions/3773891 for explanation and some diagnostic tests. My take on diagnostics is:
FWIW, "others" have tried changing
/home/egreshko 2001:b030:112f:2::0/64(rw,sync,insecure,no_root_squash,no_subtree_check)
to
/home/egreshko 2001:b030:112f:2::0/64(rw,sync,no_root_squash,no_subtree_check)
And the mount from an F33 system still succeeds.
[root@meimei ~]# mount -v cos7:/home/egreshko /mnt mount.nfs: timeout set for Fri Feb 26 01:59:49 2021 mount.nfs: trying text-based options 'vers=4.2,addr=2001:b030:112f:2::41,clientaddr=2001:b030:112f:2::2'
[root@meimei ~]# df -T | grep mnt cos7:/home/egreshko nfs4 29599744 5324544 24275200 18% /mnt
So, I suppose it could be another exports option. But, at 02:00 I'm not inclined to give it a go. :-) And, FWIW, the centos7 system is fully updated...
[egreshko@cos7 ~]$ sudo yum update . . No packages marked for update
On Thu, 25 Feb 2021 at 14:03, Ed Greshko ed.greshko@greshko.com wrote:
On 26/02/2021 01:19, George N. White III wrote:
I haven't used CentOS7 for several years, so I was ignoring this thread in the hope others could answer.
My not very trustworthy memory is that "/etc/exports" on the server needs the "insecure" option for NFS4 to work, see: https://access.redhat.com/solutions/3773891 <
https://access.redhat.com/solutions/3773891%3E for explanation
and some diagnostic tests. My take on diagnostics is:
FWIW, "others" have tried changing
/home/egreshko 2001:b030:112f:2::0/64(rw,sync,insecure,no_root_squash,no_subtree_check)
to
/home/egreshko 2001:b030:112f:2::0/64(rw,sync,no_root_squash,no_subtree_check)
And the mount from an F33 system still succeeds.
The entry for "man exports":
secure This option requires that requests not using gss originate on an Internet port less than IPPORT_RESERVED (1024). This option is on by default. To turn it off, specify insecure. (NOTE: older kernels (before upstream kernel version 4.17) enforced this re‐ quirement on gss requests as well.)
/run/sysconfig/nfs-utils had: GSS_USE_PROXY="yes" -- wonder if GSS is somehow at play here?
On 26/02/2021 04:21, George N. White III wrote:
On Thu, 25 Feb 2021 at 14:03, Ed Greshko <ed.greshko@greshko.com mailto:ed.greshko@greshko.com> wrote:
On 26/02/2021 01:19, George N. White III wrote: > I haven't used CentOS7 for several years, so I was ignoring this > thread in the hope others could answer. > > My not very trustworthy memory is that "/etc/exports" on > the server needs the "insecure" option for NFS4 to work, see: > https://access.redhat.com/solutions/3773891 <https://access.redhat.com/solutions/3773891> <https://access.redhat.com/solutions/3773891 <https://access.redhat.com/solutions/3773891>> for explanation > and some diagnostic tests. My take on diagnostics is: FWIW, "others" have tried changing /home/egreshko 2001:b030:112f:2::0/64(rw,sync,insecure,no_root_squash,no_subtree_check) to /home/egreshko 2001:b030:112f:2::0/64(rw,sync,no_root_squash,no_subtree_check) And the mount from an F33 system still succeeds.The entry for "man exports":
secure This option requires that requests not using gss originate on an Internet port less than IPPORT_RESERVED (1024). This option is on by default. To turn it off, specify insecure. (NOTE: older kernels (before upstream kernel version 4.17) enforced this re‐ quirement on gss requests as well.)
/run/sysconfig/nfs-utils had: GSS_USE_PROXY="yes" -- wonder if GSS is somehow at play here?
On my centos7 system....
[root@cos7 ~]# uname -r 3.10.0-1160.15.2.el7.x86_64
[root@cos7 ~]# cat /run/sysconfig/nfs-utils RPCNFSDARGS=" " RPCMOUNTDARGS="" STATDARGS="" SMNOTIFYARGS="" RPCIDMAPDARGS="" GSSDARGS="" BLKMAPDARGS="" GSS_USE_PROXY="yes"
nfs-utils-1.3.0-0.68.el7
And, since it is now 05:40, I modified the exports file to be simply
/home/egreshko 2001:b030:112f:2::0/64(rw)
And from an F33 client which is the host of the the centos7 VM.
[root@meimei ~]# mount -v cos7:/home/egreshko /mnt mount.nfs: timeout set for Fri Feb 26 05:45:37 2021 mount.nfs: trying text-based options 'vers=4.2,addr=2001:b030:112f:2::41,clientaddr=2001:b030:112f:2::2'
[root@meimei ~]# df -T | grep mnt cos7:/home/egreshko nfs4 29599744 5328128 24271616 19% /mnt
So, I still can't find a way to duplicate the OP's problem without specifically changing a config file.
I even when to far as to mount the file system with a F33 "located" in the USA and the centos7 system in Taiwan.
[root@acer ~]# mount -v [2001:b030:112f:2::41]:/home/egreshko /mnt mount.nfs: timeout set for Fri Feb 26 06:03:52 2021 mount.nfs: trying text-based options 'vers=4.2,addr=2001:b030:112f:2::41,clientaddr=2001:440:66:cce::2'
[root@acer ~]# df -T | grep mnt [2001:b030:112f:2::41]:/home/egreshko nfs4 29599744 5325056 24274688 18% /mnt
On Thu, 25 Feb 2021 at 18:11, Ed Greshko ed.greshko@greshko.com wrote:
On 26/02/2021 04:21, George N. White III wrote:
On Thu, 25 Feb 2021 at 14:03, Ed Greshko <ed.greshko@greshko.com
mailto:ed.greshko@greshko.com> wrote:
On 26/02/2021 01:19, George N. White III wrote: > I haven't used CentOS7 for several years, so I was ignoring this > thread in the hope others could answer. > > My not very trustworthy memory is that "/etc/exports" on > the server needs the "insecure" option for NFS4 to work, see: > https://access.redhat.com/solutions/3773891 <https://access.redhat.com/solutions/3773891%3E < https://access.redhat.com/solutions/3773891 < https://access.redhat.com/solutions/3773891%3E%3E for explanation
> and some diagnostic tests. My take on diagnostics is: FWIW, "others" have tried changing /home/egreshko2001:b030:112f:2::0/64(rw,sync,insecure,no_root_squash,no_subtree_check)
to /home/egreshko2001:b030:112f:2::0/64(rw,sync,no_root_squash,no_subtree_check)
And the mount from an F33 system still succeeds.The entry for "man exports":
secure This option requires that requests not using gss originateon an
Internet port less than IPPORT_RESERVED (1024). Thisoption is
on by default. To turn it off, specify insecure. (NOTE:older
kernels (before upstream kernel version 4.17) enforcedthis re‐
quirement on gss requests as well.)/run/sysconfig/nfs-utils had: GSS_USE_PROXY="yes" -- wonder if GSS is
somehow at play here?
On my centos7 system....
[root@cos7 ~]# uname -r 3.10.0-1160.15.2.el7.x86_64
[root@cos7 ~]# cat /run/sysconfig/nfs-utils RPCNFSDARGS=" " RPCMOUNTDARGS="" STATDARGS="" SMNOTIFYARGS="" RPCIDMAPDARGS="" GSSDARGS="" BLKMAPDARGS="" GSS_USE_PROXY="yes"
nfs-utils-1.3.0-0.68.el7
And, since it is now 05:40, I modified the exports file to be simply
/home/egreshko 2001:b030:112f:2::0/64(rw)
And from an F33 client which is the host of the the centos7 VM.
[root@meimei ~]# mount -v cos7:/home/egreshko /mnt mount.nfs: timeout set for Fri Feb 26 05:45:37 2021 mount.nfs: trying text-based options 'vers=4.2,addr=2001:b030:112f:2::41,clientaddr=2001:b030:112f:2::2'
[root@meimei ~]# df -T | grep mnt cos7:/home/egreshko nfs4 29599744 5328128 24271616 19% /mnt
So, I still can't find a way to duplicate the OP's problem without specifically changing a config file.
There is always the possibility of some "invisible character" error in a config file.
I even when to far as to mount the file system with a F33 "located" in the USA and the centos7 system in Taiwan.
[root@acer ~]# mount -v [2001:b030:112f:2::41]:/home/egreshko /mnt mount.nfs: timeout set for Fri Feb 26 06:03:52 2021 mount.nfs: trying text-based options 'vers=4.2,addr=2001:b030:112f:2::41,clientaddr=2001:440:66:cce::2'
[root@acer ~]# df -T | grep mnt [2001:b030:112f:2::41]:/home/egreshko nfs4 29599744 5325056 24274688 18% /mnt
Google turns up many ways to get "mount.nfs4: access denied by server while mounting". One that I don't recall being mentioned is mismatched user/group ID's with "sec=sys".
On 26/02/2021 07:38, George N. White III wrote:
So, I still can't find a way to duplicate the OP's problem without specifically changing a config file.There is always the possibility of some "invisible character" error in a config file.
The OP has said he hadn't modified any config files.
I even when to far as to mount the file system with a F33 "located" in the USA and the centos7 system in Taiwan. [root@acer ~]# mount -v [2001:b030:112f:2::41]:/home/egreshko /mnt mount.nfs: timeout set for Fri Feb 26 06:03:52 2021 mount.nfs: trying text-based options 'vers=4.2,addr=2001:b030:112f:2::41,clientaddr=2001:440:66:cce::2' [root@acer ~]# df -T | grep mnt [2001:b030:112f:2::41]:/home/egreshko nfs4 29599744 5325056 24274688 18% /mntGoogle turns up many ways to get "mount.nfs4: access denied by server while mounting". One that I don't recall being mentioned is mismatched user/group ID's with "sec=sys".
He also said in an earlier message
My exportfs is very simple, just /path/to/export client(rw)
Which would seem to rule that out. And, why I finally modified my exports to be that.
On Thu, 2021-02-25 at 13:19 -0400, George N. White III wrote:
On Sun, 21 Feb 2021 at 12:46, Jerome Lille jerome.lille@ownbay.net wrote:
I tried to simply change type from nfs to nfs4 in the fstab in the client. Unfortunately I then get the response
mount.nfs4: access denied by server while mounting ...
There is nothing in the server logs about this. And it is not the firewall, because even when I turned it off, I get the same access denied.
Anything else I have to change?
My exportfs is very simple, just /path/to/export client(rw)
I haven't used CentOS7 for several years, so I was ignoring this thread in the hope others could answer.
My not very trustworthy memory is that "/etc/exports" on the server needs the "insecure" option for NFS4 to work, see: https://access.redhat.com/solutions/3773891 for explanation and some diagnostic tests. My take on diagnostics is:
I modified the options from (rw) to (rw,sync,no_root_squash,insecure) and after that I finally could mount it in version 4. I've tested earlier to add the insecure option, but I'm not sure if I then remembered to restart nfs after the changes or not. Or if it is in the combination with the other options that made it possible.
Thanks for all the help /Jerome
On 26/02/2021 16:48, Jerome Lille wrote:
On Thu, 2021-02-25 at 13:19 -0400, George N. White III wrote:
On Sun, 21 Feb 2021 at 12:46, Jerome Lille jerome.lille@ownbay.net wrote:
I tried to simply change type from nfs to nfs4 in the fstab in the client. Unfortunately I then get the response
mount.nfs4: access denied by server while mounting ...
There is nothing in the server logs about this. And it is not the firewall, because even when I turned it off, I get the same access denied.
Anything else I have to change?
My exportfs is very simple, just /path/to/export client(rw)
I haven't used CentOS7 for several years, so I was ignoring this thread in the hope others could answer.
My not very trustworthy memory is that "/etc/exports" on the server needs the "insecure" option for NFS4 to work, see: https://access.redhat.com/solutions/3773891 for explanation and some diagnostic tests. My take on diagnostics is:
I modified the options from (rw) to (rw,sync,no_root_squash,insecure) and after that I finally could mount it in version 4. I've tested earlier to add the insecure option, but I'm not sure if I then remembered to restart nfs after the changes or not. Or if it is in the combination with the other options that made it possible.
Good that you now have it working.
Now, to go figure out why I can mount just fine when my exports file has
/home/egreshko 2001:b030:112f:2::0/64(rw)
"Logic" tells me it should fail like it was failing for you. :-) :-)
-
Clifford Snow -
Ed Greshko -
George N. White III -
Gordon Messmer -
Jerome Lille -
Jerry James -
Tom Horsley