Hello sssd-developers,
Allow me to introduce myself and the suggested contribution to the SSSD project. PrimaryData Inc. is a startup company in the field of distributed NAS. As such, we would like to contribute to the SSSD project by developing support for rpc.idmapd.
Below is our proposed design for this new feature.
Best regards, Noam Meltzer
---
*SSS NFS Client (rpc.idmapd plugin) - Design*
We named the client "sss_nfs" (althogh "sss_idmap" or "idmap" might have been better names, the term "idmap" is already occupied in the SSSD world).
*rpc.idmapd - background*
rpc.idmapd runs on NFSv4 servers as a userspace daemon (part of nfs-utils). It's role is to assist knfsd by providing the following 6 mapping functions:
1. (user) name to uid
2. (group) name to gid
3. uid to (user) name
4. gid to (group) name
5. principal (user) name to ids (uid + gid)
6. principal (user) name to grouplist (groups which user are member of)
rpc.idmapd provides API for developing plugins (loaded by dlopen(3)) which implements the actual mapping process.
* note: 5 + 6 are relevant only for kerberised NFSv4 servers. At the first stage we only purpose a design without kerberos support.
* note2: On the kernel level, there's a caching mechanism for the responses from the userspace daemon.
*SSSD - Responder*
The functionality required from the Responder side is a subset of the functionality provided by existing NSS Responder's commands.
As you can see below (on the client part of the design) - no changes are needed in the NSS Responder.
*SSSD - NFS Client*
*Responder-Facing Interactions (existing NSS Responder commands)*
SSS_NSS_GETPWNAM - map (user) name to uid requests SSS_NSS_GETGRNAM - map (group) name to gid requests SSS_NSS_GETPWUID - map uid to (user) name requests SSS_NSS_GETGRGID - map gid to (group) name requests
The request & reply sent to & from the responder is "standard" in terms of the NSS Responder.
The client only needs a portion of the reply. Only this portion will be extracted from the packet (i.e. uid/gid/user name/group name).
*Optimisation Techniques*
The optimisation techniques used for the NSS client will be used here as well. i.e. Fast Cache (memcache) & negative-cache.
It will be possible for the user to disable Fast Cache from the configuration file. (see below)
*Configuration File*
The configuration of the client will be part of rpc.idmap config file (/etc/idmapd.conf).
On Tue, Dec 17, 2013 at 07:40:29AM +0200, Noam Meltzer wrote:
Hello sssd-developers,
Allow me to introduce myself and the suggested contribution to the SSSD project. PrimaryData Inc. is a startup company in the field of distributed NAS. As such, we would like to contribute to the SSSD project by developing support for rpc.idmapd.
Below is our proposed design for this new feature.
Best regards, Noam Meltzer
Hi Noam,
thanks for this design, it's great to have outside contributions!
*SSS NFS Client (rpc.idmapd plugin) - Design*
We named the client "sss_nfs" (althogh "sss_idmap" or "idmap" might have been better names, the term "idmap" is already occupied in the SSSD world).
How does this client work, is it a module that the rpc.idmapd dlopens() ?
*rpc.idmapd - background*
rpc.idmapd runs on NFSv4 servers as a userspace daemon (part of nfs-utils). It's role is to assist knfsd by providing the following 6 mapping functions:
(user) name to uid
(group) name to gid
uid to (user) name
gid to (group) name
principal (user) name to ids (uid + gid)
principal (user) name to grouplist (groups which user are member of)
rpc.idmapd provides API for developing plugins (loaded by dlopen(3)) which implements the actual mapping process.
- note: 5 + 6 are relevant only for kerberised NFSv4 servers. At the first
stage we only purpose a design without kerberos support.
- note2: On the kernel level, there's a caching mechanism for the responses
from the userspace daemon.
*SSSD - Responder*
The functionality required from the Responder side is a subset of the functionality provided by existing NSS Responder's commands.
As you can see below (on the client part of the design) - no changes are needed in the NSS Responder.
*SSSD - NFS Client*
*Responder-Facing Interactions (existing NSS Responder commands)*
SSS_NSS_GETPWNAM - map (user) name to uid requests SSS_NSS_GETGRNAM - map (group) name to gid requests SSS_NSS_GETPWUID - map uid to (user) name requests SSS_NSS_GETGRGID - map gid to (group) name requests
The request & reply sent to & from the responder is "standard" in terms of the NSS Responder.
The client only needs a portion of the reply. Only this portion will be extracted from the packet (i.e. uid/gid/user name/group name).
Sure, this would work and as long as you see no performance issues, this approach is completely fine.
But please note that especially getgr* calls can be quite expensive, especially when nesting is involved. The reason is that normally the getgr* calls also download all the group members, resolving the nesting in the process.
But if you do want to optimize further (perhaps in the future), we could then add a new NSS call that would only download the group entry in order to retrieve the GID. I think the backend part of this change would be also usable for the PAC responder and we already talked about this change recently with Sumit.
But for version 1.0 I agree we can just use the existing NSS calls. A future version of the client can switch to another NSS call later, we can do that safely.
*Optimisation Techniques*
The optimisation techniques used for the NSS client will be used here as well. i.e. Fast Cache (memcache) & negative-cache.
It will be possible for the user to disable Fast Cache from the configuration file. (see below)
What problems do you envision with the Fast Cache? Or did yo mostly want to make sure kernel does its own caching and the SSSD stays out of the way?
*Configuration File*
The configuration of the client will be part of rpc.idmap config file (/etc/idmapd.conf).
I think the plan sounds great in general!
On 17 December 2013 11:55, Jakub Hrozek jhrozek@redhat.com wrote:
On Tue, Dec 17, 2013 at 07:40:29AM +0200, Noam Meltzer wrote:
Hello sssd-developers,
Allow me to introduce myself and the suggested contribution to the SSSD project. PrimaryData Inc. is a startup company in the field of
distributed
NAS. As such, we would like to contribute to the SSSD project by
developing
support for rpc.idmapd.
Below is our proposed design for this new feature.
Best regards, Noam Meltzer
Hi Noam,
thanks for this design, it's great to have outside contributions!
*SSS NFS Client (rpc.idmapd plugin) - Design*
We named the client "sss_nfs" (althogh "sss_idmap" or "idmap" might have been better names, the term "idmap" is already occupied in the SSSD
world).
How does this client work, is it a module that the rpc.idmapd dlopens() ?
yes, dlopen() is used. in more details: once "sss_nfs" is specified as a "Method" in /etc/idmapd.conf the module /usr/lib64/libnfsidmap/sss_nfs.so will be dlopen()ed (the file paths are under the assumption of fedora19 x86_64)
*rpc.idmapd - background*
rpc.idmapd runs on NFSv4 servers as a userspace daemon (part of
nfs-utils).
It's role is to assist knfsd by providing the following 6 mapping
functions:
(user) name to uid
(group) name to gid
uid to (user) name
gid to (group) name
principal (user) name to ids (uid + gid)
principal (user) name to grouplist (groups which user are member of)
rpc.idmapd provides API for developing plugins (loaded by dlopen(3))
which
implements the actual mapping process.
- note: 5 + 6 are relevant only for kerberised NFSv4 servers. At the
first
stage we only purpose a design without kerberos support.
- note2: On the kernel level, there's a caching mechanism for the
responses
from the userspace daemon.
*SSSD - Responder*
The functionality required from the Responder side is a subset of the functionality provided by existing NSS Responder's commands.
As you can see below (on the client part of the design) - no changes are needed in the NSS Responder.
*SSSD - NFS Client*
*Responder-Facing Interactions (existing NSS Responder commands)*
SSS_NSS_GETPWNAM - map (user) name to uid requests SSS_NSS_GETGRNAM - map (group) name to gid requests SSS_NSS_GETPWUID - map uid to (user) name requests SSS_NSS_GETGRGID - map gid to (group) name requests
The request & reply sent to & from the responder is "standard" in terms
of
the NSS Responder.
The client only needs a portion of the reply. Only this portion will be extracted from the packet (i.e. uid/gid/user name/group name).
Sure, this would work and as long as you see no performance issues, this approach is completely fine.
But please note that especially getgr* calls can be quite expensive, especially when nesting is involved. The reason is that normally the getgr* calls also download all the group members, resolving the nesting in the process.
This sounds problematic.
But if you do want to optimize further (perhaps in the future), we could then add a new NSS call that would only download the group entry in order to retrieve the GID. I think the backend part of this change would be also usable for the PAC responder and we already talked about this change recently with Sumit.
But for version 1.0 I agree we can just use the existing NSS calls. A future version of the client can switch to another NSS call later, we can do that safely.
Ok. lets start that way. Though I believe that we'll provide the extra NSS call soon enough...
*Optimisation Techniques*
The optimisation techniques used for the NSS client will be used here as well. i.e. Fast Cache (memcache) & negative-cache.
It will be possible for the user to disable Fast Cache from the configuration file. (see below)
What problems do you envision with the Fast Cache? Or did yo mostly want to make sure kernel does its own caching and the SSSD stays out of the way?
As you guessed, we mostly want to make sure that the kernel does its own caching and SSSD stays out the way. :-) But that depends on the specific usage pattern. For others Fast Cache might be useful.
*Configuration File*
The configuration of the client will be part of rpc.idmap config file (/etc/idmapd.conf).
I think the plan sounds great in general!
Thanks!
On Tue, Dec 17, 2013 at 03:28:45PM +0200, Noam Meltzer wrote:
On 17 December 2013 11:55, Jakub Hrozek jhrozek@redhat.com wrote:
On Tue, Dec 17, 2013 at 07:40:29AM +0200, Noam Meltzer wrote:
Hello sssd-developers,
Allow me to introduce myself and the suggested contribution to the SSSD project. PrimaryData Inc. is a startup company in the field of
distributed
NAS. As such, we would like to contribute to the SSSD project by
developing
support for rpc.idmapd.
Below is our proposed design for this new feature.
Best regards, Noam Meltzer
Hi Noam,
thanks for this design, it's great to have outside contributions!
*SSS NFS Client (rpc.idmapd plugin) - Design*
We named the client "sss_nfs" (althogh "sss_idmap" or "idmap" might have been better names, the term "idmap" is already occupied in the SSSD
world).
How does this client work, is it a module that the rpc.idmapd dlopens() ?
yes, dlopen() is used. in more details: once "sss_nfs" is specified as a "Method" in /etc/idmapd.conf the module /usr/lib64/libnfsidmap/sss_nfs.so will be dlopen()ed (the file paths are under the assumption of fedora19 x86_64)
*rpc.idmapd - background*
rpc.idmapd runs on NFSv4 servers as a userspace daemon (part of
nfs-utils).
It's role is to assist knfsd by providing the following 6 mapping
functions:
(user) name to uid
(group) name to gid
uid to (user) name
gid to (group) name
principal (user) name to ids (uid + gid)
principal (user) name to grouplist (groups which user are member of)
rpc.idmapd provides API for developing plugins (loaded by dlopen(3))
which
implements the actual mapping process.
- note: 5 + 6 are relevant only for kerberised NFSv4 servers. At the
first
stage we only purpose a design without kerberos support.
- note2: On the kernel level, there's a caching mechanism for the
responses
from the userspace daemon.
*SSSD - Responder*
The functionality required from the Responder side is a subset of the functionality provided by existing NSS Responder's commands.
As you can see below (on the client part of the design) - no changes are needed in the NSS Responder.
*SSSD - NFS Client*
*Responder-Facing Interactions (existing NSS Responder commands)*
SSS_NSS_GETPWNAM - map (user) name to uid requests SSS_NSS_GETGRNAM - map (group) name to gid requests SSS_NSS_GETPWUID - map uid to (user) name requests SSS_NSS_GETGRGID - map gid to (group) name requests
The request & reply sent to & from the responder is "standard" in terms
of
the NSS Responder.
The client only needs a portion of the reply. Only this portion will be extracted from the packet (i.e. uid/gid/user name/group name).
Sure, this would work and as long as you see no performance issues, this approach is completely fine.
But please note that especially getgr* calls can be quite expensive, especially when nesting is involved. The reason is that normally the getgr* calls also download all the group members, resolving the nesting in the process.
This sounds problematic.
But if you do want to optimize further (perhaps in the future), we could then add a new NSS call that would only download the group entry in order to retrieve the GID. I think the backend part of this change would be also usable for the PAC responder and we already talked about this change recently with Sumit.
But for version 1.0 I agree we can just use the existing NSS calls. A future version of the client can switch to another NSS call later, we can do that safely.
Ok. lets start that way. Though I believe that we'll provide the extra NSS call soon enough...
*Optimisation Techniques*
The optimisation techniques used for the NSS client will be used here as well. i.e. Fast Cache (memcache) & negative-cache.
It will be possible for the user to disable Fast Cache from the configuration file. (see below)
What problems do you envision with the Fast Cache? Or did yo mostly want to make sure kernel does its own caching and the SSSD stays out of the way?
As you guessed, we mostly want to make sure that the kernel does its own caching and SSSD stays out the way. :-) But that depends on the specific usage pattern. For others Fast Cache might be useful.
*Configuration File*
The configuration of the client will be part of rpc.idmap config file (/etc/idmapd.conf).
I think the plan sounds great in general!
Thanks!
Hi Noam,
since there were no other comments to the propsal, I think that means an ACK :-)
We typically keep the design documents on the wiki as well. Would you mind creating a page under: https://fedorahosted.org/sssd/wiki/DesignDocs
You would need a FAS account to do so. You can obtain one here: https://admin.fedoraproject.org/accounts/user/new
Feel free to ping me with questions!
On Tue, Jan 7, 2014 at 12:25 PM, Jakub Hrozek jhrozek@redhat.com wrote:
On Tue, Dec 17, 2013 at 03:28:45PM +0200, Noam Meltzer wrote:
On 17 December 2013 11:55, Jakub Hrozek jhrozek@redhat.com wrote:
On Tue, Dec 17, 2013 at 07:40:29AM +0200, Noam Meltzer wrote:
Hello sssd-developers,
Allow me to introduce myself and the suggested contribution to the
SSSD
project. PrimaryData Inc. is a startup company in the field of
distributed
NAS. As such, we would like to contribute to the SSSD project by
developing
support for rpc.idmapd.
Below is our proposed design for this new feature.
Best regards, Noam Meltzer
Hi Noam,
thanks for this design, it's great to have outside contributions!
*SSS NFS Client (rpc.idmapd plugin) - Design*
We named the client "sss_nfs" (althogh "sss_idmap" or "idmap" might
have
been better names, the term "idmap" is already occupied in the SSSD
world).
How does this client work, is it a module that the rpc.idmapd dlopens() ?
yes, dlopen() is used. in more details: once "sss_nfs" is specified as a "Method" in /etc/idmapd.conf the module /usr/lib64/libnfsidmap/sss_nfs.so will be dlopen()ed (the file paths are under the assumption of fedora19 x86_64)
*rpc.idmapd - background*
rpc.idmapd runs on NFSv4 servers as a userspace daemon (part of
nfs-utils).
It's role is to assist knfsd by providing the following 6 mapping
functions:
(user) name to uid
(group) name to gid
uid to (user) name
gid to (group) name
principal (user) name to ids (uid + gid)
principal (user) name to grouplist (groups which user are member
of)
rpc.idmapd provides API for developing plugins (loaded by dlopen(3))
which
implements the actual mapping process.
- note: 5 + 6 are relevant only for kerberised NFSv4 servers. At the
first
stage we only purpose a design without kerberos support.
- note2: On the kernel level, there's a caching mechanism for the
responses
from the userspace daemon.
*SSSD - Responder*
The functionality required from the Responder side is a subset of the functionality provided by existing NSS Responder's commands.
As you can see below (on the client part of the design) - no changes
are
needed in the NSS Responder.
*SSSD - NFS Client*
*Responder-Facing Interactions (existing NSS Responder commands)*
SSS_NSS_GETPWNAM - map (user) name to uid requests SSS_NSS_GETGRNAM - map (group) name to gid requests SSS_NSS_GETPWUID - map uid to (user) name requests SSS_NSS_GETGRGID - map gid to (group) name requests
The request & reply sent to & from the responder is "standard" in
terms
of
the NSS Responder.
The client only needs a portion of the reply. Only this portion will
be
extracted from the packet (i.e. uid/gid/user name/group name).
Sure, this would work and as long as you see no performance issues,
this
approach is completely fine.
But please note that especially getgr* calls can be quite expensive, especially when nesting is involved. The reason is that normally the getgr* calls also download all the group members, resolving the nesting in the process.
This sounds problematic.
But if you do want to optimize further (perhaps in the future), we
could
then add a new NSS call that would only download the group entry in order to retrieve the GID. I think the backend part of this change
would
be also usable for the PAC responder and we already talked about this change recently with Sumit.
But for version 1.0 I agree we can just use the existing NSS calls. A future version of the client can switch to another NSS call later, we can do that safely.
Ok. lets start that way. Though I believe that we'll provide the extra
NSS
call soon enough...
*Optimisation Techniques*
The optimisation techniques used for the NSS client will be used
here as
well. i.e. Fast Cache (memcache) & negative-cache.
It will be possible for the user to disable Fast Cache from the configuration file. (see below)
What problems do you envision with the Fast Cache? Or did yo mostly
want
to make sure kernel does its own caching and the SSSD stays out of the way?
As you guessed, we mostly want to make sure that the kernel does its own caching and SSSD stays out the way. :-) But that depends on the specific usage pattern. For others Fast Cache
might
be useful.
*Configuration File*
The configuration of the client will be part of rpc.idmap config file (/etc/idmapd.conf).
I think the plan sounds great in general!
Thanks!
Hi Noam,
since there were no other comments to the propsal, I think that means an ACK :-)
Done.
https://fedorahosted.org/sssd/wiki/DesignDocs/rpc.idmapd%20plugin
sssd-devel@lists.fedorahosted.org