Hi everyone,
I've been acquainting myself with SSSD code the last few weeks. Particularly, the NSS responder, and what it uses down the stack.
The SSSD code is sure complex and its asynchronous nature doesn't help. However, what struck me, and frustrated me the most, as a new reader of the code, was the meager amount of code documentation.
In the absence of documentation, a piece of logic, such as processing a getpwnam request, requires traversing many functions, modules, and several programs. Sure, you can guess what a function does, based on its name and somewhat on arguments, but you can't be sure without reading it and the functions it calls, and often the functions those call in turn. Same goes with data structures: to understand them you need to read the code that creates and operates them.
To the author code often seems obvious, but a new reader has a dozen questions: which state the passed object should be in, how does the function modify the object, where does it put its output, what else it affects, exactly what string should be that, what status codes the function can return and what will they mean, etc.
Documentation cements the author's intention in the mind of the reader, freeing the memory and precious attention for understanding the code at hand, as opposed to requiring going deeper and deeper into the underlying code, where the reader eventually forgets why he or she is looking at it. Reading such complicated code as can be seen in SSSD without documentation is an exercise in frustration.
We often speak about cooperation and getting more external contributors. However, we're not going to get many if they have such a mountain of code to climb without our help. We ourselves will be (and are) struggling to understand our own code, and will pay with lost time again and again when trying to modify or extend it.
In short: it is NOT OK that SSSD code has no documentation. Put bluntly, it is sloppy engineering. I'm sure we can do better.
If you have just a sentence describing each function and each of its arguments, understanding the code becomes considerably easier. You don't have to guess that much, and you don't have to verify your guesses and read so much more. Function documentation would often make reading its code unnecessary. A sentence for a data structure and each of its fields makes understanding the code operating it much easier.
I suggest several measures which could be taken.
1. Require *every* new module/function/argument/structure/field to be documented. It must be at least a sentence for each. A module or a bigger data structure could warrant more. Describe module/function/structure overall, and each of its declarations/arguments/return values/fields. 2. Next level, documentation tax: if you touch anything, document it. Modified a module header? Add a short module description. Added or removed a field in a structure? Add structure and field descriptions. Changed a function? Document it. 3. Third level, documentation duty: document a whole existing module header of your choosing, say once a month.
The format of documentation doesn't matter, as long as it's legible and agreed upon. I suggest Doxygen, but it can be anything else. We don't have to actually run the documentation extraction tool (such as "doxygen") on our code and use perfectly correct syntax (although that could be the next step). What matters is that documentation is there, it's uniform, and you don't have to think about the format when writing it.
For example of how it can look see the attached patch where I tried to document some things (likely misunderstanding some) as I read the code.
For more examples of what I mean see the tlog header files:
https://github.com/Scribery/tlog/tree/master/include/tlog
They are not the best example and there are many ways they can be improved (tell me please!), but if SSSD had just that it would be already awesome.
Please tell me what you think, object, or suggest your own solutions!
Thank you.
Nick
On Wed, Jul 20, 2016 at 03:53:54PM +0300, Nikolai Kondrashov wrote:
I suggest several measures which could be taken.
- Require *every* new module/function/argument/structure/field to be documented. It must be at least a sentence for each. A module or a bigger data structure could warrant more. Describe module/function/structure overall, and each of its declarations/arguments/return values/fields.
I'm OK with documenting interfaces and concepts, but not OK with documenting every function. I would rather spend time on making the code more readable than effectively creating two places to maintain (the code and the docs). Especially for static functions, I really don't see the value.
So I'm fine with adding documentation to headers, but I think it's busywork to add doxygen headers to every function.
I also think it's OK to document why the code does what it does, but documenting private interfaces wouldn't help in my opinon, but adding more comments to the code does.
- Next level, documentation tax: if you touch anything, document it. Modified a module header? Add a short module description. Added or removed a field in a structure? Add structure and field descriptions. Changed a function? Document it.
better: If a developer works on a module that is confusing to understand, document the code flow.
- Third level, documentation duty: document a whole existing module header of your choosing, say once a month.
The format of documentation doesn't matter, as long as it's legible and agreed upon. I suggest Doxygen, but it can be anything else. We don't have to actually run the documentation extraction tool (such as "doxygen") on our code and use perfectly correct syntax (although that could be the next step). What matters is that documentation is there, it's uniform, and you don't have to think about the format when writing it.
For example of how it can look see the attached patch where I tried to document some things (likely misunderstanding some) as I read the code.
Some pieces of the patch are beneficial, some are not. I don't really see how documenting that pvt_ctx is a private context improves the code. That's obvious from the name of the structure member, if not, we need to rename the member. If the patch added something like "this is a context that holds private data such as NSS context that contains NSS client data etc for NSS responder" if would be much better.
The check_next comment seems to be good OTOH.
Hi Jakub,
Thanks for your swift response :)!
On 07/20/2016 04:17 PM, Jakub Hrozek wrote:
On Wed, Jul 20, 2016 at 03:53:54PM +0300, Nikolai Kondrashov wrote:
I suggest several measures which could be taken.
- Require *every* new module/function/argument/structure/field to be documented. It must be at least a sentence for each. A module or a bigger data structure could warrant more. Describe module/function/structure overall, and each of its declarations/arguments/return values/fields.
I'm OK with documenting interfaces and concepts, but not OK with documenting every function. I would rather spend time on making the code more readable than effectively creating two places to maintain (the code and the docs). Especially for static functions, I really don't see the value.
So I'm fine with adding documentation to headers, but I think it's busywork to add doxygen headers to every function.
I also think it's OK to document why the code does what it does, but documenting private interfaces wouldn't help in my opinon, but adding more comments to the code does.
Documenting private interfaces is basically the same as documenting the code, and serves the same purpose: making it easier to understand the code. When you document an interface (public or private, doesn't matter) you narrow it down in people's understanding, making it easier to reason about. Besides, today's private interface is tomorrow's public interface and vice versa.
Furthermore, regarding busywork, if a function/structure/module is simple, then it's easy to document. If not, then it surely deserves the effort of documentation, and perhaps even refactoring.
Yes, documentation is duplicated knowledge, but it saves time and is worth the effort, considering the old fact that code is more often read than written.
All that said, even just the public interfaces is much better than the current situation. I, personally, will be happy to see that :)
- Next level, documentation tax: if you touch anything, document it. Modified a module header? Add a short module description. Added or removed a field in a structure? Add structure and field descriptions. Changed a function? Document it.
better: If a developer works on a module that is confusing to understand, document the code flow.
And here is the pitfall: the author is the one least qualified to judge how confusing the code is, because he/she already groks it. I'm not saying it's impossible, but bias would be expected.
- Third level, documentation duty: document a whole existing module header of your choosing, say once a month.
The format of documentation doesn't matter, as long as it's legible and agreed upon. I suggest Doxygen, but it can be anything else. We don't have to actually run the documentation extraction tool (such as "doxygen") on our code and use perfectly correct syntax (although that could be the next step). What matters is that documentation is there, it's uniform, and you don't have to think about the format when writing it.
For example of how it can look see the attached patch where I tried to document some things (likely misunderstanding some) as I read the code.
Some pieces of the patch are beneficial, some are not. I don't really see how documenting that pvt_ctx is a private context improves the code. That's obvious from the name of the structure member, if not, we need to rename the member. If the patch added something like "this is a context that holds private data such as NSS context that contains NSS client data etc for NSS responder" if would be much better.
Thanks, I'll add that!
The check_next comment seems to be good OTOH.
Ah, good.
Otherwise, this patch was just an illustration, I'll be sending it as part of my SSSD/tlog integration patchset if we agree on documentation in general and particular format. If not, I'll just drop it. Let's not discuss it here, but thanks for the comments :)
Nick
On (20/07/16 16:33), Nikolai Kondrashov wrote:
Hi Jakub,
Thanks for your swift response :)!
On 07/20/2016 04:17 PM, Jakub Hrozek wrote:
On Wed, Jul 20, 2016 at 03:53:54PM +0300, Nikolai Kondrashov wrote: better: If a developer works on a module that is confusing to understand, document the code flow.
And here is the pitfall: the author is the one least qualified to judge how confusing the code is, because he/she already groks it. I'm not saying it's impossible, but bias would be expected.
Therefore we have a review. If reviewer does not understand the code then the code should be changed. Documentation will not help.
+ @see my other response about unit test.
LS
On 07/20/2016 05:38 PM, Lukas Slebodnik wrote:
On (20/07/16 16:33), Nikolai Kondrashov wrote:
Hi Jakub,
Thanks for your swift response :)!
On 07/20/2016 04:17 PM, Jakub Hrozek wrote:
On Wed, Jul 20, 2016 at 03:53:54PM +0300, Nikolai Kondrashov wrote: better: If a developer works on a module that is confusing to understand, document the code flow.
And here is the pitfall: the author is the one least qualified to judge how confusing the code is, because he/she already groks it. I'm not saying it's impossible, but bias would be expected.
Therefore we have a review. If reviewer does not understand the code then the code should be changed. Documentation will not help.
Reviewers are usually developers, and usually *core* developers, who understand a lot about the context and code already, so they're poor judges as well.
It's easy to reason about the code which you discussed with your colleagues for the past week or even a month, talked about on meetings, etc. However, when you come back to it half a year later, you get a different value of "confusing". Not speaking about newbies.
- @see my other response about unit test.
Thanks, Lukas, replied.
Nick
On (20/07/16 15:17), Jakub Hrozek wrote:
On Wed, Jul 20, 2016 at 03:53:54PM +0300, Nikolai Kondrashov wrote:
I suggest several measures which could be taken.
- Require *every* new module/function/argument/structure/field to be documented. It must be at least a sentence for each. A module or a bigger data structure could warrant more. Describe module/function/structure overall, and each of its declarations/arguments/return values/fields.
I'm OK with documenting interfaces and concepts, but not OK with documenting every function. I would rather spend time on making the code more readable than effectively creating two places to maintain (the code and the docs). Especially for static functions, I really don't see the value.
So I'm fine with adding documentation to headers, but I think it's busywork to add doxygen headers to every function.
I also think it's OK to document why the code does what it does, but documenting private interfaces wouldn't help in my opinon, but adding more comments to the code does.
BTW we already have a documentation for public fuction in libraries. Becasue it is stable API.
Internal functions are not stable even thought they are public. The biggest issue with a documentation is outdated documentation. Outdated documetation is worse that code without documentation. Yes, we could check it as part of review but people are not perfect and we can miss that.
Therefore it is much better to write unit tests for functions. a) they cannot be outdated. (they have to pass) b) you can see a usage of functions. c) if it is a hard to write unit test than it means that functions is complicated or do a lot of things and should be refactored. => refactored code will be easier to read, to understand and to test.
- Next level, documentation tax: if you touch anything, document it. Modified a module header? Add a short module description. Added or removed a field in a structure? Add structure and field descriptions. Changed a function? Document it.
better: If a developer works on a module that is confusing to understand, document the code flow.
Partially agree, non-obvious part should be either documented in code or refactored.
LS
Hi Lukas,
Thanks for your response!
On 07/20/2016 05:28 PM, Lukas Slebodnik wrote:
On (20/07/16 15:17), Jakub Hrozek wrote:
On Wed, Jul 20, 2016 at 03:53:54PM +0300, Nikolai Kondrashov wrote:
I suggest several measures which could be taken.
- Require *every* new module/function/argument/structure/field to be documented. It must be at least a sentence for each. A module or a bigger data structure could warrant more. Describe module/function/structure overall, and each of its declarations/arguments/return values/fields.
I'm OK with documenting interfaces and concepts, but not OK with documenting every function. I would rather spend time on making the code more readable than effectively creating two places to maintain (the code and the docs). Especially for static functions, I really don't see the value.
So I'm fine with adding documentation to headers, but I think it's busywork to add doxygen headers to every function.
I also think it's OK to document why the code does what it does, but documenting private interfaces wouldn't help in my opinon, but adding more comments to the code does.
BTW we already have a documentation for public fuction in libraries. Becasue it is stable API.
Internal functions are not stable even thought they are public. The biggest issue with a documentation is outdated documentation. Outdated documetation is worse that code without documentation. Yes, we could check it as part of review but people are not perfect and we can miss that.
Depends on how much outdated it is. Even 25% outdated documentation is much better than no documentation. Besides, code gets outdated too. Some code gets updated and some of its uses don't, and we live with that, we don't have tests for *everything*. It doesn't mean we shouldn't write or update code. The fact that documentation gets outdated sometimes, doesn't mean we shouldn't write it, or try to keep it updated.
I argue that the effort of writing documentation and keeping it updated is worthy. It makes life easier for existing and new developers alike, including external contributors. Reading more often than writing, etc.
Therefore it is much better to write unit tests for functions. a) they cannot be outdated. (they have to pass) b) you can see a usage of functions. c) if it is a hard to write unit test than it means that functions is complicated or do a lot of things and should be refactored. => refactored code will be easier to read, to understand and to test.
Unit tests are great and can help understanding, but they're not a substitute for documentation. Documentation is written in natural, human language, is easier to digest, and tells *directly* what that code/structure is for.
Nick
On (20/07/16 17:41), Nikolai Kondrashov wrote:
Hi Lukas,
Thanks for your response!
On 07/20/2016 05:28 PM, Lukas Slebodnik wrote:
On (20/07/16 15:17), Jakub Hrozek wrote:
On Wed, Jul 20, 2016 at 03:53:54PM +0300, Nikolai Kondrashov wrote:
I suggest several measures which could be taken.
- Require *every* new module/function/argument/structure/field to be documented. It must be at least a sentence for each. A module or a bigger data structure could warrant more. Describe module/function/structure overall, and each of its declarations/arguments/return values/fields.
I'm OK with documenting interfaces and concepts, but not OK with documenting every function. I would rather spend time on making the code more readable than effectively creating two places to maintain (the code and the docs). Especially for static functions, I really don't see the value.
So I'm fine with adding documentation to headers, but I think it's busywork to add doxygen headers to every function.
I also think it's OK to document why the code does what it does, but documenting private interfaces wouldn't help in my opinon, but adding more comments to the code does.
BTW we already have a documentation for public fuction in libraries. Becasue it is stable API.
Internal functions are not stable even thought they are public. The biggest issue with a documentation is outdated documentation. Outdated documetation is worse that code without documentation. Yes, we could check it as part of review but people are not perfect and we can miss that.
Depends on how much outdated it is. Even 25% outdated documentation is much better than no documentation. Besides, code gets outdated too. Some code gets updated and some of its uses don't, and we live with that, we don't have tests for *everything*. It doesn't mean we shouldn't write or update code. The fact that documentation gets outdated sometimes, doesn't mean we shouldn't write it, or try to keep it updated.
outdated documentation == confusing documentation. confusing documentation => bad thing for newcomers.
I argue that the effort of writing documentation and keeping it updated is worthy. It makes life easier for existing and new developers alike, including external contributors. Reading more often than writing, etc.
It still does not make sense to me why is documentation better than unit test.
Therefore it is much better to write unit tests for functions. a) they cannot be outdated. (they have to pass) b) you can see a usage of functions. c) if it is a hard to write unit test than it means that functions is complicated or do a lot of things and should be refactored. => refactored code will be easier to read, to understand and to test.
Unit tests are great and can help understanding, but they're not a substitute for documentation. Documentation is written in natural, human language, is easier to digest, and tells *directly* what that code/structure is for.
Examples are much better than documentation of arguments. Unit tests are examples of how to use functions. The unit test should contain ordinary use case + corner cases with error codes. The biggest benefit is that you cannot have outdated unit tests.
We could write examples to comments as well. IMHO it does not worth the effort. It make sense for public libraries with stable API.
LS
On 07/20/2016 06:42 PM, Lukas Slebodnik wrote:
On (20/07/16 17:41), Nikolai Kondrashov wrote:
Depends on how much outdated it is. Even 25% outdated documentation is much better than no documentation. Besides, code gets outdated too. Some code gets updated and some of its uses don't, and we live with that, we don't have tests for *everything*. It doesn't mean we shouldn't write or update code. The fact that documentation gets outdated sometimes, doesn't mean we shouldn't write it, or try to keep it updated.
outdated documentation == confusing documentation. confusing documentation => bad thing for newcomers.
As long as most of it is right, it will still deliver a lot of useful information. Yes, somebody will get confused about some things sometimes, but they will still get the picture, and we will be here to accept the complaints and fix it.
I argue that the effort of writing documentation and keeping it updated is worthy. It makes life easier for existing and new developers alike, including external contributors. Reading more often than writing, etc.
It still does not make sense to me why is documentation better than unit test.
Because it's natural language, it's designed for humans, and it says directly what the code does and for what. You don't have to derive the knowledge by reading some more code. It's there, just read basic English.
People don't communicate in unit tests written in C, because they're hard to understand. Even for developers who do it all day, every day.
Therefore it is much better to write unit tests for functions. a) they cannot be outdated. (they have to pass) b) you can see a usage of functions. c) if it is a hard to write unit test than it means that functions is complicated or do a lot of things and should be refactored. => refactored code will be easier to read, to understand and to test.
Unit tests are great and can help understanding, but they're not a substitute for documentation. Documentation is written in natural, human language, is easier to digest, and tells *directly* what that code/structure is for.
Examples are much better than documentation of arguments. Unit tests are examples of how to use functions. The unit test should contain ordinary use case + corner cases with error codes.
Examples are great, without doubt, but they're not the same as documentation. Let's take a simple (made up) piece of information. To communicate the possible return codes in documentation, you need just a few sentences of clear, direct human language:
EOK - user added successfully, ENOMEM - memory allocation failed, EAGAIN - operation needs to continue asynchronously from the main loop, EINVAL - user name was invalid, EEXIST - a user with the same name already existed, user not added.
How much test code will you need to write and read to explain and understand all that?
The biggest benefit is that you cannot have outdated unit tests.
That is sure a very nice feature of tests, which helps make them good examples. BTW, people writing programming books often setup test systems to verify their example code, so it works both ways.
So, unit tests are good examples and help improve code, but they don't help understanding as much as natural-language documentation.
We could write examples to comments as well. IMHO it does not worth the effort. It make sense for public libraries with stable API.
I don't think we should write example code in comments, if only really rarely. And I think we can benefit greatly from code documentation, in *addition* to unit tests.
Nick
On 07/20/2016 05:28 PM, Lukas Slebodnik wrote:
c) if it is a hard to write unit test than it means that functions is complicated or do a lot of things and should be refactored. => refactored code will be easier to read, to understand and to test.
BTW, same goes with documentation :) If it's hard to write, then code needs refactoring.
Oh, and documentation is easier to write than unit tests, we can never write unit tests for everything.
Nick
On (20/07/16 18:05), Nikolai Kondrashov wrote:
On 07/20/2016 05:28 PM, Lukas Slebodnik wrote:
c) if it is a hard to write unit test than it means that functions is complicated or do a lot of things and should be refactored. => refactored code will be easier to read, to understand and to test.
BTW, same goes with documentation :) If it's hard to write, then code needs refactoring.
No, because you can write documentation even for function which is hard to unit test. Documentation might help to increase understanding of code. But it does not enforce better code.
Oh, and documentation is easier to write than unit tests, we can never write unit tests for everything.
"easier to write" does not mean better/more readable code. Unit test is much stricter than documentation. BTW It does not mean that we should not use documentation comments.
We can describe "test-case" in unit test so it will be easier to find out which use case do you want to cover with the test. Sometimes it can be find out from the name of test.
IMHO, we should use some mixed way. We can have a documentation for stable parts of code (e.g. sysdb) and unit tests for other parts of code.
LS
On 07/20/2016 06:33 PM, Lukas Slebodnik wrote:
On (20/07/16 18:05), Nikolai Kondrashov wrote:
On 07/20/2016 05:28 PM, Lukas Slebodnik wrote:
c) if it is a hard to write unit test than it means that functions is complicated or do a lot of things and should be refactored. => refactored code will be easier to read, to understand and to test.
BTW, same goes with documentation :) If it's hard to write, then code needs refactoring.
No, because you can write documentation even for function which is hard to unit test. Documentation might help to increase understanding of code. But it does not enforce better code.
Sure, although strictly speaking unit tests don't absolutely enforce better code either. They're just harder to write in general, and so provide more incentive to refactor the code, but you can still write a unit test even for a bad code, although the test itself will be more complex and will test less. There is no limit to amount of code a non-lazy programmer can write ;)
I.e., yes, unit tests enforce better code stronger than documentation.
However, my response was just an observation of minor benefit of documentation, not the main reason why we should do it.
Oh, and documentation is easier to write than unit tests, we can never write unit tests for everything.
"easier to write" does not mean better/more readable code. Unit test is much stricter than documentation. BTW It does not mean that we should not use documentation comments.
We can describe "test-case" in unit test so it will be easier to find out which use case do you want to cover with the test. Sometimes it can be find out from the name of test.
IMHO, we should use some mixed way. We can have a documentation for stable parts of code (e.g. sysdb) and unit tests for other parts of code.
I fully agree that unit tests help improve code and fully support adding more of them, especially if they're well commented. However, the effect they have on making the code easier to understand is lower and it's harder to achieve. Because they're still *code*, not natural language, they don't describe the code *directly*, and they're harder to write than documentation.
In short, if I was told: "if you want to understand this code, go and read some more of other code (tests)" or "here is how this works", I would choose the latter.
Regarding documenting only stable code there are several problems. First, how and when to define code as "stable". Then, we don't need contributions to stable code as much as to unstable. I.e. we rarely need fixes to such code, and generally will be wary of new features there.
Nick
On 07/20/2016 02:53 PM, Nikolai Kondrashov wrote:
Hi everyone,
I've been acquainting myself with SSSD code the last few weeks. Particularly, the NSS responder, and what it uses down the stack.
The SSSD code is sure complex and its asynchronous nature doesn't help. However, what struck me, and frustrated me the most, as a new reader of the code, was the meager amount of code documentation.
In the absence of documentation, a piece of logic, such as processing a getpwnam request, requires traversing many functions, modules, and several programs. Sure, you can guess what a function does, based on its name and somewhat on arguments, but you can't be sure without reading it and the functions it calls, and often the functions those call in turn. Same goes with data structures: to understand them you need to read the code that creates and operates them.
To the author code often seems obvious, but a new reader has a dozen questions: which state the passed object should be in, how does the function modify the object, where does it put its output, what else it affects, exactly what string should be that, what status codes the function can return and what will they mean, etc.
Documentation cements the author's intention in the mind of the reader, freeing the memory and precious attention for understanding the code at hand, as opposed to requiring going deeper and deeper into the underlying code, where the reader eventually forgets why he or she is looking at it. Reading such complicated code as can be seen in SSSD without documentation is an exercise in frustration.
We often speak about cooperation and getting more external contributors. However, we're not going to get many if they have such a mountain of code to climb without our help. We ourselves will be (and are) struggling to understand our own code, and will pay with lost time again and again when trying to modify or extend it.
In short: it is NOT OK that SSSD code has no documentation. Put bluntly, it is sloppy engineering. I'm sure we can do better.
If you have just a sentence describing each function and each of its arguments, understanding the code becomes considerably easier. You don't have to guess that much, and you don't have to verify your guesses and read so much more. Function documentation would often make reading its code unnecessary. A sentence for a data structure and each of its fields makes understanding the code operating it much easier.
I suggest several measures which could be taken.
- Require *every* new module/function/argument/structure/field to
be documented. It must be at least a sentence for each. A module or a bigger data structure could warrant more. Describe module/function/structure overall, and each of its declarations/arguments/return values/fields. 2. Next level, documentation tax: if you touch anything, document it. Modified a module header? Add a short module description. Added or removed a field in a structure? Add structure and field descriptions. Changed a function? Document it. 3. Third level, documentation duty: document a whole existing module header of your choosing, say once a month.
The format of documentation doesn't matter, as long as it's legible and agreed upon. I suggest Doxygen, but it can be anything else. We don't have to actually run the documentation extraction tool (such as "doxygen") on our code and use perfectly correct syntax (although that could be the next step). What matters is that documentation is there, it's uniform, and you don't have to think about the format when writing it.
For example of how it can look see the attached patch where I tried to document some things (likely misunderstanding some) as I read the code.
For more examples of what I mean see the tlog header files:
https://github.com/Scribery/tlog/tree/master/include/tlog
They are not the best example and there are many ways they can be improved (tell me please!), but if SSSD had just that it would be already awesome.
Please tell me what you think, object, or suggest your own solutions!
Thank you.
Nick
Hi Nick, when I joined SSSD I used to document everything, exactly the way you do. But I was trained by Jakub not to do it and in time I started to see benefits of documenting only public interface that is bound to be stale. Maintaining documentation is a pain and core developer doesn't need it since we can understand most of the code quite quickly and we rewrite the poor parts we don't.
However, I can see the benefits of code documentation for new comers. So the question is, would this help us to attract more contributors? Or is SSSD already too complicated for any ordinary coder to step in? I'm afraid its the latter, since serious development requires lots of knowledge in both outside and inside SSSD that simply don't come in a few days.
Do you have an example of a project that gained more contributors after creating a documentation?
I personally see more benefit in writing articles on how SSSD works (we already have some) and on our coding habits (e.g. EAGAIN mean asynchronous processing) than in documenting code.
Unfortunately, you chose probably the worse code for start that you could possible choose. NSS responder is one of the oldest code that I admit is terrible to read and contains lots of per-feature hacks. But as you dig in, you see that most of the code is just about reading data from cache and refreshing them in providers. All this complexity can be now done with one call of cache_req. Since you have the code in grasp, would you like to convert it to cache_req please?
Pavel.
Hi Pavel,
Thank you for your thoughtful and weighted response. Please see my comments below.
On 07/25/2016 11:56 AM, Pavel Březina wrote:
Hi Nick, when I joined SSSD I used to document everything, exactly the way you do. But I was trained by Jakub not to do it and in time I started to see benefits of documenting only public interface that is bound to be stale. Maintaining documentation is a pain and core developer doesn't need it since we can understand most of the code quite quickly and we rewrite the poor parts we don't.
I understand. However, this creates a closed-in culture with a high price of entry. Which is still higher for people who (feel they) can't approach and talk to us.
Is it worth paying for a person who just wants a couple fixes or small features done and doesn't want to become a full-time developer?
However, I can see the benefits of code documentation for new comers. So the question is, would this help us to attract more contributors? Or is SSSD already too complicated for any ordinary coder to step in? I'm afraid its the latter, since serious development requires lots of knowledge in both outside and inside SSSD that simply don't come in a few days.
I'm sure that documentation would lower the barrier to entry. Yes, SSSD is complicated, but not writing documentation wouldn't make it easier to understand. The question is how many people trying to modify it give up, and whether we can lower that number.
We have a hard time understanding it ourselves, and we can ask each other and tap the sacred folklore. People outside don't have that privilege, or don't feel they have.
Do you have an example of a project that gained more contributors after creating a documentation?
Nope, I don't have any concrete evidence.
I personally see more benefit in writing articles on how SSSD works (we already have some) and on our coding habits (e.g. EAGAIN mean asynchronous processing) than in documenting code.
Such documentation seems good precisely because it's so removed from the code. Yes, it needs changing rarely, but it's also much less useful for a person wanting to change a specific piece of code, as compared to becoming a general developer.
Unfortunately, you chose probably the worse code for start that you could possible choose. NSS responder is one of the oldest code that I admit is terrible to read and contains lots of per-feature hacks.
Yes, it's pretty hard to read and that made me really wish for documentation.
But as you dig in, you see that most of the code is just about reading data from cache and refreshing them in providers. All this complexity can be now done with one call of cache_req. Since you have the code in grasp, would you like to convert it to cache_req please?
I would like to do that, but I have a lot of urgent work to do on Session Recording project, sorry. I'll try to sneak some minor improvements in, though.
To summarize, yes, once you get into it, once you talk to people, figure out all the ideas, read the code again and again, get through a couple reviews, etc., you can live without documentation. However, most potential (occasional) contributors can't afford that.
And, yes, code documentation is only one variable in the equation of getting the contributors. It is not a panacea, it only helps so much. Yet, it also increases engineering discipline (a hacked-together code is harder to document), and improves and speeds up understanding of the code for core developers too.
We were all newbies with this or other projects (I still am one) and we remember the pain. Do we want our contributors to feel that too?
Nick
sssd-devel@lists.fedorahosted.org