Jonathan Gardner wrote:
Everyone is a volunteer
Everyone is a potential volunteer. You included.
We recruit the users (the "Eloi") and ask them to choose a project they are interested in, and we hook them up with those specialists
Have you looked at the fedora.redhat.com project pages...that looks like a recruitment effort to me..based on each project. Or are you talking about hiring a brass band and marching around or something similar? Recruitment efforts on a lot of fronts are in need of creative ideas. By the way, for anyone reading fedora.us QA review needs volunteers. So does the Fedora Core community triage effort, find me on #fedora-bugs on the irc network if you are interested in helping out with either of those efforts....
We recruit some more-technically minded people (the "Morlocks") and have them develop some usuability plans. These people should not be testers or developers, but people who understand the software or have the ability to communicate productively with the authors of the software
Have fun trying to identify the people who can do that well.
A third set of people will actually engage with the end-users in one- on-one sessions, following the plans that were developed
Are you keeping a running count in your head about the number of people you are talking about...and how much time you really expect these volunteers to contribute consistently? One on One sessions are expensive in terms of manhours...and you don't have anyone signed up yet..so your total number of manhours to spend are zero. Don't start building a process that is manpower expensive..until you have manpower to burn... you are going to kill the process under its own weight before you even get someone to volunteer.
The usuability sessions will be made public, so that other unrelated projects can derive some benefit from them
A more cynical view is that, making it public will provide other 'potential' volunteers the opportunity to rip apart your methodology and reinterpret the result to prove the result you derived from the sessions are invalid....be careful what you wish for.
We'll allow others who know more about this kind of thing to comment on the results, summarize it for the developers, identify the problems and suggest solutions
Unless you have a commitment from the developers to buy into the methodology that forms the basis of the summaries...this is just going to lead to a lot of argument. I tell you right now, that you aren't likely to win over development mindshare just by handing them a usability summary without their involvement with outlining at least the methodology you layout. If you are going to experiment with this, you should first find a project with developers who are open to working with you on this to see if it really will be helpful. Pushing the summaries on developers for a project if they aren't interested..is not going to help.
With enough data, we should be able to identify the biggest problems and work to solve them before FC3. Then, when FC3 comes out, we can do this all over again.
This is a very hopeful hypothesis. And I would argue, that the biggest problems will be defined differently by different segments of the userbase. It will be interesting how you build a process that matches biggest problems...to specific usage patterns...in an unbiased manner. It will be interesting to see how you guard against having your summaries being biased by a small vocal minority.
The only problems I see are getting people involved and working with each other. I worry that while we will be able to get participants, those will always be the *wrong* people we are trying to target
There are LOADS of problems...loads and loads... i like this handbook on volunteerism
http://www.sport.vic.gov.au/Web/SRV/srvimages.nsf/Images/VMPWorkbookpdf/ $File/VMPWorkbook.pdf
The summary on the first 18 pages are so, is generally useful as a guideline for any volunteer process...
However, never underestimate the amount of data that one session can produce! So maybe we don't need that much to actually happen.
the amount of data isn't the question, I fear for the quality of the data, and whether what you will be producing is something the developers want.
Maybe if you rethink this idea assuming you only have 3 or 4 people who are energized about working on the problem you see, and think about a process that can produce a result with just 3 or 4 people's worth of volunteer time. Think usability strike team, instead of usability batalion,
-jef"is still looking for someone to head up an accessibility strike team"spaleta
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Monday 01 March 2004 8:32 pm, Jef Spaleta wrote:
Jonathan Gardner wrote:
We recruit the users (the "Eloi") and ask them to choose a project they are interested in, and we hook them up with those specialists
Have you looked at the fedora.redhat.com project pages...that looks like a recruitment effort to me..based on each project. Or are you talking about hiring a brass band and marching around or something similar? Recruitment efforts on a lot of fronts are in need of creative ideas. By the way, for anyone reading fedora.us QA review needs volunteers. So does the Fedora Core community triage effort, find me on #fedora-bugs on the irc network if you are interested in helping out with either of those efforts....
I think QA and usability tests are two different things. QA people go in with certain expectations and an understanding of the software. Usuability studies include people who are familiar with the software and people who have never seen it before, and watch how they interact with the software. It exposes nitpicks and spots where the users just plain get confused or lost. QA has already absorbed the project, they already undertand the quirks and have been brainwashed that it is okay, that's the way it is supposed to work.
I don't think the FOSS community distinguishes between these two things adequately. We have this view that there are developers/project managers, and then everybody else (documentation, QA, usuability, users, all in one).
Are you keeping a running count in your head about the number of people you are talking about...and how much time you really expect these volunteers to contribute consistently? One on One sessions are expensive in terms of manhours...and you don't have anyone signed up yet..so your total number of manhours to spend are zero. Don't start building a process that is manpower expensive..until you have manpower to burn... you are going to kill the process under its own weight before you even get someone to volunteer.
So start small, introduce abstraction layers as (or if) we get bigger.
Unless you have a commitment from the developers to buy into the methodology that forms the basis of the summaries...this is just going to lead to a lot of argument. I tell you right now, that you aren't likely to win over development mindshare just by handing them a usability summary without their involvement with outlining at least the methodology you layout. If you are going to experiment with this, you should first find a project with developers who are open to working with you on this to see if it really will be helpful. Pushing the summaries on developers for a project if they aren't interested..is not going to help.
I see your point. I'll do an experiment at home. I have a lovely wife who hates computers but uses them anyway. Are there any volunteers for a project that would be willing to take the results of a usability study seriously?
Some potential candidates:
1) Configuring KMail, Mozilla Mail, or Evolution to get and send mail via my mail server (IMAPS and Sendmail with AUTH)
2) Configuring the network to browse the internet, given the IP addresses provided by my ISP, using Red Hat's internet configuration tool.
I'm open to other suggestions.
With enough data, we should be able to identify the biggest problems and work to solve them before FC3. Then, when FC3 comes out, we can do this all over again.
This is a very hopeful hypothesis. And I would argue, that the biggest problems will be defined differently by different segments of the userbase. It will be interesting how you build a process that matches biggest problems...to specific usage patterns...in an unbiased manner. It will be interesting to see how you guard against having your summaries being biased by a small vocal minority.
That is true. Usuability studies only render anecdotes. We never know if the two or three people who participate are truly representative of their population. But in my experience, even though they don't turn out a great volume of statistics with error bars or complete analysis of the target market, they do give a lot of valuable feedback.
I think if we are going to make Linux a serious contender on the desktop, we need to solve these problems sooner rather than later. In fact, I believe we can't conquer the desktop until we solve these showstoppers. We have to make Linux work for Aunt Tillie and Eric Raymond.
There are LOADS of problems...loads and loads... i like this handbook on volunteerism
http://www.sport.vic.gov.au/Web/SRV/srvimages.nsf/Images/VMPWorkbookpdf / $File/VMPWorkbook.pdf
The summary on the first 18 pages are so, is generally useful as a guideline for any volunteer process...
I browsed that guidebook and found it contains some knowledge I'd like to study in more detail. I have some familiarity with organizing people and such via church activities and Boy Scouts of America. Humm, maybe it is time to visit some friends and get their input... ;-)
- -- Jonathan Gardner jgardner@jonathangardner.net
On Tue, Mar 02, 2004 at 12:07:13AM -0800, Jonathan M. Gardner wrote:
I see your point. I'll do an experiment at home. I have a lovely wife who hates computers but uses them anyway. Are there any volunteers for a project that would be willing to take the results of a usability study seriously?
Its one thing that new users do well, sometimes inadvertantly but then don't feel confident about filing bugs.
Some potential candidates:
- Configuring KMail, Mozilla Mail, or Evolution to get and send mail via
my mail server (IMAPS and Sendmail with AUTH)
- Configuring the network to browse the internet, given the IP addresses
provided by my ISP, using Red Hat's internet configuration tool.
I'm open to other suggestions.
3) Using the supplied mailman and apache configure the lot graphically (you can't yet we provided all the bits)
There is also a ton of scope for improving error handling from errors to diagnosis
eg hitting "Print" when no printing is set up ought to tell you this and if you know the admin pw let you set it up at that point.
-
Alan Cox -
Jef Spaleta -
Jonathan Gardner