On Thu, Jan 28, 2021 at 12:36 PM Lukas Ruzicka <lruzicka@redhat.com> wrote:
I do understand the audio criterion quite differently. I believe that an application is an application, while an audio framework is an audio framework. I think that the working sound criterion should apply to audio as in audio framework and not to individual applications, that might be judged according to the "default application functionality" criterion. Therefore, limiting to gstreamer based applications is not important for my point of view.

Currently, a new situation is happening when we will have a new audio framework that should be able to support all kind of applications from gstreamer over alsa to jack, the working sound criterion at the Beta stage should actually read something like: "is the audio server working to produce audio?" Basically, what we are doing now is to test that some default application, like Videos or Rythmbox plays back music, which of course is not much different from "when I go to Gnome Settings and hit TEST on audio tab, I can hear sound coming from the speakers". The criterion I am proposing does not mention any application specifically, so basically it could be met when Gnome Session does "splash splash". We could perhaps automate some tests to check the technical stuff of PipeWire - the services are running, the info commands actually return some info.

What I am trying to say is that if some application does not play sound, while others do, it is clearly a problem in the application and not the sound framework, but if all applications based on one backend (alsa, gstreamer, jack) do not play, the framework might be to blame and therefore the situation might be regarded as blocking even if those applications are not based on gstreamer.

The most important change we are facing now is that PipeWire is here to support everything and if it does not, the operating system has major flaws.

I think there's a misunderstanding in how all the "frameworks" and stacked on top of each other. I'm not very knowledgeable in this area, but I think the layers are more or less like this:

6. Totem | Firefox | Rhythmbox | Audacity
--------------------------
5. GStreamer | FFmpeg
--------------------------
4. PulseAudio | JACK
--------------------------
3. Pipewire
--------------------------
2. ALSA | OSS
--------------------------
1. Hardware

The application from layer 6 of course doesn't need to use any framework from layer 5, it can talk to lower layers directly (it can talk directly even to ALSA on layer 2, but then you lose some benefits, like software sound multiplexing). But the important thing to notice is that gstreamer, ffmpeg, etc are way above Pipewire.

If we pick a particular app and say "app X must be able to play sound at Beta" (or alternatively "you must be able to find an app that plays sound at Beta"), that app X might talk directly to e.g. PulseAudio and while it works, you might not detect that 90% of other apps using a framework like gstreamer don't work. That's why I believe gstreamer is mentioned in the criterion, because it ensures that a) hardware works properly, and b) that a large portion of our apps are likely to work properly as well (once you test at least one of them).

Of course saying "at least one app must be able to produce sound" (which is basically what you proposed in the first email) is also valid, it's just a weaker version of that criterion (it will validate hardware and some low levels like Pipewire and ALSA, but it might or might not validate nothing above it). Audacity is a good example of an app using layer 4 at most, so layer 5 would not be covered.

Perhaps I'm misunderstanding you, but it seems to me that the current criterion is very much in line with what you're saying you want to do. It tests something from the (almost) very top all the way down to the bottom.