Hopefully it's possible to implement this with filesystem ACLs, just like PulseAudio/Wayland. Haven't looked into this in detail yet.