Record LLM messages in the OS #27
Description
@rquesada proposed that the OS record messages between the applications and their users.
This is a good idea in theory and a potentially useful service for the OS to provide, but we need to look at the use cases. Why does one assistant want to know about the messages sent/received by another, for example? What about when they are multi-modal (e.g. text/voice/video)?
I'm disinclined to do this today as it shifts the OS from the management plane squarely into the data plane. For example, in Azure you create storage accounts but the management plane is not involved in the connections between the users and the data plane. The bandwidth/latency/security/availability/etc. requirements of the two are vastly different, so until we have a way (e.g. nginx reverse proxy) to handle the data plane in/from the OS (assuming it's deemed necessary or useful), we should measure twice and cut once on this.