Update DSP mount and search path configuration

[quic-parihar]

Update DSP mount and search path configuration to
support dynamic runtime resolution

• Refactored mount logic to allow flexible path overrides
• Added support for environment-based search path injection

[lumag]

linux-msm/hexagonrpc#15 triggered the following idea: the binaries can be coming from different sources. So we might not have a single .conf file. We need to be able to parse conf.d-like directory, each file adding new entries and new machines. This way hexagon-dsp-binaries can provide configuration file for the binaries inside that package, PmOS can install additional device-specific binaries and then the user can manually repack Hexagon binaries for the WoA laptops.

[ekanshibu]

linux-msm/hexagonrpc#15 triggered the following idea: the binaries can be coming from different sources. So we might not have a single .conf file. We need to be able to parse conf.d-like directory, each file adding new entries and new machines.

I see, if the sources of binaries are going to change, then yes, a single file might not work. I think this modular configuration also covers the requirement of supporting vendor specific DSP artifacts. I don't have the complete understanding on conf.d but will check this. @quic-parihar you can also have a look into this.

This way hexagon-dsp-binaries can provide configuration file for the binaries inside that package, PmOS can install additional device-specific binaries and then the user can manually repack Hexagon binaries for the WoA laptops.

I don't completely understand the manually repacking point. Is PmOS going to maintain the device specific binaries separately? Or the bins are still maintained in hexagon-dsp-binaries and the user is somehow making the modification to use different binaries?

@quic-bkumar @Chennak-quic, seeking your inputs as well.

[lumag]

This way hexagon-dsp-binaries can provide configuration file for the binaries inside that package, PmOS can install additional device-specific binaries and then the user can manually repack Hexagon binaries for the WoA laptops.

I don't completely understand the manually repacking point. Is PmOS going to maintain the device specific binaries separately? Or the bins are still maintained in hexagon-dsp-binaries and the user is somehow making the modification to use different binaries?

PmOS, Debian, Ubuntu or just the user. For example, for Pixel or for WoA laptops it is possible to download and manually install these binaries, but it is not possible to redistribute those binaries. As such, the user might have both hexagon-dsp-binaries and external binaries.

[quic-parihar]

linux-msm/hexagonrpc#15 triggered the following idea: the binaries can be coming from different sources. So we might not have a single .conf file. We need to be able to parse conf.d-like directory, each file adding new entries and new machines.

@lumag — would appreciate your input on where to maintain this conf.d structure. Should we follow the convention of using /etc, like /etc/fastrpc/conf.d/, which is commonly used for host-specific configurations? Or, considering that /etc is meant for host-specific settings, should we place it elsewhere — for example, /usr/share/qcom/dsp/conf.d/

[lumag]

linux-msm/hexagonrpc#15 triggered the following idea: the binaries can be coming from different sources. So we might not have a single .conf file. We need to be able to parse conf.d-like directory, each file adding new entries and new machines.

@lumag — would appreciate your input on where to maintain this conf.d structure. Should we follow the convention of using /etc, like /etc/fastrpc/conf.d/, which is commonly used for host-specific configurations? Or, considering that /etc is meant for host-specific settings, should we place it elsewhere — for example, /usr/share/qcom/dsp/conf.d/

I'd suggest being simple: /usr/share/qcom/conf.d. The location is not just limited to DSP files. Sensors data or ACDB will also go to the same place, if I'm not mistaken.

[quic-parihar]

From the recent discussions, I’ve gathered the following:

1. We plan to maintain a conf.d directory to store the current DSP configuration files, as well as any future ones.
2. The use of multiple .ini/.conf files is intended to support scenarios where users may source DSP binaries from different locations.

Thinking that instead of reading all the conf files from conf.d directory for configuring dsp bin paths, shouldn't we have a centralized place like we have for linux dsp firmware binaries which tells us where to read the bins from. In this case I was thinking a file or environment variable which tells which conf/ini file to refer for configuring dsp bin paths. @lumag need your inputs on this.

[lumag]

Thinking that instead of reading all the conf files from conf.d directory for configuring dsp bin paths, shouldn't we have a centralized place like we have for linux dsp firmware binaries which tells us where to read the bins from. In this case I was thinking a file or environment variable which tells which conf/ini file to refer for configuring dsp bin paths. @lumag need your inputs on this.

Having an environment variable would defeat the purpose of conf.d. You have to read all config files and select the correct entry.

[quic-parihar]

Thinking that instead of reading all the conf files from conf.d directory for configuring dsp bin paths, shouldn't we have a centralized place like we have for linux dsp firmware binaries which tells us where to read the bins from. In this case I was thinking a file or environment variable which tells which conf/ini file to refer for configuring dsp bin paths. @lumag need your inputs on this.

Having an environment variable would defeat the purpose of conf.d. You have to read all config files and select the correct entry.

If there are two .conf files—one provided by Qualcomm and the other uploaded by the user—and both contain a machine entry for "Qualcomm QCS9100 Ride Rev3" defining different paths, how can we determine which file specifies the correct path for that entry?

[lumag]

Thinking that instead of reading all the conf files from conf.d directory for configuring dsp bin paths, shouldn't we have a centralized place like we have for linux dsp firmware binaries which tells us where to read the bins from. In this case I was thinking a file or environment variable which tells which conf/ini file to refer for configuring dsp bin paths. @lumag need your inputs on this.

Having an environment variable would defeat the purpose of conf.d. You have to read all config files and select the correct entry.

If there are two .conf files—one provided by Qualcomm and the other uploaded by the user—and both contain a machine entry for "Qualcomm QCS9100 Ride Rev3" defining different paths, how can we determine which file specifies the correct path for that entry?

Let user decide. Introduce some logical order (e.g. alphabetical) and then the later definition "wins".

[quic-parihar]

Thinking that instead of reading all the conf files from conf.d directory for configuring dsp bin paths, shouldn't we have a centralized place like we have for linux dsp firmware binaries which tells us where to read the bins from. In this case I was thinking a file or environment variable which tells which conf/ini file to refer for configuring dsp bin paths. @lumag need your inputs on this.

Having an environment variable would defeat the purpose of conf.d. You have to read all config files and select the correct entry.

If there are two .conf files—one provided by Qualcomm and the other uploaded by the user—and both contain a machine entry for "Qualcomm QCS9100 Ride Rev3" defining different paths, how can we determine which file specifies the correct path for that entry?

Let user decide. Introduce some logical order (e.g. alphabetical) and then the later definition "wins".

@lumag I understand the purpose of the conf.d directory — we're using it to manage multiple configuration files for different purposes.

However, I'm still unclear about the need to read multiple .ini files specifically for configuring dsp search paths for loading firmware libs. If a user wants to define custom paths for firmware libraries, can't they simply modify the existing machine_dsp_paths.ini file or replace it with a new file of the same name?
The only challenge I see is reverting to the default paths defined in the original .ini file. But even that can be handled by keeping a backup of the original before making changes.
Introducing multiple files for dsp search paths and relying on users to maintain them in a particular order might lead to unnecessary complexity and confusion.

Let me know what I am missing to align on same page with you.

[lumag]

Thinking that instead of reading all the conf files from conf.d directory for configuring dsp bin paths, shouldn't we have a centralized place like we have for linux dsp firmware binaries which tells us where to read the bins from. In this case I was thinking a file or environment variable which tells which conf/ini file to refer for configuring dsp bin paths. @lumag need your inputs on this.

Having an environment variable would defeat the purpose of conf.d. You have to read all config files and select the correct entry.

If there are two .conf files—one provided by Qualcomm and the other uploaded by the user—and both contain a machine entry for "Qualcomm QCS9100 Ride Rev3" defining different paths, how can we determine which file specifies the correct path for that entry?

Let user decide. Introduce some logical order (e.g. alphabetical) and then the later definition "wins".

@lumag I understand the purpose of the conf.d directory — we're using it to manage multiple configuration files for different purposes.

However, I'm still unclear about the need to read multiple .ini files specifically for configuring dsp search paths for loading firmware libs. If a user wants to define custom paths for firmware libraries, can't they simply modify the existing machine_dsp_paths.ini file or replace it with a new file of the same name?

Why would they? It's a config file owned by the corresponding package. When you apt upgrade it next time, the config file will be replaced by the one coming from the package. Likewise on a rootfs there might be multiple sets of files, installed from several sources. Each of those sets will carry its own config file.

The only challenge I see is reverting to the default paths defined in the original .ini file. But even that can be handled by keeping a backup of the original before making changes. Introducing multiple files for dsp search paths and relying on users to maintain them in a particular order might lead to unnecessary complexity and confusion.

"Have a possibility, not the policy". We do depend on users maintaining a lot of stuff in a proper order: udev rules, systemd unit files, apt configuration, etc.

Let me know what I am missing to align on same page with you.

Don't think about the users as dummies.

[quic-parihar]

Based on the internal and discussion we have had on this PR, proposing the following:
We expect packages to bring in their own config files whether it be hexagon-dsp-binaries or some other package.
Will be parsing /usr/share/qcom/conf.d/ directory to read all the .conf files present in lexicographical order. Picking the latest dsp_lib_path.

config file property will be DSP_library_path=<path_value>

For backward compatibility will also maintain a default config file with dsp_lib_paths configured in fastrpc project.

[lumag]

Based on the internal and discussion we have had on this PR, proposing the following: We expect packages to bring in their own config files whether it be hexagon-dsp-binaries or some other package. Will be parsing /usr/share/qcom/conf.d/ directory to read all the .conf files present in lexicographical order. Picking the latest dsp_lib_path.

config file property will be DSP_library_path=<path_value>

Relative to /usr/share/qcom, please.

For backward compatibility will also maintain a default config file with dsp_lib_paths configured in fastrpc project.

Why? And how? The fastrpc project provides no DSP libs.

[quic-parihar]

Based on the internal and discussion we have had on this PR, proposing the following: We expect packages to bring in their own config files whether it be hexagon-dsp-binaries or some other package. Will be parsing /usr/share/qcom/conf.d/ directory to read all the .conf files present in lexicographical order. Picking the latest dsp_lib_path.
config file property will be DSP_library_path=<path_value>

Relative to /usr/share/qcom, please.

Sure

For backward compatibility will also maintain a default config file with dsp_lib_paths configured in fastrpc project.

Why? And how? The fastrpc project provides no DSP libs.

I’d like to revise my earlier comment. Initially, I was considering a scenario where the fastrpc project is updated to include configuration changes, but the hexagon-dsp-bins package is still on an older version that doesn’t include the config file. I realized that even today that fastrpc project does not support /usr/share/qcom dsp search paths by default, rather the test apps create symlink to the DSP libraries.

One question we’ve been discussing is how to manage the conf.d path across different platforms, such as Linux and Android. Should we always assume a default path like /usr/share/qcom/ for Linux-based projects and /vendor/... path for Android in our code? Is that correct or we have any other way to do that.

[lumag]

One question we’ve been discussing is how to manage the conf.d path across different platforms, such as Linux and Android. Should we always assume a default path like /usr/share/qcom/ for Linux-based projects and /vendor/... path for Android in our code? Is that correct or we have any other way to do that.

Let users specify the path via the configure's script arg instead of hardcoding it to Linux vs Android

[quic-parihar]

The changes have been made as discussed and marked complete. @lumag @basak-qcom please have a look.