Possible to run on non mobile/laptop qualcomm platforms?

[UtsavBalar1231]

I am curious if this can be ran on qualcomm iot platform such as qcs6490 or qcs8250 platforms

[SorryMaker2022]

TL;DR: Theoretically, PowerServe cannot be deployed directly to these IoT devices but it is possilbe to run it after a hardware-specific patch.

The core issue of running PowerServe on other types of devices is the hardware version of HTP or DSP (the official name of Qualcomm NPUs).

• Snapdragon 888: HTP v68
• QCS8250: DSP v66
• QCS6490: Specifications not found, but it likely uses either v66 or v68, inferred from the product number.
  (Note that Qualcomm IoT / Automotive SoC series are usually using NPUs 2-3 generations behind their high-end mobile SoCs)

Currently, PowerServe is designed for HTP v75 Snapdragon 8Gen3 and HTP v79 Snapdragon 8 Elite SoCs. I have tried adapting it to the Snapdragon 888. There are several modifications required, both in the code and the deployment process, before the NPU inference part can run correctly (note that the speculative decoding feature has not been verified yet). These include manually fetching the Hexagon SDK from Qualcomm's website, banning some unsupported features and adjusting the model converter settings. You can refer to Qualcomm documentation (though be aware it is not always accurate) and ask questions on their support forum for further assistance.

At this time, the PowerServe team is understaffed and does not have access to the two SoCs you mentioned. We don't currently have plans to add official support for these SoCs, but we believe it's feasible to run a modified version of PowerServe on them with the appropriate changes.

We sincerely welcome contributions from the community to help support additional hardware platforms. If you are able to work on a patch or help us with testing, it would be greatly appreciated! Please feel free to reach out if you have any questions or need guidance.