Where is this repo's API doc?

[WangShuoran]

When I use the test_app, I know that different algorithms have their own execution effects.

But when I what to use esp_nn_fully_connected_s8 opened in repo/test/src/fully_connected_test_c, I don't know those parameters mean. I know this is neural network's knowledge, but there are too many parameters to try one by one.

So I want to know the API parameters meaning for, or those parameters come from?

Thank you!

[vikramdattu]

Hi @WangShuoran there is only minimal description available for APIs: https://github.com/espressif/esp-nn/blob/master/include/esp_nn_ansi_headers.h#L141

I understand that, for someone who is completely new to this would find it not sufficient. The more descriptive docs do not exist simply because the kernels are only supported by tflite-micro repo.
Hence, I didn't find need to put efforts on this. I will, however, take this as a feedback and improve the docs.

I would encourage you to use https://github.com/espressif/tflite-micro-esp-examples repo for real life examples, which already uses esp-nn for optimized kernels. That way, it would be more useful and you could start with the practical examples.

[WangShuoran]

I don't have machine learning experience, but my colleague has a lot of work experience. I'm not good at understanding the concepts while he is not good at puting existing models on ESP32, and he doesn't have Github accounts.

Fortunately, after discussion, we got the following questions and hope to get your help. If you are willing to help us, we are also willing to improve the documentation for this warehouse.

1. Does esp_ nn_ fully_ connected_s8 function works to complete the training operation or obtain the training results through the trained w and b ?

2. If it is a training operation, is there any operation example to complete the calculation in ESP-NN?

3. We have trained the full connected network and predicted on pytorch and have obtained the number of network layers, the number of neurons in each layer, the learning rate and other parameters for nn model. We need to make data prediction on esp32, so we need to implement the predict processure in pure C language. How can I enter my parameters in your API? If this API is for calculation, how can I import the w and b of this API into this API?

[vikramdattu]

1. Does esp_nn_ fully_ connected_s8 function works to complete the training operation or obtain the training results through the trained w and b ?

The above function is used in an inference for tflite-micro models.

2. If it is a training operation, is there any operation example to complete the calculation in ESP-NN?

You may take a look at examples here: https://github.com/espressif/tflite-micro-esp-examples/tree/master/examples

3. We have trained the full connected network and predicted on pytorch and have obtained the number of network layers, the number of neurons in each layer, the learning rate and other parameters for nn model. We need to make data prediction on esp32, so we need to implement the predict processure in pure C language. How can I enter my parameters in your API? If this API is for calculation, how can I import the w and b of this API into this API?

You need to convert the model to tflite-micro format (i.e., quantized tflite model) model to be able to take the benefit of esp-nn, as it supports tflite APIs only.

The document here will give you a good idea about how you can train and convert the model: https://github.com/tensorflow/tflite-micro/tree/main/tensorflow/lite/micro/examples/person_detection

If you want to continue training in pytorch, you can easily find ways to convert that model to tflite.

[WangShuoran]

@vikramdattu Thank you for your quick reply. And sorry for I am so late.

I am understood your mean, and after looking at all the existing problems in this warehouse, I think if my colleague and I have not the develop tensorflow experience, look this repo is unwise.

And this repo is use for https://github.com/espressif/tflite-micro-esp-examples/tree/master/examples, I am search lot of resource with tensorflow with esp tensorflow.

I feel the esp tensorflow is google tensorflow transplant, so if I am use tensorflow to implementation model, I think this model is very quickly move to esp32, no problems in this process. Does my feel all right?

So I follow your advice and will use tensorflow.

[vikramdattu]

Hi @WangShuoran. Yes, you're right.
tflite-micro-esp-example is based on tflite-micro by google. The micro version is meant just for inferencing and it follows the specification. To train your model, as you rightly pointed, you should use TensorFlow and then convert that model to tflite-micro format and run on esp32 using espressif/tflite-micro-esp-examples repo.

[mkisho]

I also have a question regarding documentation, so I think I will use this issue to ask my question, instead of opening a new one.

I am trying make a similar implementation of fully connected neural network for research purposes, but I am struggling to find out how a certain parameter is calculated. After training a tflite model, I can see where most of the parameters of the esp_nn_fully_connected_s8_ansi function come from, as most of them are directly taken from the model (g.e. filter_data are the quantized weights), but that out_mult value I really can't understand. The tflite model provides a floating point scale value for each layer of the network, I think this out_mult parameter is probably related to it since it seems to be a scaling value, but with an integer representation for speeding up code execution. But where exactly does this out_mult value comes from? Is it calculated somewhere in the code after loading the tflite model? Or is it somewhere in the model itself?

Thanks for your help!

[WangShuoran]

@mkisho I'm very interested in what you said.

Let us wait @vikramdattu reply, and if you solve the problem, I may also learn a lot, and even improve the full connection layer documentation.

[vikramdattu]

Hi, @WangShuoran @mkisho sorry for the late reply.

It is true that tflite's quantization params are float values, but those are quantized by tflite-micro's prepare stage. This results in multiplier that you see as the parameter to fully_connected layer.

In fact, all other OPs also get these parameters.

For example, you may trace this function to get an idea how that's done: https://github.com/espressif/tflite-micro-esp-examples/blob/master/components/tflite-lib/tensorflow/lite/micro/kernels/fully_connected.cc#L38