Building a new classifier for the ESP WHO

[caleb221]

Hello!
I have been reading through the open sourced documentation on GitHub as well as the libraries that have been developed.
I understand that the MTMN face detection model is being used on the ESP32 and it is specially designed for facial recognition.

My question would be what were the steps taken in implementing this model in order to make it fit onto the ESP32?

I would like to create another model using mobilenets v2 and MTCNN, but I am unsure as to how to port this onto the ESP.

If anyone could help I would appreciate it a lot!
Thanks!

[ESP_GXC]

When you have finished training the model with some popular frameworks like tensorflow, you can then implement the inference process to the ESP related devices. The model structure is easy to implement as we have provided some common operations and you can also rewrite what you need. The model coefficient conversion might be a hard work to do, but still can be done when understanding the format of data. The `dl_matrix3d_t` in the library is channel major layout, a.k.a NHWC. You can write a script to generate a header file that contains the coefficients in `dl_matrix3d_t` structure for GCC compiler.
In the first step, I suggest you to try float point solution to get the right functionality, then try to use fixed point solution to accelerate the process.

[caleb221]

Ok Great! Thank you!
Also, i appreciate the quick reply
Thanks again!
-Caleb

[geekboood]

Hi, I encounter a problem when I try to use the DL lib. Can you take a look at this issue?
https://github.com/espressif/esp-who/issues/106

[riccardo.brue]

When you have finished training the model with some popular frameworks like tensorflow, you can then implement the inference process to the ESP related devices. The model structure is easy to implement as we have provided some common operations and you can also rewrite what you need. The model coefficient conversion might be a hard work to do, but still can be done when understanding the format of data. The `dl_matrix3d_t` in the library is channel major layout, a.k.a NHWC. You can write a script to generate a header file that contains the coefficients in `dl_matrix3d_t` structure for GCC compiler.
In the first step, I suggest you to try float point solution to get the right functionality, then try to use fixed point solution to accelerate the process.

Can you please give more details about how to converte a trained tensorflow model in order to be used on the ESP-32? Thanks.

[caleb221]

If anyone is familiar with this framework, I am currently testing an implementation of MTCNN.
I am at this point with the code and am having a NaN issue.
more details explained here:
https://github.com/espressif/esp-face/issues/17
Thanks!
-Caleb