ESP32 Forum

i just got an esp eye and am having a bit of an issue trying to get face detection and video to work as expected. the demo app that came on it did work correctly up to it not detecting any faces. So no boxex show up around any faces. i then setup the toolchain and installed the recognition solutions example and that is when the video in the face stream fliped upside down and still has not detected any faces. some insight to why the stream is now rotated upside down and why it has not detected a face yet would be appreciated.

Hi,
The default example in the esp-eye has flipped the image in advance, for the supplied sensor being upside down. So if you install the example in the esp-who, you should do the following operation: For the detection, what is the distance between your face and the camera ? It might be better at about 30-40 cm.

I have tried an array of distances between my face and the camera between right up to it and across a room and everything in between. I installed camera_web_server example and when i started the stream it seemed to be running fine. But when i enabled recognition and detection this time i would get a box popping up around my face for a quick second here and there but not consistently. Then after a couple minutes of trying this the stream froze. to get it back i had to reboot the eye and it hasnt detected again. Also where would i add the above code in the example?

So i believe this is my main issue. It shows in the serial data when the stream disconnects:

CORRUPT HEAP: multi_heap.c:165 detected at 0x3ffc6b88
abort() was called at PC 0x40092ebf on core 0
0x40092ebf: multi_heap_assert at /home/zenn1999/esp/esp-who/esp-idf/components/heap/multi_heap.c:380
(inlined by) assert_valid_block at /home/zenn1999/esp/esp-who/esp-idf/components/heap/multi_heap.c:165

Looks like its also an idf bug. Is there a workaround or a fix?

Hi,
I'm a little confused about the uploaded image. Is the version with or without vflip setting ? And with this example, you can easily do it with a simple button.

And for the corrupt thing, can you post more info especially the lines after the one you posted ?

zenn1999 wrote: ↑

Sat Apr 06, 2019 12:07 am

i just got an esp eye and am having a bit of an issue trying to get face detection and video to work as expected. the demo app that came on it did work correctly up to it not detecting any faces. So no boxex show up around any faces. i then setup the toolchain and installed the recognition solutions example and that is when the video in the face stream fliped upside down and still has not detected any faces. some insight to why the stream is now rotated upside down and why it has not detected a face yet would be appreciated.

Hi. Phew, I'm glad (but sad too) that someone else is having the issue with face detection/recognition. I'm not having an issue with the stream rotated to upside-down, I have it with the USB cable at the top (unless that's the wrong way) - but now knowing how to flip it is handy too

The face detection/recognition issue though sounds exactly like what I'm getting. I get a great stream, all the controls seem to work as expected, but I get no coloured rectangle. I also get the same thing as you with the stream freezing after a minute or two (or three). No errors in compilation, so can't blame anything there.

Not sure where to put the blame nor how to diagnose the two issues. Even if we get the boxes working, then there's still the freezing to tackle (groan).

So yes i did later see the vflip switch in the camera_web_server example. But I would like to add it to the recognition solution code as holding the esp eye with the usb pointing up, with a short usb cable, causes it to be held at an angle which I am trying to eliminate as the problem with recognition. I tried recognition with vflip enabled on the camera_web_server example and thats the only real time I got any detection. The box would only appear for at most a second and I would move the camera and turn my head back and forth but I couldn't get any consistent detection. But I also have more output from the serial output after a crash:

CORRUPT HEAP: multi_heap.c:165 detected at 0x3ffc6b88
abort() was called at PC 0x40092ebf on core 0
0x40092ebf: multi_heap_assert at /home/zenn1999/esp/esp-who/esp-idf/components/heap/multi_heap.c:380
(inlined by) assert_valid_block at /home/zenn1999/esp/esp-who/esp-idf/components/heap/multi_heap.c:165


Backtrace: 0x4009386c:0x3ffd1700 0x40093a95:0x3ffd1720 0x40092ebf:0x3ffd1740 0x400931cb:0x3ffd1760 0x400856e2:0x3ffd1780 0x40085b89:0x3ffd17a0 0x4000bec7:0x3ffd17c0 0x40136cb7:0x3ffd17e0 0x40136d37:0x3ffd1800 0x40136d7f:0x3ffd1820 0x40138e4a:0x3ffd1840 0x4013900c:0x3ffd1860 0x40133acc:0x3ffd1880 0x4009032d:0x3ffd18b0
0x4009386c: invoke_abort at /home/zenn1999/esp/esp-who/esp-idf/components/esp32/panic.c:707

0x40093a95: abort at /home/zenn1999/esp/esp-who/esp-idf/components/esp32/panic.c:707

0x40092ebf: multi_heap_assert at /home/zenn1999/esp/esp-who/esp-idf/components/heap/multi_heap.c:380
(inlined by) assert_valid_block at /home/zenn1999/esp/esp-who/esp-idf/components/heap/multi_heap.c:165

0x400931cb: multi_heap_free_impl at /home/zenn1999/esp/esp-who/esp-idf/components/heap/multi_heap.c:380

0x400856e2: heap_caps_free at /home/zenn1999/esp/esp-who/esp-idf/components/heap/heap_caps.c:130

0x40085b89: _free_r at /home/zenn1999/esp/esp-who/esp-idf/components/newlib/syscalls.c:42

0x40136cb7: mem_free at /home/zenn1999/esp/esp-who/esp-idf/components/lwip/lwip/src/core/mem.c:151

0x40136d37: do_memp_free_pool at /home/zenn1999/esp/esp-who/esp-idf/components/lwip/lwip/src/core/memp.c:231

0x40136d7f: memp_free at /home/zenn1999/esp/esp-who/esp-idf/components/lwip/lwip/src/core/memp.c:231

0x40138e4a: sys_check_timeouts at /home/zenn1999/esp/esp-who/esp-idf/components/lwip/lwip/src/core/timeouts.c:381

0x4013900c: sys_timeouts_mbox_fetch at /home/zenn1999/esp/esp-who/esp-idf/components/lwip/lwip/src/core/timeouts.c:433

0x40133acc: tcpip_thread at /home/zenn1999/esp/esp-who/esp-idf/components/lwip/lwip/src/api/tcpip.c:483

0x4009032d: vPortTaskWrapper at /home/zenn1999/esp/esp-who/esp-idf/components/freertos/port.c:435


Rebooting...

So I deleted my old ESP-WHO and downloaded the latest ESP-WHO (as at today). Saw there was a couple of changes and thought I'd see if anything helped.

I also re-did the "menuconfig" part and cleared all cameras except the OV2640, disabled the SPI option there too and made sure "partitions" was using the "partitions.csv" file.

Re-compiled, fired up FF and waited a bit (seems an option gets culled after a few seconds). Then took a snapshot of a bright photo (woman with dark here, light complexion, against a bare and light background) , clicked "Start Stream" and then enabled "Face Recognition". Immediately saw the yellow bounding box around the face on the photo. Pointed the camera at me, nothing

I think this all means I have to brighten my face with more light and, because I'm a a balding grey-beard, I will also need to ensure a definite outline of my facial feature (maybe experiment with backgrounds).

I still am getting the occasional freezes requiring anything from a reboot to a full power cycle.

I finally got a little time to play with the eye more. I installed the camera web server as a client on my network and logged in to the camera web server.I installed the camera stationary with a longer cable, so it would be level and still. and i turned a flashlight toward my face at different angles.... FINALLY I am an intruder!!!! I guess i blend into the background well. And so far I havent had to reboot the eye due to it freezing. I had a theory that more light and stability was what i needed.