ESP32-CAM ov2640 exposure control/FREX mode

[HermannSW]

I was made aware that FREX pin is NOT brought out to the 24pin connector.
I looked into what I measured, and found a ov2640 datasheet with connector pins described in figure 1:
https://www.mpja.com/download/ov2640data%20sheet.pdf

The first pin I measured at beginning of the thread was Y7, and that was connected to pin9.
But that was my own pin numbering, and it turned out that I used reverse order, and it actually was pin 24+1-9=16, which is DATA7.

So pin10 in my numbering is pin15 in datasheet, and that is DGND, the digital ground pin.
So what I measured was that pin B2 (FREX) seems to be pulled down to DGND.

Bad news is that accessing FREX pin B2 is not possible via the flat ribbon cable connector.

Good news from the last days is that a global reset can be consistently done as the global external shutter captures proved.

In another ov2640 datasheet it says "Supports LED and flash strobe mode":
https://www.uctronics.com/download/cam_ ... 2640DS.pdf

But I measured flat ribbon connector pin1 (STROBE) with Arduino IDE Blink sketch, and pin1 does not change.

[HermannSW]

My working hypothesis was that global reset of all sensor lines needed for global external shutter frame captures was somehow triggered by a very dark image.

So I just opened the cardboard box providing the darkness more and more, until it was open 10cm high on one side:


Interestingly I was able to still get global external shutter frame captured although now quite some parts of frame are bright:


Capture was done with these global external shutter settings (suffix of http://192.168.178.156/status):

Code: Select all

...,"ges":2,"ges_gpio":0,"ges_repeat":5,"ges_vsync":0,"ges_offset":190,"ges_on":100,"ges_off":1500}

Just for comparison, from a few postings back, quite some rolling shutter effect of propeller rotating at same minimal speed (mini drone motor powered with 0.30V instead of 3.7V):

[HermannSW]

Wow, it took some tries, but I was able to get several global external shutter captures after I completely removed the cardboard box covering the scene So it is not darkness that determines rolling vs. global shutter capturing ... (in this frame you can see the ESP32-CAM module builtin flash light active in the mirror):

[HermannSW]

To get an idea of why the camera behaves as it does, decoding the I2C traffic between ESP32 and ov2640 sensor would be beneficial. I did start my work on enabling high framerate video capturing (from 90fps/120fps for v1/v2 Raspberry camera to 750fps/1007fps) with doing exactly that. I did solder big balls of tin-solder onto test pads PP39 and PP40, and with those capturing I2C traffic with logic analyzer was easy:
https://www.raspberrypi.org/forums/view ... 3#p1237647


I recently did order 10 spring test probe tips from aliexpress (bottom right):
https://www.aliexpress.com/item/32819065349.html

I could use sharp knife and remove top plastic of pin3 (SDIOD) and pin5 (SDIOC) lines as I did before with FREX line (top right), or solder two more cables to pin3 and pin5 of ESP32-CAM connector as I did with pin15 before (bottom left):

[HermannSW]

Just realized that the ESP32-CAM module I most often worked with did have some heat issue with the flash and got black:


Luckily it was recoverable with a simple knife:


I am pretty sure that the multiple exposure captures I did with each exposure in µs range, or in ms range maximally, cannot be the cause of this. It seems that the issue was caused by using the Flash toggle and keeping it on for too long time -- so be careful when using Flash feature in FCameraWebServer!

[HermannSW]

I did use "shots" tool again with these settings:

Code: Select all

...,"ges":2,"ges_gpio":0,"ges_repeat":3,"ges_vsync":0,"ges_offset":190,"ges_on":100,"ges_off":1000}

This time I had the exact same scene to capture (only the hand moved slightly between the two captured frames). Depending on when I did press "doit" button, the generated three 100µs flashes 1000µs apart got captured in the frame or not. When the flashes got captured, result was global shutter frame. Otherwise result was rolling shutter frame -- that is what I thought when doing these two captures.

Successful global external shutter frame:


Rolling shutter frame:


Later I increased the light onto propeller so that rolling shutter effect was not just a blur, but many horizontal lines as shown few postings before. That frame was done when propeller motor was powered with 5V and had 27272rpm. The battery I used here (not at home, cannot generate 0.3V for slower rotation here) has 1.5V, so rpm value should be much less than 27272rpm. I was able to capture the following frame after many experiments. On top it shows the rolling shutter effect with the many horizontal thin lines, but then there is a not so small range where you can see the global shutter capture of propeller at two positions. So ov2640 seems to do rolling/global shutter depending on the amount of light per region(!):

[HermannSW]

Current state of ov2640 global shutter experiments is, that soldering cable to ESP32-CAM flat ribbon cable connector was not necessary.
In addition, ov2640 sensor allows for:

• some kind of "implicit" global shutter mode
• global shutter capturing at daylight without external shutter
• mixed mode global+rolling shutter frames

I added new section "TODOs+ideas" to github repo, also related to Raspberry cameras; for now:
https://github.com/Hermann-SW/Raspberry ... todosideas
TODOs+ideas

• fully understand how+why (implicit) global shutter capturing for ov2640 works without being triggered explicitely
• Raspberry v1 camera ov5647 is successor of ov2640; does it have an implicit global shutter mode as well?
• Raspberry v2 camera Sony imx219 sensor is different; does it allow for implicit global shutter capturing as well?

[HermannSW]

Now that even daylight global shutter captures are possible, I wanted to add "frequency" feature allowing to specify frequency [Hz] and On [µs] and get permanent flashes of the specified length with the given frequency.

Just having started I thought that this can be done with "shots" option without any additional implementation, and did test with that.
I used this menu settings:


On and Off time together are 20ms, so this corresponds to 50Hz frequency.
Repeat is just a high number.
Offset is a number higher than the values normally needed, no problem since there are many flashes.

This is a frame captured with those settings. and it again shows effects I currently cannot explain.
There are 4 regions, up to line 202, up to line 370, up to line 538 and up to line 600.
370-202 = 168 = 538-370, but why is there no region border at 202-138 = 64?


P.S:
I made room dark and turned "Wait for next VSYNC" off, and now get consistent regions 168 lines high.
Region up to line 42, up to line 210, up to line 378, up to line 546 and up to line 600.
Obviously lines per region is dependent on brightness of the scene.
Just as reminder, propeller does roughly (27272/60 rps) / 50 Hz = 9.09 rotations between consecutive flashes:

[HermannSW]

I think I made a good capture, and a good observation as well.

This was the global shutter setup, 2 shots, 50µs flashes, 950µs apart. And offset 170ms:

Code: Select all

...,"flash":0,"ges":2,"ges_gpio":0,"ges_repeat":2,"ges_vsync":0,"ges_offset":170,"ges_on":50,"ges_off":950}

The observation I made is, that when pressing "doit" initiates the flashes, and then stops the value of offset milliseconds later (170ms here), the output in Arduino IDE Serial monitor tells exactly how long processing for the last frame took:

Code: Select all

20:27:33.636 -> MJPG: 19012B 153ms (6.5fps), AVG: 279ms (3.6fps), 0+0+0+0=0 0

It was a capture without covering box again,and it took quite some time to get external lighting so that the characteristic rolling shutter horizontal lines for fast rotating propeller did show in camera image in browser:


Since 170ms offset is longer than the 153ms of last frame capture, maybe an overlay effect of 2 frames might explain the shown mixed rolling shutter+global shutter frame ...

[HermannSW]

I looked into many (preliminary) ov2640 datasheets on the web, and all show this same SVGA Frame Timing diagram, which is inconsistent:


Diagram says that t_LINE is 1190 t_p, and summing up all components gives 866416 t_p. But 672×t_LINE is 799680 t_p, with a difference of 66736 t_p. So this diagram is of no help, as it is not clear what all is wrong.

I did measure that getting a typical 800×600 frame JPEG of size around 19KB from ov2640 (with "fb = esp_camera_fb_get();") takes between 100ms and 145ms in FCameraWebServer app_httpd.cpp.

Unfortunately this all does not allow to tell where in the lifecycle of the frame the flashes initiated immediately and taking 2×50µs+950µs=1.05ms happen (in previous frame capturing of 600 rows phase, or in the time after that). What is known is that the flashes for previous posting frame happened 170ms-153ms=17ms before the final frame started.