ESP32-CAM ov2640 exposure control/FREX mode

[HermannSW]

Yesterday I learned how to add new features to ESP32-CAM CameraWebServer example sketch, and provided steps and needed tools:
https://www.esp32.com/viewtopic.php?f=1 ... 507#p45507

I added a "Flash" toggle


allowing to capture with flash off and on:



Today I learned how to do the equivalent of Raspbian pigpio waveforms at µs precision with ESP32 using the RMT component. Find description and sketch RMTWrite.ino here:
https://www.raspberrypi.org/forums/view ... 7#p1487157



Now that the basics are there (cable soldered allowing access to ov2640 B2 FREX pin, adding features to CameraWebServer and µs precision waveform generation with ESP32), integrating all together and building global external shutter solution for ESP32-CAM is next ...

[HermannSW]

I had the GUI and did know how to do the "shot" captures based on streaming video and not based on still images yesterday:
https://www.esp32.com/viewtopic.php?f=1 ... 710#p45710

Today I solved the issues left, biggest was to realize that doing "window.stop" to stop MJPEG streaming did show the last captured frame nicely only with Firefox (sorry Chrome).

I had learned earlier how to do µs precision waveforms with ESP32 RMT component.

Today I did wire all together, and built complete "miniature global external shutter" (I started with complete room dark with v1 camera, then reduced to a dark moving box, and now I ended up with a 22cm×16cm×11cm cardboard box):


This is the inside, just ESP32-CAM module and fast rotating mini drone propeller:


I still did not make use of the cable soldered to flex cable connector pin10 of ESP32-CAM module, so no global shutter videos yet. But a single shot scenario with rolling shutter does work already, with these menu settings:


Yesterday I thought that I do not need the offset input, today I realized that I do need it, but had to change its unity from µs to ms, I will explain below what that value is used for.

And this is the 800x600 frame captured with above settings (after many tries, because I have no mechanism yet to sync the timings to frame start (that is likely to come for free with FREX mode later). It stayed after having pressed doit button, that did a 550µs length flash with GPIO4 connected ESP32-CAM internal flash led only(!). Mini drone propeller normally get powered from 3.7V lipo, but powering from laptop USB with 5V does not do harm besides just rotating faster (more than 27000rpm instead 20000rpm):


Before describing the new features implementation in more detail, we can use this frame for propeller rpm determination. The 550µs flash length lights roughly 1/4 of a full rotation. Speed is therefore 1000000/(550µs*4 )= 454rps = 27272rpm (that is 48.5m/s or 174.6km/h radial speed at blade tips).


The changes to index.html are:

• adding a doit button and changing offset unity from µs to ms
• make "function updateConfig (el)" correctly deal with type="number" input
• hide "Flash" group when enetering any global external shutter mode
• add doit button JS running in browser, doing synchronous sleep for offset milliseconds before stopping stream

  Code: Select all

  +  const doitButton = document.getElementById('ges_doit')
  +  const ges_offset_value = document.getElementById('ges_offset')
  +
  +  function sleep(ms) {
  +    return new Promise(resolve => setTimeout(resolve, ms));
  +  }
  +
  +  async function doit(ms) {
  +    updateConfig(doitButton)
  +    await sleep(ms)
  +    stopStream();
  +  }
  +
  +  doitButton.onclick = () => {
  +    doit(ges_offset_value.value)
  +  }
  +
  

The changes to app_httpd.cpp are:

• adding only "shot" mode for doit button for now
• use an array of two rmt objects, one for GPIO4, the other for GPIO13

The complete changes can be found in this (not that big) commit:
https://github.com/Hermann-SW/Raspberry ... 8da9cce6cd

[HermannSW]

Any try to attach an ISR (interrupt service routine) to GPIO25 (VSYNC_GPIO_NUM for ESP32-CAM module) resulted in FCameraWebserver starting, connecting to WLAN and emitting connected messages to Serial, and then immediately rebooting.

So I ended up doing poor mans copy of GPIO25 to GPIO16 in order to see VSYNC signal on 100Msps logic analyzer, added these lines as last lines in FCameraWebServer.ino setup():

Code: Select all

  pinMode(16, OUTPUT);
  for(;;) { digitalWrite(16, digitalRead(VSYNC_GPIO_NUM)); }

This is what logic analyzer shows, inverted signal compared to figure 15 of ov2640 datasheet, for 1620x1200:


Now I will try to add a function to app_httpd.cpp that can be called from JS in index.html to wait for start of next frame (VSYNC low) and return then. Hopefully that will give more recreatable flash experiments.

[HermannSW]

Added and wired in new "Wait for VSYNC" toggle:
https://github.com/Hermann-SW/Raspberry ... a24bdd7851

Gives recreatable results most of the time -- will have to understand why not always.
I was victim of "compiler optimizing away stuff" (until I prevented by printing the loop cnt value):

[HermannSW]

I learned how to pass vsync pin define from FCameraWebServer.ino to app_httpd.cpp, just store into global variable.

And I eliminated need for "Serial.printf()" of count variable, just declaring it volatile prevents compiler optimizing code away.

Finally I made sure that now more than one full frame will be skipped, and it gives greatly more recreateable good behavior. Still not always, though. Done with last commit, this is better "wait for vsync" code:
https://github.com/Hermann-SW/Raspberry ... 9b68c65c72

Code: Select all

            volatile int cnt = 0;
            while (digitalRead(pin_vsync) != 0) { // skip current frame
                ++cnt;
            }
            while (digitalRead(pin_vsync) == 0) { // skip vsync
                ++cnt;
            }
            while (digitalRead(pin_vsync) != 0) { // skip next frame
                ++cnt;
            }
            while (digitalRead(pin_vsync) == 0) { // skip vsync
                ++cnt;
            }
            // Serial.printf("Waited for VSYNC: '%d'\n", cnt);

Next step is to try for global external shutter capture utilizing the cable soldered to pin10 of camera flat ribbon cable connector.

[HermannSW]

I am baffled, just played with the last version of FCameraWebServer and reduced builtin camera flash time to only 50µs:


I took 800x600 frames as before and got this frame without haveing changed code nor having done anything with cable soldered to pin10 of ov2640 flat ribbon cable connector:


Yes, the frame is dark (hey, the little internal flash light was on for 50µs(!) only).
And I could connect the 5000lm led tiggered by MOSFETs and powered by 50W led driver to GPIO13 for much more light.
But there is absolutely NO rolling shutter effect -- this seems to be my first ov2640 global external shutter frame.
And the propeller did rotate with more than 400 rotations per second while the frame was captured.

The (not connected!) cable soldered to pin10 must have something to do with this.
Because for comparison, this is image taken with ESP32-CAM ov2640 sensor from very first posting in this thread, where mini drone propeller did rotate with minimal speed possible (just measured as 45rps), and there rolling shutter effect did show extremely:


Since my second ESP32-CAM module does not allow to capture MJPEG stream nor single frames after too much soldering at pin10, I will have to wait for new module ordered to arrive on Tuesday ...

[HermannSW]

I determined that VSYNC signal (GPIO25 per camera_pins.h) is exposed on pin18 of ov2640 flat ribbon cable connector. I did take a logic analyzer capture of CIF mode streaming (400x296) because that was discussed in datasheet. Again I used Raspberry camera as big magnifying glass (HDMI monitor) in order to get reading for 0.5mm spaced pin18:


This is what logic analyzer showed:


Besides that VSYNC signal is inverted compared to datasheet, the timings wrt t_p and t_LINE in datasheet do not match what I measured. So [F]CameraWebServer example does different timings.
Min/Max values determined from exported data are (0.03549481/0.03549822) and (0.00448625/0.00448966) seconds.
While VSYNC frequency is 25.01Hz, the reported framerate for 400x296 on serial console is only 12.7fps on average.
Only when going to smallest CIF (sub-)resolution 160x120 reported average in seral console is 25.6fps.

The plan was to capture vsync for the two other native ov2640 resolutions as well (800x600, 1600x1200). But I did shorten some pins on the connector accidentally and that had bad effects. First after rebooting ESP32-CAM only "Get Still" did work, later streaming did work, but not always. So I will not do more measurements while module is up and running.

[HermannSW]

Today I finished "shots" and "5shots" implementation in addition to "shot" in doit:
https://github.com/Hermann-SW/Raspberry ... 0d5bff3655

I tested that all three effects work on GPIO4 (connected to builtin ESP32-CAM flash light) as well as GPIO13 with logic analyzer.

And I did capture first multi exposure frame with these menu settings (4 flashes with GPIO4 builtin flash light, 50µs on and 200µs apart):


And here you see the result, an even better proof that ov2640 sensor did global external shutter capture. You see each propeller position without any rolling shutter effect. And you see 4 different positions, proving that propeller was rotating:


Another slightly more than 200µs and the next propeller capture would overlay the first, That means than 4*(50µs+200µs)=1000µs=1ms is slightly less than needed time for half rotation of propeller. So rotations per second (rps) are slightly less than 1000/2=500 (in fact 454rps as we have calculated in previous posting).

I really like above multiple exposure frame -- because it was done with the tiny builtin ESP32-CAM flashlight and not with 5000lm led triggered via GPIO13!

I am really excited for the two ESP32-CAM modules I ordered on Saturday to arrive (hopefully tomorrow), so that I can figure out what exactly is responsible for being able to capture global external shutter frames (is it really the cable soldered to pin10 of ESP32-CAM connector, that is not connected to anything?) ...


P.S:
I really have to update github README.md, until then, this is the "working" global external shutter setup (dark inside):


Opening the cardboard shows that not much is inside:


This near shot shows the details, mini drone propeller in left 3rd hand, ESP32-CAM in right 3rd hand, cable soldered to pin10 of ESP32-CAM connector is NOT connected, 3rd hand base is on top of mini breadboard to reduce the noise when propeller rotates with 27272rpm:

On left border of ESP32-CAM module you can see the tiny yellow builtin flashlight that is enough for capturing simple multiple exposure global external shutter frames. The logic analyzer is still connected to GND and GPIO13 from testing new shots and 5shots effects implementation done before.

[HermannSW]

Today the two newly ordered ESP32-CAM modules arrived.
Both had Wifi issues, although I did not change location.
Adding external Wif antenna fixed the issue:



Both modules work like the module I did solder a cable to pin10 of connector.
It was just a little more difficult to find working offsets.
Here is a capture similat to the multiple exposure capture done yesterday:

Some cables may be where they should not be, I think that is the explantion for the disturbing horizontal lines.

But as you can see the new modules can take global external shutter frames, without any change or soldering!

So I went back to what I did in the very first posting of this thread and did redo exactly what I described there.
Still I can see the strong rolling shutter effect (propeller shows up disconnected top left as well):


That was done with "AEC Sensor" off. I did turn it on again and got this differently looking rolling shutter effect:


Then I turned back to first setup and increased propeller motor voltage from minimal to 3.3V and got this extreme rolling shutter effect:



Summary:

• untouched ESP32-CAM modules can capture global external shutter frames(!)
• under normal conditions they still show rolling shutter behavior

So what is the explanation for the different behavior?
I don't have one yet.
The multiple exposure frame is proof that "somehow" a global reset had happened, otherwise the only 50µs exposures would not show up in lower lines.

[HermannSW]

I thought that now I do not need the soldered cable to pin10 of connector anymore.
But I did use it now once, I connected logic analyzer to ESP32-CAM GND and the cable soldered to pin10.
The idea was to see whether pin B2(FREX) gets changed while doing successful global external shutter captures.
I would have had no explanation besides "darkness" inside the small cardbox for such a signal though.

I did 6 consecutive global shutter captures and did record pin10 signal with logic analyzer.
During the whole 40s of recording logic analyzer showed all low signal.
So there was no FREX mode triggering signal on sensor pin B2.

Therefore still no explanation on why global external shutter captures do work although normal behavior is rolling shutter.