Adding features to Examples->ESP32->Camera->CameraWebServer

[HermannSW]

Yesterday I did solder a cable to 0.5mm spaced pin10 of ov2640 flat ribbon cable connector on ESP32-CAM module. Via that cable I can control FREX pin B2 (one of 38) of ov2640 image sensor. I want to create global external shutter captures with ov2640 FREX mode:
https://www.esp32.com/viewtopic.php?f=1 ... 445#p45445

I did global external shutter captures extensively with Raspberry v1 camera FREX mode already. For that I created a set of tools that can be executed separately and/or together to achieve various effects:
https://github.com/Hermann-SW/Raspberry ... tter#tools

Since there is no shell on ESP32 module, I wanted to add the new FREX features to CameraWebServer example from ESP32 Arduino IDE. I had no idea what needed to be done, and decided to add a small feature first to find out which files(s) to change and which tools are needed. This posting will describe how I added a "Flash" checkbox to CameraWebServer applicaton of ov2640 sensor. It allows to turn the ESP32-CAM builtin flash on and off for lighting the captured scene:


CameraWebServer example directory under Linux can be found here:

Code: Select all

~/.arduino15/packages/esp32/hardware/esp32/1.0.2/libraries/ESP32/examples/Camera/CameraWebServer

There are 4 files:

• app_httpd.cpp — the WebServer app
• camera_index.h — contains index.html.gz as byte array for serving
• camera_pins.h — #defines for 16 camera GPIO pins needed for different ESP32 models
• CameraWebServer.ino — the CameraWebServer Arduino sketch

I did create subdirectory orig that contains unmodified copy of the files before my changes.
And subdirectory index that is used to decode the index.html webpage, modify it and gzip it again.

First I needed a tool that extracts index.html.gz from top section of camera_index.h.
Find tool index/d.c attached, as well as tool index/c.c for conversion of gzipped modified index.html into byte array.

After looking into the files I was surprised that adding new feature is so easy.
This is the diff for app_http.cpp, just add one more case to existing "cmd_handler()":

Code: Select all

$ diff -c2 orig/app_httpd.cpp app_httpd.cpp 
*** orig/app_httpd.cpp	2019-06-26 14:15:12.654361609 +0200
--- app_httpd.cpp	2019-06-26 15:54:14.682359257 +0200
***************
*** 521,524 ****
--- 521,529 ----
          }
      }
+     else if(!strcmp(variable, "flash")) {
+ #define LED_BUILTIN 4
+         pinMode(LED_BUILTIN, OUTPUT);
+         digitalWrite(LED_BUILTIN, atoi(value));
+     }
      else {
          res = -1;
$ 

And this is the (small !) diff for index.html:

Code: Select all

$ diff -c2 index/index.html.orig index/index.html
*** index/index.html.orig	2019-06-26 14:16:10.122873710 +0200
--- index/index.html	2019-06-26 14:20:54.560402494 +0200
***************
*** 535,538 ****
--- 535,545 ----
                              </div>
                          </div>
+                         <div class="input-group" id="flash-group">
+                             <label for="flash">Flash</label>
+                             <div class="switch">
+                                 <input id="flash" type="checkbox" class="default-action">
+                                 <label class="slider" for="flash"></label>
+                             </div>
+                         </div>
                          <section id="buttons">
                              <button id="get-still">Get Still</button>
$ 

That's it, loading CameraWebServer.ino into Arduino IDE, adjusting "ssid" and "password" to my WLAN, uncomment CAMERA_MODEL_WROVER_KIT define and commenting out CAMERA_MODEL_AI_THINKER, selecting "Tools->Partition Scheme" as "Huge App (3MB...", and then compiling and flashing to ESP32-CAM module!

This is screenshot with Flash off:


And this with Flash on:


The ESP32-CAM module is connected with USB2TTY adapter to laptop USB, and there seems to be not enough power for continuously on ESP32-CAM flash. As human I see a lot of flickering, a little can that be seen on short video taken with smartphone camera:
https://www.youtube.com/watch?v=aZberpm ... e=youtu.be


Now have fun adding your own features to CameraWebServer application ...

P.S:
I am really impressed of the ESP32-CAM flash brightness.
Android Light Meter app shows 25604 lux at 3cm distance between flash and smartphone:


P.P.S:
Partially broken ESP32-CAM module described in the other thread, flash on eliminates side shadows:

All screenshots are a bit smaller than 320x240 because "DCW (Downsize EN)" option was enabled.

[HermannSW]

Now that I am pretty sure that global external shutter for ESP32-CAM ov2640 sensor is doable, I created a new sub project on github:
https://github.com/Hermann-SW/Raspberry ... aWebServer

The two new tools get_index_html and put_index_html automate the whole process of extracting index.html for modification from camera_index.h and the reverse process for putting changed index.html into camera_index.h.

I did submit new "Flash" toggle feature and decided to do the modifications in 2nd ESP32->Camera example FCameraWebServer. Only one symbolic link needs to be created:

Code: Select all

cd ~/.arduino15/packages/esp32/hardware/esp32/1.0.2/libraries/ESP32/examples/Camera
ln -s ~/Raspberry_v1_camera_global_external_shutter/FCameraWebServer

This allows to have FCameraWebServer as github sub project (with its own README.md) and being able to use that in Arduino IDE as if it was installed with arduino-esp32 examples. README.md contains all current information and has some "tbd"s for next steps.

[HermannSW]

For adding the new group of "Global external shutter" elements I had to learn how to show/hide based on selected global external shutter mode selected. And I had to learn how to do type="number" inputs because until now there are none.
Because I wanted to have a diff of the different index.html versions and not only the diff of gzipped C byte arrays I first checked in the two versions of index.html sofar before, and modified [get|put]_index_html tools. Yes, that file is redundant, but having it in github diffs is invaluable in my eyes.
I just pushed the new version to github, still amazed how small the diffs are to get the complex set of gui elements with all the logic:
https://github.com/Hermann-SW/Raspberry ... a499b2b5cb
It was a very good choice to do my work based on CameraWebServer -- I learned so much from it.
On startup the only new input seen is "Global external shutter" with value None:


The other global external shutter modes (corresponding to the Raspbian v1 camera tools)
https://github.com/Hermann-SW/Raspberry ... ster/tools
show up like below, input showing/hiding dependent on the mode. I initially allow the 5000lm led to be mosfet driven by GPIO13 (default) or GPIO4 (ESP32-CAM builtin flash light). Both pins behave nicely on bootup of ESP32. While with Raspberry v1 camera I execute shot/shots/5shots tools at any time while the Pi records a (typically 1fps) video, for ESP32-CAM I will start based on "Get Still". For that a configurable time offset after starting frame capturing for the tool effects seems to make sense.







Now that the GUI is ready, next step will be to do real global shutter captures based on the cable soldered to pin10 of ov2640 flat ribbon cable connector on ESP32-CAM module. It seems to be fine to connect this cable to a fixed ESP32-CAM GPIO pin (to be selected), no GUI input needed for that. Progress will be added to the other thread:
https://www.esp32.com/viewtopic.php?f=1 ... 445#p45445

[HermannSW]

Perhaps the offset input can be removed, since I found out how to do what is needed based on streaming video rather than on still image.

This is current code for closeButton, that is used to stop streaming:

Code: Select all

  closeButton.onclick = () => {
    stopStream()
    hide(viewContainer)
  }

I did comment out the hide command and got exactly what is needed:
Video gets (mjpeg) streamed until closeButton is pressed, then streaming stops, BUT last frame keeps being displayed!


Even more, while the displayed size of that frame does not match the capturing dimension, right clicking the image and doing "copy image" and then in gimp "create image from clipboard" gets image with exactly the size requested (eg. 1600x1200).


Next I added a small "doit" button that will trigger shot/shots/5shots effects, and like the other buttons it calls out to app_httpd.c code via a webrequest:

Code: Select all

$ git diff index.html
diff --git a/FCameraWebServer/index/index.html b/FCameraWebServer/index/index.html
index f1fae5f..7999646 100644
--- a/FCameraWebServer/index/index.html
+++ b/FCameraWebServer/index/index.html
@@ -556,6 +556,7 @@
                                 <input id="ges_gpio" type="checkbox" class="default-action">
                                 <label class="slider" for="ges_gpio"></label>
                             </div>
+                            <button id="ges_doit">doit</button>
                         </div>
                         <div class="input-group" id="ges_offset-group">
                             <label for="ges_offset">Offset [µs]</label>
@@ -879,6 +880,12 @@ document.addEventListener('DOMContentLoaded', function (event) {
     }
   }
 
+  const doitButton = document.getElementById('ges_doit')
+
+  doitButton.onclick = () => {
+    updateConfig(doitButton)
+  }
+
 })
 
         </script>
$

I used below code and that just works (pressing "doit" just turns on builtin flash led for 1s and then turns it off again). Clicking doit does so every time. That is the code in "if (1)" part, but what I really want is executing the code in else part based on ESP32 RMT component:

Code: Select all

$ git diff ../app_httpd.cpp
diff --git a/FCameraWebServer/app_httpd.cpp b/FCameraWebServer/app_httpd.cpp
index 3e52a2d..b6f9d39 100644
--- a/FCameraWebServer/app_httpd.cpp
+++ b/FCameraWebServer/app_httpd.cpp
@@ -539,6 +539,40 @@ static esp_err_t cmd_handler(httpd_req_t *req){
     else if(!strcmp(variable, "ges_gpio")) {
         ges_gpio = atoi(value);
     }
+    else if(!strcmp(variable, "ges_doit")) {
+#include "esp32-hal-rmt.h"
+
+#define assert(expr) if (!(expr)) {                   \
+  Serial.begin(115200); Serial.println(#expr);        \
+  Serial.println(__FILE__); Serial.println(__LINE__); \
+  for(;;) {}                                          \
+}
+
+if (1)
+{
+        pinMode(LED_BUILTIN, OUTPUT);
+        digitalWrite(LED_BUILTIN, 1);
+        delay(500);
+        delay(500);
+        digitalWrite(LED_BUILTIN, 0);
+} else {
+  rmt_data_t data[]={
+    {      2, 0, ges_on, 1 },
+    {      2, 0, 0, 0 }
+  };
+
+  static rmt_obj_t* rmt_send = NULL;
+
+  if (!rmt_send) {
+    assert((rmt_send = rmtInit(ges_gpio?13:4, true, RMT_MEM_64)) != NULL)
+
+    assert(1000 == rmtSetTick(rmt_send, 1000)) // 1µs resolution
+  }
+
+  assert(rmtWrite(rmt_send, data, sizeof(data)/sizeof(data[0])))
+
+  // assert(4==5);
+}
+    }
     else if(!strcmp(variable, "ges_repeat")) {
         ges_repeat = atoi(value);
$

Unfortunately that code hangs when clicking "doit", and even kills the webserver (no browser page reload possible).
I will have to debug what is wrong with that code (that runs fine standalone in Arduino sketch).
In case you know, I would like to hear about.


P.S:
Problems resolved, but I ended here and thought I was crazy because I was not able to recreate this today:

I did comment out the hide command and got exactly what is needed:
Video gets (mjpeg) streamed until closeButton is pressed, then streaming stops, BUT last frame keeps being displayed!

I tried thousand of things, went back to previous version, nothing helped.
Then finally I realized that I used Firefox yesterday while Google Chrome today.
For this "feature" to work Chrome browser is no option but Firefox has to be used (I don't know how to avoid Chrome writing all black frame as last step).

[HermannSW]

The problems were solved, and description of the changes as well as first rpm determination for fast rotating mini drone propeller can be found in this posting:
https://www.esp32.com/viewtopic.php?f=1 ... 780#p45780

Async await was used for synchronous wait in index.html browser JS.
app_httpd.cpp allows for delay() calls when needed (runs on ESP32).
FCameraWebServer.ino does not see changes besides ssid, password and module version.

[ESP32Cool4753]

Hello from France,
I just saw your tutorial (https://www.esp32.com/viewtopic.php?t=11190)
Really a great tutorial, thank you very much it works perfectly.

Just a question:
I wanted to run 2 servomotors (Pan-Tilt) with the "ESP32Servo.h" library, by taking inspiration from an example that works really well in a simplified version, that is to say without the menu, (see this link : https://randomnerdtutorials.com/esp32-c ... ilt-2-axis)
This works fine as there are pins 14 and 15 available on the ESP32-CAM, but I would like to do the same while keeping the menu as you present it.

But I'm lost because there are these three tabs, and I get lost, I don't know where to put this or that writing, can you orient me please?

I managed to modify the program in html to have the 4 buttons (Up, Down, Left, Right) in the menu, but for the rest, I don't know where to put all this to make the Pan-Tilt work. . (I am a beginner...).