VGA RGB Digitizer/Grabber with ESP32-S3?

[squix78]

I'd like to stream RGB VGA from old computers and consoles over WiFi to a desktop computer. There are chips that can digitize the analog RGB VGA signal into 3x8bit pixel information, like the AD9883A or TVP7002. Open source projects normally have the 24 bit lines + clock signals fed into an FPGA but can't the ESP32 process these bitstreams directly with I2S subsystem?
So the plan would be to connect the 24bit color information directly to an ESP32-S3. The I2S subsystem would then pump the data into the PSRAM. From there my software would encode the frames into JPEG/MJPEG. A client could then source the MJPEG stream from the ESP32's web server. Could that work?
A retro device producing 640x480 at 50Hz would result in a bit frequency of 15.36MHz. Is the I2S subsystem capable of process this? Will I have enough CPU power to encode the frame into JPEG? 20FPS output is enough for my purpose. Any help/opinion appreciated...

[ESP_igrr]

Hi squix78,

ESP32-S3 LCD_CAM peripheral can support pixel clocks up to 40 MHz. It is designed for the 8- or 16-bit DVP interface. Since your ADC converter outputs 24-bit parallel data, you will need to do some conversion to get 8- or 16-bit result. This can be done either using latches, if you want to preserve the full 24-bit color, or by simply discarding 8 bits of color information and wiring the only the 16 bits in an RGB 565 format.

The practical pixel clock will likely be higher than 15.36 MHz since the standard VGA signal occupies just a portion of the entire frame time, due to the blanking intervals. For example, for a 640*480 60 Hz output the pixel clock is 25.175 MHz.

Doing something useful with this data might be harder, though. At 16bpp, a 640*480 frame would be too large to fit into the internal RAM of the ESP32-S3, so you'd have to transfer it to PSRAM. Assuming you are okay with a 20fps refresh rate, this would require 640*480*16*20(fps) = 98 Mbits/s for one way transfer, or about 200 Mbits/s since we will eventually need to read the data back.
PSRAM bandwidth (in 4-line mode, 80 MHz) is around 280 Mbits/s. You can get approximately twice that much if you can get a module with octal PSRAM. So if you manage to read the data from PSRAM in such a way that the bandwidth is used efficiently (avoiding cache trashing), it is possible to write and read back the data in time.

Regarding the JPEG encoding, I'm not sure if it is viable to do this fast enough. Perhaps with an optimized JPEG encoding algorithm based on ESP32-S3 instruction extensions, it could be done.