ESP32 S3 video buffer filling

[Slinisa]

I want to use ESP32 S3 to generate VGA signals+data. I modified existing bitluni "ESP32 S3 VGA" project and it's working fine so far. It allocates frame buffer and displays it using DMA using linked descriptors.
I modified the project in order to use only 16K buffer. Linked descriptors re-use 16K buffer so instead of using 640x480 bytes (the displayed image is in 8-bit format) the same buffer is showed 640x480/16384 times (~19).
The plan is to fill-up 16K buffer during vertical blanking phase and then every time a line is displayed re-fill that line with data for the next scanline. For example, I fill the "scanlines" 0-18 of the buffer during vertical blanking phase. After line 0 is displayed, I fill it with data for line 19 so that when line 19 is to be displayed, it's ready with fresh video data.
There are few ways that this could be done and I'm not that proficient with ESP32 S3 programming to figure out which one is possible.
1. The preferred way would be busy-waiting, without using interrupts. In order to have that solution, I need a way to find out which line is being drawn by a LCD-CAM peripheral, or which buffer is being DMA-transferred to a peripheral. Any of these two would be fine, if they are possible.
2. Having an interrupt after each horizontal sync or horizontal line. I couldn't find that interrupt, only vertical sync in the technical manual.

I did found a struct dma_descriptor_t member "owner" that is supposed to change the owner from DMA to CPU after it's done, but I'm not sure how to use it. I tried tweaking it from one value to another and then serial print the value during run-time and it doesn't seem to change the value automatically.
I have BPI-PICOW-S3 board and the examples run just fine. The example project is written in C/C++.
Any suggestion (or even example code) would be appreciated.

[ESP_Sprite]

I don't know Bitluni's code, but each DMA descriptor has an 'EOF' flag. You can set this flag; while it does not affect the data flow to the peripheral in any way, you can set up a DMA interrupt that triggers every time the DMA channel reads a descriptor with that flag set. You could use this to implement a front/backbuffer for a set of lines and more-or-less get what you want.

[Slinisa]

Thank you for your reply. That seems like what I was looking for for the solution with interrupts. Do you have an example of setting up the interrupt vector? And do you know approximately what is the interrupt response time? I've read that for the external interrupt for ESP32 is in the order of milliseconds, which is incredibly slow. I don't know the situation with S3, non-external interrupts.

[ESP_Sprite]

It's not milliseconds, that's not what it is on any uC, it's microseconds at worst. For an EOF interrupt example, you could look at esp_lcd_panel_rgb in ESP-IDF, especially in bounce buffer mode it closely matches what you're trying to do.

[Slinisa]

Thanks again for your reply. I did find some example in the meantime (since I was waiting for one day for them to approve my post) and it worked like a charm. I do have two questions though:

1. I noticed the third parameter in a callback function is user data:
dma_callback(gdma_channel_handle_t dma_chan, gdma_event_data_t *event_data, void *user_data)
how do I send this parameter? Setting a callback doesn't mention any parameters:
gdma_tx_event_callbacks_t tx_cbs = {
.on_trans_eof = dma_callback
};
Can I set different parameters for each descriptor or all descriptors share the same user data?

2. How do I set a callback function for vertical sync?

[ESP_Sprite]

1. I noticed the third parameter in a callback function is user data:
dma_callback(gdma_channel_handle_t dma_chan, gdma_event_data_t *event_data, void *user_data)
how do I send this parameter? Setting a callback doesn't mention any parameters:
gdma_tx_event_callbacks_t tx_cbs = {
.on_trans_eof = dma_callback
};

The gdma_register_tx_event_callbacks() function takes the callback argument.

Can I set different parameters for each descriptor or all descriptors share the same user data?

They all share the same argument.

2. How do I set a callback function for vertical sync?

You'd do that on the LCD peripheral. The esp_lcd_panel_rgb.c sources do it; search for LCD_LL_EVENT_VSYNC_END and you'll find it.

[Slinisa]

Hi,

I've found that one, tried it and couldn't make it working. Below is the code. It triggers GDMA interrupt after each line correctly, but not VSYNC part. I've commented out the code that is related to VSYNC. I tried all the different combinations off the lines below. For example, both lcd_ll_enable_interrupt and esp_intr_enable (including LCD_cam.lc_dma_int_ena.lcd_vsync_int_ena). Not all at the same time. It didn't enter lcd_default_isr_handler at all.
Is there a simpler version? I'm not planning to use bounce buffers so the whole lcd_ll abstraction seems complicated. All I need to do is activate VSYNC interrupts and define a callback. Is it possible without the said abstraction?

static IRAM_ATTR void lcd_default_isr_handler(void *args)
{
Serial.println("VSYNC");
}

.
.
.

gdma_tx_event_callbacks_t tx_cbs = {
.on_trans_eof = dma_callback
};
gdma_register_tx_event_callbacks(dma_chan, &tx_cbs, NULL);
// esp_intr_alloc_intrstatus(ETS_LCD_CAM_INTR_SOURCE, ESP_INTR_FLAG_IRAM | ESP_INTR_FLAG_INTRDISABLED | ESP_INTR_FLAG_SHARED | ESP_INTR_FLAG_LOWMED, (uint32_t)lcd_ll_get_interrupt_status_reg(&LCD_CAM), LCD_LL_EVENT_VSYNC_END, lcd_default_isr_handler, NULL, &intr);
// lcd_ll_enable_interrupt(&LCD_CAM, LCD_LL_EVENT_VSYNC_END, false);
// esp_intr_disable(intr);
// lcd_ll_clear_interrupt_status(&LCD_CAM, UINT32_MAX);
// lcd_ll_enable_interrupt(&LCD_CAM, LCD_LL_EVENT_VSYNC_END, true);
// LCD_CAM.lc_dma_int_ena.lcd_vsync_int_ena=1;
// esp_intr_enable(intr);

[TBHKpmy]

ArduinoIDE Ver1.8.19でESP32s3の開発をしているのですが
LCD_CAMモジュールの割り込みを登録する際に「esp_intr_alloc」を使用して割り込みハンドラー関数を下記のように登録しているのですが、CISからのVSYNCを受信してもLCD_CAMモジュール割り込みが発生しない。

※LCD_CAMモジュールのレジスタ設定で「LCD_CAM.lc_dma_int_ena.cam_vsync_int_ena = 1;」と設定しているのでLCD_CAMモジュールでCISのVSYNCを受信すると割り込みが発生するようになっている？

//◇LCD_CAM_LC_DMA_INT_ENA
LCD_CAM.lc_dma_int_ena.lcd_vsync_int_ena = 0;
LCD_CAM.lc_dma_int_ena.lcd_trans_done_int_ena = 0;
LCD_CAM.lc_dma_int_ena.cam_vsync_int_ena = 1;
LCD_CAM.lc_dma_int_ena.cam_hs_int_ena = 0;

// Allocate LCD_CAM interrupt, keep it disabled
esp_intr_alloc(ETS_LCD_CAM_INTR_SOURCE,
ESP_INTR_FLAG_INTRDISABLED | ESP_INTR_FLAG_LEVEL1 | ESP_INTR_FLAG_IRAM,
&VSYNCHandler, NULL, &lcd_cam_intr_handle);

esp_intr_enable(lcd_cam_intr_handle);

LCD_CAMモジュール割り込みを発生させるにはどうしたらいいでしょうか？

よろしくお願いいたします。


関連するソースコード全体です

void ESP32CAM_Init(ESP32CAM_PIN *param , uint16_t horizon_size , uint16_t vertical_size)
{

dma_buf_size = horizon_size;

periph_module_enable(PERIPH_LCD_CAM_MODULE); // LCD_CAMの有効
periph_module_reset(PERIPH_LCD_CAM_MODULE); // LCD_CAMのリセット

// Reset CAM bus
LCD_CAM.cam_ctrl1.cam_reset = 1;

//①CAM_CLKの設定(レジスタ設定)
// CAM module registers configuration
//◇LCD_CAM_CAM_CTRL
LCD_CAM.cam_ctrl.cam_stop_en = 0;
LCD_CAM.cam_ctrl.cam_vsync_filter_thres = 0;
LCD_CAM.cam_ctrl.cam_byte_order = 0;
LCD_CAM.cam_ctrl.cam_bit_order = 0;
LCD_CAM.cam_ctrl.cam_line_int_en = 1;
LCD_CAM.cam_ctrl.cam_vs_eof_en = 1;
LCD_CAM.cam_ctrl.cam_clkm_div_num = 2; // CAM_CLK_IDX = 20MHz
LCD_CAM.cam_ctrl.cam_clkm_div_b = 0;
LCD_CAM.cam_ctrl.cam_clkm_div_a = 0;
LCD_CAM.cam_ctrl.cam_clk_sel = 1; // XTAL_CLK source (40MHz)

//◇LCD_CAM_CAM_CTRL1
LCD_CAM.cam_ctrl1.cam_rec_data_bytelen = horizon_size;
LCD_CAM.cam_ctrl1.cam_line_int_num = 0; // hs割り込みを生成するhsyncの数
LCD_CAM.cam_ctrl1.cam_clk_inv = 0;
LCD_CAM.cam_ctrl1.cam_vsync_filter_en = 1;
LCD_CAM.cam_ctrl1.cam_2byte_en = 0;
LCD_CAM.cam_ctrl1.cam_de_inv = 0;
LCD_CAM.cam_ctrl1.cam_hsync_inv = 0;
LCD_CAM.cam_ctrl1.cam_vsync_inv = 0;

//◇LCD_CAM_CAM_RGB_YUV
LCD_CAM.cam_rgb_yuv.cam_conv_8bits_data_inv = 0;
LCD_CAM.cam_rgb_yuv.cam_conv_yuv2yuv_mode = 0;
LCD_CAM.cam_rgb_yuv.cam_conv_yuv_mode = 0;
LCD_CAM.cam_rgb_yuv.cam_conv_protocol_mode = 0;
LCD_CAM.cam_rgb_yuv.cam_conv_data_out_mode = 1;
LCD_CAM.cam_rgb_yuv.cam_conv_data_in_mode = 1;
LCD_CAM.cam_rgb_yuv.cam_conv_mode_8bits_on = 1;
LCD_CAM.cam_rgb_yuv.cam_conv_trans_mode = 0;
LCD_CAM.cam_rgb_yuv.cam_conv_bypass = 0;


//②ピン設定（カメラモジュールのデータ線がどのピンなのかを設定）
// Route CAM signals to GPIO pins
const struct
{
uint32_t pin;
uint8_t signal;
} mux[] = {
{ param->d0_pin, CAM_DATA_IN0_IDX }, // CAM_DATA_IN0をd0_pinに接続
{ param->d1_pin, CAM_DATA_IN1_IDX }, // CAM_DATA_IN1をd1_pinに接続
{ param->d2_pin, CAM_DATA_IN2_IDX }, // CAM_DATA_IN2をd2_pinに接続
{ param->d3_pin, CAM_DATA_IN3_IDX }, // CAM_DATA_IN3をd3_pinに接続
{ param->d4_pin, CAM_DATA_IN4_IDX }, // CAM_DATA_IN4をd4_pinに接続
{ param->d5_pin, CAM_DATA_IN5_IDX }, // CAM_DATA_IN5をd5_pinに接続
{ param->d6_pin, CAM_DATA_IN6_IDX }, // CAM_DATA_IN6をd6_pinに接続
{ param->d7_pin, CAM_DATA_IN7_IDX }, // CAM_DATA_IN7をd7_pinに接続
{ param->pclk_pin, CAM_PCLK_IDX }, // CAM_PCLKをpclk_pinに接続
{ param->href_pin, CAM_H_ENABLE_IDX }, // CAM_H_ENABLEをhref_pinに接続
{ param->vsync_pin, CAM_V_SYNC_IDX }, // CAM_V_SYNC_IDXをvsync_pinに接続
{ GPIO_MATRIX_CONST_ONE_INPUT, CAM_H_SYNC_IDX }, // CAM_H_SYNCをGPIO_MATRIX_CONST_ONE_INPUTに接続
{ GPIO_MATRIX_CONST_ONE_INPUT, CAM_V_SYNC_IDX } // CAM_V_SYNCをGPIO_MATRIX_CONST_ONE_INPUTに接続
};

for (int i = 0; i < 13; i++)
{
if (mux.pin != GPIO_MATRIX_CONST_ONE_INPUT)
{
esp_rom_gpio_pad_select_gpio(mux.pin);
}
esp_rom_gpio_connect_in_signal(mux.pin, mux.signal, false);
}

esp_rom_gpio_pad_select_gpio(param->xclk_pin);
esp_rom_gpio_pad_set_drv(param->xclk_pin, 0); //ドライブスレングスを設定 (- 0: 5mA - 1: 10mA - 2: 20mA - 3: 40mA)
esp_rom_gpio_connect_out_signal(param->xclk_pin, CAM_CLK_IDX, false, false);

//③HSYNCに従ってLCD_CAM_CAM_VH_DE_MODE_ENの値を設定する。
LCD_CAM.cam_ctrl1.cam_vh_de_mode_en = 1;

//④LCD_CAM_CAM_UPDATEの値を設定する。
LCD_CAM.cam_ctrl.cam_update = 1; // Update registers

//⑤リセット
LCD_CAM.cam_ctrl1.cam_reset = 0; //カメラモジュールリセット信号
LCD_CAM.cam_ctrl1.cam_afifo_reset = 0; //Async Rx FIFOリセット信号

//⑥割り込み設定
//◇LCD_CAM_LC_DMA_INT_ENA
LCD_CAM.lc_dma_int_ena.lcd_vsync_int_ena = 0;
LCD_CAM.lc_dma_int_ena.lcd_trans_done_int_ena = 0;
LCD_CAM.lc_dma_int_ena.cam_vsync_int_ena = 1;
LCD_CAM.lc_dma_int_ena.cam_hs_int_ena = 0;

// Allocate LCD_CAM interrupt, keep it disabled
esp_intr_alloc(ETS_LCD_CAM_INTR_SOURCE,
ESP_INTR_FLAG_INTRDISABLED | ESP_INTR_FLAG_LEVEL1 | ESP_INTR_FLAG_IRAM,
&VSYNCHandler, NULL, &lcd_cam_intr_handle);

esp_intr_enable(lcd_cam_intr_handle);

//◇LCD_CAM_LC_DMA_INT_RAW_REG(Read Only)
//LCD_CAM_LCD_VSYNC_INT_RAW
//LCD_CAM_LCD_TRANS_DONE_INT_RAW
//LCD_CAM_CAM_VSYNC_INT_RAW
//LCD_CAM_CAM_HS_INT_RAW

//◇LCD_CAM_LC_DMA_INT_ST(Read Only)
//LCD_CAM_LCD_VSYNC_INT_ST
//LCD_CAM_LCD_TRANS_DONE_INT_ST
//LCD_CAM_CAM_VSYNC_INT_ST
//LCD_CAM_CAM_HS_INT_ST

//◇LCD_CAM_LC_DMA_INT_CLR(Write Only)
//LCD_CAM_LCD_VSYNC_INT_CLR
//LCD_CAM_LCD_TRANS_DONE_INT_CLR
//LCD_CAM_CAM_VSYNC_INT_CLR
//LCD_CAM_CAM_HS_INT_CLR

//◇LCD_CAM_LC_REG_DATE //※必要かわからない timer.hをインクルードして日時を設定するレジスタ？
//LCD_CAM_LC_DATE = ;


//⑦GDMA設定(レジスタ設定)
//■SYSTEM

if (REG_GET_BIT(SYSTEM_PERIP_CLK_EN1_REG, SYSTEM_DMA_CLK_EN) == 0) {
REG_CLR_BIT(SYSTEM_PERIP_CLK_EN1_REG, SYSTEM_DMA_CLK_EN);
REG_SET_BIT(SYSTEM_PERIP_CLK_EN1_REG, SYSTEM_DMA_CLK_EN); //1,SYSTEM_PERIP_CLK_EN1_REGレジスタのSYSTEM_DMA_CLK_Eを設定し、DMAクロックを有効にする。
REG_SET_BIT(SYSTEM_PERIP_RST_EN1_REG, SYSTEM_DMA_RST); //2,SYSTEM_PERIP_RST_EN1_REGレジスタのSYSTEM_DMA_RSTを設定し、DMAをリセットする。
REG_CLR_BIT(SYSTEM_PERIP_RST_EN1_REG, SYSTEM_DMA_RST);
}

//■DMA

//GDMAチャネルの初期化
GDMA.channel[cam_dma_num].in.int_clr.val = ~0;
GDMA.channel[cam_dma_num].in.int_ena.val = 0;

//◇GDMA_IN_CONF0_CHn_REG
GDMA.channel[cam_dma_num].in.conf0.indscr_burst_en = 0;
GDMA.channel[cam_dma_num].in.conf0.in_data_burst_en = 0;
GDMA.channel[cam_dma_num].in.conf0.mem_trans_en = 1;

//◇GDMA_IN_LINK_CHn_REG
GDMA.channel[cam_dma_num].in.link.auto_ret = 1;
GDMA.channel[cam_dma_num].in.link.stop = 0;
GDMA.channel[cam_dma_num].in.link.restart = 0;

//3,GDMAの受信チャネルとFIFOポインタのステートマシンをリセットするためにGDMA_IN_CONF0_CHx_REGレジスタのGDMA_IN_RST_CHnをまず1に設定し、次に0に設定する。
GDMA.channel[cam_dma_num].in.conf0.in_rst = 1;
GDMA.channel[cam_dma_num].in.conf0.in_rst = 0;

err = initDMAdesc();
if (err != ESP_OK) {
//Serial.println("Failed to initialize I2S and DMA");
Serial.println("Failed to initialize DMA");
}

//4,インリンクをロードし、最初の受信記述子のアドレスでGDMA_IN_LINK_CHn_REGレジスタのGDMA_INLINK_ADDR_CHn を構成する。
GDMA.channel[cam_dma_num].in.link.addr = ((uint32_t)dma_desc) & 0xfffff;

//チャネルの所有者の確認
GDMA.channel[cam_dma_num].in.conf1.in_check_owner = 0;

//◇GDMA_IN_PERI_SEL_CHn_REG
//5,GDMA_IN_PERI_SEL_CHn_REGレジスタのGDMA_PERI_IN_SEL_CHnをLCD_CAMに設定する。
GDMA.channel[cam_dma_num].in.peri_sel.sel = 5;

//6,GDMA_IN_LINK_CHn_REGレジスタのGDMA_INLINK_START_CHn を設定してGDMA の受信チャンネルをデータ転送用に有効にする。
GDMA.channel[cam_dma_num].in.link.start = 1;

//7,データ・フレームまたはパケットを受信したことを示す、GDMA_IN_SUC_EOF_CHn_INTの割り込みを待つ。
GDMA.channel[cam_dma_num].in.int_clr.in_suc_eof = 1; //割り込みフラグクリア
GDMA.channel[cam_dma_num].in.int_ena.in_suc_eof = 1; //EOF割り込み有効化

// Allocate GMDA interrupt, keep it disabled
esp_intr_alloc(ETS_DMA_IN_CH0_INTR_SOURCE,
ESP_INTR_FLAG_INTRDISABLED | ESP_INTR_FLAG_LEVEL2 | ESP_INTR_FLAG_IRAM,
&ReceiveHandler, NULL, &gdma_intr_handle);

esp_intr_enable(gdma_intr_handle);

//④LCD_CAM_CAM_UPDATEの値を設定する。
LCD_CAM.cam_ctrl.cam_update = 1; // Update registers

//⑤リセット
LCD_CAM.cam_ctrl1.cam_reset = 0; //カメラモジュールリセット信号

LCD_CAM.cam_ctrl1.cam_afifo_reset = 0; //Async Rx FIFOリセット信号

Serial.println("initialize LCD_CAM & DMA");

}