ESP32 crashes when reading bytes from file on FAT partition using fgetc

[IgrBckr]

I built a FAT partition inside the ESP32 flash using the "wear-leveling" example. I created a custom partition csv file as well as modified the CMakeLists.txt file to create the image. The problem seems to be the usage of the fgetc function when reading bytes from the files.

Here's the relevant code:

Code: Select all

#include "fatfs.h"
#include "webservice.h"
#include <stdlib.h>
#include <stdio.h>
#include "esp_vfs.h"
#include "esp_vfs_fat.h"
#include "esp_system.h"
#include "nvmem/nvmem.h"
#include "string.h"

//module name
const char *fatfs_mod_name = "FAT-FS MGMT";

//root directory name
const char *base_path = "/root";

//files paths
const char home_path[] = "/root/pages/home.htm";
const char reg_path[] = "/root/pages/reg.htm";
const char cmds_path[] = "/root/pages/cmds.htm";
const char meas_path[] = "/root/pages/meas.htm";
const char log_path[] = "/root/pages/log.htm";

char *file_paths[5]={&home_path,&reg_path,&cmds_path,&meas_path,&log_path};

//Handle of the wear levelling library instance when unlocked for file writing...
//static wl_handle_t s_wl_handle = WL_INVALID_HANDLE;

FILE *home_page = NULL;
FILE *reg_page = NULL;
FILE *cmds_page = NULL;
FILE *meas_page = NULL;
FILE *log_page = NULL;

FILE *file_list[5];

char home_page_buff[MAX_FILE_SZ];
char reg_page_buff[MAX_FILE_SZ];
char cmds_page_buff[MAX_FILE_SZ];
char meas_page_buff[MAX_FILE_SZ];
char log_page_buff[MAX_FILE_SZ];

char *html_pages_list[5]={&home_page_buff,&reg_page_buff,&cmds_page_buff,&meas_page_buff,&log_page_buff};

uint32_t home_sz, reg_sz, cmds_sz, meas_sz, log_sz;
uint32_t *file_sz_list[5]={&home_sz,&reg_sz,&cmds_sz,&meas_sz,&log_sz};


void FileSystem_Init() {
    
    ESP_LOGI(fatfs_mod_name, "Mounting FAT filesystem");

    const esp_vfs_fat_mount_config_t mount_config = {
            .max_files = 10,
            .format_if_mount_failed = false,
            .allocation_unit_size = CONFIG_WL_SECTOR_SIZE
    };

    esp_err_t err;
    err = esp_vfs_fat_spiflash_mount_ro(base_path, FATFS_PARTITION_NAME, &mount_config);
    if (err != ESP_OK) {
        ESP_LOGE(fatfs_mod_name, "Failed to mount FATFS (%s)", esp_err_to_name(err));
        return;
    }

    //Load files contents
    uint_fast8_t c = 0;
    uint_fast32_t i = 0;
    for(uint_fast8_t k=0; k<5; k++) {
        file_list[k] = fopen(file_paths[k],"r");

        if(file_list[k] == NULL) {
            ESP_LOGE(fatfs_mod_name, "Failed to open file %u for reading",k);
            continue;
        }

        while(c != EOF) {
            c = fgetc(file_list[k]);
            if(c != EOF) html_pages_list[k][i++] = c;
        }
        html_pages_list[k][i] = '\0';
        *file_sz_list[k] = i;
        c = 0;
        i = 0;

        fclose(file_list[k]);
    }
    

    //Point webservice page pointers to buffers
    p_home_page_buff = home_page_buff;
    p_reg_page_buff = reg_page_buff;
    p_cmds_page_buff = cmds_page_buff;
    p_meas_page_buff = meas_page_buff;
    p_log_page_buff = log_page_buff;
}

Backtrace:

I (622) main_task: Calling app_main()
I (647) FAT-FS MGMT: Mounting FAT filesystem
Guru Meditation Error: Core 0 panic'ed (LoadProhibited). Exception was unhandled.

Core 0 register dump:
PC : 0x4008aec9 PS : 0x00060730 A0 : 0x800830ec A1 : 0x3ffbcbd0
0x4008aec9: xQueueTakeMutexRecursive at C:/Espressif/frameworks/esp-idf/components/freertos/FreeRTOS-Kernel/queue.c:776

A2 : 0x01000000 A3 : 0xffffffff A4 : 0x3ffcaf88 A5 : 0x00000080
A6 : 0x00000000 A7 : 0x00000100 A8 : 0x8008c3fd A9 : 0x3ffbcbc0
A10 : 0x00000003 A11 : 0x00060723 A12 : 0x00060720 A13 : 0x00000000
A14 : 0x00000000 A15 : 0xb33fffff SAR : 0x00000018 EXCCAUSE: 0x0000001c
EXCVADDR: 0x01000008 LBEG : 0x4000c2e0 LEND : 0x4000c2f6 LCOUNT : 0x00000000
0x4000c2e0: memcpy in ROM

0x4000c2f6: memcpy in ROM


Backtrace: 0x4008aec6:0x3ffbcbd0 0x400830e9:0x3ffbcbf0 0x400831f1:0x3ffbcc20 0x4008327e:0x3ffbcc40 0x401585a6:0x3ffbcc70 0x401585ff:0x3ffbcca0 0x400d993f:0x3ffbccc0 0x400d8b1c:0x3ffbccf0 0x401706cf:0x3ffbcd10 0x4008cf6d:0x3ffbcd40
0x4008aec6: xQueueTakeMutexRecursive at C:/Espressif/frameworks/esp-idf/components/freertos/FreeRTOS-Kernel/queue.c:769 (discriminator 1)

0x400830e9: lock_acquire_generic at C:/Espressif/frameworks/esp-idf/components/newlib/locks.c:144

0x400831f1: _lock_acquire_recursive at C:/Espressif/frameworks/esp-idf/components/newlib/locks.c:158

0x4008327e: __retarget_lock_acquire_recursive at C:/Espressif/frameworks/esp-idf/components/newlib/locks.c:314

0x401585a6: _fgetc_r at /builds/idf/crosstool-NG/.build/HOST-x86_64-w64-mingw32/xtensa-esp32-elf/src/newlib/newlib/libc/stdio/fgetc.c:93 (discriminator 2)

0x401585ff: fgetc at /builds/idf/crosstool-NG/.build/HOST-x86_64-w64-mingw32/xtensa-esp32-elf/src/newlib/newlib/libc/stdio/fgetc.c:114

0x400d993f: FileSystem_Init at C:/Users/XXX/Documents/Repo/XXXXXXX/FW/main/nvmem/fatfs.c:122

0x400d8b1c: app_main at C:/Users/XXX/Documents/Repo/XXXXXXX/FW/main/main.cpp:21

0x401706cf: main_task at C:/Espressif/frameworks/esp-idf/components/freertos/app_startup.c:208 (discriminator 13)

0x4008cf6d: vPortTaskWrapper at C:/Espressif/frameworks/esp-idf/components/freertos/FreeRTOS-Kernel/portable/xtensa/port.c:162

I can post the core dump if necessary

Thanks in advance

[MicroController]

fgetc(...) returns an int, with EOF being a valid int but not a valid uint8_t value. Because uint_fast8_t c, c != EOF will not be true or false in the way your code expects it to be.

[IgrBckr]

fgetc(...) returns an int, with EOF being a valid int but not a valid uint8_t value. Because uint_fast8_t c, c != EOF will not be true or false in the way your code expects it to be.

I've tried using int c instead of uint_fast8_t but the problem remains (with the same back-trace).
I've also tested a modified version of the wear-leveling example that instead of reading the lines of the files with fgets, I read the bytes using fgetc(...), I get a similar crash to this but with a different backtrace...

I suspect it has something to do with not placing memcpy into IRAM, but I can't do this without enabling external flash (if I enable external flash and place memcpy in IRAM, the program crashes because there's no external flash...). I'm using ESP-IDF 5.1.1

[IgrBckr]

I found the problem: it was an overflow in the char buffer that received the bytes from the file, the buffer was smaller than the file size. I found the backtrace was a bit misleading though...

[MicroController]

memcpy is in ROM, so no worries there.

And yes, once your code has corrupted 'random' locations in RAM, the time and location of a subsequent exception become less useful.