File not opening from SPIFFS

[bcolbert]

I'm trying to read a file from SPIFFS but getting NULL from fopen. I'm able to stat the file, but open isn't working.

partitions.csv:

Code: Select all

# ESP-IDF Partition Table
# Name,		Type,	SubType,	Offset,		Size,	Flags
  nvs,		data,	nvs,		0x9000,		24K,
  phy_init,	data,	phy,		0xf000,		4K,
  factory,	app,	factory,	0x10000,	1500K,
  spiffs,	data,	spiffs,		,		1M,	,

Data directory is called spiffs and contains:

Code: Select all

total 288
-rw-r--r--  1 bcolbert  staff    20K Jul 23 08:41 drip.raw
-rw-r--r--  1 bcolbert  staff   118K Jul 23 08:41 sonar.raw

Image built with:

Code: Select all

mkspiffs -c spiffs -b 4096 -p 256 -s 0x100000 spiffs.bin

Written to flash with:

Code: Select all

esptool.py --chip esp32 write_flash -z 0x187000 spiffs.bin

Code that tries to read the file:

Code: Select all

   ESP_LOGI(BT_APP_CORE_TAG, "Initializing SPIFFS");

    esp_vfs_spiffs_conf_t conf = {
      .base_path = "/spiffs",
      .partition_label = NULL,
      .max_files = 5,
      .format_if_mount_failed = false
    };

    // Use settings defined above to initialize and mount SPIFFS filesystem.
    // Note: esp_vfs_spiffs_register is an all-in-one convenience function.
    esp_err_t ret = esp_vfs_spiffs_register(&conf);

    if (ret != ESP_OK) {
        if (ret == ESP_FAIL) {
            ESP_LOGE(BT_APP_CORE_TAG, "Failed to mount or format filesystem");
        } else if (ret == ESP_ERR_NOT_FOUND) {
            ESP_LOGE(BT_APP_CORE_TAG, "Failed to find SPIFFS partition");
        } else {
            ESP_LOGE(BT_APP_CORE_TAG, "Failed to initialize SPIFFS (%s)", esp_err_to_name(ret));
        }
        return;
    }

    size_t total = 0, used = 0;
    ret = esp_spiffs_info(conf.partition_label, &total, &used);
    if (ret != ESP_OK) {
        ESP_LOGE(BT_APP_CORE_TAG, "Failed to get SPIFFS partition information (%s)", esp_err_to_name(ret));
    } else {
        ESP_LOGI(BT_APP_CORE_TAG, "Partition size: total: %d, used: %d", total, used);
    }

    struct stat st;
    if (stat(filename, &st) == 0) {
        // Delete it if it exists
        ESP_LOGI(BT_APP_CORE_TAG, "!!! %s is there", filename);
    }

    ESP_LOGI(BT_APP_CORE_TAG, "Opening file %s", filename);
    FILE* fd = fopen(filename, "r");
    if (fd) {
        ESP_LOGE(BT_APP_CORE_TAG, "Failed to open file for reading");
        return;
    }
    else{
        while (!feof(fd)) // to read file
        {
            int n = fread(snd_buff, 1, BLOCK_SIZE, fd);
            ESP_LOGI(BT_APP_CORE_TAG, "%d bytes read and written %02x,%02x", n, ((uint16_t*)snd_buff)[0], ((uint16_t*)snd_buff)[1]);
        }
        fclose(fd);
    }

    // All done, unmount partition and disable SPIFFS
    esp_vfs_spiffs_unregister(conf.partition_label);
    ESP_LOGI(BT_APP_CORE_TAG, "SPIFFS unmounted");

Log output (snippets):

I (60) boot: Partition Table:
I (63) boot: ## Label Usage Type ST Offset Length
I (71) boot: 0 nvs WiFi data 01 02 00009000 00006000
I (78) boot: 1 phy_init RF data 01 01 0000f000 00001000
I (86) boot: 2 factory factory app 00 00 00010000 00177000
I (93) boot: 3 spiffs Unknown data 01 82 00187000 00100000
I (101) boot: End of partition table

I (11261) BT_APP_CORE: Initializing SPIFFS
I (13761) BT_APP_CORE: Partition size: total: 956561, used: 143572
I (13881) BT_APP_CORE: !!! /spiffs/drip.raw is there
I (13881) BT_APP_CORE: Opening file /spiffs/drip.raw
E (13881) BT_APP_CORE: Failed to open file for reading

Any tips? How do I read a file from SPIFFS?

[bcolbert]

Wow! My BIG mistake on testing the opening of the file. I ported the code from using open() to fopen() and mucked up my success test.

Fixed now, opening the file, but still have a major problem. Getting 0 bytes read. I may make that another forum post.

Code: Select all

    ESP_LOGI(BT_APP_CORE_TAG, "Opening file %s", filename);
    FILE* fd = fopen(filename, "r");
    if (fd == NULL) {
        ESP_LOGE(BT_APP_CORE_TAG, "Failed to open file for reading");
        return;
    }
    else {
        ESP_LOGI(BT_APP_CORE_TAG, "%s was opened FD = %p", filename, fd);
        while (!feof(fd)) // to read file
        {
            int n = fread(snd_buff, BLOCK_SIZE, 1, fd);
            ESP_LOGI(BT_APP_CORE_TAG, "%d bytes read 0x%02x,0x%02x", n, ((uint16_t*)snd_buff)[0], ((uint16_t*)snd_buff)[1]);
        }
        fclose(fd);
    }

I (25681) BT_APP_CORE: Initializing SPIFFS
I (27831) BT_APP_CORE: Partition size: total: 956561, used: 143572
I (27971) BT_APP_CORE: !!! /spiffs/drip.raw is there
I (27971) BT_APP_CORE: Opening file /spiffs/drip.raw
I (27971) BT_APP_CORE: /spiffs/drip.raw was opened FD = 0x3ffb6d2c
I (27981) BT_APP_CORE: 0 bytes read 0x00,0x00
I (27981) BT_APP_CORE: 0 bytes read 0x00,0x00
...

[ESP_Sprite]

Not sure if that's the problem, but note that fread() always tries to read an integer multiple of `size` blocks (and returns the amount of blocks read, not the amount of bytes as you assume). In your code, if BLOCK_SIZE is larger than the size of the file, you get 0 as a result. To read bytes, change your fread call to fread(snd_buff, 1, BLOCK_SIZE, fd);

[bcolbert]

Thank you for the reply ESP_Sprite. I tried both forms, what I posted and

Code: Select all

fread(snd_buff, 1, BLOCK_SIZE, fd);

and it still returns 0 with no data and feof() never triggers. It's all very disappointing.

[ESP_Sprite]

Hm, that should work. What version of esp-idf are you using? Can you also replicate this in a minimal example?

[bcolbert]

Thank you again for the reply.

I'm using IDF version:

% idf.py --version
ESP-IDF v4.4-dev-1849-g8e3e65a47-dirty

I'll put together a simple version and see what happens.

[mbratch]

Be careful with `feof`. Assuming it works in the standard way in the ESP32, `feof` is true after you have attempted to read past the end of file. It is not true if you happen to be at the end of file but have not attempted to read past it yet.

If for some (currently unknown) reason `fread` isn't reading bytes from the file, then of course `feof` will never be true in any event.

[ndjurov]

I just had a small issue with reading from spiffs file.
I use it to store the application debug logs, so at the start of application run I open it for appending only.
Once I want to read it I do separate fopen() with "r" mode and get another file pointer, read the log and send it to a host.
It turns out that before reading I forgot to close the appending FILE *, and reading went way beyond the end of the file into the territory where it certainly shouldn't be. After changing to code to make sure that the appending pointer is closed, all reading was correct.
I really don't know if this is expected behavior, but I doubt. Is it supposed to be "undefined" - I also don't know.