Store and read some static files using SPIFFS

[dlawnstjd]

I am a beginner in embedded programming. I’m programming with the ESP32DevKitC(https://www.adafruit.com/products/3269). I want to flash some static files using a SPIFFS filesystem, and try to read those files in run-time. To do that, I made a SPIFFS filesystem image that contains some static files using mkspiffs(https://github.com/igrr/mkspiffs), flashed that filesystem image, and mount SPIFFS to VFS via esp_vfs_register() and SPIFFS_mount() functions.
I succeeded in opening a new file, write some text to the file, and read the text from the file again. However, when I try to open the static files that uploaded by mkspiffs using SPIFFS_open() function, I got a error code(-10002, NOT FOUND).

I've tried several different things, but it's not working. Here are some snippet code I wrote.

- make spiffs image and flash it.
<Makefile>

Code: Select all

flashfs:
	mkspiffs -c filesystem -b 65536 -p 256 -s 524288 $(BUILD_DIR_BASE)/spiffs.img
	$(ESPTOOLPY_WRITE_FLASH) 0x180000 $(BUILD_DIR_BASE)/spiffs.img

- mount vfs
<my_spiffs.c>

Code: Select all

void vfs_spiffs_register() {
    esp_vfs_t vfs = {
        .fd_offset = 0,
        .flags = ESP_VFS_FLAG_DEFAULT,
        .write = &vfs_spiffs_write,
        .open = &vfs_spiffs_open,
        .fstat = &vfs_spiffs_fstat,
        .close = &vfs_spiffs_close,
        .read = &vfs_spiffs_read,
        .lseek = &vfs_spiffs_lseek,
        .stat = &vfs_spiffs_stat,
        .link = NULL,
        .unlink = &vfs_spiffs_unlink,
        .rename = &vfs_spiffs_rename,
    		.mkdir = &vfs_spiffs_mkdir,
    		.opendir = &vfs_spiffs_opendir,
    		.readdir = &vfs_spiffs_readdir,
    		.closedir = &vfs_spiffs_closedir
    };

    ESP_ERROR_CHECK(esp_vfs_register("/spiffs", &vfs, NULL));

    // Mount spiffs file system
    spiffs_config cfg;
    int res = 0;

    cfg.phys_size = 512*1024;
    cfg.phys_addr = 0x180000;
    cfg.phys_erase_block = 65536;
    cfg.log_block_size = 65536;
    cfg.log_page_size = 256;
    cfg.hal_read_f = (spiffs_read)low_spiffs_read;
    cfg.hal_write_f = (spiffs_write)low_spiffs_write;
    cfg.hal_erase_f = (spiffs_erase)low_spiffs_erase;

    res = SPIFFS_mount(&fs,
     		&cfg,
     		spiffs_work_buf,
     		spiffs_fds,
     		sizeof(spiffs_fds),
     		spiffs_cache_buf,
     		sizeof(spiffs_cache_buf),
     		NULL);

    if (res < 0) {
      ESP_LOGD(TAG, "spiffs mount error");
      return;  
    }
    ESP_LOGD(TAG, "spiffs mounted successfully");
}

- success code for creating a new file, writing, reading
<success.c>

Code: Select all

static void test_spiffs() {
    char buf[12];

    spiffs_file fd = SPIFFS_open(&fs, "my_file", SPIFFS_CREAT | SPIFFS_TRUNC | SPIFFS_RDWR, 0);
    if (SPIFFS_write(&fs, fd, (u8_t *)"Hello world", 12) < 0) printf("errno %i\n", SPIFFS_errno(&fs));
    SPIFFS_close(&fs, fd);

    fd = SPIFFS_open(&fs, "my_file", SPIFFS_RDWR, 0);
    if (SPIFFS_read(&fs, fd, (u8_t *)buf, 12) < 0) printf("errno %i\n", SPIFFS_errno(&fs));
    SPIFFS_close(&fs, fd);

    printf("--> %s <--\n", buf);
}

- fail code for reading a static file in spiffs filesystem image that flashed at build time.
<client.c>

Code: Select all

spiffs_file src = SPIFFS_open(&fs, "/init.txt", SPIFFS_RDONLY, 0); // SPIFFS_open returns -10002

Any help will be greatly appreciated.

[ESP_igrr]

First of all, does the configuration of the SPIFFS library you add to the ESP32 project match the configuration of the SPIFFS library used in mkspiffs? That includes same options in the config header files, same version, and so on.

Second (unrelated to your problem, but an issue nevertheless): you are initalizing members of esp_vfs_t with esp_vfs_write, esp_vfs_read, and so on. Have you noticed that these functions don't have the right signature (set of arguments)?

As mentioned in VFS docs, esp_vfs_read/write/open/close/.. functions are used by newlib (C library) to talk to the VFS library. Basically, these functions implement system calls (read/write/open/close).
The functions you need to fill into the esp_vfs_t structure, on the other hand, are the functions which the VFS library will use to talk to the filesystem driver (in your case, SPIFFS).

So the chain of calls looks roughly like:
fopen -> open -> ('_open' member of syscall table) -> esp_vfs_open -> ('open' member of esp_vfs_t structure) -> my_spiffs_open (the function you need to write) -> SPIFFS_open.

For an example of functions which go into the esp_vfs_t structure, see the functions used to connect FATFS to the VFS:
https://github.com/espressif/esp-idf/bl ... /vfs_fat.c

[WiFive]

Maybe you want to consider the spiffs from mongoose-os or one of the lua projects for esp32

[dlawnstjd]

First of all, does the configuration of the SPIFFS library you add to the ESP32 project match the configuration of the SPIFFS library used in mkspiffs? That includes same options in the config header files, same version, and so on.

Second (unrelated to your problem, but an issue nevertheless): you are initalizing members of esp_vfs_t with esp_vfs_write, esp_vfs_read, and so on. Have you noticed that these functions don't have the right signature (set of arguments)?

As mentioned in VFS docs, esp_vfs_read/write/open/close/.. functions are used by newlib (C library) to talk to the VFS library. Basically, these functions implement system calls (read/write/open/close).
The functions you need to fill into the esp_vfs_t structure, on the other hand, are the functions which the VFS library will use to talk to the filesystem driver (in your case, SPIFFS).

So the chain of calls looks roughly like:
fopen -> open -> ('_open' member of syscall table) -> esp_vfs_open -> ('open' member of esp_vfs_t structure) -> my_spiffs_open (the function you need to write) -> SPIFFS_open.

For an example of functions which go into the esp_vfs_t structure, see the functions used to connect FATFS to the VFS:
https://github.com/espressif/esp-idf/bl ... /vfs_fat.c

Thanks you for your advice!!

[aaquilina]

Can anyone explain how to determine an appropriate memory space to flash 1MB of data using esptool.py? I tried looking at the Address Mapping shown below and I see that the external memory is defined from 0x3F400000 to 0x3F7FFFFF. However, I noticed @dlawnstjd flashed it to 0x180000. Can anyone guide me through the process of determining an appropriate memory offset please?

[WiFive]

Can anyone explain how to determine an appropriate memory space to flash 1MB of data using esptool.py? I tried looking at the Address Mapping shown below and I see that the external memory is defined from 0x3F400000 to 0x3F7FFFFF. However, I noticed @dlawnstjd flashed it to 0x180000. Can anyone guide me through the process of determining an appropriate memory offset please?

http://esp-idf.readthedocs.io/en/latest ... ables.html

[kolban]

If we think of flash memory ... say 4MBytes of it ... as having an address from 0x00 0000 to 0x3F FFFF ... when we use the flash tool, this is where our data is stored. Putting that aside, now think of the CPU that runs the ESP32 ... this has an address map within it which is the addressable sections of memory seen by the CPU. The flash memory is "mapped" into the address range of the CPU ... so byte 0x00 0000 maps to the CPU address at 0x3F40 0000. Putting it simpler, there are two (or more) addressing systems ... 1 for the flash memory and one for the CPU.

[aaquilina]

ah okay, makes sense. Then when using esptool.py, you're using the flash addressing. So if I understand it correctly, I can therefore write it to 0x10 0000 and assuming my code isn't larger than 1M-0x1 0000 (the app flash offset) I shouldn't have any problems.

I also wanted to ask if anyone has come across a SPIFFS_ERR_INDEX_REF_LU error while reading using the spiffs_object_read function. This is called from the SPIFFS_read but I don't understand what the error means. I know for sure that the image I flashed using esptool is present because while debugging the SPIFFS_read function, fd->size which I assume the is the size of the file pointed to by the file descriptor i require is exactly the same size i get using mkspiffs.

[ESP_igrr]

You should use the partition table to define the region of flash used for SPIFFS, and then use esp_partition_{read,write,erase_range} functions to work with it.

The mapping between the physical addresses in Flash and the CPU address space is not linear (as described above) and can be configured at runtime using Flash MMU. See the Technical Reference Manual for details about Flash MMU. ESP-IDF provides esp_partition_mmap function to set up mapping between Flash and CPU address space if you wish to read from the partition via the cache.