ESP IDF Upload .html file into SPIFS while programming image.

[ketan vadodariya]

Hi,

I am using ESP32 custom board and I planning to have inbuilt web-page into esp32 device. I have to installed the .html file and javascript files into the spiffs partition while programming.

So how to preload the web pages files into spiffs partition table. ?

[ESP_Sprite]

You probably want to check the docs: if you put 'spiffs_create_partition_image(my_spiffs_partition my_folder FLASH_IN_PROJECT)' in one of the component CMakeLists.txt, it'll automatically build and flash the spiffs partition.

[ketan vadodariya]

Hi,
Thank you for you update.

Yes I tried with the same option but getting error while building the project.

I have a added following line into the CMakeList.txt file and created folder where CMakeList.txt is exisit.

Code: Select all

cmd.exe /C "cd /D D:\Ketan\Wimera\ESP32\RemonApplication32\build\esp-idf\main && python C:/Espressif/frameworks/esp-idf-v4.4.1/components/spiffs/spiffsgen.py 0xc800 D:/Ketan/Wimera/ESP32/RemonApplication32/main/webpage D:/Ketan/Wimera/ESP32/RemonApplication32/build/storage.bin --page-size=256 --obj-name-len=32 --meta-len=4 --use-magic --use-magic-len"
Traceback (most recent call last):
  File "C:/Espressif/frameworks/esp-idf-v4.4.1/components/spiffs/spiffsgen.py", line 604, in <module>
    main()
  File "C:/Espressif/frameworks/esp-idf-v4.4.1/components/spiffs/spiffsgen.py", line 591, in main
    spiffs = SpiffsFS(image_size, spiffs_build_default)
  File "C:/Espressif/frameworks/esp-idf-v4.4.1/components/spiffs/spiffsgen.py", line 397, in __init__
    raise RuntimeError('image size should be a multiple of block size')
RuntimeError: image size should be a multiple of block size

[ESP_Sprite]

It tells you the issue:

Code: Select all

RuntimeError: image size should be a multiple of block size

Make sure your spiffs partition size is a multiple of 4096 bytes and this should go away.

[ketan vadodariya]

Hi,

I changed partition size then it's worked.

After the flashing the web-page into the spiffs, then later I have modify the web-page, then I can update the bin file without flashing through UART ?. I have a OTA application which is running in application.

[mbratch]

After the flashing the web-page into the spiffs, then later I have modify the web-page, then I can update the bin file without flashing through UART ?. I have a OTA application which is running in application.

I just went through this with my application. The OTA only downloads one bin file, so it cannot do a SPIFS partition separately. Therefore, your code needs to handle the update of the SPIFS, if needed, when it boots up.

There are probably a several different ways of handling this. My SPIFS web page partition has several files, so what I did was:

1. Changed my build process so that it creates the spiffs partition bin explicitly
2. With the spiffs file as a dependency, I add the binary spiffs image to my final bin using `target_add_binary_data`
3. When my application boots, rather than try to manage image versions or anything like that, it simply compares the binary data attached to the application, byte for byte, with the current spiffs partition content. If they don't match, I write the attached binary spiffs data to the spiffs partition with `esp_partition_erase_range` and `esp_partition_write`.
   
   This assumes that the size of my partition doesn't ever change later on, so I allocated as much as I think I'd ever need. If the size of the partition doesn't match the size of the attached data, I just generate an error and move on. I could have gotten into adjusting the partition size at run time, and, therefore, adjust the partition table at run time, but it is an unnecessary complexity in this case. I just use a fixed partition table for my application which allocates the OTA partitions, etc.

My `CMakeList.txt` at the project level creates and attaches the spiffs partition file as a binary blob to my main application binary. My web page files are all under a project subfolder called `web_root`, and the spiffs partition file name is `www.bin`.

Code: Select all

set(WWW_SIZE "0x30000")
set(WWW_DIR ${CMAKE_CURRENT_SOURCE_DIR}/web_root)
set(SPIFFSGEN_DIR ${IDF_PATH}/components/spiffs)
set(WWW_FILE "www.bin")

file(GLOB_RECURSE WWW_SRC ${WWW_DIR}/*.*)

add_custom_command(OUTPUT ${BUILD_DIR}/${WWW_FILE}
    COMMAND python ${SPIFFSGEN_DIR}/spiffsgen.py ${WWW_SIZE} ${WWW_DIR} ${BUILD_DIR}/${WWW_FILE}
                    --page-size=256 --obj-name-len=64 --meta-len=4 --use-magic --use-magic-len DEPENDS ${WWW_SRC}
)

target_add_binary_data(${PROJECT_NAME}.elf ${BUILD_DIR}/${WWW_FILE} BINARY)
add_custom_target(www_fs DEPENDS ${BUILD_DIR}/${WWW_FILE})
add_dependencies(${PROJECT_NAME}.elf www_fs)

If you Google search you can find the documentation for the particular commands used. I looked at what the tool chain used by default when it calls `spiffsgen.py` in order to determine the arguments I needed when I used `spiffsgen.py`.

If you don't have a lot of file content, you could just include your files as literal text data in your C program and create the spiffs partition and create the files. For me, what I did above was pretty easy. The time taken to write the new spiffs partition is just a few seconds.

[ketan vadodariya]

Hi,

Thank you for the details explanation, but I am not much expert on ESP32 device. This is the first time I used esp32 so not much knowledge on partition handling and spiffs stuff.

I just went through this with my application. The OTA only downloads one bin file, so it cannot do a SPIFS partition separately. Therefore, your code needs to handle the update of the SPIFS, if needed, when it boots up.

Would be mind to share the some sample code for handling such kind of stuff. ?

[mbratch]

Hi,

Thank you for the details explanation, but I am not much expert on ESP32 device. This is the first time I used esp32 so not much knowledge on partition handling and spiffs stuff.

I just went through this with my application. The OTA only downloads one bin file, so it cannot do a SPIFS partition separately. Therefore, your code needs to handle the update of the SPIFS, if needed, when it boots up.

Would be mind to share the some sample code for handling such kind of stuff. ?

I figured this out by reading the ESP-IDF documentation online, which is pretty good documentation.

The `CMakeList.txt` file excerpt I showed in my prior post is what's needed to do the steps #1 and #2 I mentioned.

To do step #3, you need to be able to access the data that was attached in steps #1 and #2. This is described in the ESP-IDF documentation for the Build System, under Embedding Binary Data. It gives a pretty clear example of the simple mechanism used to access the data.

Once you can access the data (which in this case, represents what you want your new SPIFFS partition contents to be), you'll want to compare that to the existing SPIFFS partition, one block of memory at a time. I picked a block size of 512 just so that I could stop the compare process pretty quickly if a difference is found (because then it's confirmed it's different, and the new partition data needs to be written).

Functions for accessing the partition data are described in ESP-IDF documentation under SPI Flash API. Here, you'll find the following functions that you'll need:

`esp_partition_find_first` - finds a partition with specified attributes, such as partition type (so you can find the first SPIFFs partition, for example). It returns a pointer to a partition descriptor (a "handle", basically) which is type `esp_partition_t`.

`esp_partition_read` - reads data from the partition specified by the given partition descriptor

`esp_partition_erase_range` - erases a range of addresses in the given partition. If you check the ESP-IDF documentation, it says you need to erase the partition before you can re-write it.

`esp_partition_write` - writes data to the partition specified by the given partition descriptor.

These are the essential building blocks you need. Have a go at it and read the documentation. If you're not totally sure you can always try something as an experiment and see if what you tried works.

[ketan vadodariya]

Hi,

Thank you for explanation.

I think working after made changed on CMakeList.txt.

I have a modify My CMakeList.txt files as per you showed.