How to upload "data" to an ESP32 under Arduino 2.0

[ullixesp]

Uploading a "data" dir under Arduino 1.x is easily possible with this plugin: https://github.com/me-no-dev/arduino-esp32fs-plugin . It integrates into the Ar IDE and can then be used from its menu.

However, this plugin is written in Java, as is the old Ar IDE 1.x. The new Arduino 2.0 is written in different language(s), and Java is not part of it. I see no chances that the old plugin might work; it might even ruin an existing installation.

Is there a new plugin available fitting into the Ar IDE 2.0 environment?

If not, what options do exist as alternative? I use OTA successfully, but it takes a huge overhead to program when all you want to do is a simple test.

[chegewara]

1. command line tool
2. if you by any chance also using vs code with platformio then you can use it to flash spiffs/fatfs

[ullixesp]

1. command line tool

great idea. Which one?

[chegewara]

1. command line tool

great idea. Which one?

spiffsgen.py

https://github.com/espressif/esp-idf/tr ... /spiffsgen
https://docs.espressif.com/projects/esp ... piffs.html

Then you can flash with esptool or with flash download tool on windows.

[ullixesp]

Ok, thanks. That seems to be doable for SPIFFS.

Am I right in thinking that this is ONLY for the SPIFFS file system? SPIFFS is a catastrophe when the Flash fills up https://forum.arduino.cc/t/esp32-mit-sp ... phe/621573

so my only options are LittleFS and FFat. Anything available for those?

[lbernstone]

https://github.com/lorol/arduino-esp32f ... /README.md
This is not going to be a one touch solution until someone figures out the new plug-in method (is it even fully baked yet?). The logic is in lorol's repo, so it would just be a matter of porting to whatever the new Arduino plug-in system uses.

[chegewara]

so my only options are LittleFS and FFat. Anything available for those?

There is another way, but very hacky and not elegant and can be used with SPIFFS and FFat.
Steps:
- use arduino-esp32 v1.0.x to flash data using plugin, you can use empty schetch to do it,
- use esptool to dump that partition to a file,
- flash that partition to your v2.0.0 project esp32 using one of methods i mentioned earlier.

[ullixesp]

@chegewara Undoubtedly, this is a "hacky" approach . I think I rather suffer the need to put in the code for my OTA.

It just underlines the need for "someone to figure out..." as stated by @ibernstone.

[palmerr23]

I have written a utility to overcome this issue

https://github.com/palmerr23/ESP32-OTA-and-File-Manager

It provides a web-based interface for
* OTA
* File system formatting
* FS directory listing
* File uploading
* File editing
* File deletion

[DOugL-]

I often will pop into a commandline to quickly tweek an ESP32 project instead of bringing up the Arduino IDE 2.x so I created a Makefile to compile some of my projects and because of the SPIFFS /LITTLEFS issue of the missing IDE 2.x GUI tool I added the capability to the Makefile. You will have to tweak you device port and some of the paths to where you put things but here's a semi-generic Makefile I copy to each of my ESP32 project directories. If on Windows you'll need to run it from a Linux container or something since I use make, grep, cat, cut, xargs, printf and maybe some other standard *nix tools to figure out flash filesystem settings based on chips used.

I compile using: make build
I compile and flash using: make build flash
I create my SPIFFS or LittleFS image using: make filesystem.bin
I upload my filesystem image using: make flash-fs

Code: Select all

#
# Usage: make build | flash | filesystem.bin | flash-fs | clean
#
CHIP        := esp32
sketch      := YOUR_SKETCH_HERE.ino
CORE        :=esp32:esp32
# Flashing on  an "ESP32 Dev Module" board
boardconfig :="${CORE}:${CHIP}"

ARDUINO_PATH=~/Projects/Arduino/arduino-ide_nightly-20230206_Linux_64bit/resources/app/node_modules/arduino-ide-extension/build/
ESP32_PATH=~/.arduino15/packages/esp32/hardware/esp32/2.0.7/
ARDUINO_CLI ?= $(ARDUINO_PATH)/arduino-cli
TOOL_PATH=~/.arduino15/packages/esp32/tools/
MKSPIFFS    ?= ~/.arduino15/packages/esp32/tools/mkspiffs/0.2.3/mkspiffs
MKLITTLEFS  ?= ~/.arduino15/packages/esp32/tools/mklittlefs/3.0.0-gnu12-dc7f933/mklittlefs

# Partition table configuration options:
# default default_ffat ffat huge_app large_spiffs_16MB max_app_8MB 
# minimal min_spiffs noota_3g
# app3M_fat9M_16MB bare_minimum_2MB default_16MB default_8MB
# noota_3gffat no_ota noota_ffat
# rainmaker
PARTITIONTYPE := default
#PARTITIONTYPE := min_spiffs
PARTITIONSIZE := "${PARTITIONTYPE}.csv"
PARTITIONMAX := `cat $(ESP32_PATH)/tools/partitions/${PARTITIONSIZE}  | grep app0 | cut -f5 -d',' | xargs printf "%d" | cut -f1 -d'('`
PARTITION_TABLE=$(ESP32_PATH)/tools/partitions/${PARTITIONSIZE}

partitioncmd := --build-property build.partitions=${PARTITIONTYPE} --build-property upload.maximum_size=${PARTITIONMAX}


DEVICE :=/dev/ttyUSB0
#DEVICE :=/dev/ttyACM0

.PHONY: build
build:
        $(ARDUINO_CLI) compile --verbose --fqbn $(boardconfig) $(sketch) $(partitioncmd)

.PHONY: flash
flash:
        $(ARDUINO_CLI) upload --verbose -p ${DEVICE} --fqbn ${boardconfig} ${sketch} $(partitioncmd)

.PHONY: filesystem.bin
.ONESHELL:
filesystem.bin:
        PROPS=$$($(ARDUINO_CLI) compile --fqbn $(boardconfig) --show-properties)
        BUILD_FS_BLOCKSIZE=4096
        BUILD_FS_PAGESIZE=256
        BUILD_FS_START_HEX=$$(cat ${PARTITION_TABLE} | grep "^spiffs"|cut -d"," -f4 | xargs)
        #echo "BUILD_FS_START_HEX = $${BUILD_FS_START_HEX}"
        BUILD_FS_START=`printf "%d" $${BUILD_FS_START_HEX}`
        BUILD_FS_SIZE_HEX=$$(cat ${PARTITION_TABLE} | grep "^spiffs"|cut -d, -f5 | xargs)
        BUILD_FS_SIZE=`printf "%d" $${BUILD_FS_SIZE_HEX}`
        $(MKSPIFFS) -c data --page $$BUILD_FS_PAGESIZE --block $$BUILD_FS_BLOCKSIZE --size $$BUILD_FS_SIZE $@
        #$(MKLITTLEFS) -c data --page $$BUILD_FS_PAGESIZE --block $$BUILD_FS_BLOCKSIZE --size $$BUILD_FS_SIZE $@

.PHONY: flash-fs
.ONESHELL:
flash-fs: filesystem.bin
        BUILD_FS_START_HEX=$$(cat ${PARTITION_TABLE} | grep "^spiffs"|cut -d, -f4 | xargs)
        python ~/.arduino15/packages/esp32/tools/esptool_py/4.5.1/esptool.py --chip ${CHIP} \
          --port ${DEVICE} \
          --baud 460800 \
          --before default_reset \
          --after hard_reset \
          write_flash $${BUILD_FS_START_HEX} filesystem.bin


.PHONY: clean
clean:
        rm -rf build
        rm -f filesystem.bin