HELP NEEDED: Arduino ESP32 Support RC1 test (version 2.0.0)

[pataga]

This isn't a bug report but a message of thanks

I was previously using a complex build setup using esp-idf with arduino as a component, spefically esp-idf 3.3.5 and arduino-esp32 v1.06 with menuconfig tweaks required to get things working.

I just refactored the code to work with Visual Studio Code + PlatformIO + Espressif esp32dev platform + Arduino framework using the platformio.ini dependencies noted in this thread.

Code: Select all

[env:esp32dev]
;platform = espressif32
platform = https://github.com/platformio/platform-espressif32.git#feature/idf-master

platform_packages =	framework-arduinoespressif32 @ https://github.com/espressif/arduino-esp32#2.0.0-rc1

board = esp32dev
framework = arduino
board_build.f_cpu = 80000000L

; 4MB flash, spiffs partition = 0x30000 bytes, app partition = 0x1E0000 bytes x 2 (OTA firmware update support)
board_build.partitions = min_spiffs.csv 
upload_port = /dev/ttyUSB*
upload_speed = 921600
monitor_port = /dev/ttyUSB*
monitor_speed = 115200
monitor_filters = esp32_exception_decoder
build_flags = 
    -DCORE_DEBUG_LEVEL=ARDUHAL_LOG_LEVEL_DEBUG
    -DCONFIG_FREERTOS_HZ=1000
    -DCONFIG_MAIN_TASK_STACK_SIZE=16384
lib_deps = 
    https://github.com/me-no-dev/AsyncTCP.git
    https://github.com/me-no-dev/ESPAsyncWebServer.git

The project repository is
https://github.com/har-in-air/ESP32_IMU_BARO_GPS_VARIO

And it works ... the VSC IDE with PlatformIO is so much easier to work with including git integration. The only problem I am facing is a SPIFFS related issue. I am using ESP32AsyncWebServer. When I try to upload a file to the SPIFFS partition and it already exists, deleting the existing file and opening a new file to write does not work. The upload completes without error, but the resulting file listing shows a 0bytes file.

However, if I execute two separate webserver transactions, one to delete the file and the second to upload the replacement file, it works. If you are aware of SPIFFS file replacement issues or tips, please let me know ...

Thanks again

[ullixesp]

I didn't know one can set values found in `sdkconfig.h` from pio' ini file. That was helpful for me!

Re SPIFFS, I can only strongly suggest to reconsider the choice of SPIFFS, and suggest to use LittleFS. SPIFFS performance is terrible once the Flash is half-full https://forum.arduino.cc/t/esp32-mit-sp ... phe/621573

[pataga]

Re SPIFFS, I can only strongly suggest to reconsider the choice of SPIFFS, and suggest to use LittleFS.

I checked it out, and I saw a recommendation not to use LittleFS if the files were small, as there was too much overhead per file with LittleFS. I have a 192kBytes SPIFFS partition, the files being read / written are generally less than a kByte in size, maybe 10 files worst case.

I also had a problem with watchdog resets generated by AsyncTCP task when downloading data logs from external spi flash - these can be up to 16Mbytes in size, and I use the chunked download method. I ended up inserting a "feed_watchdog" routine for each call to the download chunk routine. That seems to have fixed the issue, though I am not sure if this is the 'correct' way to do it. I saw advice on StackOverflow that lengthy calls should not be in the AsyncRequest callbacks, the main loop should handle them triggered by flags set in the callbacks. But I don't see how that will work for this callback as it needs to return with the number of bytes read for the current chunk ... :

[ullixesp]

@pataga Since you have that few and that small files, why not use NVS? Easier and more robust than any File system. I use it for some dozen of variables, plus two long strings. Strings can be 4k, blobs even >500k(!).

Can you provide more details on your WDT approaches? When downloading a file or a string/String of fixed size then chunking should not be needed at all, as the final size is already known. I did chunking, but found the same WDT problem when the chunks became too big. With small chunks, it takes wayyyy too long to download.

I looked into the command processing in ESPAsyncWebserver: it has finished processing long before the WDT crash comes, and has forwarded to AsyncTCP. This also has long finished when the crash comes. This next step seems to be the crucial step, and there I am stuck. I believe WiFi is the culprit.

[pataga]

Here's the relevant code bits for the chunked spi flash data log download (external SPI flash, up to 16Mbytes). I logged the callbacks, the AsyncRequest callback is asking for approximately 1 to 2 kbytes per chunk.

Prior to refactoring the code, I was using blocking code that sent 256 bytes at a time and yielded using vTaskDelay for one FreeRTOS tick. That worked fine to keep the WDT happy. But now I have switched to EspAsyncWebServer, and it's kind of magic - not sure what's going on in the background, especially when issues like this bite

Watchdog feed code

Code: Select all

#include <driver/timer.h>

void feed_watchdog() {
    TIMERG0.wdt_wprotect = TIMG_WDT_WKEY_VALUE;
    TIMERG0.wdt_feed = 1;
    TIMERG0.wdt_wprotect = 0;
    }

Chunked read from spi flash

Code: Select all

// chunked download for the IMU/GPS data log stored in external spi flash. 
// returns buffer with up to maxLen bytes read
static int datalog_chunked_read(uint8_t* buffer, size_t maxLen, size_t index) {
  int bytesRemaining = (int)(FlashLogFreeAddress - (uint32_t)index);
  if (bytesRemaining) {
    int numBytes =  bytesRemaining > maxLen ? maxLen : bytesRemaining;  
    spiflash_readBuffer(index, buffer, numBytes);
    return numBytes;
    }
  else {
    return 0;
    }
  }

  Server configuration 
  ...
  server->on("/datalog", HTTP_GET, [](AsyncWebServerRequest * request) {
    ESP_LOGD(TAG,"Client: %s %s",request->client()->remoteIP().toString().c_str(), request->url().c_str());
    if (FlashLogFreeAddress) {
      AsyncWebServerResponse *response = request->beginResponse("application/octet-stream", FlashLogFreeAddress, [](uint8_t *buffer, size_t maxLen, size_t index) -> size_t {
        feed_watchdog(); // prevent watchdog resets when downloading large files
        return datalog_chunked_read(buffer, maxLen, index);
      });
      response->addHeader("Content-Disposition", "attachment; filename=datalog");
      response->addHeader("Connection", "close");
      request->send(response);
      }
    else {
      request->send(400, "text/plain", "ERROR : no data log found");
      }
    });
    ...
    

[pataga]

Re SPIFFS, I can only strongly suggest to reconsider the choice of SPIFFS, and suggest to use LittleFS.

I checked it out, and I saw a recommendation not to use LittleFS if the files were small, as there was too much overhead per file with LittleFS. I have a 192kBytes SPIFFS partition, the files being read / written are generally less than a kByte in size, maybe 10 files worst case.

I ended up changing to LittleFS, and it works reliably. No issue with uploading replacement files like I had with SPIFFS.

I currently only have 6 files totalling < 4kbytes, LittleFS is using ~20kBytes. But I can live with the overhead.

[pataga]

I'm seeing an exception that I don't understand at all. Using Visual Studio Code + Platformio + esp32dev platform using Arduino framework with arduino-esp322.0.0-rc1.

Code: Select all

[env:esp32dev]
;platform = espressif32
platform = https://github.com/platformio/platform-espressif32.git#feature/idf-master

platform_packages =	framework-arduinoespressif32 @ https://github.com/espressif/arduino-esp32#2.0.0-rc1

board = esp32dev
framework = arduino
board_build.f_cpu = 80000000L

; 4MB flash, LittleFS partition = 0x28000 bytes, app partition = 0x1E0000 bytes x 2 (OTA firmware update support)
board_build.partitions = partitions.csv 
upload_port = /dev/ttyUSB*
upload_speed = 921600
monitor_port = /dev/ttyUSB*
monitor_speed = 115200
build_type = debug
monitor_filters = esp32_exception_decoder
build_flags = 
    -DCORE_DEBUG_LEVEL=ARDUHAL_LOG_LEVEL_DEBUG
lib_deps = 
    https://github.com/me-no-dev/AsyncTCP.git
    https://github.com/me-no-dev/ESPAsyncWebServer.git
    https://github.com/lorol/LITTLEFS.git
extra_scripts = ./littlefsbuilder.py

The application was working fine, I then decided to change the lcd spi interface to use non-blocking queued transactions for extra performance.

I used the example lcd spi code from esp-idf as a base and tested the code in a different framework (esp-idf 3.3.5 with arduino 1.06 as a component). That works fine, am able to draw to the frame buffer and transmit it to the LCD via queued transactions indefinitely without issues. I then introduced the same lcd driver into this application (VSC + platformio, arduino framework using 2.0.0-rc1). It has no problem initializing the lcd and going through a number of screen re-draws as the app initializes different software modules, but then crashes for a seemingly unrelated reason (LittleFS file transaction). I tried quadrupling the task stack size, but the crash still occurs at the same place. If I replace the lcd driver code with my old blocking byte by byte transmission, no problems.

What I find strange is that the exception decoder is pointing to a ESP32-S2 function call but I have defined the platform as esp32dev. Also, why is the decoder only able to decode one function call, in Arduino I get the full stack trace ?

Code: Select all

[ 24117][I][mpu9250.cpp:456] mpu9250_calibrateGyro(): Calibrating gyro
[ 24836][D][calib.cpp:150] calib_save(): deleting calib.txt
Guru Meditation Error: Core  1 panic'ed (LoadProhibited). Exception was unhandled.

Core  1 register dump:
PC      : 0x40081ef1  PS      : 0x00060230  A0      : 0x800f80d1  A1      : 0x3ffd8700  
A2      : 0x3ffc0864  A3      : 0x3ffbbc7c  A4      : 0x003d9000  A5      : 0x00000080  
A6      : 0x0404087c  A7      : 0x00000013  A8      : 0x00000000  A9      : 0x3ffd8910  
A10     : 0x3ffd870c  A11     : 0x3ffbcaad  A12     : 0x3ffd8934  A13     : 0x3ffd8b2c  
A14     : 0x00ff0000  A15     : 0x00000003  SAR     : 0x0000000f  EXCCAUSE: 0x0000001c  
EXCVADDR: 0x04040884  LBEG    : 0x40090e2c  LEND    : 0x40090e37  LCOUNT  : 0x00000000  

Backtrace:0x40081eee:0x3ffd87000x400f80ce:0x3ffd8740 0x400f0f41:0x3ffd8770 0x400ed167:0x3ffd87a0 0x400edd49:0x3ffd87d0 0x400f0211:0x3ffd8860 0x400f0bad:0x3ffd88c0 0x400f0ce9:0x3ffd8910 0x400ec359:0x3ffd8930 0x400fe87d:0x3ffd8a60 0x401f153d:0x3ffd8a80 0x401e8427:0x3ffd8aa0 0x401e8609:0x3ffd8ad0 0x401fdef9:0x3ffd8b50 0x400d51c1:0x3ffd8b70 0x400da2f2:0x3ffd8c00 0x400d4686:0x3ffd8c50 0x400d4f96:0x3ffd8c80 
  #0  0x40081eee:0x3ffd87000 in esp_flash_read at /Users/ficeto/Desktop/ESP32/ESP32S2/esp-idf-public/components/spi_flash/esp_flash_api.c:674

ELF file SHA256: 0000000000000000

Rebooting...