Esp32 crash on nvs_set_blob (memcpy in ROM)

[brebisson]

Hello,

When attempting to save data in flash using NVS API, I get a crash.
My code will first call the begin function bellow, which seems to work..
But calls to save crashes. The object size is 168 bytes...
Here is the code and the error from ESP32-IDF.

Does anyone have any idea what is happening and why I get this crash?
Thanks,
Cyrille

Code: Select all

class SomeObject { public:
    void save()
    {
        nvs_handle_t my_handle;
        if (nvs_open("CdB500Brains", NVS_READWRITE, &my_handle)!=ESP_OK) return;
        printf("TSavedData::save %d %d\r\n", int(this), sizeof(*this));
        nvs_set_blob(my_handle, "save", (void*)this, sizeof(*this));
        nvs_commit(my_handle);
        nvs_close(my_handle);
    }
void begin()
    {
        nvs_handle_t my_handle;
        esp_err_t err = nvs_flash_init();
        if (err == ESP_ERR_NVS_NO_FREE_PAGES || err == ESP_ERR_NVS_NEW_VERSION_FOUND) { ESP_ERROR_CHECK(nvs_flash_erase()); err = nvs_flash_init(); }
        ESP_ERROR_CHECK(err);
        ESP_ERROR_CHECK(nvs_open("CdB500Brains", NVS_READWRITE, &my_handle));
        size_t size = sizeof(*this); err= nvs_get_blob(my_handle, "save", (void*)this, &size);
        if (err!=ESP_OK || size!=sizeof(*this)) { reset(); ESP_ERROR_CHECK(nvs_set_blob(my_handle, "save", (void*)this, sizeof(*this))); }
        ESP_ERROR_CHECK(nvs_commit(my_handle));
        nvs_close(my_handle);
    }
}
    

Code: Select all

TSavedData::save 1073426588 168  // This is my print function. It shows the object address and size (168 bytes)
Guru Meditation Error: Core  0 panic'ed (Cache disabled but cached memory region accessed). 

Core  0 register dump:
PC      : 0x400e2308  PS      : 0x00060034  A0      : 0x8008588b  A1      : 0x3ffb1040  
--- 0x400e2308: adc_hal_get_reading_result at C:/Users/cydeb/Documents/esp/v5.3.2/esp-idf/components/hal/adc_hal.c:232

A2      : 0x00000000  A3      : 0x3ffb2850  A4      : 0x80083c04  A5      : 0x3ffc4110
A6      : 0x3ffc5e94  A7      : 0x00000002  A8      : 0x800857c0  A9      : 0x00060623
A10     : 0x3ffc5e98  A11     : 0x3ffc60bc  A12     : 0x3ffb1044  A13     : 0x3ffb1048
A14     : 0x00000020  A15     : 0x00060623  SAR     : 0x00000018  EXCCAUSE: 0x00000007
EXCVADDR: 0x00000000  LBEG    : 0x4000c2e0  LEND    : 0x4000c2f6  LCOUNT  : 0x00000000
--- 0x4000c2e0: memcpy in ROM
0x4000c2f6: memcpy in ROM

Backtrace: 0x400e2305:0x3ffb1040 0x40085888:0x3ffb1080 0x40083099:0x3ffb10a0 0x4000c2dd:0x3ffc40f0 0x4008b92d:0x3ffc4100 0x4008bd25:0x3ffc4130 0x4008f9fb:0x3ffc4150 0x40084567:0x3ffc41d0 0x400e1666:0x3ffc4210 0x400dcb9b:0x3ffc4240 0x400f23e6:0x3ffc4260 0x400dd90a:0x3ffc4280 0x400dbcd2:0x3ffc42e0 0x400dc6f1:0x3ffc4340 0x400dc99d:0x3ffc43b0 0x400db743:0x3ffc43d0 0x400d7152:0x3ffc4400 0x400d71b2:0x3ffc4420 0x400d807a:0x3ffc4440 0x400f36e4:0x3ffc4460 0x400886b1:0x3ffc4490
--- 0x400e2305: adc_hal_digi_dma_link at C:/Users/cydeb/Documents/esp/v5.3.2/esp-idf/components/hal/adc_hal.c:229
0x40085888: adc_dma_intr_handler at C:/Users/cydeb/Documents/esp/v5.3.2/esp-idf/components/esp_adc/esp32/adc_dma.c:36
0x40083099: _xt_lowint1 at C:/Users/cydeb/Documents/esp/v5.3.2/esp-idf/components/xtensa/xtensa_vectors.S:1240
0x4000c2dd: memcpy in ROM
0x4008b92d: spi_flash_ll_get_buffer_data at C:/Users/cydeb/Documents/esp/v5.3.2/esp-idf/components/hal/esp32/include/hal/spi_flash_ll.h:137
0x4008bd25: spi_flash_hal_read at C:/Users/cydeb/Documents/esp/v5.3.2/esp-idf/components/hal/spi_flash_hal_common.inc:215
0x4008f9fb: spi_flash_chip_winbond_read at C:/Users/cydeb/Documents/esp/v5.3.2/esp-idf/components/spi_flash/spi_flash_chip_winbond.c:61
0x40084567: esp_flash_read at C:/Users/cydeb/Documents/esp/v5.3.2/esp-idf/components/spi_flash/esp_flash_api.c:899
0x400e1666: esp_partition_read at C:/Users/cydeb/Documents/esp/v5.3.2/esp-idf/components/esp_partition/partition_target.c:38
0x400dcb9b: nvs::NVSPartition::read(unsigned int, void*, unsigned int) at C:/Users/cydeb/Documents/esp/v5.3.2/esp-idf/components/nvs_flash/src/nvs_partition.cpp:37
0x400f23e6: nvs::Page::readEntry(unsigned int, nvs::Item&) const at C:/Users/cydeb/Documents/esp/v5.3.2/esp-idf/components/nvs_flash/src/nvs_page.cpp:855
0x400dd90a: nvs::Page::findItem(unsigned char, nvs::ItemType, char const*, unsigned int&, nvs::Item&, unsigned char, nvs::VerOffset) at C:/Users/cydeb/Documents/esp/v5.3.2/esp-idf/components/nvs_flash/src/nvs_page.cpp:908
0x400dbcd2: nvs::Storage::eraseMultiPageBlob(unsigned char, char const*, nvs::VerOffset) at C:/Users/cydeb/Documents/esp/v5.3.2/esp-idf/components/nvs_flash/src/nvs_storage.cpp:744 (discriminator 1)
0x400dc6f1: nvs::Storage::writeItem(unsigned char, nvs::ItemType, char const*, void const*, unsigned int) at C:/Users/cydeb/Documents/esp/v5.3.2/esp-idf/components/nvs_flash/src/nvs_storage.cpp:447
0x400dc99d: nvs::NVSHandleSimple::set_blob(char const*, void const*, unsigned int) at C:/Users/cydeb/Documents/esp/v5.3.2/esp-idf/components/nvs_flash/src/nvs_handle_simple.cpp:44
0x400db743: nvs_set_blob at C:/Users/cydeb/Documents/esp/v5.3.2/esp-idf/components/nvs_flash/src/nvs_api.cpp:441
0x400d7152: TSavedData::save() at C:/Users/cydeb/Desktop/Brains/main/main.cpp:184
0x400d71b2: CMWiFi::setAPName(char const*, char const*, bool) at C:/Users/cydeb/Desktop/Brains/main/messages.h:322
0x400d807a: app_main at C:/Users/cydeb/Desktop/Brains/main/main.cpp:672
0x400f36e4: main_task at C:/Users/cydeb/Documents/esp/v5.3.2/esp-idf/components/freertos/app_startup.c:208
0x400886b1: vPortTaskWrapper at C:/Users/cydeb/Documents/esp/v5.3.2/esp-idf/components/freertos/FreeRTOS-Kernel/portable/xtensa/port.c:139
    

[MicroController]

Does anyone have any idea what is happening and why I get this crash?

Code: Select all

Guru Meditation Error: Core  0 panic'ed (Cache disabled but cached memory region accessed). 

Access to a "cached memory region", i.e. flash or external RAM, while the NVS was writing to flash.

Code: Select all

--- 0x400e2308: adc_hal_get_reading_result at C:/Users/cydeb/Documents/esp/v5.3.2/esp-idf/components/hal/adc_hal.c:232

Ok, maybe something with the ADC.

Code: Select all

--- 0x400e2305: adc_hal_digi_dma_link at C:/Users/cydeb/Documents/esp/v5.3.2/esp-idf/components/hal/adc_hal.c:229
0x40085888: adc_dma_intr_handler at C:/Users/cydeb/Documents/esp/v5.3.2/esp-idf/components/esp_adc/esp32/adc_dma.c:36
0x40083099: _xt_lowint1 at C:/Users/cydeb/Documents/esp/v5.3.2/esp-idf/components/xtensa/xtensa_vectors.S:1240

Ah yes, an ADC-DMA interrupt fired and the handler likely tried to run from or access data in flash while the NVS write had the cache disabled.

The flash driver should be disabling normal interrupts during flash writes. Apparently, the ADC-DMA interrupt was erroneously registered with ESP_INTR_FLAG_IRAM although it actually is not fully IRAM, possibly because an ADC callback was registered which isn't in IRAM.

[brebisson]

Hello,

There is indeed an ADC running in my system with a call back!

And I just tried disabeling it and the crash seems to be gone.....

What am I doing wrong in it's declaration?

Code: Select all

static bool IRAM_ATTR s_conv_done_cb(adc_continuous_handle_t handle, const adc_continuous_evt_data_t *edata, void *user_data)
        {
            CADC *This= (CADC*)user_data;
            if (edata->size==nbChannels*nbSamples*2) // We have 2*8 samples here, so we do 16 additions, which should be fast enough for IRQ time...
            {
                uint16_t *d= (uint16_t*)(edata->conv_frame_buffer);
                uint16_t v[nbChannels]= {0};
                for (int i=nbSamples; --i>=0;) for (int j=nbChannels; --j>=0;) v[j]+= *d++;
                for (int j=nbChannels; --j>=0;) This->res[j]= v[j];
            }
            return pdFALSE;
        }

[MicroController]

Looks fine to me.

I noticed that the HAL functions, adc_hal_get_reading_result() and adc_hal_digi_dma_link(), which seem to be pointed to by the PC/backtrace, are not in IRAM though. But these actually don't seem to be called by the ISR.

How/where are nbChannels and nbSamples declared and defined?

[brebisson]

Hello,

> How/where are nbChannels and nbSamples declared and defined?
They are static const int in a class...
as in:
class CADC { public:
static int const nbChannels= 1;
static int const nbSamplesShift= 3;
static int const nbSamples= 1<<nbSamplesShift;

Thanks for taking the time to look at this and help me!

Cyrille

[MicroController]

> How/where are nbChannels and nbSamples declared and defined?
They are static const int in a class...
as in:
class CADC { public:
static int const nbChannels= 1;
static int const nbSamplesShift= 3;
static int const nbSamples= 1<<nbSamplesShift;

Hmm. This may or may not be a problem. Their values may be stored as data in flash, and the callback may try to load the values from flash.
As a test, can you try and replace the references to nbChannels and nbSamples in the callback with their respective literals, i.e. 1 and (1<<3)?

Or, preferably, just make them "static constexpr int nbChannels = ...".

[ESP_Sprite]

FWIW, the memcpy is a red herring; the registers that decodes to that are LBEG/LEND which is are zero-overhead loop registers that seemingly last were used in memcpy().

[boarchuz]

Is CONFIG_ADC_CONTINUOUS_ISR_IRAM_SAFE enabled?

https://docs.espressif.com/projects/esp ... #iram-safe

[brebisson]

Hello,

Well, moving to all type #define did seem to solve the problem... Strange that const was not enough!
I did not even know of constexpr... They seem to just add stuff to C++ every decade

Thanks for your help,
Cyrille

[brebisson]

Hello,

Ok, I am back and the crash is back

Whenever I enable my continuous ADC read, I get nvs crashes (for example on WiFi connect which seems to use nvs)...

My latest code does NOT use any call backs (althrough I am assuming that there is still an IRQ going on)...
I schedule a continuous ADC read of 16 read on 3 channels...
I then read the result as part of a regular task which gets results...
Note that the results are NOT read as fast as they are generated. I am assuming that once the queue/given space is full, the ADC driver overrides the oldest data... But the doc does not explain that...
Anyhow, the ADC will do around 300 rounds of 3*16 convertions per second, and I will read around 20 of them...

Note that when they are no nvs access happening, the ADC code works well and provides correct values... so the "logic" and "math" seems good...

Thanks for your help.
Cyrille

Code: Select all

class CADC { public:
    static int const nbChannels= 3;
    static int const nbSamples= 16;
    static int const sampleSize= nbChannels*nbSamples*2;
    uint8_t adcChannelToPinId[7]= {0}; // This will contain the "back index" from adc channel to pin index (as in pin 32 is adc 4 which is my convertion channel 0)
    adc_continuous_handle_t handle = NULL;
    void begin() 
    {
        uint8_t const adcPins[nbChannels]={32, 33, 34}; // 3 pins  will be used through ADC...
        adc_continuous_handle_cfg_t adc_config = { .max_store_buf_size = sampleSize*4, .conv_frame_size = sampleSize, .flags= 0 };
        ESP_ERROR_CHECK(adc_continuous_new_handle(&adc_config, &handle));
        adc_channel_t channel[nbChannels];
        adc_digi_pattern_config_t adc_pattern[nbChannels];
        for (int i=nbChannels; --i>=0;) 
        {
            adc_unit_t unused; adc_continuous_io_to_channel(adcPins[i], &unused, &channel[i]); adcChannelToPinId[channel[i]]= i; 
            adc_pattern[i].atten = ADC_ATTEN_DB_12;
            adc_pattern[i].channel = channel[i] & 0x7;
            adc_pattern[i].unit = ADC_UNIT_1;
            adc_pattern[i].bit_width = 12;
        }
        adc_continuous_config_t dig_cfg = { .pattern_num= nbChannels, .adc_pattern = adc_pattern, .sample_freq_hz = 20 * 1000, .conv_mode = ADC_CONV_SINGLE_UNIT_1, .format = ADC_DIGI_OUTPUT_FORMAT_TYPE1 };
        ESP_ERROR_CHECK(adc_continuous_config(handle, &dig_cfg));
        ESP_ERROR_CHECK(adc_continuous_start(handle));
    }
    void read(int *res)
    {
        uint32_t ret_num = 0;
        uint8_t P[sampleSize], *p= P;
        esp_err_t ret = adc_continuous_read(handle, p, sampleSize, &ret_num, 0); if (ret!=ESP_OK || ret_num!=sampleSize) return;
        int res2[nbChannels]; memset(res2, 0, sizeof(res2)); // populated by call back function...
        int res2cnt[nbChannels]; memset(res2cnt, 0, sizeof(res2cnt)); // populated by call back function...
        for (int i = nbChannels*nbSamples; --i>=0;) 
        {
            uint8_t b1= *p++; uint8_t b2= *p++; // Decode 16 bit of ADC DMA data and get the value and channel for each...
            int chan= adcChannelToPinId[(b2>>4)&7];
            res2[chan]+= b1|(int(b2&0xf)<<8);
            res2cnt[chan]++;
        }
        for (int i=nbChannels; --i>=0;) if (res2cnt[i]==nbSamples) res[i]= res2[i]/nbSamples; // Average the results...
    }
} adc;