file.write issues with mode "w+b" using SD

[aferust]

Using ESP32S3 rev2 board on windows Arduino IDE.

I am creating a port to osgeo shapelib [1, 2]. My port lives here [3]. The library reads and creates shapefiles to extract geographical information (GIS vector layers) such as polygon maps, GPS points, polylines, etc. I modified the library to work with the SD library reading files from the SD card. The reading part works great so far. Maybe it is enough to be able to read with esp32. However, I want to push it a bit forward, and I also want to achieve the writing functionality.

The problem is:
Guru Meditation Error: Core 1 panic'ed (LoadProhibited). Exception was unhandled.

After some debugging, I found out that the problem occurs when a file.write runs with mode "w+b". Particularly here:

if (psSHP->sHooks.FWrite(pabyRec, nRecordSize, 1, psSHP->fpSHP) < 1) // https://github.com/aferust/ShapeLibForE ... .cpp#L1757

which invokes this function:

[https://github.com/aferust/ShapeLibForE ... ks.cpp#L33]
...
SAOffset SADFWrite(const void *p, SAOffset size, SAOffset nmemb, SAFile &file)
{
size_t bytesWritten = file.write((const uint8_t *)p, size * nmemb); // problem here
...

Of course, I checked if there are null or unallocated buffers, but everything is what it is supposed to be. I am stuck in this problem and need help.

Thanks in advance!

1: http://shapelib.maptools.org/
2: https://github.com/OSGeo/shapelib
3: https://github.com/aferust/ShapeLibForESP32


A test code to reproduce the problem:
```
#include "shapefil.hpp"
#include "SD.h"

void setup() {
Serial.begin(115200);

if (!SD.begin()) {
Serial.println("Card Mount Failed");
return;
}

SHPsetFileSystem(SD);

// Open SHP and DBF files for writing
SHPHandle hSHP = SHPCreate("/p1.shp", SHPT_POLYGON);
//DBFHandle hDBF = DBFCreate("/p1.dbf");
if (hSHP == nullptr /*|| hDBF == nullptr*/) {
Serial.println("Failed to create SHP or DBF file!");
return;
}
/*
// Add a field to the DBF file
if (DBFAddField(hDBF, "Name", FTString, 20, 0) == -1) {
Serial.println("Failed to add field to DBF!");
return;
}
*/
// Create a polygon
double xCoords[] = {0.0, 1.0, 1.0, 0.0, 0.0}; // X coordinates
double yCoords[] = {0.0, 0.0, 1.0, 1.0, 0.0}; // Y coordinates
int partStart[] = {0}; // One part starting at index 0

SHPObject* shp = SHPCreateObject(
SHPT_POLYGON, -1, 1, partStart, NULL, 5, xCoords, yCoords, NULL, NULL);

// Write the polygon to the SHP file
if (SHPWriteObject(hSHP, -1, shp) == -1) { // The program crashes here !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Serial.println("Failed to write SHP object!");
} else {
Serial.println("Polygon written to SHP file.");
}

SHPDestroyObject(shp);
/*
// Write a record to the DBF file
if (!DBFWriteStringAttribute(hDBF, 0, 0, "Example Polygon")) {
Serial.println("Failed to write to DBF!");
} else {
Serial.println("Record written to DBF file.");
}
*/
// Close files
SHPClose(hSHP);
//DBFClose(hDBF);

Serial.println("Files written successfully.");
}

void loop() {}
```

[lbernstone]

w+b is not a valid mode. C (and C++) considers all files binary.
Arduino only knows R,W, and A. ESP-IDF is posix and uses fopen under the covers

[aferust]

"w+" does not work either. Anyway, thank you for your response. We will only have reading functionality, which is nice, though.

[lbernstone]

You should get a backtrace from the error. If you decode that backtrace, it should give you a file+line where the underlying failure occurs.

[aferust]

I have already detected the crash lines of my code using my poor man's debugging method. Using the tool you suggested, it only helped to reveal ...portable/xtensa\port.c:162, and ...Arduino15\packages\esp32\hardware\esp32\3.0.7\cores\esp32\main.cpp:59 which I could infer anything.

Decoding stack results
0x40100191: is in SADFWrite(void const*, unsigned long, unsigned long, fs::File&) (c:\Users\user\Documents\Arduino\libraries\shapelib\src\fshooks.cpp:28).
0x400d28e0: is in SHPWriteObject(SHPHandle, int, SHPObject const*) (c:\Users\user\Documents\Arduino\libraries\shapelib\src\shpopen.cpp:1757).
0x400d1c05: setup() at D:\projects\ESP32Arduino\shpwrite\shpwrite.ino:37
0x400d8016: loopTask(void*) at C:\Users\user\AppData\Local\Arduino15\packages\esp32\hardware\esp32\3.0.7\cores\esp32\main.cpp:59
0x4008baf6: vPortTaskWrapper at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/FreeRTOS-Kernel/portable/xtensa\port.c:162

[aferust]

Btw the problem is "r+".

[aferust]

Hey, I resolved my issue. I was so stupid not to consider the lifetime of an instance of class fs::File returning from fs::open. With the previous code, ref variables of fs::File were roaming around, which was a problem for the internal pointer of File (std::shared_ptr<FileImpl>). I had to allocate and return a copy of fs::File to have full control over the lifetime of my copy.

Everything works great now. Sorry for taking your time. But now I know that the Exception Decoder exists and is useful.

Code: Select all

SAFile *SADFOpen(const char *pszFilename, const char *pszAccess,
                 void *pvUserData)
{
    (void)pvUserData;

    auto file = new fs::File(shp_fs->open(pszFilename, pszAccess));
    if (!(*file))
    {
        delete file;
        return nullptr;
    }
    return file;
}
...
int SADFClose(SAFile *file)
{
    if (file)
    {
        file->close();
        delete file;
        return 0;
    }
    return -1; // Geçersiz dosya
}