Dynamic class allocate on External RAM

[zekageri]

HI dear forumers!

Is there some way that i could allocate my classes on external ram?

Curerntly iam doing this:

Code: Select all

#define MAX_CLASSES 20
class exampleClass {
    public:
    void sayHello(){
        Serial.println("Hello");
    }
};
exampleClass dynamicClass;
exampleClassArr*  exampleClasses[MAX_CLASSES] EXT_RAM_ATTR;

void allocateNewClasses(){
    for(int i = 0; i < MAX_CLASSES; i++){
        exampleClasses[i] = new exampleClass();
        exampleClasses[i]->sayHello();
    }
}

void setup(){
    allocateNewClasses();
}

In this example, iam allocating an array in external ram to hold pointers to dynamically created classes.
Is there a way that i could allocate my new exampleClass(); in external ram instead of the internal heap?

Iam using PlatformIO with Arduino framework.

Thank for your answers.

[lbernstone]

That's not what EXT_RAM_ATTR is for. That flag has been deprecated, but AFAIK was intended for allocating firmware data into the bss memory- that is not a good idea for your userspace data.
Class data does not exist until you instantiate the object, so if psram is available at the time your create your object, it will use it. "new" is going to follow the rules set by the memory mapping of the OS, which will allocate anything over 16K (or 4K in newer models) into psram. It's not clear to me from your example how big your array is, and if it is not clear to the compiler, then it is likely to assume a small array. If you want more control over where that is allocated, you will need to ps_malloc and then free the data yourself when you are done with it. If you want something dynamic, without having to roll your own, I'd suggest looking at the standard libraries. Here's an example that shows how to use a custom allocator to create a std:vector array in psram. https://github.com/lbernstone/psram_custom_allocator

[zekageri]

Thank you for your reply. If i understand you correctly, the compiler decides if my array is going to the PSram or not based on the size of the array? It's okay if my array of class pointers is in my internal ram, i want my class that my pointer is pointing to, to be in external ram. Because as i said in my post, i know that this array of class pointers is really low memory compared to the actual memory taken up by the dynamically allocated classes. So you say if my class is bigger than 16k then the compiler will alllocate it in ext ram? That's good to hear. ( i didn"t knew about the EXT_RAM deprecation tho thanks. )

The thing is that if i allocate my array of class pointers in ext ram, the new class itself will be in internal ram isn't it?

Code: Select all

#define MAX_CLASSES 20
class exampleClassSystem{}
exampleClassSystem* classArray[MAX_CLASSES];

void setup(){
    // This array will be in ps_ram.
    classArray= (exampleClassSystem*) ps_malloc (MAX_CLASSES * sizeof (exampleClassSystem));
    // but will the new class will be in psram, because it looks to me that it will be in internal ram.
    for(size_t i = 0; i < MAX_CLASSES; i++){
        classArray[i] = new exampleClassSystem();
        // So classArray[i] will be in external ram, because i allocated the array of pointers in external ram.
        // But will the class space be allocated in external ram too? Not just it's pointer?
    }
    
}

Thanks for your reply. I appreciate your help.

[zekageri]

Oh my latest example does not compile actually. xd Sorry for that.

How could i use your custom allocator for dynamic classes in dynamic arrays?

[zekageri]

I have a bunch of different classes. All of the different classes has it's own array. If i allocate them at the start of the program like this:

Code: Select all

void zSystem::allocateComponentArrays(){
    *lightsArray    = (lightSystem*)ps_malloc(MAX_LIGHTS * sizeof(lightSystem));
    *shadersArray   = (shaderSystem*)ps_malloc(MAX_LIGHTS * sizeof(shaderSystem));
    *boilersArray   = (boilerSystem*)ps_malloc(MAX_LIGHTS * sizeof(boilerSystem));
    *hmvsArray      = (hmvSystem*)ps_malloc(MAX_LIGHTS * sizeof(hmvSystem));
    *thermsArray    = (thermSystem*)ps_malloc(MAX_LIGHTS * sizeof(thermSystem));
    *rgbsArray      = (rgbSystem*)ps_malloc(MAX_RGBS * sizeof(rgbSystem));
    *ledsArray      = (ledSystem*)ps_malloc(MAX_LEDS * sizeof(ledSystem));

    memset(lightsArray,0,MAX_LIGHTS);
    memset(shadersArray,0,MAX_LIGHTS);
    memset(boilersArray,0,MAX_LIGHTS);
    memset(hmvsArray,0,MAX_LIGHTS);
    memset(thermsArray,0,MAX_LIGHTS);
    memset(rgbsArray,0,MAX_RGBS);
    memset(ledsArray,0,MAX_LEDS);
}

And later push the classes inside like this:

Code: Select all

for (JsonPair componentRef : lightsObj) {
    if( lightCount >= MAX_LIGHTS ){break;}
    const char* componentID  = componentRef.key().c_str();
    JsonObject component     = lightsObj[componentID];
    lightsArray[lightCount]  = new lightSystem();
    lightsArray[lightCount]->update(component,componentID,lightCount,zoneID);
    lightCount++;
}

My heap is the same as i did not allocate it in ext ram.

If i do this:

Code: Select all

for (JsonPair componentRef : lightsObj) {
    if( lightCount >= MAX_LIGHTS ){break;}
    const char* componentID  = componentRef.key().c_str();
    JsonObject component     = lightsObj[componentID];
    lightsArray[lightCount]  = (lightSystem*)ps_malloc(sizeof(lightSystem)); //new lightSystem();
    lightsArray[lightCount]->update(component,componentID,lightCount,zoneID);
    lightCount++;
}

The app just restarts without an error.

[lbernstone]

You don't seem to understand data structures. When you define a class, all you get back is a pointer. Storing that in psram would save you 4 bytes, with a huge penalty for moving it back and forth across the SPI bus. Store your data in psram instead.
In this example I build a data structure to describe my individual elements, and then a class to hold a collection of those elements. The purpose of a class is having the methods available to the "this->" data (that's what object-oriented programming is about).
When you instantiate the small list (1000 elements), only 3000 bytes are needed. Since this is below the limit we have set for automatic psram allocation, it will use the SRAM. When we then request 100.000 elements, we need 300K, which is obviously more than we can get from the main heap, so it uses the psram. If you wanted to ensure that this always used psram, you could call ps_malloc instead of regular malloc.

Code: Select all

const char* fmtMemCk = "Free: %d\tMaxAlloc: %d\t PSFree: %d\n";
#define MEMCK Serial.printf(fmtMemCk,ESP.getFreeHeap(),ESP.getMaxAllocHeap(),ESP.getFreePsram())

typedef struct {
  uint8_t red;
  uint8_t blue;
  uint8_t green; 
} pixel;

class PsTest
{
  public:
      PsTest();
      ~PsTest();
      bool begin(uint32_t elements);
      bool end();

  private:
      uint32_t _elements;
      pixel* _pixels;
};

PsTest::PsTest() : _elements(10000) {}

PsTest::~PsTest() {end();}

bool PsTest::end() {
  if (_pixels) {
    free(_pixels);
    return true;
  }
  return false;
}

bool PsTest::begin(uint32_t elements) {
  _elements = elements;
  _pixels = (pixel*)malloc(_elements * sizeof(pixel));
  if (!_pixels) {
    log_d("Not enough memory to allocate %d elements", _elements);
    return false;
  }
  memset(_pixels, 0, _elements * sizeof(pixel));
  log_d("allocated %d elements", _elements);
  return true;
}

PsTest ps_test;
void setup() {
  Serial.begin(115200);
  Serial.println();
  MEMCK;
  ps_test.begin(1000);
  MEMCK;
  ps_test.end();
  MEMCK;
  ps_test.begin(100000);
  MEMCK;
}

void loop() {delay(-1);}