MQTT Outbox Malloc from PSRAM

[NotMyRealName]

I have an application I am developing for a commercial project that uses an ESP32 with SPI PSRAM.

I've just realised that messages I queue in the MQTT outbox are duplicated in a buffer allocated from a standard malloc call. With the way I have it setup this will come from the internal RAM. Would it be possible/practical to modify the library to have a config option to prefer allocation of all outbox buffers from the external RAM?

I realise there is a custom outbox implementation config option, I haven't found examples for that yet, but might have a go at that too.
That's not as simple of course...

[xien551]

Directly wirite or read the values through the address of external ram. Maybe there are examples in IDF.

Or do "idf menuconfig", maybe you could configure it as your wish.

[NotMyRealName]

Thanks for the suggestion. I understand what you are suggesting, but I can already write or read external PSRAM just fine.

I've already been through the MQTT code. The issue is in the implementation of the MQTT library. Anything you pass to the MQTT publish function will be copied to an internal outbox. This is a linked list of dynamically allocated buffers. These buffers are allocated using malloc.

In other component libraries there is an option in the config to prefer allocation from external memory. There is no option like this for MQTT.

It basically means that I could in theory buffer and prepare a large message (e.g. 10-50KB) in external ram just fine but then be unable to send it because there might not be enough internal memory to duplicate it in the outbox.

Another solution (harder to implement) would be a zero copy publish.

[ESP_igrr]

Hi NotMyRealName,
Your suggestion does make sense, could you please open a feature request at https://github.com/espressif/esp-mqtt/issues? This way it is going to be tracked by the team working on esp-mqtt.

[NotMyRealName]

Thanks ESP_igrr,

Link to the issue in case anyone wants to find it:
https://github.com/espressif/esp-mqtt/issues/242