ESP32 Forum

Hi. I am trying to compress some Lua source code and store it in flash memory. Anything I tried didn't work. Eventually I found the issue (ChatGPT was not completly useless for once ) and got it working. The issue was I had a stack overflow. Enabling PSRAM and allocating on heap fixed it. Apparently the tdefl_compressor has a size of 167744 bytes (168kB). I am not well versed in compression algorithms but this seems extremly high to me. Even after I skimmed the miniz.h definitions I think the structure should be smaller than 100kB (which also seems a lot to me).
So is this normal? Am I doing something wrong? Is there an easier way to compress text? I have spent hours on this already and I am at a loss.

idk_some_username wrote: ↑

Sat Mar 02, 2024 12:30 am

Is there an easier way to compress text?

Heatshrink requires very little RAM and may compress good enough for you.

MicroController wrote: ↑

Sat Mar 02, 2024 11:26 am

Heatshrink requires very little RAM and may compress good enough for you.

That is indeed good enough. Thank you!

In that case I'd invite you to check if my ESP32-optimized version of heatshrink might be of use to you:
https://github.com/BitsForPeople/esp-heatshrink

Wow, 14 to 22x speed increase, that's impressive!

Turns out, to unlock the S3's speed mode you have to type some secret cheat codes like "EE.LD.128.USAR.IP"

Unfortunately I'm not quite getting to the speed of heatshrink's 'index', but the memory tradeoff of the index becomes much less worthwhile when you have the PIE available.
In the end, I'm still pretty happy that the more DSP-focused PIE can be 'bent' to benefit other applications as well.