rom/miniz heap space requirements

[lerebel103@gmail.com]

Hi all,

Can rom/miniz be used on a reference ESP32 (520KB ram), which offers just under 300KB of usable heap in user space? When running the following snippet as a test on esp-idf v3.2.2:

Code: [Select all] [Expand/Collapse]

1. #include <stdio.h>
2. #include <rom/miniz.h>
3. #include <lwip/opt.h>
4.  
5. void app_main() {
6.     printf("Free heap memory: %d\n", esp_get_free_heap_size());
7.     printf("   8-BIT blocks available: %d\n", heap_caps_get_largest_free_block(MALLOC_CAP_8BIT));
8.     printf("   32-BIT blocks available: %d\n", heap_caps_get_largest_free_block(MALLOC_CAP_32BIT));
9.  
10.     printf("Memory needed for a tdelf compressor: %d\n", (int) sizeof(tdefl_compressor));
11.     tdefl_compressor* comp = (tdefl_compressor*)malloc(sizeof(tdefl_compressor));
12.     if (comp == NULL) {
13.         printf("Failed to allocate compressor!!!!!\n");
14.     } else {
15.         printf("Free heap memory: %d", esp_get_free_heap_size());
16.         printf("Compressor allocated.\n");
17.     }
18. }

GeSHi © Codebox Plus Extension

I can see:

Code: [Select all] [Expand/Collapse]

1. Free heap memory: 296484
2.    8-BIT blocks available: 126412
3.    32-BIT blocks available: 126412
4. Memory needed for a tdelf compressor: 167744
5. Failed to allocate compressor!!!!!

GeSHi © Codebox Plus Extension

It looks as thought about 167KB of contiguous memory is needed to allocate tdefl_compressor, which fails. Many others have successfully used miniz on ESP32, what am I missing please?

Thanks!

[ESP_Angus]

I think the others may be decompressing, which doesn't need as much contiguous RAM.

The only options I can think of are:

- Recompile miniz from source and decouple some of these buffers (so that the tdefl buffer members are pointers to other data structures which are allocated independently). May need some other changes in the miniz code if code like sizeof(buffer member) is ever used.

- Enable PSRAM. The structure will fit easily in 4MB of external RAM.

Sorry I can't offer a better solution.

[lerebel103@gmail.com]

Nah, that's great thanks for confirming this @ESP_Angus, I could see many success stories out there and wasn't sure if I had missed something embarrassingly simple. Sure thing, I can see that these could be allocated separately, great tip:

Code: [Select all] [Expand/Collapse]

1.   mz_uint8 m_dict[TDEFL_LZ_DICT_SIZE + TDEFL_MAX_MATCH_LEN - 1];
2.   mz_uint16 m_huff_count[TDEFL_MAX_HUFF_TABLES][TDEFL_MAX_HUFF_SYMBOLS];
3.   mz_uint16 m_huff_codes[TDEFL_MAX_HUFF_TABLES][TDEFL_MAX_HUFF_SYMBOLS];
4.   mz_uint8 m_huff_code_sizes[TDEFL_MAX_HUFF_TABLES][TDEFL_MAX_HUFF_SYMBOLS];
5.   mz_uint8 m_lz_code_buf[TDEFL_LZ_CODE_BUF_SIZE];
6.   mz_uint16 m_next[TDEFL_LZ_DICT_SIZE];
7.   mz_uint16 m_hash[TDEFL_LZ_HASH_SIZE];
8.   mz_uint8 m_output_buf[TDEFL_OUT_BUF_SIZE];

GeSHi © Codebox Plus Extension

Cheers.