How to use miniz to create a compressed file that can be decompressd by gzip?

shriya_dhar
Posts: 5
Joined: Tue May 02, 2023 7:04 am

How to use miniz to create a compressed file that can be decompressd by gzip?

Postby shriya_dhar » Tue Nov 07, 2023 3:15 pm

I am facing a challenge when attempting to decompress data compressed using the miniz library on the ESP32 platform. My objective is to compress data using miniz library in flasher_stub component of esp-idf and then decompress the data on the cloud side using some python script that can be compatible with miniz decompression style like zlib.

As per the documentation and information available, the miniz library is described as a "lossless, high-performance data compression library" that implements the zlib (RFC 1950) and Deflate (RFC 1951) compressed data format specification standards. However, I am encountering difficulties when attempting to decompress the compressed data using Python.

Specifically, when using the Python zlib module, which should be compatible with the zlib and Deflate formats, I am receiving an "unknown compression method" error. I have verified that the file and data are correct, and they were compressed using the miniz library on the ESP32. Despite this, I am unable to successfully decompress the data.

I would greatly appreciate any help or any insights that you can provide regarding how to properly decompress data that has been compressed with the miniz library using Python.

I have already tried:
    to create a compressed file with zlib compatible HEADERS but that doesn't work when trying to decompress on python side.
      use compress2() as described here https://stackoverflow.com/questions/322 ... sd-by-gzip

      So far with no luck!

      ClockToshi
      Posts: 8
      Joined: Thu Nov 16, 2017 9:09 pm

      Re: How to use miniz to create a compressed file that can be decompressd by gzip?

      Postby ClockToshi » Tue Nov 07, 2023 10:41 pm

      Not sure about compressing on esp32 and decompressing elsewhere but this works for me to compress on my linux host (and then i decompress on the esp32)

      Code: Select all

      python -c "import zlib; import sys; open(sys.argv[2], 'wb').write(zlib.compress(open(sys.argv[1], 'rb').read(), 9))" uncompress_file.bin compressed_file.bin
      9 is the compression level in the example above.

      hope it helps

      shriya_dhar
      Posts: 5
      Joined: Tue May 02, 2023 7:04 am

      Re: How to use miniz to create a compressed file that can be decompressd by gzip?

      Postby shriya_dhar » Mon Nov 20, 2023 3:45 pm

      I aim to compress and store data on a file system, with the intention of later decompressing it on the cloud side. Since Miniz employs zlib-style compression, I initially assumed that I could decompress it using any standard compressor. However, I'm currently facing challenges with this aspect.

      Who is online

      Users browsing this forum: Google [Bot] and 148 guests