ESPTool deflate algorithm firmware flash

[vanBassum]

Hello,

We had some problems with the ESPtool in our production environment. So I created a native library in C# to flash the ESP32 without use of the ESPTool. This works great when uploading uncompressed firmware. But in the production environment I would like to use the compressed upload functions to decrease production time. I have tried the DeflateStream and GZIPStream, see below, but I receive an inflation error from the stubloader. So I think I'm using the wrong compression algoritm altough I couln't find more information about the specific algoritm to use.

The only information I could find in the following link states: "the data is compressed using the gzip Deflate algorithm"
https://github.com/espressif/esptool/wi ... l-Protocol

The complete library can be found here, beware a lot of things still need to be done. Including documentation.
https://github.com/KooleControls/ESPTool

Code: Select all

public static byte[] Compress(byte[] data)
        {
            byte[] compressedBytes;

            using (var uncompressedStream = new MemoryStream(data))
            {
                using (var compressedStream = new MemoryStream())
                {
                    // setting the leaveOpen parameter to true to ensure that compressedStream will not be closed when compressorStream is disposed
                    // this allows compressorStream to close and flush its buffers to compressedStream and guarantees that compressedStream.ToArray() can be called afterward
                    // although MSDN documentation states that ToArray() can be called on a closed MemoryStream, I don't want to rely on that very odd behavior should it ever change
                    using (var compressorStream = new GZipStream(compressedStream, CompressionLevel.Optimal, true))
                    {
                        uncompressedStream.CopyTo(compressorStream);
                    }

                    // call compressedStream.ToArray() after the enclosing DeflateStream has closed and flushed its buffer to compressedStream
                    compressedBytes = compressedStream.ToArray();
                }
            }

            return compressedBytes;
        }

[vanBassum]

So, I used the nuget package zlib.net instead of the gzipstream from microsoft. This library does work!
Apperantly there are some incompatabilities between the two.

I used this code to compress the image before flashing, I have also pushed the code to git (7ca907fc4af88c3dfe9ee14e5ae23db44eb6a7af)

Code: Select all

        public static byte[] Compress(byte[] data)
        {
            byte[] outdata;
            using (MemoryStream outMemoryStream = new MemoryStream())
            using (ZOutputStream outZStream = new ZOutputStream(outMemoryStream, zlibConst.Z_DEFAULT_COMPRESSION))
            using (Stream inMemoryStream = new MemoryStream(data))
            {
                CopyStream(inMemoryStream, outZStream);
                outZStream.finish();
                outdata = outMemoryStream.ToArray();
            }
            return outdata;
        }