Stripped-down libstdc++-psram-workaround.a

[p-rimes]

Holy hell! I used C++ std::*stream in my app and all of a sudden my libstdc++ usage ballooned from ~7k to ~200k.

Merely instantiating a std::stringstream does this, and investigation of the map file reveals that a lot of that is related to pulling in locale and formatting symbols which I will never use, but which are being loaded anyway.

So, my reasoning is that I should produce a new libstdc++, patching the stream classes to not use locale at all, and then to link this instead of the toolchain provided libstdc++. (I am very open to other suggestions to avoid this approach -- I did investigate uSTL but found the implementation too simplistic, and needed the full std:: streams/rdbuf interface.)

However, let's say that I need to reproduce the libstdc++-psram-workaround.a binary that is contained in esp-idf... how would I do that? A total guess is to copy the upstream gcc/libstdc++-v3/**/*.cc files into a new component in my app, with the following flags in component.mk:

Code: Select all

CFLAGS+=-mfix-esp32-psram-cache-issue
CXXFLAGS+=-mfix-esp32-psram-cache-issue
LDFLAGS:=$(filter-out -lstdc++,$(LDFLAGS))

Is that all that is necessary? Any other suggestions, or if other people have run into this situation before? I suppose I could avoid the STL, but I'm not aware of a good replacement for std::*streams. Thanks for your advice!

[kolban]

I'm a heavy user of the standard libraries in C++. I haven't looked at the additional code size that is linked in. However, let's assume that it is 200K of additional, would we consider that "prohibitive"? My thinking is that the increase is in linked object files that don't contribute to RAM utilization unless leveraged/used. If we assume that there is 4MBytes of flash then an additional 200K is 5%.

It is extremely likely that I am missing some major concept here. Could you help me understand the significance of 200K being considered too much?

[p-rimes]

Well... firstly I noticed the issue when my app went over the default 1M factory partition size, so it actually refused to boot (!). That was a shock. Now I have added a call to `make size-components` to my CI process, so at least I am logging that info now and I can track regressions.

It is true that I have 4MB to play with on all my current dev boards, but my understanding of the typical OTA-using partition table (with one factory app, and one additional known-good OTA image) is:
0 ~ 0x10000 <system/bootloader/reserved partitions, let's say negligible, leaving ~3.9MB>
1MB factory
1MB ota_1
1MB ota_2
~0.9MB (spare)

So I could resize the three app partitions up to ~1.3MB each with the spare space, but then I will only have 300k left to play with. Which does seem like a lot! But considering that just touching std::stringstream was an extra 200k, it makes me a bit worried to continue with all C++ (which, I love dearly. As well as the STL.)

If it helps for comparison, I was extensively using STL vector, map, string, algorithms, std::function, and coming in at under 10k. But the addition of a single stream pulled in an additional 190k.

[ESP_Sprite]

To answer your question: as long as you compile libstdc++ with -mfix-esp32-psram-cache-issue in the compiler flags (for _all_ object files in the library, mind you; run a 'make clean' first to make sure), you should be good.

[p-rimes]

Thanks ESP_Sprite!

I was able to successfully extract the relevant sources and headers from gcc 5.2.0 obtained from crosstool-NG. I walked through the gcc Makefile.am's and figured out the right compilation options for each file to support both ABIs (c++98 and c++11). Quite finicky, tbh.

But it works(!), and I now have a drop-in libstdc++ component that passes my smoke tests! And it shaved 15k off the final build size (which is... O.K., I did expect a slight change from the one provided in esp-idf, but I haven't actually stripped anything out yet.)

I made sure to add '-nostdinc++' in my app's top-level Makefile to make sure that all the idf components are compiled using my libstdc++ headers, and removed the '-lstdc++' from LDFLAGS which is there by default.

I've added my component as a github repo here:
https://github.com/paulreimer/esp32-libstdc-v3
(* it requires the esp-idf PR applied here: https://github.com/espressif/esp-idf/pull/1318. Otherwise I guess you could just rename all the files to *.cpp? The stock gcc ones end with *.cc)

To be clear, this is just a stock/drop-in libstdc++ that does still have the locale bloat. But I believe I can now chop away at the *.cc files to address that (and so could anyone who wants to customize a libstdc++).

[p-rimes]

Just an update here, I was able to whittle it down to 33k (as in this branch: https://github.com/paulreimer/esp32-lib ... /stripped1)

The biggest reductions were from disabling wchar support (~50%), and then from removing/stripping the STL facets for money, time, and collate features (another 50% after that). As well, as mentioned above, just copying the sources from gcc into a component already reduced the binary size quite a bit.

I'm not sure if I will actually end up using this, but it is nice to know it can be done, if I run out of flash space and need a custom build.

It still does feel odd to me how bloated the STL is in the streams area... And that there isn't an size-constrained C++ alternative for this (e.g. just using byte streams, no encoding/conversion).