CONFIG_COMPILER_OPTIMIZATION_PERF=y - causes compiler to crash

[srjasz]

I am trying to change the optimization settings to CONFIG_COMPILER_OPTIMIZATION_PERF. From the manual it says change sdkconfig
from

CONFIG_COMPILER_OPTIMIZATION_DEBUG=y
# CONFIG_COMPILER_OPTIMIZATION_SIZE is not set
# CONFIG_COMPILER_OPTIMIZATION_PERF is not set
# CONFIG_COMPILER_OPTIMIZATION_NONE is not set

to

# CONFIG_COMPILER_OPTIMIZATION_DEBUG is not set
# CONFIG_COMPILER_OPTIMIZATION_SIZE is not set
CONFIG_COMPILER_OPTIMIZATION_PERF=y
# CONFIG_COMPILER_OPTIMIZATION_NONE is not set

When I do the compiler crashes during the build and gets (usually) to this point.

Bootloader binary size 0x5250 bytes. 0x2db0 bytes (36%) free.
[427/956] Building C object esp-idf/lwip/CMakeFiles/__idf_lwip.dir/lwip/src/core/ip.c.obj

and then freezes there.

Is this in fact the correct method to use to change the optimization setting, and am I doing it correctly?
I am using an -S3 chip.

Thanks