WiFi transfer speed starts out great, then crashes down: can the ESP32's WiFi dynamic rate scaling algorithm be tuned?

[PanicanWhyasker]

ESP32-WROVER-32E based data logger is collects data during the day and uploads it via WiFi during the night.
Strangely, the bulk transfer speed starts out great, but reduces greatly after a few minutes (sometimes 1 minute, sometimes 5 minutes, with 2 minutes being typical).

Sample1:


Sample2:


Circumstances:
- The logger and the AP antenna are quite far apart, about 90 metres;
- This is in an open field, direct line of sight, very quiet (radio-wise) environment;
- The RSSI does not change as you can see, it is about -74 dBm
- Same device with the same code achieves more than 10 Mbps when it's closer to the AP

I'm not sure what causes this sudden drop in transfer speed, but it smells of PHY issue, and I suspect it is the WiFi dynamic rate scaling feature (I'm reading here about it). Likely the ESP32 PHY starts out sending packets at higher bitrate, sees some lost packets, adjust speeds and thus we observe 2-4 varying Mbps in the first few minutes. Then it suddenly gives up and fixates on 0.64 Mbps which maybe causes no packet loss.

The other thing that happens is that the ESP32 consumes more power in the 0.64 Mbps mode. Computed roughly by the drop in battery voltage (internal resistance & co) I think the total ESP32 consumption jumps from ~150mA to about ~350mA.

In general it sucks to be stuck in the 0.64 Mbps mode, as it's worse both speed-wise and power-wise. And it doesn't get unstuck by itself: unless the device is physically moved, it almost never ever improves above 0.6-0.8 Mbps. But if you restart it, it begins with the faster speed again, and then similarly crashes after a few minutes.

Question
Is it possible to tune the PHY's dynamic rate scaling algorithms for WiFi? I want to instruct it to prioritize throughput at the cost of higher packet loss and retries. I.e. "please, only reduce the bitrate if the packet loss is awfully bad".

[PanicanWhyasker]

Any ideas?

[WiFive]

Have you tried esp_wifi_internal_set_fix_rate

[PanicanWhyasker]

Wow, this seems exactly like the tweeky type of API I'll need. Many thanks, WiFive, will try and report my findings.
I'll need to disable AMPDU first, right?

[ESP_Jack]

esp_wifi_internal_set_fix_rate is a internal debug API.
Use esp_err_t esp_wifi_config_11b_rate(wifi_interface_t ifx, bool disable); will be easier.

Well, It is hard to say the reason. But I can try to explain it.
The dynamic rate scaling of ESP32 is based on the PER(packet error ratio) and RSSI. The algorithm make it more easier to adjust rate lower than adjust it higher. When the device use 1Mbps rate, it will introduce some vicious spiral:
1. 1Mbps packet is with long TX time(1500 bytes with 12ms), it will be more easier to be interfered in the open air
2. bad PER make it hard to use a higher rate

To escape from the vicious spiral, we also use RSSI to adjust the rate. However, RSSI it not a feedback information and also not so stable. So we are conservative when we are use the RSSI.

If you can provide the version here, we may help debug this. Since in some version, we try to avoid use 1Mbps in some cases.

[PanicanWhyasker]

Many thanks, ESP_Jack! My ESP-IDF version is 3.2, thus esp_wifi_config_11b_rate() is not available. I am a bit hesitant to change the ESP-IDF, since I have some private patches on top of v3.2, which enable more detailed crash reporting. But if you think that v3.2 is too old and a newer version will drastically reduce the speed problems I'm observing, I will of course rebase my changes.

It does seem like a vicious spiral. As I said it rarely ever escapes the 0.6-0.8 Mbps transfer speed once it's stuck there. It sometimes does, but very rarely.

[ESP_Sprite]

3.2 is pretty old and not supported anymore, sorry; if you used a more recent supported version I could ask for a backport. Can I ask what improvements in crash reporting you did and if you perhaps want to share them? If you do and they're universally usable, we may think about integrating it in mainline esp-idf.

[PanicanWhyasker]

@ESP_Sprite, the changes to the 3.2 release tree were an ability to hook into the panic handler, so you could fetch info about the crash (backtrace, crash reason) in user app code. Essentially, add a function to provide a callback, and invoke that callback in the panic handler in components/esp32/panic.c

On the app-side I had code to store the retrieved backtrace into a RTC_NOINIT_ATTR area, so on the next restart, you could see that the previous invocation crashed, and you could send the backtrace/crash data over WiFi for further developer analysis.
I remember you had something that approached this functionality, but it wasn't convenient for my application, so I rolled out my own. I haven't checked if there's been developments in more recent IDFs.

I'm OK to share it if you think that would be useful.

[ESP_Sprite]

If you can post a link to the changes in code, I can see if it's actually possible to do this in recent versions of ESP-IDF; if not, I'll create a ticket to look into it. It sounds like an useful feature.