ESP-NOW (WiFi) appears to run out of memory

[plajjd]

I have 6 ESP32 devices broadcasting ESP-NOW (WiFi) packets to a "master" ESP32 device. I am using the standard ESP-NOW function esp_now_send() with the "broadcast_mac" address. My little network is connectionless. The devices ignore messages not destined to them.

My "master" device simply listens for packets from the 6 "slave" devices, and sends a single "ack" packet back to the sender for each packet received.

The slaves are transmitting 7 packets per second, each containing around 120 bytes. (So the master is receiving 42 packets per second, and sending 42 acks per second)

So - the Master is receiving around 5K bytes per second (total from all 6 slaves). The master does NOT save the data, nor does it allocate dynamic memory to handle it. It just receives the data in the receive callback function, and then discards it.

After running for 30 minutes, the Master started giving errors like this:

Code: Select all

E (1808955) wifi: esf_buf: t=3 l=56 max:8, alloc:8 no eb
W (1808955) wifi: alloc eb len=56 type=3 fail, heap:227864

.[0;31mE (1808955) EspNow_Server: Send error.[0m
E (1808965) wifi: esf_buf: t=3 l=56 max:8, alloc:8 no eb
W (1808965) wifi: alloc eb len=56 type=3 fail, heap:228152

Can someone help me decipher the error?

I assume it means "no empty blocks", but the heap shows over 200K left. Is this an issue of memory fragmentation?

Are there suggestions for how to receive this much data over ESP-NOW without running out of memory?

Thanks!

[chromebin]

Hi plajjd, did you manage to solve this issue?


I too use ESP_NOW:
- exclusively (no Wifi)
- in broadcast (as it doesn't support simple private (group) broadcasts yet)
- while receiving packets only (from a firmware update)
- 220B packets come in every 6ms

And after some time the serial is flooded with lines, possibly once for each packet, like this:

Code: Select all

E (189875) wifi: esf_buf: t=8 l=1600 max:32, alloc:32 no eb, TXQ_BLOCK=4000		# mostly
E (189881) wifi: esf_buf: t=8 l=1600 max:32, alloc:32 no eb, TXQ_BLOCK=0		# some

https://github.com/espressif/esp-idf/issues/2545 suggests this might be due to state machine violations. But as far as I known ESP_NOW is connectionless, and I don't use any other Wifi functionality, so what might be the issue?

Note this occurs on the receiving side. The sending side doesn't error.


Note: I upgraded a week ago to a new IDF. but I had never tested the transfer with big files, so let's not jump to conclusions.
Note: what do these lines mean? TXQ_BLOCK is apparently a IDF 3.3 addition, nowhere to be found.


Edit: I'm checking out if I'm low on disk space.

[chromebin]

I suspect this has to do with a high packets-received rate. Which is unfortunate, as a high rate of broadcasts could potentially induce the error in all devices in the vicinity, potentially causing reboots or other failures.

Which means that broadcasts should not be used, except for pairing. But even devices that adhere to this rule will be subject to flooding due to devices that don't.

In particular I use broadcasts to update all devices on the network with a single packet. This is a legitimate usage and reduces the total number of packets (a good thing for everybody), but given the above, this could cause problems (in all devices in the vicinity).

The solution would be to enable "private broadcasts", that would at least mean that this usage would not cause other devices to choke. In a private broadcast all devices transmit to a network MAC they agree on. So each device gets packets received on only 3 MACs: their own MAC, the broadcast (0xFFFFFFFF) and the newly created network MAC.

I think this would be a nice and apparently simple upgrade to ESP NOW, making it better for all users.

[plajjd]

chromebin - I apologize for the late response - I haven't been checking the forum in recent weeks.

I think our solution was to increase the WiFi buffer sizes in the sdkconfig.

We have implemented a large (> 32 devices) ESP-NOW network using ONLY broadcast packets - no "peer" connections. So far, it seems to work well.

Phil.