(solved) LWIP/sockets memory leak?

[mzimmers]

Hi all -

App runs Wifi every 5 minutes to send a heartbeat message to a server. Sequence is:

1. Start Wifi.
2. Create multicast socket; set options.
3. Create and send heartbeat message.
4. Close socket.
5. Disconnect and stop Wifi.

I recently noticed that I'm losing about 800 bytes of heap with each iteration. I removed step 3 to try to isolate the problem, and it still occurs. I know it sounds crazy, but could there possibly be a memory leak of this size in the allocation and de-allocation of socket resources?

ESP32-WROVER, IDF 3.1.1.

[mzimmers]

Here's an example:

In one of my tasks, I create and delete a socket.

Code: Select all

            for (;;)
            {
                startComms();
                while (state != WIFI_STATE_SOCKET_CREATED)
                {
                    vTaskDelay(100);
                    state = getCommsState();
                }
                ESP_LOGI(TAG, "************************* state is %s. heap is %d. *************************",
                         WIFI_STATE_TAGS[state].c_str(),
                         m_tasks->worker->reportHeap());
                stopComms();
                while (state != WIFI_STATE_STOPPED && state != WIFI_STATE_DISCONNECTED)
                {
                    vTaskDelay(100);

                    state = getCommsState();
                }
            }

Here's the startComms()/stopComms() code:

Code: Select all

int32_t Wifi::startComms()
{
    int32_t rc = 0;

    //ESP_LOGI(TAG, "entering startComms().");

    // only send the start if we're not already started (or running).
    if (!isActive())
    {
        ESP_ERROR_CHECK(esp_wifi_start());
        setCommsState(WIFI_STATE_STARTING);
    }
    else {
//        ESP_LOGW(TAG, "startComms(): commsState is %s. Not attempting to start.",
//                 (const char *) COMMS_STATE_TAGS[m_commsState].c_str());
    }
    return rc;
}

int Wifi::stopComms()
{
    int rc = 0;
    esp_err_t err;
    WifiState state;

    state = getCommsState();
    if (isActive())
    {
        setCommsState(WIFI_STATE_STOPPING);
    }
   
    // shut down the socket opened on wifi.
    socketClose();

    switch (state)
    {
    case WIFI_STATE_CONNECTED_TO_AP:
    case WIFI_STATE_IP_ADDRESS_ASSIGNED:
    case WIFI_STATE_SOCKET_CREATED:
        //ESP_LOGI(TAG, "stopComms(): calling esp_wifi_disconnect().");
        err = esp_wifi_disconnect();
        // do not break; just fall through.

    case WIFI_STATE_UNKNOWN:
    case WIFI_STATE_STARTING:
    case WIFI_STATE_STARTED:
    case WIFI_STATE_DISCONNECTED:
        //ESP_LOGI(TAG, "stopComms(): calling esp_wifi_stop().");
        err = esp_wifi_stop();
       
        break;
    case WIFI_STATE_STOPPED:
    case WIFI_STATE_STOPPING:
    default:
        ESP_LOGW(TAG, "stopComms() called but wifi is already stopped or stopping.");
        break;
    }
    return rc;

(edited for brevity)

Each iteration of the forever loop shows a loss of ~800 bytes of heap, per heap_caps_get_free_size(MALLOC_CAP_INTERNAL | MALLOC_CAP_8BIT). The exact number varies slightly between iterations.

Running this code will run my ESP32 out of heap in a matter of minutes.

[ESP_Angus]

Running this code will run my ESP32 out of heap in a matter of minutes.

Does this actually happen or does heap usage stabilise after a few iterations?

There are some resources (TCP sockets for example) which continue to occupy some memory after being closed (because of the TIME_WAIT state). In this case you may see a "leak" for the first few iterations, but the memory usage will stabilise (either because of TIME_WAIT period timing out and resources being cleaned up, or because the max socket limit is reached so sockets are timed out early.)

You can follow the steps shown here to enable heap tracing and diagnose a memory leak:
https://docs.espressif.com/projects/esp ... mory-leaks

(Also note the section on "False Positive Memory Leaks" a little further down the page.)

[mzimmers]

Hi Angus -

Yes it actually happens. I've stripped away 90% of my application, eliminating everything but the Wifi. Upon startup, it begins at about 150K of heap, and loses 800 bytes per loop iteration, until there isn't enough heap to do something (usually a malloc() inside the i2c driver), at which point the CPU panics, restarts and the process begins again. It sheds memory as fast as it can start and stop wifi (with the socket creation inside). I've also eliminated any dynamic heap usage at the application layer (no more new/delete calls).

I've looked through the page on memory leaks, and can't trace it any further than the socket creation/deletion.

Originally, my app was taking 5 minutes per loop iteration, so there's more than enough time for the system to do any garbage collection.

[WiFive]

Is it still there if you don't stop/start wifi? Although 3.1.1 includes https://github.com/espressif/esp32-wifi ... 32f332c4e7 so maybe not.

[aureoleday]

I too find the lwip memory leak in the tcp server example.
Connecting and disconnect the tcp server with a client will reduce the heap about 100 byte every circle.

BTW, I'm using the console cmd "free" to check the heap .

[mzimmers]

I modified my test code so that it no longer opens a socket; it just starts/connects/disconnects/stops wifi. This doesn't produce any heap loss.

So, the problem is somewhere in the socket management code.

UPDATE: found it.

The example that I used to create my socket stuff didn't call freeaddrinfo(). Since I'm a networking newbie, I didn't know I was supposed to do that. Problem is solved.