ESP32-S3 TVM Example crash after 1st inference

[goksanisil23]

I'm trying to run the sample model.onnx provided in esp-who/components/esp-dl/tutorial/tvm_example/model.onnx using esp-who/components/esp-dl/tutorial/tvm_example/script.sh

Quantization, TVM code generation and deployment are all fine. However, I cannot run the inference loop more than once for some reason.
The "template_project_for_model" calls tvmgen_default_run only once, so in my app_main.c, I've changed it so that its called in the while loop instead as below:

Code: Select all

static const char *TAG = "APP_MODEL";

static void inference_task()
{
  struct tvmgen_default_inputs inputs = {
      .input_1 = input_data,
  };
  struct tvmgen_default_outputs outputs = {
      .output = output_data,
  };

  while (1)
  {
    uint64_t elapsed_time = 0;

    uint64_t time1, time2;
    time1 = esp_timer_get_time();
    int ret_val = tvmgen_default_run(&inputs, &outputs);
    time2 = esp_timer_get_time();
    elapsed_time = time2 - time1;
    printf("\ntime: %lld us\n", elapsed_time);

    for (int i = 0; i < output_len; i++)
    {
      printf("%f, ", output_data[i]);
    }
    printf("\n");
    vTaskDelay(1000 / portTICK_PERIOD_MS);
  }
}

void app_main()
{
  xTaskCreatePinnedToCore(inference_task, TAG, 8 * 1024, NULL, 6, NULL, 0);
}

However, after the 1st iteration of the while loop, the 2nd call to the tvmgen_default_run never finishes and I believe the application crashes.

What am I missing here.
Thanks and regards

[goksanisil23]

I would like to add that, after some debug printouts, I can see that the 2nd inference iteration freezes at the function call below:

esp_dl_nn_conv2d_s8_wrapper, which is called by tvmgen_default_esp_main_8 within default_lib1.c

[michelberg]

Hello I got similar issues. I tried the same by running the inference in a loop. But somehow it is like a “silent” freeze. I even tried to run like small tasks in the background to print something every second. This works until the inference task is finished.
I also get errors with conv2d. I used to build different model architectures but exporting with TVM works but the runtimes crashes shortly afterward.
I wonder if you got similar issues with “custom models”.

[BlueSkyB]

This problem can be reproduced. The cause of this issue is that the memory space for the model's output is declared as const, leading to some undefined behavior. The modification method can be referred to this commit: 97e1c52 in https://github.com/espressif/esp-dl.git.

In addition, the support for TVM in ESP-DL is not very mature, and there may be exceptions in the deployment of some custom models. Moreover, the main branch of TVM has undergone significant updates, so this implementation needs to be refactored. However, due to our current manpower constraints, we won't start the refactoring work in the short term; we need to complete other tasks first. If any exceptions occur, you may need to handle them on your own in the short term. Sorry for the inconvenience.

[dimonab]

Hello!

I encountered the same problem with the standard tvm_example model when running it in a loop. It hangs during the second evolition in tvmgen_default_run(). I have not tested non-standard models yet, but it is worth checking.

Please write here if you manage to solve the problem or find a way to work around it.

[goksanisil23]

Thanks @BlueSkB, removing the const static output in export_onnx_model.py solves the problem.

Best regards