redirect app trace into sd card

[Gfast2]

Hi ESP-IDF,

Is there some implementation on this topic?
Many linux boxes are using their SD Card as their trace file bucket. How should I do the trick on ESP32?

I did invest time on brewing my own solution. Till now I figured out, there are offically only two possible way for the logging stuff:
(esp_app_trace.h)

Code: Select all

typedef enum {
	ESP_APPTRACE_DEST_TRAX = 0x1,  ///< JTAG destination
	ESP_APPTRACE_DEST_UART0 = 0x2, ///< UART destination
} esp_apptrace_dest_t;

And the description on how this works in app_trace.c is huge. I'm hanging on it.

Any suggestion for me?


Cheers

Gfast2

[Gfast2]

Hi ESP-IDF,

I saw example (app_trace) that demostrate "How Slower" can the while loop can be lamed by a ESP_LOGx().

1. I just wonder, from the global point of view, would be the whole freeRTOS so drastically slowed down?

• Is this caused by generating human readable log text, or really more caused by the performance of the UART

Cheers

Gfast2

[ESP_igrr]

For simple logging (not apptrace) you can use esp_log_set_vprintf function to redirect log messages to your custom handler. Which would, for example, write to a file on SD card.

[Gfast2]

Hi igrr,

Thanks for the Tip. It seems like I'd try some idea just like your answers, before posting questions. To tell the truth, I always struggling if I should "RTFM" before I start to try something out. It's pretty a experience thing.

Thanks again. If my implementation as you said works well, I won't comes back. If not, I will ask a more concrete question.

Cheers

Gfast2

[Gfast2]

Hi iggr,

sorry, I can't! I think I've unstandstand, what I wanna achive is no more redirect app trace into sd card, but more about how to redirect stdout to a file.

Some ideas for me?

Cheers

Su Gao

[ESP_igrr]

Here you go: https://gist.github.com/igrr/82055d824c ... 217e0710b9.

This is a slightly modified sd_card example, it shows how to redirect logging messages (which come from ESP_LOG) and, optionally, stdout (which is used by 'printf') to a file on SD card.

[Gfast2]

Here you go: https://gist.github.com/igrr/82055d824c ... 217e0710b9.

This is a slightly modified sd_card example, it shows how to redirect logging messages (which come from ESP_LOG) and, optionally, stdout (which is used by 'printf') to a file on SD card.

Hello ESP_igrr,

OH MY GOD!!!

What a nice touch is that!!! It's no more the answer for my question, its the answer for all questions possibly can be stated about logging data to other destination!!!

How AMAZING! It works!!!

P.s. :
How you think about these extra stuffs only for color rendering. I know even in deployed systems, they are not been removed, Is that OK so?

Some really awesome log entries from my log.txt

Code: Select all

[0;32mI (232) SPI_FLASH: Partition size: total: 463346, used: 1757[0m
[0;32mI (238) SPI_FLASH: Opening file[0m
[0;32mI (239) SPI_FLASH: NOW Read from SPIFF:[0m

Cheers

Gfast2

[Gfast2]

Here you go: https://gist.github.com/igrr/82055d824c ... 217e0710b9.

This is a slightly modified sd_card example, it shows how to redirect logging messages (which come from ESP_LOG) and, optionally, stdout (which is used by 'printf') to a file on SD card.

Hey ESP_Igrr,

Just don't know how to make a pull request for your Gist.

My change is after line 32 ( int res = vfprintf(log_file... ):
(This will make all logs still being visible when the stdout still not been redirected & some how balanced the performance penalty caused by fsync() )

Code: Select all

	static int syncCounter = 0;
	printf("syncCounter=%d, Message: %s", syncCounter, fmt);

#define MAX_SYNC_MSG_BUF 5
	if( syncCounter++ > MAX_SYNC_MSG_BUF) {
		syncCounter = 0; // reset counter
		printf("syncCounter reset");
		fsync(fileno(log_file)); // do the acutal sync to file
	}

I hope this will be helpful if some one has the similar problem.

Cheers

Gfast2

[Gfast2]

Hallo Folks,

Slowly I start to understanding, why people say C is not a very safe language.

I did implementate this snippet as I describt right above. And it seems like one of my task didn't have enough Stack (Are these Logs saved in the stack of the task which created them?), have crash down the whole system, and even corrupted the whole OS.

From this point of view, I believe what I made is not a safe enough sollution. The better way should take care of these potential problem.

Did you have had the similar issues?

Cheers

Gfast2

[ESP_igrr]

Just don't know how to make a pull request for your Gist.

i don't think there's a way to do pull requests in gist, but you can "fork" any gist and do modifications in the fork, then post the link to your forked version for others to see.

Regarding stack size selection, indeed it is a not an easy problem, and there's no bulletproof solution.
In IDF, there are three methods for detecting stack overflows:
1. check that the stack pointer is valid when doing a context switch
2. check that stack canary is not corrupted when doing a context switch
3. set a debug watchpoint on the address at the end of the stack, this allows executing panic handler when stack overflow happens
We recommend keeping #2 and #3 enabled (which is the default behavior)
Corresponding Kconfig options are mentioned here: http://esp-idf.readthedocs.io/en/latest ... ckoverflow.
These methods are not 100% immune from failure. It is still possible to overshoot the stack end by such an amount that the canary check and watchpoint check will not be triggered. Such cases can indeed be hard to debug.
Looking forward, we plan to have more robust hardware mechanisms for stack bounds checking in future chips.

Related is a question about choosing the right stack size for a task. You need to have a stack which is large enough to allow for the worst case (deepest code path). At the same time, you don't want the stack to be much larger than required. We don't have a good method for making such choice automatically, but most often developers will use uxTaskGetStackHighWaterMark function to determine stack usage of their tasks, while testing the application to run through all possible code paths.
In the future, we are planning to look into automatic (dynamic) stack sizing through split-stacks feature of GCC. However, at the moment we don't have a timeline for such feature.