Timer group performance

[wobblyboots]

Hi all,

We are porting one of our LED panel driver boards to the ESP32, as much to learn about this wonderful chip as anything.

We have hit a snag.

LED panels are not complex - you disable the display, pump out the lit pixels over a shift register, select which row this should be, then re-enable the display. It's just that this process needs to be done very regularly, otherwise it becomes very obvious to humans looking at the display.

This process - updating the display - is called panel_refresh() in our code. It's not really important, but it does need to be called, as we say, very regularly.

Now, if we put a loop calling vtaskDelay(1) and then setting some EventGroup bits, and in another task, waiting on the refresh bit, then calling panel_refresh(). This works perfectly fine. But we'd like to update the display more often than every 1ms.

So, we've jumped into timers, which we have working.

This works - but every 12 to 18 seconds (depending on how it's set up, the time is the same for any given configuration), the display flickers. We believe this means that the update is not happening regularly.

In fact, what it feels like is that some other task is interrupting the panel_refresh, since the flicker is more significant and then goes away over a period of about a second or two.

We've tried wrapping the panel_refresh in a critical section, we've tried different timer groups (0 and 1) and different timers within a group (0 and 1).

We've tried putting the panel_refresh and even the ISR on the other CPU.

The task that waits for eventgroupbits and calls panel_refresh() is running at the max priority.

We've tried using Timer 1 (int 15, level 3) vs the default (Timer 0 int 6, level 1) via make menuconfig. I'm not sure which timers this refers to, since there's only 4 hardware timers.

I'm reasonably comfortable that the actual timer itself is not interrupting panel_refresh (we used another eventgroup bit to signal that refresh was taking place, and changed the ISR to check if it was refreshing, and then set yet another eventgroup bit called refresh_clash, this is never set).

So something is getting in the way. Next step is to get our hardware analysers to confirm or deny some irregularly in regard to timer triggering, but I wanted to see if anyone in this group had come across this before.

kind regards
Ian.

[ESP_igrr]

You may find SystemView helpful for debugging such issues. It offers a graphical representation of program events, such as interrupts and task switches. It should be easy to see what "gets in the way" of your refresh routine.

http://esp-idf.readthedocs.io/en/latest ... systemview

(You will need a board with high-speed JTAG to collect SystemView trace)

Regarding RTOS timer options: it refers to CPU internal counters, based on CCOMPARE and CCOUNT registers of Xtensa architecture. These are independent from Timer Group hardware timers.

[ESP_Sprite]

Also, I suggest you use the hardware for this. The I2S hardware, for example, is pretty powerful and uniquely capable of spitting out lots of pixels from DMA memory unattended, so you can spend all your computing power actually calculating the next image.

Hmm, I still have one or two RGB LED matrices laying around.. maybe it's time to make a demo app for this.

[wobblyboots]

Thanks guys, I appreciate the suggestions. It does feel like the FreeRTOS clock is getting int the way somehow, although I'm not sure even if we discover this how we change this, or give the timer group priority over the FreeRTOS clock (for example).

Our next task is to actually get the i2s module working as a 4 bit (and then a 6 bit) parallel port. Most LED panels effectively have two rows of green LEDs and two rows of red LEDs, all of which get shifted in at once. So we've been using the PIC32 parallel port using DMA for this up to this point. RGB panels have 2x3 = 6 bits that are shifted in. So an SPI equivalent isn't going to work, but we are hopeful that the I2S subsystem can do what we need. Indeed, then we'll not have to be sitting there bit-banging them in like we are now.

My reading on this I2S module makes me believe this is on the 'edge' of what is documented right now, although some people have got close with cameras and the like.

kind regards
Ian.

[ESP_Sprite]

This is true. The LCD mode of the I2S controller, unfortunately, is somewhat documented but there are no great examples for it. We hope to remedy this soon. I'll keep you updated, as I said I have some LED-panels I want to get working as well. (They're the sexy RGB ones; I should be able to abuse binary coding toi get some nice bit depth out of them without overloading the CPU.)

[wobblyboots]

Very happy to have a go at the i2s setup and post the results for assistance, thereby increasing the community knowledge, unless someone is already actively working on this.

kind regards
Ian.

[ESP_Sprite]

I'm actually working on porting some ancient code I have from back when the ESP32 was no more than a config file for an FPGA to the current ESP-IDF/hardware. Should be done this week, if I'm not interrupted by other things. I'll post the result here if I have something at least slightly working.

[wobblyboots]

Thanks so much.

In the mean time...I've got jtag on a wrover kit working, I'm at the point of generating systemview data - at least, I'm getting sysview files but neither Segger's Systemview nor Impuse displays anything.

I telnet 4444 to the openocd on localhost and run:

Code: Select all

esp32 sysview start file://pro-cpu.SVDat file://app-cpu.SVDat

The data looks like this:

Code: Select all

3b0d 0a3b 2056 6572 7369 6f6e 0909 5345
4747 4552 2053 7973 7465 6d56 6965 7765
7220 5632 2e34 320d 0a3b 2041 7574 686f
7209 0945 7370 7265 7373 6966 2049 6e63
0d0a 3b0d 0a00 0000 0000 0000 0000 000d
0aae c988 9406 180e 80b4 8913 80d0 a54c
8080 80fa 0300 8102 0e32 4e3d 4672 6565
5254 4f53 2041 7070 6c69 6361 7469 6f6e
2c44 3d45 5350 3332 2c43 3d58 7465 6e73
612c 4f3d 4672 6565 5254 4f53 c903 0e0b
4923 353d 5379 7354 6963 6baf 550e 0c49
2336 3d57 4946 495f 4d41 43a2 110e 0c49
2337 3d57 4946 495f 4e4d 49fd 040e 0b49
2338 3d57 4946 495f 4242 f504 0e0d 0a49
2339 3d42 545f 4d41 43ed 040e 0d0a 4923
3130 3d42 545f 4242 9019 0e0e 4923 3131
3d42 545f 4242 5f4e 4d49 dd05 0e09 4923
3132 3d52 5742 54bd 050e 0d0a 4923 3133
3d52 5742 4c45 c005 0e0d 4923 3134 3d52
5742 545f 4e4d 49dc 050e 0e49 2331 353d
5257 424c 455f 4e4d 49dc 050e 0949 2331
363d 534c 4330 ac06 0e09 4923 3137 3d53
4c43 31be 050e 0d0a 4923 3138 3d55 4843
4930 bf05 0e0d 0a49 2331 393d 5548 4349
31c1 050e 1149 2332 303d 5447 305f 5430
5f4c 4556 454c f505 0e11 4923 3231 3d54
4730 5f54 315f 4c45 5645 4cf5 050e 1249
2332 323d 5447 305f 5744 545f 4c45 5645
4c81 060e 1349 2332 333d 5447 305f 4c41
4354 5f4c 4556 454c 8606 0e11 4923 3234
3d54 4731 5f54 305f 4c45 5645 4cf7 050e
1149 2332 353d 5447 315f 5431 5f4c 4556
454c f405 0e12 4923 3236 3d54 4731 5f57
4454 5f4c 4556 454c fc05 0e13 4923 3237
3d54 4731 5f4c 4143 545f 4c45 5645 4c89
060e 0949 2332 383d 4750 494f bc05 0e0d
4923 3239 3d47 5049 4f5f 4e4d 49b0 060e
0e49 2333 303d 4652 4f4d 5f43 5055 30e3
050e 0e49 2333 313d 4652 4f4d 5f43 5055
31e1 050e 0e49 2333 323d 4652 4f4d 5f43
5055 32e0 050e 0e49 2333 333d 4652 4f4d
5f43 5055 33e1 050e 0949 2333 343d 5350
4930 bb05 0e09 4923 3335 3d53 5049 31be
050e 0949 2333 363d 5350 4932 b805 0e09
4923 3337 3d53 5049 33b9 050e 0949 2333
383d 4932 5330 be05 0e09 4923 3339 3d49
3253 31be 050e 0d0a 4923 3430 3d55 4152
5430 be05 0e0d 0a49 2334 313d 5541 5254
31c1 050e 0d0a 4923 3432 3d55 4152 5432
c305 0e0e 4923 3433 3d53 4449 4f5f 484f
5354 dd05 0e0c 4923 3434 3d45 5448 5f4d
4143 d205 0e09 4923 3435 3d50 574d 308f
060e 0949 2334 363d 5057 4d31 b905 0e09

..etc.. in app-cpu.SVDat

but both systemview and impuse basically show no events. Any clues?

kind regards
Ian.

[ESP_Alexey]

Hi wobblyboots,

Unfortunately I can not reproduce your problem. Could you provide logs from openocd console and also, please, re-upload you SVDat files as usual ones, not in hexdump format.

Regards,
Alexey

[wobblyboots]

Thank you so much for your help Alexey.

OpenOCD console here - works with GDB / Eclipse debugging.

Code: Select all

Open On-Chip Debugger 0.10.0-dev-ga859564 (2017-07-24-16:18)
Licensed under GNU GPL v2
For bug reports, read
        http://openocd.org/doc/doxygen/bugs.html
none separate
adapter speed: 20000 kHz
force hard breakpoints
Error: libusb_open() failed with LIBUSB_ERROR_NOT_SUPPORTED
Error: libusb_open() failed with LIBUSB_ERROR_NOT_SUPPORTED
Info : ftdi: if you experience problems at higher adapter clocks, try the command "ftdi_tdo_sample_edge falling"
Info : clock speed 20000 kHz
Info : JTAG tap: esp32.cpu0 tap/device found: 0x120034e5 (mfg: 0x272 (Tensilica), part: 0x2003, ver: 0x1)
Info : JTAG tap: esp32.cpu1 tap/device found: 0x120034e5 (mfg: 0x272 (Tensilica), part: 0x2003, ver: 0x1)
Info : accepting 'telnet' connection on tcp/4444
Info : Open file pro-cpu.SVDat
Info : Open file app-cpu.SVDat
App trace params: from 2 cores, size 4294967295 bytes, stop_tmo -1 s, poll period 1 ms, wait_rst 0, skip 0 bytes
Info : stat=e00 ctrl=80
Info : memadrstart=0 memadrend=fff traxadr=0 memsz=4000
Info : Total trace memory: 16384 bytes
Connect targets...
Info : Target halted. PRO_CPU: PC=0x400D1220 (active)    APP_CPU: PC=0x00000000
esp32: target state: halted
Info : Resume targets
Info : Targets connected.
16162
32367
48567
64769
80965
Info : Target halted. PRO_CPU: PC=0x400D1220 (active)    APP_CPU: PC=0x400D1220
esp32: target state: halted
Info : Resume targets
Disconnect targets...
Info : Target halted. PRO_CPU: PC=0x400D1220 (active)    APP_CPU: PC=0x400D1220
esp32: target state: halted
Info : Resume targets
Info : Targets disconnected.
Tracing is STOPPED. Size is 91788 of 4294967295 @ 2.664923 (2.369275) KB/s
Data: blocks incomplete 0, lost bytes: 0
TRAX: block read time [40.033001..44.000000] ms
TRAX: block proc time [0.000000..0.501000] ms
Info : Trace data processor thread exited with 0

Telnet log:

Code: Select all

$ telnet localhost 4444
Trying ::1...
Connection failed: Connection refused
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
Open On-Chip Debugger
> esp32 sysview start file://pro-cpu.SVDat file://app-cpu.SVDat
Open file pro-cpu.SVDat
Open file app-cpu.SVDat
App trace params: from 2 cores, size 4294967295 bytes, stop_tmo -1 s, poll period 1 ms, wait_rst 0, skip 0 bytes
stat=e00 ctrl=80
memadrstart=0 memadrend=fff traxadr=0 memsz=4000
Total trace memory: 16384 bytes
Connect targets...
Target halted. PRO_CPU: PC=0x400D1220 (active)    APP_CPU: PC=0x00000000
esp32: target state: halted
Resume targets
Targets connected.
16162
32367
48567
64769
80965
> esp32 sysview stop
Target halted. PRO_CPU: PC=0x400D1220 (active)    APP_CPU: PC=0x400D1220
esp32: target state: halted
Resume targets
Disconnect targets...
Target halted. PRO_CPU: PC=0x400D1220 (active)    APP_CPU: PC=0x400D1220
esp32: target state: halted
Resume targets
Targets disconnected.
Tracing is STOPPED. Size is 91788 of 4294967295 @ 2.664923 (2.369275) KB/s
Data: blocks incomplete 0, lost bytes: 0
TRAX: block read time [40.033001..44.000000] ms
TRAX: block proc time [0.000000..0.501000] ms
Trace data processor thread exited with 0
>

And svdat files attached as zip, the forum wouldn't allow a straight upload of svdat files.

make menuconfig confirms tracing enabled:

Code: Select all

 [*] SystemView Tracing Enable                                                                                                   │ │
  │ │                                                              ESP32 timer to use as SystemView timestamp source (Timer 1, Group 1)  --->                                                  │ │
  │ │                                                          [*] Trace Buffer Overflow Event                                                                                                 │ │
  │ │                                                          [*] ISR Enter Event                                                                                                             │ │
  │ │                                                          [*] ISR Exit Event                                                                                                              │ │
  │ │                                                          [*] ISR Exit to Scheduler Event                                                                                                 │ │
  │ │                                                          [*] Task Start Execution Event                                                                                                  │ │
  │ │                                                          [*] Task Stop Execution Event                                                                                                   │ │
  │ │                                                          [*] Task Start Ready State Event                                                                                                │ │
  │ │                                                          [*] Task Stop Ready State Event                                                                                                 │ │
  │ │                                                          [*] Task Create Event                                                                                                           │ │
  │ │                                                          [*] Task Terminate Event                                                                                                        │ │
  │ │                                                          [*] System Idle Event                                                                                                           │ │
  │ │                                                          [*] Timer Enter Event                                                                                                           │ │
  │ │                                                          [*] Timer Exit Event                                                                                                            │ │
  │ │                                                                                 [*] SystemView Tracing Enable                                                                                                   │ │
  │ │                                                              ESP32 timer to use as SystemView timestamp source (Timer 1, Group 1)  --->                                                  │ │
  │ │                                                          [*] Trace Buffer Overflow Event                                                                                                 │ │
  │ │                                                          [*] ISR Enter Event                                                                                                             │ │
  │ │                                                          [*] ISR Exit Event                                                                                                              │ │
  │ │                                                          [*] ISR Exit to Scheduler Event                                                                                                 │ │
  │ │                                                          [*] Task Start Execution Event                                                                                                  │ │
  │ │                                                          [*] Task Stop Execution Event                                                                                                   │ │
  │ │                                                          [*] Task Start Ready State Event                                                                                                │ │
  │ │                                                          [*] Task Stop Ready State Event                                                                                                 │ │
  │ │                                                          [*] Task Create Event                                                                                                           │ │
  │ │                                                          [*] Task Terminate Event                                                                                                        │ │
  │ │                                                          [*] System Idle Event                                                                                                           │ │
  │ │                                                          [*] Timer Enter Event                                                                                                           │ │
  │ │                                                          [*] Timer Exit Event                                                                                                            │ │
  │ │                                                                                

kind regards
Ian.