Can I work-around device lockup/hang when changing Serial baudrates?

[PanicanWhyasker]

It seems I've put myself in a well-engineered trap

I'm using ESP IDF v3.2 and I have initialization code like this:

Code: [Select all] [Expand/Collapse]

1. void setup() {
2.     /*
3.        ...
4.        setup several pins
5.        ...
6.     */
7.     int serialBaud = 115200;
8.     Serial.begin(serialBaud);
9.  
10.     /*
11.        ...
12.        ... initialize a bunch of HW and read some config from the flash, including
13.            a variable named newBaud ...
14.        ...
15.     */
16.    
17.     // change serial baudrate, if required:
18.     if (newBaud >= 1200 && newBaud <= 1500000 && newBaud != serialBaud) {
19.         Serial.printf("Custom serial baud set, changing baud-rate to %d\n", newBaud);
20.         Serial.flush();
21.         delay(2);
22.         Serial.end();
23.         serialBaud = newBaud;
24.         Serial.begin(serialBaud);
25.     }
26.    
27.     // I2C init:
28.     Wire.begin();
29. ...

GeSHi © Codebox Plus Extension

The reason for doing this is because the UART is connected to another MCU in the production device - while programming or debugging the other MCU is not present around, but in the target app they are connected. And the other device cannot function at 115200. "newBaud" is actually 57600.

Now, I'm encountering a hang similar to issue 2015, basically the CPU locks up just after the "Custom serial baud set..." message.
We had this sort of hang during development, but I was under the impression that adding the flush and the delay(2) fixed it. And when testing it didn't show up either. Curiously enough, the hang doesn't happen if the ESP is executing the factory app, only if it's running off ota_0 or ota_1.
I was able to reproduce the issue and it happens every time locally.
The problem is that I applied an OTA update to a device that is 2000 km away... As it now boots ota_0, it hits the hang and is bricked now

It seems that the hang is dependent on some kind of timing, which explains the factory vs ota_0/ota_1 behaviour.

My question is:
Is there a way to instruct people in the field to do something that can modify the ESP's behaviour?
Like:

- any possibility to force the ESP boot into the factory app (I searched and it seems I need to have a special bootloader for that)?
- a way to alter the timings, or forbid the Serial printout?

If I'm able to get the device boot just once, I'll update it with the code that apparently fixes issue 2015:

Code: [Select all] [Expand/Collapse]

1. void setup() {
2. ...  
3.     // change serial baudrate, if required:
4.     if (newBaud >= 1200 && newBaud <= 1500000 && newBaud != serialBaud) {
5.         Serial.printf("Custom serial baud set, changing baud-rate to %d\n", newBaud);
6.         Serial.flush();
7.         delay(2);
8.  
9.         serialBaud = newBaud;
10.         Serial.updateBaudRate(serialBaud);
11.     }
12.  
13. ...

GeSHi © Codebox Plus Extension

The people in the field aren't tech-savvy, and don't have programmers or any development tools for that matter. Cables, pincers, screwdrivers is all they have At least our PCB pins out a lot of ESP's pins, so they can be accessed.

I had the idea of grounding GPIO15 during boot, as the docs say:

"If driven Low, silences boot messages printed by the ROM bootloader. Has an internal pull-up, so unconnected = High = normal output."

However it doesn't change a thing on my devices for some reason.

The only other possibility otherwise would be to send out another device with the issue fixed, but it would be a huge embarrassment from the business side. We may come off as unreliable and lose this client...

[ESP_Sprite]

That is a doozy... there's no check for GPIO state somewhere above that code in setup() that you may be able to bypass this or at least have some hope of introducing some timing oddity that may sidestep the issue? Alternately, perhaps shorting Tx and Rx perhaps introduces enough of a timing difference (because Rx interrupt) that the issue doesn't show?

Also, moving this to the Arduino subforum.

[PanicanWhyasker]

@ESP_Sprite, thanks for the pointer. I tried that, but it didn't help. The only type of intervention that doesn't immediately go into the hang is if I ground pin 0 (BOOT) when I power cycle the thing, in this case it gets in serial bootloader mode. The people in the field could do this, but of course without a programmer I doubt this will be of any use.