Thread safe or thread safe?

[zliudr]

Although the issue arises from an esp-azure library I think I could use some general guidance. So the issue at hand is a function that parses a json string to extract a value. Several tasks have needs to parse json. One is a callback function and another is in the main task, for instance. If the callback function calls parseJson() and the cpu switches to main task, which also calls parseJson() from a parson library, then what happens to parseJson()? Does the second call clone the function? If I WANT to write a thread-safe function, how should I write it? Thank you!

[PeterR]

I did not find parseJson() in the IDF and so cannot answer your specific question.

In general terms however:
For a library to be thread safe then there must be no 'memory' between function calls contained within library variables. It must not be possible for the library to share some resource (internal variable and/or device) with another thread.
Executable code is not the issue, it is the shared variables and/or shared device.

So a library which works this way:

Code: Select all

pMyJsonDecode = startJson(aString);
nextValue = nextJsonValue(pMyJsonDecode);

Is likely to be thread safe as the parsing 'context' is being stored in pMyJsonDecode. The libary itself has no internal memory. The library gives you the item to be remembered and on next use you remind the library what it was up too.

If however a function uses data previously setup by itself or another library function (& not passed back to the libary user) then it is likely that the library is not thread safe.
Typically if a libary contains file scope variables and/or static variables within a function then the library is probably not thread safe (a number of exceptions including when the library uses semaphore etc to control access to shared items).

The typical 'thread unsafe' library example is strtok() http://www.cplusplus.com/reference/cstring/strtok/. In the old days strtok() was not thread safe as you initialised the process by passing your string and then continued the process by passing NULL. The library function strtok() remembered your string and so was not thread safe.

Note that these days strtok() is typically thread safe as the supporting plaform keeps a local copy of library variables for each thread.
This thread memory only works for some standard C library functions however.

[zliudr]

Thanks PeterR. So essentially if the function is "detached" from the resources it uses, it is safe. It gets cloned so it runs on a different memory block each time it needs more copies. Just curious I wonder how an OS tells whether certain function needs to be cloned (re-entered). Regarding the parse json function, it's from a library called parson. It appeared in esp-azure sample code so it could either be library of choice by the author(s) of the sample code or that of the author(s) who ported the sample code to esp. It's probably not part of azure. The comment in the sample code could have been written by multiple authors:

Code: Select all

// Copyright (c) Microsoft. All rights reserved.
// Licensed under the MIT license. See LICENSE file in the project root for full license information.

// This sample shows how to translate the Device Twin json received from Azure IoT Hub into meaningful data for your application.
// It uses the parson library, a very lightweight json parser.

// There is an analogous sample using the serializer - which is a library provided by this SDK to help parse json - in devicetwin_simplesample.
// Most applications should use this sample, not the serializer.

// WARNING: Check the return of all API calls when developing your solution. Return checks ommited for sample simplification.

There might be a memory usage concern over the serializer? There is a serializer in azure-iot-sdk-c folder with some sample code. Maybe this serializer is thread safe. I'll have to read the parson library source code to see if it stores anything internally. Hopefully not.

[PeterR]

I cannot tell from the comment. Post a link to the library and I may have a look.

Just curious I wonder how an OS tells whether certain function needs to be cloned (re-entered).

The OS doesn't know. The OS may however work in a way that re-entrancy is not an issue e.g. Javascript or Ardunio init/loop structure (sans interrupts).
The code works on variables which are auto (stack), heap (dynamic allocation) or static. Auto variable are created on the calling tasks stack and so are safe. Heap variables are created from a common pool. Heap variable may be safe if they essentially belong to one caller or they are protected using semaphores etc. Similarly static variables.
The author codes one way or another. Its entirely up to the author.
Memory usage is indeed a common reason why re-entrancy is dropped. Also it takes more thinking. But you would have to be quite memory hungry to find memory a reason to drop re-entrancy on an ESP32 IMHO.
JSON serialisation will probably use a lot of memory(be careful) but coding for re-entrancy does not itself use more memory. Its is only when you have more than one client serialising at the same time that you would use more memory EDIT: e.g with two tasks serialising you would need 2 massive task stacks to hold all the JSON workings.

[zliudr]

Maybe like the old DOS that has a busy byte so the caller knows some other process is calling INT21? The deserialization is not urgent but should be done safely. Maybe I can implement this DOS busy byte with a wrapper function with a static variable so it will read its own busy byte and return "busy" and the caller may try again later using vTaskDelay().

[ESP_Sprite]

If you find that that is needed, the more proper approach is to actually use a FreeRTOS mutex instead of trying to reinvent that wheel.

[PeterR]

Sprite is right - just share a semaphore with all users.
And before you ask; the ESP32 FreeRTOS semaphores know all about multiple cores.

[zliudr]

mutex and semaphores. Great! Now I have these keywords and I can read about them.