Mixing asynchio with synchronous (blocking) code

[garykuipers]

I have developed some asyncio classes that play well together. LoRa and queues and an OLED display all play nicely together. Then I invited class MFRC522 to the party (https://github.com/Lennyz1988/micropython-mfrc522) and it refuses to share processing time.

I think I am expecting that something that shuttles among processing logic that can await should be able to wrap a synchronous piece inside asynchronous task and "share". Apparently not.
First I would ask for a bit of education: My reading indicates that asyncio.create_task combined in a gather SHOULD each receive some processing time. This is not the case. When I launch the program below, the part using MFRC522 class comes up and you can see the security_loop messages appear. rfid_lop reads the card and sits. Issuing a ctrl-C allows the security_loop to process:

queues = QUEUES()
lora = LORA(cfg, queues)
display = OLED(cfg, queues)
rfid = RFID(cfg, queues)
security = SecurityPosts(cfg, queues, display)

async def temp_wait_execute():
    global rfid
    try:
        print('waiting for rfid')
        await asyncio.wait_for(rfid.main_loop(), timeout=10)
    except asyncio.TimeoutError as ex:
        print("Gave up waiting, task canceled")
        print(f"[ERRROR] --> {ex}")

async def main():
    lora_sender = asyncio.create_task(lora.lora_send())
    lora_receiver = asyncio.create_task(lora.lora_receive())
    rfid_loop = asyncio.create_task(temp_wait_execute())
    # rfid_loop = asyncio.create_task(rfid.main_loop())
    security_loop = asyncio.create_task(security.main_loop())
    #test = asyncio.create_task(security.test_rfid_sent_data())
    await asyncio.gather(lora_sender, lora_receiver, rfid_loop, security_loop, return_exceptions=False)
    await asyncio.sleep_ms(10_000)

asyncio.run(main())

I had started with :
queues = QUEUES()
lora = LORA(cfg, queues)
display = OLED(cfg, queues)
rfid = RFID(cfg, queues)
security = SecurityPosts(cfg, queues, display)

async def main():
    lora_sender = asyncio.create_task(lora.lora_send())
    lora_receiver = asyncio.create_task(lora.lora_receive())
    rfid_loop = asyncio.create_task(rfid.main_loop())
    security_loop = asyncio.create_task(security.main_loop())
    #test = asyncio.create_task(security.test_rfid_sent_data())
    await asyncio.gather(lora_sender, lora_receiver, rfid_loop, security_loop, return_exceptions=False)
    await asyncio.sleep_ms(10_0

And also tried this aproach:
import asyncio
import time

class SynchronousClass:
def do_something(self):
print("Doing something synchronously...")
time.sleep(5)
print("Done.")

async def async_function():
synchronous_object = SynchronousClass()
loop = asyncio.get_event_loop()
await loop.run_in_executor(None, synchronous_object.do_something)

async def main():
await async_function()
but is seems
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
loop.close()

The example is malformed (loo?)

[peterhinch]

Blocking methods are incompatible with uasyncio because it is a cooperative scheduler. For the time that a method blocks, the scheduler is stalled and nothing else gets to run. The only solution is to use _thread to run the blocking routine. There is some guidance on how this can be done here.

Incidentally uasyncio does not have loop.run_in_executor - the above link provides a workround to do something similar. Incidentally I haven't tested threading on ESP32 - I'm not sure if it's enabled by default.

[garykuipers]

Thanks Peter. I tried _thread and failed (it s one of the attempts I did not list). I will try _thread and let you know. Quick Q: How would I "enable _thread" (I thought importing something automatically enabled it)

[garykuipers]

I adapted my code to your suggestion. ImportError: no module named 'threadsafe'.
I then: upip.install('threadsafe')
and get: Error installing 'threadsafe': , packages may be partially installed

Please do not think me unintelligent, simply in need of education. About 95% of the time I use upip.install, the above is the response. The next step is to root about to see how to get that from gitbub micropython.

Here is the question: I have looked at many pages in google trying to find a simple "to find something in the github micropython project (like the threadsafe module) do this ...".I have not landed on a good way to do it. Perhaps there is a tutorial? Or instruction list? Or maybe a comprehensive list of what can be found in the project?

[peterhinch]

If import _thread fails then it means that the firmware build does not have threading enabled. Enabling this requires setting a compile option and rebuilding the firmware.

The upip package is obsolete and has been replaced by mip. However not all packages are on PyPi. To use modules from threadsafe and primitives you need to clone this repo from git and copy the directories and contents from here to your device.

However the unblock function is dependent on threading, so you must run a firmware build with threading enabled.

[garykuipers]

Thanks Peter: Please check my logic:

I commented

from threadsafe import Message

import _thread
Laving the _thread import.
It complained that it could not find message.

I then created a dummy class Message with an init that does a pass
The program started to run.

From this I conclude that the _thread is available in the build I am using: esp32-20220618-v1.19.1.bin

SO the next step is to get threadsafe from somewhere, I am going with the mip suggestion and when I do ...

import mip
Traceback (most recent call last):
File "", line 1, in
ImportError: no module named 'mip'

import upip
upip.install('mip')
Installing to: /lib/
Warning: micropython.org SSL certificate is not validated
Error installing 'mip': memory allocation failed, allocating 96 bytes, packages may be partially installed

So we have to go with the clone repo suggested in your message

[garykuipers]

Hi Peter: Your "Asynchronous programming in MicroPython" is whre I will concentrate my efforts re: asyncio, thank you!

[steveh127]

The MFRC522 read functionality doesn't block, it just tries for quite a long time. After all a card reader that blocked wouldn't be much use but trying for a while makes sense.

If you look at the MFRC522 code in the _tocard method there is a loop that tries to read the card 2000 times it follows 'i = 2000'. I reduced that to 200 and could then create a coroutine that worked with asyncio.

If your other elements are time critical and require very frequent processing you may still need to follow the threading route.

[garykuipers]

Thank you for the reply: Since I am just learning, would it be possible to have you share how you " create a cooroutine that worked with asyncio"?

[steveh127]

You have already created one. Your async def temp_wait_execute should work if you substantially reduce the MFRC522 delay. As a first test I'd remove the timeout trap, it should run, eventually, even without changing the delay.

[rarefeen]

hi @steveh127 , I am facing the same issue with MFRC522, it consumes too much time for trying to read the rfid in every loop. so all of my other concurrent task is being delayed.
did you resolve the issue or find any way?
.................
try:
while True:
uid = ""
print("start", utime.ticks_ms() / 1000)
(stat, tag_type) = rdr.request(rdr.REQIDL)
print("end read_rfid", utime.ticks_ms() / 1000)
if stat == rdr.OK:
(stat, raw_uid) = rdr.anticolAl()
if stat == rdr.OK:
uid = ("0x%02x%02x%02x%02x" % (raw_uid[0], raw_uid[1], raw_uid[2], raw_uid[3]))
print("uid ", uid)
rfid_list.append(uid)
await asyncio.sleep_ms(50)
except KeyboardInterrupt:
print("Bye")
........................
difference between start print and end print is almost >500ms :D

[peterhinch]

As a last resort blocking functions can be interfaced to uasyncio with threading: see this ref.

[bixb922]

Since the driver is a .py, you could try to yield control at some points inserting asyncio.sleep(0). A candidate is the loop line 92 after i = 2000. This might not work if you have several mrfc devices or if it messes up the timing of the hardware.