syntax of push_data_ready_event

# 1 year ago
DHollatz	Dear Tango community, I would like to use the data_ready_event for a camera image. I want to push the event with `self.push_data_ready_event('camera_frame')` but I recieve the following error: `File "C:\DeviceServers\Kameras\AlliedVisionCamera.py", line 158, in _acquire_frame self.push_data_ready_event('camera_frame') Boost.Python.ArgumentError: Python argument types in DeviceImpl.push_data_ready_event(AlliedVisionCamera, str) did not match C++ signature: push_data_ready_event(class Tango::DeviceImpl {lvalue} self, class boost::python::str {lvalue} attr_name, long c` The online documentation of PyTango says the push_data_ready_event method needs only the attribute name as input, so I don't know what I did wrong. Pytango documentation How do I push data_ready_events properly? Thanks, Dominik Edit: I use Tango 9.4.1, PyTango 9.4.1, Windows 10, POGO 9.8.0 Edit #2: If I give the counter aswell, I get a different error: `File "C:\DeviceServers\Kameras\AlliedVisionCamera.py", line 158, in _acquire_frame self.push_data_ready_event('camera_frame', 2) PyTango.DevFailed: DevFailed[ DevError[ desc = Not able to acquire serialization (dev, class or process) monitor origin = void __cdecl Tango::TangoMonitor::get_monitor(void) at (D:\bld\cpptango_1678646605551\work\cppapi\include\tango/server/tango_monitor.h:141) reason = API_CommandTimedOut severity = ERR] ]` Edited 1 year ago Attachments: 15Kb

# 1 year ago

Dear Tango community,

I would like to use the data_ready_event for a camera image.
I want to push the event with

self.push_data_ready_event('camera_frame')

but I recieve the following error:

File "C:\DeviceServers\Kameras\AlliedVisionCamera.py", line 158, in _acquire_frame
    self.push_data_ready_event('camera_frame')
Boost.Python.ArgumentError: Python argument types in
    DeviceImpl.push_data_ready_event(AlliedVisionCamera, str)
did not match C++ signature:
    push_data_ready_event(class Tango::DeviceImpl {lvalue} self, class boost::python::str {lvalue} attr_name, long c

The online documentation of PyTango says the push_data_ready_event method needs only the attribute name as input, so I don't know what I did wrong.

Pytango documentation

How do I push data_ready_events properly?

Thanks,
Dominik

Edit: I use Tango 9.4.1, PyTango 9.4.1, Windows 10, POGO 9.8.0

Edit #2: If I give the counter aswell, I get a different error:

File "C:\DeviceServers\Kameras\AlliedVisionCamera.py", line 158, in _acquire_frame
    self.push_data_ready_event('camera_frame', 2)
PyTango.DevFailed: DevFailed[
DevError[
    desc = Not able to acquire serialization (dev, class or process) monitor
  origin = void __cdecl Tango::TangoMonitor::get_monitor(void) at (D:\bld\cpptango_1678646605551\work\cppapi\include\tango/server/tango_monitor.h:141)
  reason = API_CommandTimedOut
severity = ERR]
]

Edited 1 year ago

# 1 year ago
Anton	Hi Dominik Thanks for the clear issue report. This looks like an error in the documentation. The counter isn't optional (and we also can't use it as a keyword argument). `self.push_data_ready_event('camera_frame', 0)` /Anton

# 1 year ago
DHollatz	Hi Anton, Thanks for the reply. I edited my post at the time you wrote your reply. I still get the PyTango.DevFailed error as shown above even when using: `self.push_data_ready_event('camera_frame', 0)` Dominik

# 1 year ago
DHollatz	Hi Anton, I played around a bit and I found out, that I can not push data_ready_events from a different thread in Python other than the main thread. The push_data_ready_event('camera_frame', 0) works just fine when executed by the main thread for example in a command. Is this intended? I acquire the camera frames in a separate thread because they may need a long time to acquire or wait for a trigger. Cheers, Dominik Edited 1 year ago

# 1 year ago

DHollatz

Hi Anton,

I played around a bit and I found out, that I can not push data_ready_events from a different thread in Python other than the main thread. The push_data_ready_event('camera_frame', 0) works just fine when executed by the main thread for example in a command.

Is this intended? I acquire the camera frames in a separate thread because they may need a long time to acquire or wait for a trigger.

Cheers,
Dominik

Edited 1 year ago

# 1 year ago
Anton	Hi Dominik The Not able to acquire serialization (dev, class or process) monitor error typically happens if we are trying to do things concurrently that need the "monitor lock". This lock is used to implement the serialization model in the Tango device server. https://tango-controls.readthedocs.io/en/latest/development/advanced/threading.html#serialization-model-within-a-device-server Attribute read/write handlers, command handlers, and pushing events are all things that need the monitor lock. Consider a device that is busy handling a command and that takes a few seconds - the command handler has the lock. During this time, a background thread decides to push an event. The event cannot be pushed while the command handler is busy, so the push method blocks waiting for the lock. If it takes too long, we can get the error (default timeout is 3.2 seconds). If the events are being pushed from a standard Python thread (including an asyncio callback), then that could also be a problem since cppTango's thread implementation is not "aware" of these threads. There is a context handler called EnsureOmniThread that might be useful. The documentation mentions its use for clients, but it can also be useful with servers. The idea is to ensure that the body of the thread pushing events is running within this handler. The handler must not be created each time an event is about to be pushed - it must be long-lived. https://pytango.readthedocs.io/en/v9.4.1/howto.html?highlight=ensureomnithread#using-clients-with-multithreading /Anton

# 1 year ago

Anton

Hi Dominik

The Not able to acquire serialization (dev, class or process) monitor error typically happens if we are trying to do things concurrently that need the "monitor lock". This lock is used to implement the serialization model in the Tango device server. https://tango-controls.readthedocs.io/en/latest/development/advanced/threading.html#serialization-model-within-a-device-server

Attribute read/write handlers, command handlers, and pushing events are all things that need the monitor lock. Consider a device that is busy handling a command and that takes a few seconds - the command handler has the lock. During this time, a background thread decides to push an event. The event cannot be pushed while the command handler is busy, so the push method blocks waiting for the lock. If it takes too long, we can get the error (default timeout is 3.2 seconds).

If the events are being pushed from a standard Python thread (including an asyncio callback), then that could also be a problem since cppTango's thread implementation is not "aware" of these threads. There is a context handler called EnsureOmniThread that might be useful. The documentation mentions its use for clients, but it can also be useful with servers. The idea is to ensure that the body of the thread pushing events is running within this handler. The handler must not be created each time an event is about to be pushed - it must be long-lived. https://pytango.readthedocs.io/en/v9.4.1/howto.html?highlight=ensureomnithread#using-clients-with-multithreading

/Anton

# 1 year ago
DHollatz	Hi Anton, Thank you for your help. I was able to solve the issue. The error was on my side, not on the Tango side. If anyone has a similar issue in the future, I will describe it here for future reference. The explanation of cppTango threads and separate python threads was very usefull, however I was using the EnsureOmniThread already. After some investigation I realized that my problem was caused by something else. As you described above, attribute read/write, command and event handlers require the monitor lock. I implemented my device server in a specific way that was not compatible with the monitor lock rules. The reason for this implementation was "historical". Some explanation: I communicate with the camera hardware using the python package "socket". It takes about 3 seconds to establish the connection, which is too long to do this within every Tango command execution. My solution was to open the socket connection in a separate python thread and keep it open. The main device server thread that executes Tango commands would then communicate with this "worker_thread" to send and recieve data from the camera. This comunication is inherently asynchronously. However, to make attribute read and write methods, for example exposure time, convenient for a Tango client, the hardware information should come as a return value from the Tango command, e.g. the communication should be synchronous. To solve this, I made the chain from Tango command –> worker command –> hardware request –> hardware answere –> worker command return –> Tango command return synchronous again with some waiting routines. This leads to the scenario that whenever the worker thread communicates with the hardware, the main device server thread is "stuck" within the command execution and has the monitor lock. Thus the worker_thread can not push data_ready_events. The solution for me was easy (after I knew what was going on): I implemented a command dependent synchronous/asynchronous flag. This allows synchronous calls for things like reading exposure time and asynchronous calls to support aquiring images with long exposure time or a "waiting for trigger" scenario. I created a synchronous read_camera_frame method and an asynchronous aquire_frame method. The synchronus read_camera_frame method returns the most recent frame stored in the camera_frame attribute. The asynchronous aquire_frame command can wait for the hardware and push a data_ready_event upon completion and write the new frame into the camera_frame attribute. Thanks again for the help. Dominik Edited 1 year ago

# 1 year ago

DHollatz

Hi Anton,

Thank you for your help. I was able to solve the issue. The error was on my side, not on the Tango side.
If anyone has a similar issue in the future, I will describe it here for future reference.

The explanation of cppTango threads and separate python threads was very usefull, however I was using the EnsureOmniThread already. After some investigation I realized that my problem was caused by something else. As you described above, attribute read/write, command and event handlers require the monitor lock. I implemented my device server in a specific way that was not compatible with the monitor lock rules. The reason for this implementation was "historical". Some explanation:

I communicate with the camera hardware using the python package "socket". It takes about 3 seconds to establish the connection, which is too long to do this within every Tango command execution. My solution was to open the socket connection in a separate python thread and keep it open. The main device server thread that executes Tango commands would then communicate with this "worker_thread" to send and recieve data from the camera. This comunication is inherently asynchronously. However, to make attribute read and write methods, for example exposure time, convenient for a Tango client, the hardware information should come as a return value from the Tango command, e.g. the communication should be synchronous. To solve this, I made the chain from Tango command –> worker command –> hardware request –> hardware answere –> worker command return –> Tango command return synchronous again with some waiting routines. This leads to the scenario that whenever the worker thread communicates with the hardware, the main device server thread is "stuck" within the command execution and has the monitor lock. Thus the worker_thread can not push data_ready_events. The solution for me was easy (after I knew what was going on):

I implemented a command dependent synchronous/asynchronous flag. This allows synchronous calls for things like reading exposure time and asynchronous calls to support aquiring images with long exposure time or a "waiting for trigger" scenario. I created a synchronous read_camera_frame method and an asynchronous aquire_frame method. The synchronus read_camera_frame method returns the most recent frame stored in the camera_frame attribute. The asynchronous aquire_frame command can wait for the hardware and push a data_ready_event upon completion and write the new frame into the camera_frame attribute.

Thanks again for the help.

Dominik

Edited 1 year ago

# 1 year ago
Anton	Hi Dominik Good news. Thanks for reporting back that the problem is solved, and for providing the details. Hopefully this will be useful for future readers. /Anton