Polling VS Events: how to understand and use correctly?

# 3 years ago
mariocaptain	Dear all, I have been into Tango for a couple of months and am in the middle of building a real system now using mostly taurus & pytango. Though I have read the documentations of tango, pytango and taurus, I am still a bit confused on the co-existence of polling and events. I read that since events are much more efficient they are recommended over polling, but I also notice that polling are implemented in most device server sample code and tango clients and forum snippets etc. I am so far so good with polling but I would like to understand events and use them in place of polling as much as possible to optimize the system. So any insights or advice into this matter would be greatly appreciated. My understanding: 1. Polling In this model, device servers are polled by tango every "interval" (e.g. 1 second, this can be changed). This can create bottle necks. I notice that my synoptic viewer (jdraw built in synoptic viewer) can be laggy when there are many devices involved. So polling is good if used wisely and moderately. 2. Events In this model, device servers push events out, and clients can get these events by registering some listeners. I wrote some test client to test and it's all good. My questions: 1. Is it possible to not use polling at all and only use events to achieve max efficiency? 2. In device servers, where to push events? Say in my device, I have a `read_voltage` command that reads the value of the voltage attribute. In this function, I of course will need to write I/O code to get the voltage from the real hardware. I see that if we push the changed value of voltage in this function, the push may not be triggered at all, since there is no polling to trigger the read_voltage command (?) Also for that reason, pushing the value out in this function would make no point at all since the function is polled regularly anyway, is that correct? The other way I have done is that to implement events, I just leave the read_voltage function blank, and create an "voltage_update" thread inside the device server. In this thread, the voltage value is read regularly and push out. This is fine (though I have no idea if this is the proper way or not), but I notice that when the number of devices in this class are numerous, there will be so many threads created, and in my experiment that can lead to Segmentation Fault when starting the device server. Also creating so many threads just for this reason is not very healthy IMHO. 3. If (1) is possible, how to disable polling, or maximize the use of events? What are the use cases of polling vs events? For my tango clients that I create from my own pytango code, of course I can register to events the client is interested in only. But in this case, how clients such as jive, synoptic viewer, or Taurus Gui widgets are supposed to communicate with the device servers? I am sorry if the questions are not totally clear since I am still confused. Edited 3 years ago

# 3 years ago

Dear all,

I have been into Tango for a couple of months and am in the middle of building a real system now using mostly taurus & pytango.

Though I have read the documentations of tango, pytango and taurus, I am still a bit confused on the co-existence of polling and events. I read that since events are much more efficient they are recommended over polling, but I also notice that polling are implemented in most device server sample code and tango clients and forum snippets etc. I am so far so good with polling but I would like to understand events and use them in place of polling as much as possible to optimize the system.

So any insights or advice into this matter would be greatly appreciated.

My understanding:
1. Polling
In this model, device servers are polled by tango every "interval" (e.g. 1 second, this can be changed). This can create bottle necks. I notice that my synoptic viewer (jdraw built in synoptic viewer) can be laggy when there are many devices involved. So polling is good if used wisely and moderately.

2. Events
In this model, device servers push events out, and clients can get these events by registering some listeners. I wrote some test client to test and it's all good.

My questions:
1. Is it possible to not use polling at all and only use events to achieve max efficiency?

2. In device servers, where to push events? Say in my device, I have a

read_voltage

command that reads the value of the voltage attribute. In this function, I of course will need to write I/O code to get the voltage from the real hardware. I see that if we push the changed value of voltage in this function, the push may not be triggered at all, since there is no polling to trigger the read_voltage command (?) Also for that reason, pushing the value out in this function would make no point at all since the function is polled regularly anyway, is that correct?

The other way I have done is that to implement events, I just leave the read_voltage function blank, and create an "voltage_update" thread inside the device server. In this thread, the voltage value is read regularly and push out. This is fine (though I have no idea if this is the proper way or not), but I notice that when the number of devices in this class are numerous, there will be so many threads created, and in my experiment that can lead to Segmentation Fault when starting the device server. Also creating so many threads just for this reason is not very healthy IMHO.

3. If (1) is possible, how to disable polling, or maximize the use of events? What are the use cases of polling vs events? For my tango clients that I create from my own pytango code, of course I can register to events the client is interested in only. But in this case, how clients such as jive, synoptic viewer, or Taurus Gui widgets are supposed to communicate with the device servers?

I am sorry if the questions are not totally clear since I am still confused.

Edited 3 years ago

# 3 years ago
Leonard	I have been using PyTango for a while now but never have worked with Polling, so I may also be slightly wrong, but here is my understanding: Polling in a general sense is not a nice design, but this refers to the client! I.e. a client wants to know the voltage dropped below a safe limit, so it has to check every 1s or so. Events would be the solution here, your device sends out an event when a new reading is available/ a changed reading is available or (even better) the voltage dropped below a limit configured on the device server. If your device server has a way of sending out these events without polling something in turn, that would be nice, but is hardly ever the case. For example a voltage source with RS232 will usually wait for commands, and e.g. return the voltage when requested. Thus, you need polling somewhere, to regularly get the state/voltage reading of the supply, interpret it in Tango, and react on it appropriately. Your solution is to manually poll (with a thread calling `voltage_update`) the data, allowing you to generate events at the device server level. If you only implement a simple `read_voltage` and no events, the client has to poll this attribute or command and react itself. Tango now offers an "in-between", where you write a simple code just implementing `read_voltage`, Tango takes care of requesting the value every 1s or so, and can even generate events when a defined threshold or value change is detected! The client/user can either read the value periodically or subscribe to a change event which Tango handles for you. According to the docs, two further benefits are a (limited) value history since there can be a Polling ring buffer and secondly, faster response times for slow hardware if e.g. reading the pressure takes 300ms but it would be sufficient to return a cached value from the polling buffer, the response would be almost instant. I have never actually tried this, so maybe someone more experienced from the community can add to my comment… Edited 3 years ago

# 3 years ago

Leonard

I have been using PyTango for a while now but never have worked with Polling, so I may also be slightly wrong, but here is my understanding:

Polling in a general sense is not a nice design, but this refers to the client! I.e. a client wants to know the voltage dropped below a safe limit, so it has to check every 1s or so.
Events would be the solution here, your device sends out an event when a new reading is available/ a changed reading is available or (even better) the voltage dropped below a limit configured on the device server.

If your device server has a way of sending out these events without polling something in turn, that would be nice, but is hardly ever the case. For example a voltage source with RS232 will usually wait for commands, and e.g. return the voltage when requested. Thus, you need polling *somewhere*, to regularly get the state/voltage reading of the supply, interpret it in Tango, and react on it appropriately.

Your solution is to manually poll (with a thread calling `voltage_update`) the data, allowing you to generate events at the device server level. If you only implement a simple `read_voltage` and no events, the client has to poll this attribute or command and react itself.

Tango now offers an "in-between", where you write a simple code just implementing `read_voltage`, Tango takes care of requesting the value every 1s or so, and can even generate events when a defined threshold or value change is detected!
The client/user can either read the value periodically or subscribe to a change event which Tango handles for you.
According to the docs, two further benefits are a (limited) value history since there can be a Polling ring buffer and secondly, faster response times for slow hardware if e.g. reading the pressure takes 300ms but it would be sufficient to return a cached value from the polling buffer, the response would be almost instant.

I have never actually tried this, so maybe someone more experienced from the community can add to my comment…

Edited 3 years ago

# 3 years ago
lorenzo	Hi Leonard, Mariocaptain, basically, you're correct. At device level, talking about server side now, Tango exploits "device polling" for continuously monitoring the equipment (e.g. reading the voltage), comparing the current value with the previous one and sending the relevant events (change, archive,…) to the registered clients whenever the difference is beyond the threshold. Thresholds can be configured on a per-attribute basis. This is the default mechanism for sending events, and it is just matter of configuration, no code required. Tango provides also a mechanism to "push events in the code", e.g. allows the developer to trigger sending events on demand. This can be useful, for instance, if you have an equipment that's sending data asynchronously to the Tango device and you want to react immediately to the new data, informing the clients. This, however, requires writing a few lines of code in the relevant methods. Back to question 3, yes, you can decide not to configure device polling and provide the necessary code to push all the required events, but this can have undesired effects. Proper clients, e.g. clients specifically written to subscribe to events will behave correctly, and so will clients based on the general purpose Tango libs (for instance ATK), which will first try to subscribe to events. But if a client just reads your "voltage", e.g. invokes read_attribute, this will cause the device server to execute the whole method stack, possibly down to accessing the equipment (unless special care is taken in the code…). Cheers, Lorenzo

# 3 years ago

lorenzo

Hi Leonard, Mariocaptain,
basically, you're correct. At device level, talking about server side now, Tango exploits "device polling" for continuously monitoring the equipment (e.g. reading the voltage), comparing the current value with the previous one and sending the relevant events (change, archive,…) to the registered clients whenever the difference is beyond the threshold. Thresholds can be configured on a per-attribute basis. This is the default mechanism for sending events, and it is just matter of configuration, no code required.
Tango provides also a mechanism to "push events in the code", e.g. allows the developer to trigger sending events on demand. This can be useful, for instance, if you have an equipment that's sending data asynchronously to the Tango device and you want to react immediately to the new data, informing the clients. This, however, requires writing a few lines of code in the relevant methods.
Back to question 3, yes, you can decide not to configure device polling and provide the necessary code to push all the required events, but this can have undesired effects. Proper clients, e.g. clients specifically written to subscribe to events will behave correctly, and so will clients based on the general purpose Tango libs (for instance ATK), which will first try to subscribe to events. But if a client just reads your "voltage", e.g. invokes read_attribute, this will cause the device server to execute the whole method stack, possibly down to accessing the equipment (unless special care is taken in the code…).
Cheers,
Lorenzo

# 3 years ago
mariocaptain	Hi Lorenzo & Leonard, Thanks for your input, which makes it clearer for me now. Also, reading the Device Polling part from the tango documentation again makes it clear that polling is the underlying and must have for device servers. The events mechanism also need this underlying polling to work. The best news is that tango handles all this automatically, so my understanding to share is that let tango takes care of all the polling, and we only define events as needed in our clients.

# 3 years ago

mariocaptain

Hi Lorenzo & Leonard,

Thanks for your input, which makes it clearer for me now.
Also, reading the Device Polling part from the tango documentation again makes it clear that polling is the underlying and must have for device servers. The events mechanism also need this underlying polling to work.

The best news is that tango handles all this automatically, so my understanding to share is that let tango takes care of all the polling, and we only define events as needed in our clients.