No heartbeat error on Event Subscription

# 9 years ago
Vatsal	Dear Andy, Yes I have switched the firewall OFF on my windows system. Also I opened the error log from the ATK Panel for my device. There are no errors in the error log. I checked in the Diagnostic Window. It says "tango://192.168.118.210:20000/dserver/JDeviceForEvent/jdEvt1 has no event channel defined in the database 192.168.118.210:20000 May be the server is not running." I have attached the snapshot of the Diagnostic Window and Error Log. Could the issue be because the event channel is not defined ? If that is the case then please suggest the way of defining the event channel in the database. In the Appendix D (Section D.3 and D.4) of the Tango Control System manual (v8.1) it is mentioned that the event channel is required for tango release prior to version 8. I've installed tango v 8.1.2 and I believe it uses ZMQ for events. Also the Tango jar (Tango-9.0.3 jar) file which I'm using has all the API for ZMQ. So, I'm not able to understand why the Diagnostic window is showing this error. Also is there any way of specifying the logging level on the client side ? The detailed log on the client side might help you in understanding the issue better. Also I want to know if I am the only one who is facing this issue ? Are you not able to reproduce the issue on any system ? Regards, Vatsal Attachments:

# 9 years ago

Dear Andy,

Yes I have switched the firewall OFF on my windows system.

Also I opened the error log from the ATK Panel for my device. There are no errors in the error log.

I checked in the Diagnostic Window. It says "tango://192.168.118.210:20000/dserver/JDeviceForEvent/jdEvt1 has no event channel defined in the database 192.168.118.210:20000 May be the server is not running."

I have attached the snapshot of the Diagnostic Window and Error Log.

Could the issue be because the event channel is not defined ? If that is the case then please suggest the way of defining the event channel in the database.

In the Appendix D (Section D.3 and D.4) of the Tango Control System manual (v8.1) it is mentioned that the event channel is required for tango release prior to version 8. I've installed tango v 8.1.2 and I believe it uses ZMQ for events. Also the Tango jar (Tango-9.0.3 jar) file which I'm using has all the API for ZMQ.

So, I'm not able to understand why the Diagnostic window is showing this error.

Also is there any way of specifying the logging level on the client side ? The detailed log on the client side might help you in understanding the issue better.

Also I want to know if I am the only one who is facing this issue ? Are you not able to reproduce the issue on any system ?

Regards,
Vatsal

# 9 years ago
Andy	Dear Vatsal, I have downloaded your server and test client and run them on my Ubuntu system. The events work on my system. One minor problem was the while loop in your client does not sleep and uses 100% of the cpu! So the events problem is not with your server or client but rather with your setup. We still have to understand why. I used JTangoServer-1.1.7-all.jar which I got from the sourceforge download site. I had errors compiling with the version of the server you pointed to in your initial post (log4j etc were missing). Here is the screenshot of the server and client running in my eclipse workbench. I have included the relevant windows for jive and atkpanel showing how they should look when events work. Edited 9 years ago Attachments:

# 9 years ago

Andy

Dear Vatsal,

I have downloaded your server and test client and run them on my Ubuntu system. The events work on my system. One minor problem was the while loop in your client does not sleep and uses 100% of the cpu!

So the events problem is not with your server or client but rather with your setup. We still have to understand why.

I used JTangoServer-1.1.7-all.jar which I got from the sourceforge download site. I had errors compiling with the version of the server you pointed to in your initial post (log4j etc were missing).

Here is the screenshot of the server and client running in my eclipse workbench. I have included the relevant windows for jive and atkpanel showing how they should look when events work.

Edited 9 years ago

# 9 years ago
Andy	Vatsal, some more answers inline: Alucard Yes I have switched the firewall OFF on my windows system. Good Alucard Also I opened the error log from the ATK Panel for my device. There are no errors in the error log. I checked in the Diagnostic Window. It says "tango://192.168.118.210:20000/dserver/JDeviceForEvent/jdEvt1 has no event channel defined in the database 192.168.118.210:20000 May be the server is not running." I don't know where this error is coming from. It sounds suspiciously like an error message from the old corba events system. Are you sure you are not including an old TangORB in your classpath? Alucard Also is there any way of specifying the logging level on the client side ? The detailed log on the client side might help you in understanding the issue better. Yes there is via environment variables. I have forgotten how … Alucard Also I want to know if I am the only one who is facing this issue ? Are you not able to reproduce the issue on any system ? I cannot reproduce it on Linux for now. But by persevering we will get to the bottom of this. Andy

# 9 years ago

Andy

Vatsal,

some more answers inline:

Alucard
Yes I have switched the firewall OFF on my windows system.

Good

Alucard
Also I opened the error log from the ATK Panel for my device. There are no errors in the error log.

I checked in the Diagnostic Window. It says "tango://192.168.118.210:20000/dserver/JDeviceForEvent/jdEvt1 has no event channel defined in the database 192.168.118.210:20000 May be the server is not running."

I don't know where this error is coming from. It sounds suspiciously like an error message from the old corba events system. Are you sure you are not including an old TangORB in your classpath?

Alucard
Also is there any way of specifying the logging level on the client side ? The detailed log on the client side might help you in understanding the issue better.

Yes there is via environment variables. I have forgotten how …

Alucard
Also I want to know if I am the only one who is facing this issue ? Are you not able to reproduce the issue on any system ?

I cannot reproduce it on Linux for now. But by persevering we will get to the bottom of this.

Andy

# 9 years ago
Andy	I have managed to reproduce the problem! When using the TangORB-9.0.3a.jar I get the same error from your client as you: home/goetz/workspaces/tango/JDeviceForEvent/bin:/home/goetz/tango/jevents/TangORB-9.0.3-a.jar:/home/goetz/tango/jevents/JTangoServer-1.1.7-all.jar:/usr/share/java/jayatanaag.jar Device name: JD/Evt/1 ====================== ZMQ (3.22) event system is available ============================ tcp://127.0.0.1:51103 Connected !!!! Tue Aug 11 10:05:01 CEST 2015 Event Name: JD/Evt/1/speed Event: change Event Type: 0 Event Source: 0 Event Error: false Attribute Value: 0.0 ——————————————————————– tango://pc35.home:10000/dserver/jdeviceforevent/test Not found tango://pc35.home:10000/dserver/jdeviceforevent/test Not found Tue Aug 11 10:05:21 CEST 2015 Event Name: JD/Evt/1 Event: change Event Type: 0 Event Source: 0 Event Error: true Error list size: 1 Error 1 Description: No heartbeat from dserver/jdeviceforevent/test Error 1 Severity: ERR Error 1 Reason: API_NoHeartbeat Error 1 Origin: ZmqEventConsumer.checkIfHeartbeatSkipped() I propose you replace this jar with the latest stable one from sourcforge. You can download it from here: http://sourceforge.net/projects/tango-cs/files/JTango/JTangoServer-1.1.7-all.jar/download Let me know if this fixes your problem. Andy

# 9 years ago

Andy

I have managed to reproduce the problem! When using the TangORB-9.0.3a.jar I get the same error from your client as you:

home/goetz/workspaces/tango/JDeviceForEvent/bin:/home/goetz/tango/jevents/TangORB-9.0.3-a.jar:/home/goetz/tango/jevents/JTangoServer-1.1.7-all.jar:/usr/share/java/jayatanaag.jar
Device name: JD/Evt/1
====================== ZMQ (3.22) event system is available ============================
tcp://127.0.0.1:51103 Connected !!!!
Tue Aug 11 10:05:01 CEST 2015
Event Name: JD/Evt/1/speed
Event: change
Event Type: 0
Event Source: 0
Event Error: false
Attribute Value: 0.0


——————————————————————–


tango://pc35.home:10000/dserver/jdeviceforevent/test Not found
tango://pc35.home:10000/dserver/jdeviceforevent/test Not found
Tue Aug 11 10:05:21 CEST 2015
Event Name: JD/Evt/1
Event: change
Event Type: 0
Event Source: 0
Event Error: true
Error list size: 1
Error 1 Description: No heartbeat from dserver/jdeviceforevent/test
Error 1 Severity: ERR
Error 1 Reason: API_NoHeartbeat
Error 1 Origin: ZmqEventConsumer.checkIfHeartbeatSkipped()

I propose you replace this jar with the latest stable one from sourcforge. You can download it from here:

http://sourceforge.net/projects/tango-cs/files/JTango/JTangoServer-1.1.7-all.jar/download

Let me know if this fixes your problem.

Andy

# 9 years ago
Vatsal	Dear Andy, I downloaded the new stable JTangoServer-1.1.7-all.Jar from the link provided by you. I have also modified the client code as per your suggestion. Instead of infinite loop I have added the following code: `while(true) { Thread.sleep(20000); }` I have attached the new client code and the same device server code. It produced some strange result. The event channel error in the ATKPanel Diagnostic Window is now resolved. I've attached the snapshot of the ATKPanel and now it looks exactly same as yours. On the client side I don't get the Heartbeat error at every 10 seconds but I get the value of the Speed variable even though it has not been changed. So behavior seems like that the event is periodic in nature (although it is not). I have attached the client log and the device server log. The device server log indicates that the "ZmqEventSubscriptionChange" command of the admin device is called at fixed interval by the client and it results in re-subscription of the event and a synchronous read of the attribute value. I inferred it from the regular repetition of the following line in the device server log `REQUEST 2015-08-11 15:59:09,816 [dserver/JDeviceForEvent/jdEvt1] - Operation command_inout_4 (cmd = ZmqEventSubscriptionChange) from cache_device requested from PC5-HP.ncra.tifr.res.in (Java client with main class org.tango.console.TestClient.TestClient_Console - PID=7456)` I appreciate the efforts the members of Tango community are putting in to resolve the issue. Regards, Vatsal Trivedi

# 9 years ago

Vatsal

Dear Andy,

I downloaded the new stable JTangoServer-1.1.7-all.Jar from the link provided by you. I have also modified the client code as per your suggestion. Instead of infinite loop I have added the following code:

while(true)
{
    Thread.sleep(20000);
}

I have attached the new client code and the same device server code. It produced some strange result.

The event channel error in the ATKPanel Diagnostic Window is now resolved. I've attached the snapshot of the ATKPanel and now it looks exactly same as yours.

On the client side I don't get the Heartbeat error at every 10 seconds but I get the value of the Speed variable even though it has not been changed. So behavior seems like that the event is periodic in nature (although it is not).

I have attached the client log and the device server log. The device server log indicates that the "ZmqEventSubscriptionChange" command of the admin device is called at fixed interval by the client and it results in re-subscription of the event and a synchronous read of the attribute value. I inferred it from the regular repetition of the following line in the device server log

REQUEST 2015-08-11 15:59:09,816 [dserver/JDeviceForEvent/jdEvt1] - Operation command_inout_4 (cmd = ZmqEventSubscriptionChange) from cache_device requested from PC5-HP.ncra.tifr.res.in (Java client with main class org.tango.console.TestClient.TestClient_Console - PID=7456)

I appreciate the efforts the members of Tango community are putting in to resolve the issue.

Regards,
Vatsal Trivedi

# 9 years ago
Pascal	Hi You wrote: Hostname:-> PC5-HP TANGO_HOST :-> 192.168.118.210:20000 Did you try with: TANGO_HOST=PC5-HP:20000 ?

# 9 years ago
Vatsal	Hi, I was not able to attach the client log and the Device Server log due to the size issue along with my previous post so I have attached the logs with this post. I also want to tell that currently the system I'm using is in a workgroup and not in a domain. Also there is no DNS mapping corresponding to the hostname and IPAddress of my system in the DNS server. I have manually added the entry in the hosts file of my system. All the systems in my office are on the same LAN and they internally uses Link-Local Multicast Name Resolution(LLMNR) protocol to resolve the hostname and IPAddress. I'm not sure whether the information I provided in the above paragraph will be useful to you. I just thought that it might help you in understanding my network configuration better. Regards, Vatsal Trivedi Attachments: 820Kb 5Kb 15Kb

# 9 years ago

Vatsal

Hi,

I was not able to attach the client log and the Device Server log due to the size issue along with my previous post so I have attached the logs with this post.

I also want to tell that currently the system I'm using is in a workgroup and not in a domain. Also there is no DNS mapping corresponding to the hostname and IPAddress of my system in the DNS server. I have manually added the entry in the hosts file of my system. All the systems in my office are on the same LAN and they internally uses Link-Local Multicast Name Resolution(LLMNR) protocol to resolve the hostname and IPAddress.

I'm not sure whether the information I provided in the above paragraph will be useful to you. I just thought that it might help you in understanding my network configuration better.

Regards,
Vatsal Trivedi

# 9 years ago
Andy	Why do you specify the TANGO_HOST with the ip address instead of the ip name? Is the name resolution working? If this is not working due to your setup then indeed it will be difficult for the server to contact the client using the ip name. This would explain why events aren't working … Try to make ip name resolution work for your PC or try on another PC. Andy Edited 9 years ago

# 9 years ago
Vatsal	Hi, @Pascal I tried by using Hostname (PC5-HP) instead of IP Address in the TANGO_HOST environment variable. It does not resolve the issue. @Andy I also tried your suggestion of changing the hostname to PC5HP. It does not help either. If some new ideas come to your mind let me know I'll keep on trying. Thanks for your continuous support. Regards, Vatsal Trivedi

# 9 years ago
Andy	I am not sure I understood. Can you resolve the HP-PC5 hostname from a client e.g. does ping HP-PC5 work? When you changed the TANGO_HOST to the ip name what changed? Does jive and atkpanel still work? In your last screenshot you showed the TANGO_HOST=192.168.118.210:20000. Can you still use jive and your client with TANGO_HOST=PC5-HP.ncra.tifr.res.in:20000 i.e. the Fully Qualified Domain Name (FQDN)? The way the network connection works for TANGO events is that the device server will try to build a connection to the client using the FQDN hostname of the client. If it cannot resolve this name then the server cannot send events to the client. I am more and more convinced this is your problem. Possible solutions are: (1) use /etc/hosts and add an alias for PC5-HP.ncra.tifr.res.in for the ip address of the pc (2) use /etc/hosts and change the hostname to be PC-HP5 and have an entry in /etc/hosts for this host (3) make DNS work correctly so that you can resolve the FQDN to the ip address In ALL cases the host name displayed in the log output must be resolvable for events to work. Andy Edited 9 years ago

# 9 years ago

Andy

I am not sure I understood. Can you resolve the HP-PC5 hostname from a client e.g. does ping HP-PC5 work?

When you changed the TANGO_HOST to the ip name what changed? Does jive and atkpanel still work? In your last screenshot you showed the TANGO_HOST=192.168.118.210:20000. Can you still use jive and your client with TANGO_HOST=PC5-HP.ncra.tifr.res.in:20000 i.e. the Fully Qualified Domain Name (FQDN)?

The way the network connection works for TANGO events is that the device server will try to build a connection to the client using the FQDN hostname of the client. If it cannot resolve this name then the server cannot send events to the client. I am more and more convinced this is your problem.

Possible solutions are:

(1) use /etc/hosts and add an alias for PC5-HP.ncra.tifr.res.in for the ip address of the pc

(2) use /etc/hosts and change the hostname to be PC-HP5 and have an entry in /etc/hosts for this host

(3) make DNS work correctly so that you can resolve the FQDN to the ip address

In ALL cases the host name displayed in the log output must be resolvable for events to work.

Andy

Edited 9 years ago