Tango Logging Service and Python Logging

# 6 years ago
Lochan	Hi all, I am working on the logging aspect in Tango. I have used both Tango Logging Service (for viewing logs) and Python logging (for syslog storage). But to achieve this, I need to write two statements in the device server code using both services like: `#For TLS (viewer) self.fatal_stream("%s", logmsg) #For python (syslog) logger.fatal("%s", logmsg)` Similarly for other levels (DEBUG, INFO, ERROR, WARN). The intention is to merge the two logging services. That is, establishing one logging channel, either Tango or Python to log into different targets (Tango device and syslog). Example: `self.fatal_stream("%s", logmsg)` should send messages to both syslog and logviewer. I explored some alternatives. Like Python Log Viewer (2011 version), that displays logs and also filters logs on the basis of different levels. I have been going through these links and trying to explore more. The idea behind using this, is that, if we use Python logging for syslog and Python log viewer for viewing, our both purposes might be fulfilled without writing statements for two services separately. The other alternative is adding syslog as a target in TLS apart from file, console and device. Let me know, in case anyone has tried it before, and, whether it is a right approach to move forward. Also, I wanted to know: Has Tango community already planned to update Tango Logging Service with additional features in next release (v10) ? If we opt for modifying the existing Tango logging service to include syslog as an additional target, how should we go for it? -Lochan

# 6 years ago

Hi all,

I am working on the logging aspect in Tango. I have used both Tango Logging Service (for viewing logs) and Python logging (for syslog storage). But to achieve this, I need to write two statements in the device server code using both services like:

#For TLS (viewer)
self.fatal_stream("%s", logmsg)

#For python (syslog)
logger.fatal("%s", logmsg)

Similarly for other levels (DEBUG, INFO, ERROR, WARN).

The intention is to merge the two logging services. That is, establishing one logging channel, either Tango or Python to log into different targets (Tango device and syslog).

Example:

self.fatal_stream("%s", logmsg)

should send messages to both syslog and logviewer.

I explored some alternatives. Like Python Log Viewer (2011 version), that displays logs and also filters logs on the basis of different levels. I have been going through these links and trying to explore more. The idea behind using this, is that, if we use Python logging for syslog and Python log viewer for viewing, our both purposes might be fulfilled without writing statements for two services separately.

The other alternative is adding syslog as a target in TLS apart from file, console and device.

Let me know, in case anyone has tried it before, and, whether it is a right approach to move forward.

Also, I wanted to know:

Has Tango community already planned to update Tango Logging Service with additional features in next release (v10) ?
If we opt for modifying the existing Tango logging service to include syslog as an additional target, how should we go for it?

-Lochan

# 6 years ago
Reynald	Lochan 1. Has Tango community already planned to update Tango Logging Service with additional features in next release (v10) ? As far as I remember, there is nothing big planned for the moment on that topic… but contributions and ideas are of course welcome. Lochan 2. If we opt for modifying the existing Tango logging service to include syslog as an additional target, how should we go for it? Log4Tango was originally based on log4cpp project. I noticed there seems to be already a special appender (RemoteSyslogAppender) for syslogs in log4cpp project. It might be interesting to have a look at it and see whether we could integrate it and adapt it into our cppTango project where log4tango code is currently stored. It looks like this RemoteSyslogAppender was already existing in the log4cpp version which was used to build log4tango but it seems this appender has been removed. I don't know why. Maybe one of the dinosaurs or great archeologist from our community knows why? Hoping this helps a bit. Kind regards, Reynald Rosenberg's Law: Software is easy to make, except when you want it to do something new. Corollary: The only software that's worth making is software that does something new.

# 6 years ago

Reynald

Lochan
1. Has Tango community already planned to update Tango Logging Service with additional features in next release (v10) ?

As far as I remember, there is nothing big planned for the moment on that topic… but contributions and ideas are of course welcome.

Lochan
2. If we opt for modifying the existing Tango logging service to include syslog as an additional target, how should we go for it?

Log4Tango was originally based on log4cpp project.
I noticed there seems to be already a special appender (RemoteSyslogAppender) for syslogs in log4cpp project.
It might be interesting to have a look at it and see whether we could integrate it and adapt it into our cppTango project where log4tango code is currently stored.

It looks like this RemoteSyslogAppender was already existing in the log4cpp version which was used to build log4tango but it seems this appender has been removed. I don't know why. Maybe one of the dinosaurs or great archeologist from our community knows why?

Hoping this helps a bit.
Kind regards,
Reynald

Rosenberg's Law: Software is easy to make, except when you want it to do something new.
Corollary: The only software that's worth making is software that does something new.

# 6 years ago
nleclercq	Lochan, In order to solve the problem without having to wait for someone to reinject the RemoteSyslogAppender into Tango, you could write a python logging handler forwarding your logging events to the TLS. All you need is a handler owning a reference to your tango device (or simply to its logger - in case PyTango exposes it). BTW, we decided to remove most of the log4cpp appenders cause our design dogma was 100% Tango centric - any single byte exchanged between two Tango entities was supposed to be carried by the Tango protocol. It was also a way to lighten the dependency. However, it should not be a big deal to add the RemoteSyslogAppender back to log4tango. Hope this help. n. Edited 6 years ago

# 6 years ago

nleclercq

Lochan,

In order to solve the problem without having to wait for someone to reinject the RemoteSyslogAppender into Tango, you could write a python logging handler forwarding your logging events to the TLS. All you need is a handler owning a reference to your tango device (or simply to its logger - in case PyTango exposes it).

BTW, we decided to remove most of the log4cpp appenders cause our design dogma was 100% Tango centric - any single byte exchanged between two Tango entities was supposed to be carried by the Tango protocol. It was also a way to lighten the dependency. However, it should not be a big deal to add the RemoteSyslogAppender back to log4tango.

Hope this help.

n.

Edited 6 years ago

# 6 years ago
Lochan	Thank you for your response Reynald and Nicolas. Reynald As far as I remember, there is nothing big planned for the moment on that topic… but contributions and ideas are of course welcome. I read about specifying different logging levels per target here. I was just wondering if such feature is finalized to be added into Tango 10. nleclercq In order to solve the problem without having to wait for someone to reinject the RemoteSyslogAppender into Tango, you could write a python logging handler forwarding your logging events to the TLS. All you need is a handler owning a reference to your tango device (or simply to its logger - in case PyTango exposes it). Yes Nicolas, I have done the similar thing. `logger = logging.getLogger() logger.setLevel(logging.DEBUG) syslog = SysLogHandler(address='/dev/log', facility= 'syslog') formatter = logging.Formatter('%(name)s: %(levelname)s %(module)s %(message)r') syslog.setFormatter(formatter) logger.addHandler(syslog)` But this again forces me to use both Tango logging and Python logging in same source code. I intend to use any one of them. -Lochan Edited 6 years ago

# 6 years ago

Lochan

Thank you for your response Reynald and Nicolas.

Reynald
As far as I remember, there is nothing big planned for the moment on that topic… but contributions and ideas are of course welcome.

I read about specifying different logging levels per target here. I was just wondering if such feature is finalized to be added into Tango 10.

nleclercq
In order to solve the problem without having to wait for someone to reinject the RemoteSyslogAppender into Tango, you could write a python logging handler forwarding your logging events to the TLS. All you need is a handler owning a reference to your tango device (or simply to its logger - in case PyTango exposes it).

Yes Nicolas, I have done the similar thing.

logger = logging.getLogger()
logger.setLevel(logging.DEBUG)
syslog = SysLogHandler(address='/dev/log', facility= 'syslog')
formatter = logging.Formatter('%(name)s: %(levelname)s %(module)s %(message)r')
syslog.setFormatter(formatter)
logger.addHandler(syslog)

But this again forces me to use both Tango logging and Python logging in same source code.
I intend to use any one of them.

-Lochan

Edited 6 years ago

# 6 years ago
nleclercq	Lochan, I didn't clearly explain my proposal. What I propose is to write your own Python handler - i.e. to implement a TangoHandler class inheriting from Python logging.Handler. You have to __init__ialize your TangoHandler instance passing a reference to your Tango device and you're almost done - i.e. in TangoHandler.emit just forward the log record to the TLS. See https://dzone.com/articles/python-custom-logging-handler-example for an example. n. Edited 6 years ago

# 6 years ago

nleclercq

Lochan,
I didn't clearly explain my proposal. What I propose is to write your own Python handler - i.e. to implement a TangoHandler class inheriting from Python logging.Handler. You have to __init__ialize your TangoHandler instance passing a reference to your Tango device and you're almost done - i.e. in TangoHandler.emit just forward the log record to the TLS.
See https://dzone.com/articles/python-custom-logging-handler-example for an example.
n.

Edited 6 years ago

# 6 years ago
Lochan	Got your point Nicolas. Would this work even if the devices are written in C++/Java? Since we are using Python logging for the new handler? -Lochan

# 5 years ago
nleclercq	You can talk to any device from python whichever is its implementation language However, reading your last question I have a doubt about my understanding of your use case… Hope everything is clear on your side Edited 5 years ago