Panic-The Alba Alarm System
srubio avatar
sergi_rubio

At ALBA we have several ways to get logs from PyAlarm:

* Using the SNAP database, but as it is not multihost you can't use it.

* Reading the PastAlarms attribute of the PyAlarm devices. It contains all the alarms triggered since the device was restarted. It should work for you, as taurus can get attributes from other tango hosts .

* Generate an Alarm log file; using the LogFile property of the PyAlarm server you can save all PyAlarm actions to a file. As it is file-based you won't need to care about tango hosts; but it is very hard to maintain in the long term.

* Using a mail archive. We have some internal email addresses (e.g. alarms@vacuum) that record every single mail sent to them (just like the tango mailing list archives). Adding this address to all alarms allow us to have a full-log of all messages sent. The advantage of this method is that it keeps record of both the message and the receivers in CC. All logs are centralized in the mail service, so it is much easier to maintain.


If you want my advice, I would use a mix of the 2nd (for taurus gui's) and 4th method (for long-term logs).
Keep on dancing,

http://www.tango-controls.org/resources/howto/how-fandango/
vhardion avatar
hardion
Hi,

We are also using the Alba Alarm system at MAX IV.
For the long term storage of the alarm we use an elastic search database + kibana for the UI. Our user are quite happy about kibana although this is not integrated into any taurus gui (for that we are using Panic).
This has the advantage to easily define query and to see the events over a time.

my 20cts,
Vincent
Vincent Hardion
Control System
MAX IV Laboratory
avatar par défaut
johfor
Just to expand on Vincent's reply:

we've added a tiny feature to PyAlarm which pushes each alarm event as a JSON document to a simple "logger" device (using a command), which in turn stores the event in elasticsearch. The historical data can then be viewed through the kibana web UI, where users can do various filtering and also set up specific views. So far it has been pretty solid, with very low maintenance.

I'm attaching a kibana screenshot from our controlroom. The UI is a bit strange but powerful once you get used to it. However, the main benefit is that we're not developing it ourselves :)

Caveat: we're currently using ES 1.X and kibana 3, but the current version of ES is 2.X and kibana 3 is no longer compatible. Kibana 4 is a complete rewrite and works quite differently, with an even more confusing UI. We're not sure whether to migrate or how.
Attachments:
srubio avatar
sergi_rubio
Very nice tool, Johan

It looks very nice, and we would be very glad to integrate this tools in the main branch of PyAlarm (maybe as an optional feature like the smslib).

I'm working in the next Panic major release, so if you can send me some link and/or notes I will give it a try.

Thanks
Keep on dancing,

http://www.tango-controls.org/resources/howto/how-fandango/
avatar par défaut
CoraF
Hello,

I just started to try out PANIC. Has there been any news on what you discussed last for logging alarms?
In principle I am not so much interested in archiving, but since this SNAP is a feature of the PANIC gui, I would like to understand how I can make it work.
What I did was downloading Tango Java Archiver (I only sourced SNAp-related sql-files from db-folder)
installed MYSQL-python
installed PyTangoArchiving
I changed the properties of my PyAlarm DS to 'UseSnap: True', 'CreateNewContexts: True'

Now, when I start PyAlarm, one message says: "Creating SnapAPI object …/n ERROR:snapArchiver.DbHost property not defined"
I guess that means I need to add this property to the PyAlarm DS but am unsure of what it needs to be.
In the PANIC gui clicking on "Alarm History Viewer" a dialog-window tells me that snapshots are not available and more specifically, when I click "Open Snapshot" the message is "Could not establish connection to SnapManagerDS. Please check if DS is running or if credentials are correct".
I certainly didn't have this DS running. Where is this one? What do I need to do here? Do I need additional packages? How do these snaps look like in the gui? I imagine that they would just look like the last-report of certain alarms, only logged so that the reports are kept.

Thanks for enlightening me and sorry if there is already an easy-to-understand documentation that I missed.
avatar par défaut
pgoryl
Hi CoraF,

It sounds like the SNAP archiving itself is not properly configured.
Have you tried to run Bensikin (https://tango-controls.readthedocs.io/en/latest/tools-and-extensions/archiving/bensikin.html?highlight=Bensikin) to check if you can manually create some contexts and then take some snapshots?
You may need to dig into SNAP documentation which is included in ArchivingRoot.zip, for how to setup SNAP.

Below are example settings from the TangoBox VM:

SnapManager class properties:
  • DbHost: tangobox-archive
  • DbName: snap
  • DbSchema: snap
  • isRac: false
And archiving/snap/snapmanager.1 device propereties:
  • DbHost: tangobox-archive
  • DbName: snap
  • DbPassword: snapmanager
  • DbSchema: snap
  • DbUser: snapmanager

And SnapArchiver archiving/snap/snaparchiver.01_01 properties (I use to setup both device and class):
  • beansFileName: beansBeamline.xml
  • DbHost: tangobox-archive
  • DbName: snap
  • DbPassword: snaparchiver
  • DbSchema: snap
  • DbUser: snaparchiver

Best regards,
Piotr
avatar par défaut
CoraF
Thanks, Piotr!

I don't know anything about archiving tools, so really my dumb question: How to run those three Snap device servers and how to run bensikin? If I am not blind the doc on SNAP is mostly a user guide on Bensikin and doesn't explain how to launch it actually. Where are the executables? How to setup the database (ok, for this I would take your input for the properties)?
Thanks!
avatar par défaut
pgoryl
Hi, I didn't manage to answer quickly and unfortunately (for this answer) I am starting week of vacations (which is fortunate for me smile ). Yes, there is missing documentation on how to make SNAP running. It is on pending list for documentation project.
So, to be quick, you may look on TangoBox archiving container related scripts for reference: https://github.com/tango-controls/tangobox/tree/master/docker/archive, be careful there is (as far as I remember) a bug for beansFileName property value. The working value is as the above: beansBeamline.xml

Best regards,
Piotr
abence avatar
ABence
I'm testing PANIC 6.5.1 in tangoBox 9.2.
I create devices with same pattern like in network Machine Control at SOLEIL
the pattern is X/Y/Z.<number>
but alarm with comprehensionList doesn't work

the same pattern without dot work

('obj', Alarm(TC_UpperFixedValue:test/pyalarm/1:1539779848.0))
('device', 'test/pyalarm/1')
('formula', 'any( [ s > 21 for s in FIND( test/VI/TC*/Value )])')
('test', 'any( [ s > 21 for s in FIND( test/VI/TC*/Value )])')
TangoEval: TangoEval.parse_formula: Replacing FIND( test/VI/TC*/Value ) with [test/vi/tc.01/value, test/vi/tc.02/value, test/vi/tc.03/value, test/vi/tc.04/value, test/vi/tc.05/value, test/vi/tc.06/value]
[b]TangoEval: parse_variables(…): [('test/vi/tc', '', ''), ('test/vi/tc', '', ''), ('test/vi/tc', '', ''), ('test/vi/tc', '', ''), ('test/vi/tc', '', ''), ('test/vi/tc', '', '')][/b]


('obj', Alarm(one_TC_UpperFixedValue2:test/pyalarm/1:1539782149.0))
('device', 'test/pyalarm/1')
('formula', 'any( [ s > 21 for s in FIND( test/FC/TC*/Value )])')
('test', 'any( [ s > 21 for s in FIND( test/FC/TC*/Value )])')
TangoEval: TangoEval.parse_formula: Replacing FIND( test/FC/TC*/Value ) with [test/fc/tc01/value, test/fc/tc02/value, test/fc/tc03/value]
TangoEval: parse_variables(…): [('test/fc/tc01', 'value', ''), ('test/fc/tc02', 'value', ''), ('test/fc/tc03', 'value', '')]

the fomula is good ? or i make a mistake?
avatar par défaut
pgoryl
Hi,
Unfortunately, it looks like it is a way TangoEval deals with dots in names, at this moment.
Looking through fandango code, "parse_variables" is accepting only alphanumeric (a-zA-Z0-9-_) for device name parts.

Fortunately, It seems that it is enough to extend a regex in the line:
redev = '(?P<device>(?:'+alnum+':[0-9]+/{1,2})?(?:'+'/'.join([alnum]*3)+'))'
.
To: 
redev = '(?P<device>(?:'+alnum+':[0-9]+/{1,2})?(?:'+'/'.join([alnum]*3)+'(?:\.[0-9]+)?))'

in fandandgo/tango/defaults.py and tangoeval.py

I will prepare a related pull request for fandango.
 
Register or login to create to post a reply.